It may seem that bug testing is the sole responsibility of developers. But more and more, teams are finding that additional value can be extracted by involving the entire company. If the first real user of your product is a paying customer, you may have missed an opportunity to uncover issues in your software before it gets in their hands. That’s where a company-wide Bug Bash can help.
A Bug Bash is a process popularised by Ron Patton way back in 2001. A Bug Bash is an opportunity for the entire team – including designers, marketing, support – to “pound on the product”. So a Bug Bash isn’t so much about bashing bugs, it’s about bashing the product to find any undiscovered bugs.
What makes a Bug Bash effective?
No matter how diligent your developers are, how accurate your unit tests are, or how complete your integration testing is, there will always be hidden problems. What makes sense to a developer does not mean it is correct from a user point of view. The same goes for unit tests, integrations test, just because the code is right, doesn’t mean the product is. A Bug Bash is a chance to test workflows, inputs and outputs to see if they make sense for a human. It’s beta testing, acceptance testing and limit testing all at once.
The most important part of a Bug Bash is to ensure you have a diverse team of people doing the bashing. Does an email field allow email address with unusual domains? Do name fields appropriately handle “non-western” formats? Is the app usable by the colour blind, or vision impaired or mobility impaired? Do workflows and processes make sense to a novice user? If a user does something unexpected, does the software recover correctly? While you should cater for these things during the design and development process, the reality of software is that sometimes issues are missed. You don’t want to find out once it’s in the hands of the end user.
The most obvious example of this sort of process is when an agency runs an acceptance testing process with a client. Not only do we care that we get what we paid for, as a customer, but we also care that our users will be able to use the product we’ve paid to have built! So there are multiple levels of “right”. There’s what we paid for, what we wanted, and most importantly, what makes sense for our users. The best process in the world isn’t perfect, and a Bug Bash program is a great “last line of defence”.
How do you run a Bug Bash?
There are a bunch of ways to run Bug Bash programs. You can run them with beta users, alpha users or internal teams. You can run programs as competitions or games, or as a private “crowd test” session. The motivators don’t matter, although it can certainly help make the process more fun for your testers. The important thing is that testers:
a) understand the scope of the testing exercise (i.e. which bits of the product they should be testing)
b) have a timeline for testing (i.e. that they have two days to test)
c) have a consistent, easy to understand process for reporting issues (i.e. you don’t want to put your CEO in front of a complicated bug tracker)
A Bug Bash is never going to be a replacement for proper testing processes. Unit tests and integration tests are critical to ensuring bug-free code, but a Bug Bash is an excellent addition to your test line up. If you can pull one together, you have a quick and relatively cheap way to ensure your product will be ready to go by the time it gets into the hands of end-users. And of course, BugHerd is a great way to get one done. Try it now!