In FAQ 5 I wrote about game testers, the unsung heroes of the game industry. In FAQ 17 Matthew Burns wrote about the view from inside game QA. In this article, my goal is to teach you how to write a good bug report. I teach video game quality assurance at the University of Southern California, one of the top game schools in the world. I've read a lot of bug reports in my time, and written a lot of them too. If you ever want to get a QA job at a game company, you may need to write a bug report (also called a "bug") as a test of your communication skills.
QA departments use an online database for bug reporting. There are several different tools used for this: DevTrack, Mantis, JIRA, FogBugz, to name just a few. When a tester encounters a problem in the game he (or she) is testing, s/he then writes a report of the problem in the database. More about that in a bit (that being the main focus of this article).
Once some new bugs have been written, the lead tester (the senior tester who leads the test effort on the game) filters the bugs. That means that the lead tester reads the bugs to make sure they're "good" bugs. A "good" bug is not a duplicate of a bug already written by someone else, it describes a real problem worth looking into, and it contains enough information so that the developer (more on what this means below) can see the problem and fix it.
The game's producer or project manager also filters the bugs. It wouldn't do for bad bugs to be reported to the development team. Bad bugs waste time.
Once the bugs have been filtered, they are released to the development team. Development leads (the lead artist, the lead programmer, the lead designer) read the bugs appropriate to their own areas of responsibility and assign them to individual developers to fix. So if it's an art bug, the lead artist knows which artist is responsible for that art asset, so the lead artist assigns the bug to that artist. If it's a programming bug, the lead programmer can probably tell which programmer wrote the code that needs to be fixed.
The developer (the person to whom the bug was assigned, be s/he an artist, designer, programmer) is notified that a bug has been assigned. Then the developer opens the database and reads the bug. Assuming the bug is well-written enough that the developer understands and accepts the validity of the problem, the developer then fixes the bug, and clicks the "Fixed" selection in the database.
If the developer reads the bug and has a question that needs to be answered in order to fully understand the problem, the developer can post the question directly in the bug and send it back to the tester. Once the tester answers the question, the bug bounces back into the developer's court. This little game of badminton continues until the developer has enough information and can fix the bug. Hopefully, the tester provides enough information in the initial report so that no back-and-forth is even necessary.
Once a bug has been fixed, the bug goes back to the tester who wrote it for retesting. If the tester determines that the bug is indeed fixed, he or she clicks the "Verified Fixed" selection, and the bug is on its way to being "Closed." If the tester determines that there is still a problem, he or she "Reopens" the bug, and it goes back to the developer to try again to fix.
There's more to it than that, but hopefully you get the idea. The database is a communication tool that the testers and developers use to track and resolve issues with the game.
I can hear voices in the distance, crying "Finally!!" Well, I figured it was helpful to set the stage first. Good to have the context, right?
A bug contains the following five main parts:
Let's examine those in detail.
Writing in first person point of view, describe the problem. Provide enough information pertinent to the problem so that anyone reading the bug report will understand what happened. Write in complete sentences. Use proper punctuation, grammar, syntax, and spelling.
I was playing the Dodgers versus the Zombies, at Pebbledon Downs. I was at bat, and I hit a high fly ball. The ball flew high and long, but it never came down. Instead, the ball got stuck in my onscreen score. The ball flew right into the O and stayed there.
Now that you've described the problem, explain why it's a problem. What did you expect to happen, instead of the thing that actually happened? And what is the basis of that expectation? It may seem to you that it's a waste of time to have to explain why it's a problem -- maybe you think it's self-evident that it's a problem (that any idiot can plainly see why it's a problem, and what any idiot would expect to happen instead). Well, it's your job as a QA tester to communicate. So communicate! What seems self-evident to you may not be so self-evident to your reader. And that even extends to the basis for the expectation.
The onscreen score is an artificial construct; it's not really a part of the physical world being represented in the game. The score remains stationary on the screen while the camera pans around the sports arena. The score doesn't smack any sports fans in the head (the sports fans are not affected by the relative motion of the onscreen score), and the score doesn't collide with structures in the sports arena. So it's unexpected for the ball to interact with the onscreen score in any way.
Now it's dead easy for anyone reading your bug to understand what you expected to happen (and why you expected that to happen).
Great, so you found a bug and you're proud of yourself for spotting it. But if the developer can't make the problem happen, how can he or she fix it? You have to tell the developer how to recreate (replicate) the problem. Writing in second person point of view, give the developer complete instructions about how to make the problem happen, so he can see it for himself. And to really communicate thoroughly, conclude with an observation step. Tell the developer what to look for, and what he will see, when he has performed your steps.
The problem has only been observed when playing in Pebbledon Downs, with the Dodgers and the Zombies. Your onscreen score has to be zero, and you have to aim for the zero. It can be tricky to make this problem happen, because it requires a good strong hit, a fly ball, and a trajectory that intersects perfectly with the middle of the O in the score. But I was able to make it happen a second time by simply being persistent. I recommend using the practice mode (that way the test is not prolonged by inning changes, vampire attacks, and timeouts). Using the left stick, position the batter's feet so that a line drawn across both shoetips points directly towards the O. Press the right trigger just as the ball crosses the 5-point line, and hit the ball, then watch as it flies. When the ball flies into the O, observe that it gets stuck in the O.
See how that's written in second person (I told the developer what he has to do). And notice that in the observation step I told him what to look for.
Now that you've described the problem, why it's a problem, and how to recreate the problem, you need to say how severe the problem is, and why you say it's that severe. You can't just say "it's a bad bug and needs to be fixed." You have to communicate the reason why it's bad. And you should use the A-B-C-D scale described in FAQ 5. (Remember: A is worst, B is bad, C is normal, D is minor.)
When the problem happens, it's quite severe, because the ball is stuck and the game can not continue. The only way out of it (other than rebooting) is to hit Start, then Quit the game. But the problem happens only on those extremely rare occasions when the ball perfectly enters the O. So I'm assigning this a severity of C.
I'll bet you thought I made a mistake when I didn't list this above! The reason I put this last is that the best time to write the summary is after you've written the rest of the report. A summary needs to be short, and it needs to summarize the problem you're reporting.
Ball gets stuck in the O
A bug report summary is a lot like the subject line of an email. It has to be short, and it has to give an idea of what the email is about (in this case, what the bug is). You'll find that it's easier to write the most concise summation of the problem after you've already written the full bug description.
If you haven't yet, be sure to read FAQ 5, "Testers: The Unsung Heroes of Games," and FAQ 17, "More About Testing."
Got a question or comment about this article? Email your comments to - you'll get a response on the Sloperama Game Design bulletin board.
Click here to go to the previous article.
Click here to return to the Sloperama Game Advice main page.
Updates to these articles are logged at log.html.
© 2013-2015 Tom Sloper. All rights reserved. May not be re-published without written permission of the author.