May 1, 2006

Writing Issue Reports That Work

It's important not only to write Issue Reports, but to write them well.

Issue Reports (Bug Reports) are one of the main communication methods that QAers use.
You are creating a statement to your stakeholders - "I have found what I think is a problem, and here's my clear explanation of what it is and how you can see it too.  Please look into this".
  • Understand the Audience for the Report
It's important to know who is going to read your Issue Reports, and what they are expected to do with them.

For some shops, the only readers will be yourself and one or more developers.  If that's true, you can use all sorts of jargon and abbreviations that you each understand.

But in many shops, there will be lots of stakeholders who need to read what you write - other QAers, Developers, Support, Product Management, Documentation, Management, etc.  It may become more important to use less jargon, and add more details.

There are a few special cases that must be kept in mind as well. 

For example, if offshore testers or developers must read your Issue Reports, you'll need to pay special attention not to use confusing jargon or colloquialisms in your writing.

If customers will eventually read your Issue Reports, you'll need to be very, very careful in choosing your words.  (By the way, this is not something I'd recommend, without first sending the Issue Reports to a skilled writer for "sanitizing" first.)  You may even be better off having two different descriptions of the bug - one for internal consumption, and one for customers.
  • Choose a Good Summary/Title
Since the one-line Summary or Title is often what prompts someone to decide to read your Issue Report further, and is often the only piece of information about the bug showing up on summary reports, it's important to put some thought into this field.

The title should be short (because it may become truncated) and to-the-point.
It's tempting to cut corners and write just a few words here "Program x crashed" - but clearly that's not useful. copyrightjoestrazzere

Also, you don't need to include text here that is already included in other fields of the bug tracking system.  For most systems, items such as severity/priority, product, component, etc are tracked in their own fields.  No use wasting valuable space in the Summary/Title on these.
  • Describe the problem concisely and effectively
In a paragraph or two, describe the problem.  Here you can use more words than the Summary/Title will allow, but still avoid wordiness.
  • Include Steps to Reproduce the Problem you are Seeing
Not every bug is fully reproducible.  But, it's important to try to find out a relatively minimal set of steps to reproduce the problem and to note them in the Issue Report.
This gives the developer a fighting chance of seeing what you are seeing, and actually getting it fixed quickly.

Avoid steps that don't matter - those which have nothing to do with reproducing the bug.  Including too many steps can waste time and lead to confusion.

Include all steps which actually seem to matter.  Write them in a clear style, in a manner which avoids guesswork.

It takes a bit longer to narrow down the steps, but it's usually time well spent.

If the bug is not fully reproducible, indicate that clearly in the issue report.
  • Include the Results you Expected
Often, QAers know what is expected out of a sequence of steps better than Developers.

Sometimes, their expectations are right, sometimes they are wrong.  Either way, include what you expected to see.
  • Include the Results you Actually Observed
Since this is an Issue Report, we assume that something unexpected occurred.  Note what actually occurred.

If you observed an error message, include that.

If you observed something significant in a log file, include that portion of the log.
  • Include Enough Details for Searching
Try to think like a Support person, or a Manager, or a new QAer who wasn't familiar with this Issue.  What would you search for if you saw these same symptoms happen?  If you are lucky enough to have a defect tracking system that includes full-text searching, then make sure to include these important keywords somewhere in the text of your Issue Reports.

If your defect tracking system has a "keywords" field, put them in there.
  • Explain the Effects on the Customer
Somewhere along the line, your Issue Report will be analyzed for Priority and/or Severity.

If you explain the effects of this problem on the customer, you'll have a better chance that the Issue is properly analyzed.  And if the problem makes further testing impossible, make sure to indicate that fact (in this context, you are a customer, too). 
  • Attach Anything Else that Could Help
Attach anything that could help clarify and/or debug the problem.

As they say "a picture is worth 1000 words".  So, often attaching a picture of the problem can be very helpful.  Log files, test files, etc, can also be attached if they will help reproduce or debug the problem.

Think about whether something is better off being attached (in order to avoid an overly-wordy Issue Report), or included within the body of the Report itself (in order to be useful during a search).
  • Avoid speculation
Report the facts - what you saw, what you expected to see, and any other symptoms.  But in general, avoid speculation as to the root cause of the problem, unless you have sufficient facts to back up your speculation.

Speculation can send the reviewer on a wild goose chase and waste their time.  It can also make this bug report appear as a search result for cases where it's not relevent.
  • Be careful of the tone of your report
Don't use a negative tone in your writing.  Remember, your job is to describe the bug so that it can be fixed effectively and efficiently.  There's no benefit in criticizing the developer here, or the designer - you are their partners, you are both on the same side.  Be objective and respectful.
  • Avoid duplication - search first
You don't want to waste people's time reading issue reports that have already been covered by someone else.  And in some shops, you may be penalized for writing such bug reports.  So, search first, using the kinds of keywords that you would write into your report if needed.

If the problem has already been written into an issue report, it's sometimes useful to add additional facts if you have them.