Writing bug reports is a frequent task in the tester’s work. Here are my tips on how to write good and clear bug reports that properly underline: what issue can be observed, what behavior should have been observed instead, and how to reproduce it.
First of all, make sure you only describe one problem per bug report. Don’t mix in several different issues, with totally different root causes, into the same report, as it will make retesting and closing the report difficult. If you know for sure that the root cause of several different behaviors is the same one, than you can specify all the observed behaviors in the same report, as long as they are related.
This section is not always provided in a bug report, simply because it is either not needed, or because it is included in the ‘steps to reproduce’ section. Useful information that could be added to the pre-requisites section includes:
- the version of the application being tested
- the environment on which the bug was encountered
- the setup, which includes: OS, browser version, type of device used, etc.
Steps to reproduce
- this is probably the most underrated part of the entire bug report. So many bug reports have a very short and unclear ‘steps to reproduce’ section, which leads to confusion regarding: how to find the items in the application that you need to interact with to reproduce the bug, what is the exact data or type of data to be used, what are the exact steps that need to be performed, and so on.
- a very important aspect to consider when writing the steps to reproduce is to write them as detailed as possible. This helps newcomers to the team understand how to reproduce the bug, or test the fix once it is available, without them needing to constantly ask for feedback from the person who created the report. It also helps developers understand how to reproduce the issue they are trying to fix, in order to understand the circumstances and state of the application when the bug occurs. Also, it helps anyone who needs to retest the fix of the bug, in preparation for releases months or years later, to understand what needs to be tested. When writing a bug report, don’t assume you will remember all its’ details months later, because you won’t. Therefore write the steps to reproduce as detailed as possible, to help anyone test it at any time, without prior knowledge of its’ existence. Also, if you are creating an automated test for this bug, a detailed ‘steps to reproduce’ section helps write the best test for it.
- write the smallest number of mandatory steps. The first time you reproduced a bug, you might have performed more steps than needed to actually reproduce that bug. For example, you might have wanted to test that adding 20 items to a shopping cart works properly, but only one particular item was causing an error. In this case, that particular item is the only one of interest for reproducing this bug, since all other items were added properly to the cart. Therefore, the steps to reproduce only need to contain the steps required to add that item, making the process of reproducing the bug shorter and more relevant than the first time it was encountered. This also helps to have a shorter duration for running the automated test when the fix is tested, or with every CI run, if this bug needs to be checked for within the CI process.
- number the steps and make them as granular as possible, so that each step represents a distinct action. This helps correlate each step with an expected result.
Clearly specify in this section what is the expected output of the steps executed. Phrases like ‘it should work’ or ‘it should be successful’ are very generic, and anyone reading these can interpret them according to their own perception of the application under test. Therefore, write the expected results in a concise and complete manner. Most times the bug found is a direct contradiction of some acceptance criteria written for the part of the application you are testing. In this case you should write the expected results to reflect this acceptance criteria. Consult with product owners if the expected result is not clear but the behavior observed in the system does not seem to be a valid one from the perspective of a user.
In this section specify all the details of the actual behavior of the system you are testing. Be specific, and don’t write actual results in the form of ‘is not working’ or ‘is failing’. That is ambiguous and leaves room for interpretation. Additionally, it is useful to grab some screenshots of the behavior, or to add some information from the logs that reflect what has occurred. This helps developers quickly pinpoint where the problem is.
Additional useful information
- Workaround: if a workaround to a very disruptive behavior is known, specify it. This can be communicated to customers if the issue affects their activity, to be used until a proper fix can be performed.
- Severity: this can be specified by a tester if known. It is easy to determine whether a bug is critical or severe, when important parts of the application under test are unavailable to customers. A discussion with the product owner can shed light on what severity the issue has, when it is not critical or severe.
- Context: in some cases, context is very helpful. It can contain information like: what was being tested when the issue was encountered, how the issue impacts other user flows, why the issue is disruptive to the customers’ experience.