Software Testing- Bug Reporting
Defect reports are among the most important deliverables to come out of test. They are as important as the test plan and will have more impact on the quality of the product than most other deliverables from test. Thus, it is well worth the effort to learn how to write effective defect reports.
Effective defect reports can:
The objective is not to write the perfect defect report but to write an effective defect report that conveys the proper message and simplifies the process for everyone. This article focuses on two aspects of defect reports: the remarks (or description) and the abstract. First, let’s take a look at the essentials for writing effective remarks.
Defect Rremarks
Checklist
Here is a mnemonic to use as a mental checklist for self-inspecting your defect report. The first letter of each item on the checklist spells CAN PIG RIDE. These ten points ensure that your defect reports answer the questions that will be of most benefit to your organization:
It is not just good technical writing skills that lead to effective defect reports. It is more important to make sure that you have covered the essential items that will be of most benefit to the intended audience of the defect report.
Essentials for Eeffective Defect Remarks
Condense
Say it clearly but briefly. First, eliminate unnecessary wordiness, as shown in table 1. Second, don’t add in extraneous information. It is important that you include all relevant information, but make sure that the information is relevant. In situations where it is unclear how to reproduce the problem or the understanding of the problem is vague, you will probably need to capture more information. Keep in mind that irrelevant information can be just as problematic as too little relevant information.
Accurate
Make sure that what you are reporting really is a bug. You can lose credibility very quickly if you get a reputation of reporting problems that turn out to be setup problems, user errors, or misunderstandings of the product. Before you write up the problem, make sure that you have done your homework. Consider the following:
Make sure that you understand the numerous test-case influences and consider their roles in the perceived bug you are reporting. This is one area that quickly separates the experienced tester from the novice. If you are unsure about the validity of the problem, it may be wise to consult with an experienced tester or developer prior to writing up the problem.
Don't be afraid to write up problems, but do your best to ensure that they are valid problems. When you discover that you have opened an incorrectly reported problem, make sure that you learn from the experience.
Neutralize
State the problem objectively. Don't try to use humor, and don’t use emotionally charged zingers. What you think is funny when you write the defect may not be interpreted as funny by a developer who is working overtime and is stressed by deadlines. Using emotionally charged statements doesn't do anything for fixing the problem. Emotional statements just create barriers to communication and teamwork. Even if the developers doubted you and returned your previous defect and now you have proof that you are correct and they are wrong, just state the problem and the additional information that will be helpful to the developer. In the long run, this added bit of professionalism will gain you respect and credibility. Read over your problem description before submitting it, and remove or restate those comments that could be interpreted as being negative toward a person.
Table 2 is a response to a developer’s returning a defect for more information and requesting more details on what values caused the problem.
Precise
The person reading the problem description should not have to be a detective to determine what the problem is. Right up front in the description, describe exactly what you perceive the problem to be. Some descriptions detail a series of actions and results. For example, "I hit the Enter key and action A happened. Then I hit the back arrow and action B happened. Then I entered the ‘xyz’ command and action C happened." The reader may not know if you think all three resulting actions were incorrect or which one, if any, is incorrect. In all cases, but especially if the description is long, you need to summarize the problem at the beginning of the description. Don't depend on an abstract in a different field of the defect report to be available or used by everyone who reads the problem description. Don’t assume that others will draw the same conclusions that you do. Your goal is not to write a description that it is possible to understand but to write a description that won't be misunderstood. The only way to make that happen is to explicitly and precisely describe the problem rather than just giving a description of what happened, as shown in table 3.
Isolate
Each organization has its own expectations of how much the tester is required to isolate the problem. Regardless of what is required, a tester should always invest some reasonable amount of effort into isolating the problem. Consider the following when isolating problems:
In the problem description, describe the exact inputs used. For example, if you found a problem while printing a PostScript document, even if you think the problem occurs with any PostScript document, specify the exact document that you used to find the problem.
Your ability to isolate in large part defines your value-add as a tester. Effective isolation saves everyone along the line a great deal of time. It also saves you a lot of time when you have to verify a fix.
Generalize
Oftentimes, the developers will fix exactly what you report, without even realizing the problem is a more general problem that needs a more general fix. For example, I may report that my word processor "save file" function failed and the word processor aborted when I tried to save the file "myfile." A little more investigation may have revealed that this same failure occurs any time I save a zero-length file. Perhaps on this release it aborts on every save to a remote disk, a read-only disk, and so forth. To already know this when you write the report will save the developer a lot of time and enhance the possibility of a better fix to handle the general case.
When you detect a problem, take reasonable steps to determine if it is more general than is immediately obvious, as shown in table 4. 
Re-create
Some bugs are easy to re-create, and some are not. If you can re-create the bug, you should explain exactly what is required to do the re-create. You should list all the steps, include the exact syntax, file names, and sequences that you used to encounter or re-create the problem. If you believe that the problem will happen with any file, any sequence, etc., then mention that but still provide an explicit example that can be used to do the re-create. If in your effort to verify that the bug is re-creatable you find a shorter and more reliable means of re-creating, document the shortest, easiest means of re-creation.
If you cannot re-create the problem, or if you suspect that you may not be able to re-create the problem, gather all the relevant information possible that may provide useful information to the person who has to try to fix the problem. This may be a time when you consider asking a developer if he wants to examine the system while it is still in the problem state or if there is any particular information that should be captured before cleaning up the problem state and restoring the system. Don't assume that it can be re-created if you haven't verified that it can be re-created. If you cannot or have not re-created the problem, it is important to note that in the defect remarks.
Impact
What is the impact if the bug were to surface in the customer environment? The impact of some bugs is self-evident: "Entire system crashes when I hit the Enter key." Some bugs are not so obvious. For example, you may discover a typo on a window. This may seem trivial unless you point out that every time someone uses your product this is the first thing he sees, and the typo results in an offensive word. In this case, even though it is just a typo, it may be something that absolutely must be fixed prior to shipping the product. Make your best judgment. If you think it is possible that this defect will not get sufficient priority, then state the potential impact and sell the importance of the defect. Don't oversell, but make sure the readers of the defect report have an accurate understanding of the probable impact on the customer.
Debug
What will the developer need to be able to debug this problem? Are there traces, dumps, logs, and so forth that should be captured and made available with this defect report? Document what has been captured and how it can be accessed.
Evidence
What evidence exists that will prove the existence of the error? Have you provided both the expected results and the actual results? Is there documentation that supports your expected results? Since you are writing a problem report, it is obvious that you believe there is a problem. Provide anything you can that will convince others that this is a valid problem. Evidence may take the form of documentation from user guides, specifications, requirements, and designs. It may be past comments from customers, de facto standards from competing products, or results from previous versions of the product. Don’t assume everyone sees things the same way you do. Don't expect people to read between the lines and draw the same conclusions as you. Don’t assume that three weeks from now you will remember why you thought this was a bug. Think about what it is that convinced you that this is a bug and include that in the report. You will have to provide even more evidence if you think there is a chance that this situation may not be readily accepted by all as a valid bug.
Defect Abstracts
The short, one-line abstract that gets associated with most defects can be a very powerful communication tool. Oftentimes, the abstract is the only portion of the defect that gets read by the decision makers. It is the abstract and not the full description that gets included in reports. It is the abstract that the project managers, screeners, team leads, and other managers look at when trying to understand the defects associated with the product.
The abstract must be concise and descriptive and convey an accurate message. The abstract is usually very limited in length. Because of the space limitations, abbreviations are OK and short, accurate messages take priority over good grammar. A good use of keywords is essential since many searches are based on the abstract. Keywords such as abort, hang, typo, and so forth are both descriptive and useful as search words. Where space permits, it is helpful to mention the environment, the impact, and any of the who, what, when, where, and why questions that you can address in such a short space.
Be as specific as possible. For example, the following abstract is true but doesn’t provide nearly as much information as it could:
Abstract: Problems found when saving and restoring data member.
Perhaps this would be a more descriptive abstract:
Abstract: xyz's save/restore of data member on WinNT fails, data corrupted.
You can never get everything you want in an abstract, so here is a list of items and tips that I try always to include:
Mandatory:
Recommended (space permitting):
Summary
Testers spend a significant amount of time seeking out and discovering software problems. Once detected, it greatly enhances productivity to report the defect in such a way as to increase the likelihood of getting the problem fixed with the least amount of effort. Making sure that the proper information is provided is more important than superior writing skills. The ten topics described in this paper will go a long way toward helping you provide the right information in every defect report.
The better you are at writing defect reports and abstracts, the more likely it is that problems will actually get fixed in a timely manner. Your credibility and value-add to the business will increase as developers, managers, and other testers are better able to do their jobs because your defect reports are well written and reliable.. {end}
Kelly Whitmill has more than twenty years of experience in testing software. He is currently a senior software engineer at InfoPrint Solutions Company. Kelly's project experience includes client/server and Web-based applications, large mainframe applications, and a variety of other applications on multiple platforms. He has worked on teams of all sizes, which has provided an understanding of the varying team dynamics. Kelly has special interests in test automation, system testing, and practical test techniques and practices to improve testing. He has written many defect reports, both good and bad, and can provide practical tips on how to avoid the bad and embrace the good.
This article was published in a Site http://www.stickyminds.com by kelly .Kelly can be reached at kc_whitmill@yahoo.com.
Posted byPrashanthNaik at 8:18 PM
Labels: Software Testing
Posts
