Agile Testing: A Practical Guide for Testers and Agile Teams

There are reasons to track defect rates. There are also reasons why you wouldn’t track a defect. For example, we don’t think that a bug should be counted as a defect if it never makes it out of the iteration. This, of course, brings up another discussion about what should we track and why, but we won’t discuss that here.

Chapter 18, “Coding and Testing,” explores metrics related to defect rates.

Traceability

Another reason we’ve heard for having a DTS is traceability, linking defects to test cases. We’re not sure that this is a valid reason. Not all defects are linked to test cases, nor should they be. For example, errors like spelling mistakes might not need specific test cases. Maybe the product was not intuitive to use; this is a very real bug that often goes unreported. How do you write a test to determine if something is usable? Exploratory testing might find bugs in edge conditions that are not worth the effort of creating automated tests.

If it is an automated test case that caught a bug, then the need to record that defect is further reduced, because it will be caught again if ever reintroduced. The need for traceability is gone. So, maybe we don’t need to track defects.

Why Shouldn’t We Use a DTS?

Agile and Lean provide us with practices and principles that help reduce the need for a DTS. If the process is solid, and all of the people are committed to delivering a quality product, defects should be rare and very simply tracked.

As a Communication Tool

Defect tracking systems certainly don’t promote communication between programmers and testers. They can make it easy to avoid talking directly to each other.

Waste of Time and Inventory

We tend to put lots of information into the DTS in addition to all of the steps to reproduce the defect. Depending on the bug, it can take a long time to write these steps so that the programmer can reproduce it as well. Then there is the triage, and someone has to make comments, interpret the defect, attempt to reproduce it, (ideally) fix it, write more comments, and assign it back to the person who reported it. Finally, the fix can be verified. This whole cycle can double if the programmer misunderstood the problem in the first place. The cost of a single defect report can become exorbitant.

How much easier would it be if we as testers could just talk to the programmer and show what we found, with the developer then fixing the defect right away? We’ll talk more about that later.

In Chapter 15, “Coding and Testing,” we’ll explain how tester and programmers work together on bugs.

Defects in a DTS become a queue or a mini product backlog. According to lean principles, this inventory of defects is a waste. As a team, we should be thinking of ways to reduce this waste.

Janet’s Story

In 2004, Antony Marcano, author of TestingReflections.com, wrote a blog post about the idea of not using a bug-tracking system. When it was discussed on mailing lists, he was flamed by many testers as introducing something similar to heresy. He finds he has a different reception now, because the idea is making its way into the mainstream of agile thinking.

He suggests that bug-tracking systems in agile teams are just “secret backlogs.”

—Janet

Antony will share his ideas about the hidden backlog when we cover iteration planning in Chapter 18, “Coding and Testing.”

Defect Tracking Tools

If you do decide to use a DTS, choose it carefully. Understand your needs and keep it simple. You will want everyone on the team to use it. If it becomes overhead or hard to use, people will find ways to work around it. As with all tools used by your agile development team, you should consider the whole team’s opinion. If anyone from the customer team enters bug reports, get his or her opinion too.

One of the simplest tools that Janet has used is Alcea’s FIT IssueTrack. It is configurable, does not make you follow a predefined process, and is easy to get metrics out of. Do your homework and find the tool that works for you. There are a variety of open source defect-tracking systems, hosted systems, and integrated enterprise systems available.

Whether or not you use a DTS, you want to make defects as visible as possible.

We use a commercial DTS, but we find value in keeping bugs visible. We color-code bugs and include them as tasks in our story board, shown in Figure 5-1. Yellow cards denote normal bugs, and red cards denote either high production bugs or “test stopper” development bugs—both categories need to be addressed right away. A quick look at the board lets us see how many bugs are in the To Do, WIP, Verify and Done columns. Other cards are color-coded as well: blue for story cards, green for test task cards, and white for development tasks. Striped cards are for tasks added after iteration planning. Yellow and red bug cards stand out easily.