Agile Testing: A Practical Guide for Testers and Agile Teams

Every morning during our Scrum, we further ensured parity between the functional groups within the team through simple dialogue. Communication was ridiculously good—we sat close to each other, often even at the same computer. When a defect was discovered, the developer was right there observing, taking notes, and talking through the requirements. A business analyst was always nearby to further validate our thinking. Often within minutes a resolution was checked-in, deployed to the test environment, and verified.

Both developers and testers had to be committed to this approach or it wouldn’t have worked. Without discipline, the developers could have easily moved forward onto more features and let the bugs slide until the end of the project, risking an incomplete iteration. If we were not co-located as we were, communication would have suffered; likely a bug tracking system or email would have become our primary means of communicating defects, resulting in longer turn-around times and an increased probability of rework.

As part of any development, you will always need to make trade-offs. Your team may decide to release with some outstanding bugs because it is deemed more important to get new functionality out the door than to fix low-level bugs.

It’s All about Choices

Teams have solved the problem of how to handle defects in many different ways. Some teams put all of their bugs on task cards. Other teams have chosen to write a card, estimate it, and schedule it as a story. Still others suggest adding a test for every bug—that way you don’t have to record the defect, just the test.

Is there one right way? Of course not! But, how do you know what is right for your team? We have some suggestions to help you choose and decide what is right for you. Think about your team and your product and what might work in your situation. First, we’ll talk about what defects we should log, then we’ll talk a bit about when you should fix them, and finally we’ll look at what media to choose. The right combination will depend on how far along your team is in its agile journey and how mature your product is.

Decide Which Bugs to Log

Not all bugs need to be logged, but teams often struggle with which ones should be recorded and which ones don’t need to be. We recommend that you avoid creating a defect report if possible. Have a conversation with a real person first, and only produce a defect report if it is truly a real problem that demands a change to the product or the programmers just can’t get to it right away.

Unit Test Failures

Don’t log unit test failures. If you are part of a team that is practicing TDD (test-driven development) and has good coverage with its unit tests, you know that failed tests during the build should not be logged. A failed test during the continuous integration build is a signal for the programmers to address the problem right away. Logging these bugs would be redundant and a waste of time.

Failures in Higher-Level Regression Tests

Many teams have builds that run regression tests above the unit level, such as tests behind the GUI and tests through the GUI. When one of these builds fails, should you log the bug in a DTS?

Lisa’s Story

We have two builds, an “ongoing build” that runs only unit tests, and a “full build” that runs the functional tests behind and through the GUI. When the “full build” breaks, if a developer investigates and tackles the problem right away as sometimes happens, usually no bug is logged. The problem is fixed quickly. At other times, the failure is not straightforward. One of the testers investigates, narrows down the problem, and files a bug that either states the name of the failing test or provides manual steps to recreate the problem.

In either case, tests are written that reproduce the bug, and the code is fixed to make the tests pass. The tests become part of one of the builds.

—Lisa

Failing tests in themselves are a type of recorded bug. But sometimes, as in Lisa’s case, more information needs to be added to allow for an effective and clean fix, so logging the defect is warranted.

Story Bugs within the Current Iteration

Don’t log bugs that can be fixed immediately, especially if you would otherwise record them in an electronic DTS. If your team is working closely with the programmers and is practicing pair testing as soon as a story is completed, we strongly recommend that you don’t log those bugs as long as the programmer addresses them right away. As you notice issues, talk them over with the programmer and decide whether they are real issues or not. Talk to the customer if you need to, but make a couple of notes so you remember what you saw so you can adjust your tests if needed.

If you are using index cards to log bugs, you may want to put an index card up on the task board (or a card on your electronic board) just as a reminder.