Agile Testing: A Practical Guide for Testers and Agile Teams

When writing programmer task cards, make sure that coding task estimates include time for writing unit tests and for all necessary testing by programmers. A card for “end-to-end” testing helps make sure that programmers working on different, independent tasks verify that all of the pieces work together. Testers should help make sure all necessary cards are written and that they have reasonable estimates. You don’t want second-guess estimates, but if the testing estimates are twice as high as the coding estimates, it might be worth talking about.

Some teams keep testing tasks to a day’s work or less and don’t bother to write estimated hours on the card. If a task card is still around after a day’s work, the team talks about why that happened and writes new cards to go forward. This might cut down on overhead and record-keeping, but if you are entering tasks into your electronic system, it may not. Do what makes sense for your team.

Estimating time for bug fixing is always tricky as well. If existing defects are pulled in as stories, it is pretty simple. But what about the bugs that are found as part of the iteration?

Janet’s Story

With new agile teams, I found that they always seem to end up with time spent on bugs that wasn’t allotted as part of their estimates for the stories. Over time, programmers learn how much time they typically spend fixing bugs from a story, and can just add half a day or a couple hours to their tasks for that purpose. Retesting bug fixes adds time to tester’s estimates as well.

Until the team members get a handle on this, it may be appropriate to track the time spent on fixing and testing bugs separately. My current team adds a story in XPlanner with tasks for fixing and testing those bugs that didn’t get caught immediately. We are tracking the time so we can better estimate down the road.

—Janet

However your team chooses to estimate time spent for fixing defects during the iteration, whether it is included in the story estimate or tracked separately, make sure it is done consistently.

Another item to consider when estimating testing tasks is test data. The beginning of an iteration is almost too late to think about the data you need to test with. As we mentioned in Chapter 15, “Tester Activities in Release or Theme Planning,” think about test data during release planning, and ask the customers to help identify and obtain it. Certainly think about it as you prep for the next iteration. When the iteration starts, whatever test data is missing must be created or obtained, so don’t forget to allow for this in estimates.

Lisa’s Story

We worked on a theme related to quarterly account statements for participants in retirement plans. We were modifying a monthly job that takes a “snapshot” of each participant’s account on the specified date. The snapshot relies on a huge amount of data in the production database, including thousands of daily transactions. We planned ahead.

For the first iteration, we did a few stories in the theme knowing we could only test a few cases using some individual retirement plans for which we had data in the test database. We also knew we needed a larger-scale test, with all of the retirement plans in the database and at least an entire month’s worth of data. We wrote a task card to copy enough production data to produce one monthly “snapshot” and made sure the data was scrubbed to protect privacy.

Then we planned the full-blown test in the next iteration. This data enabled the testers to find problems that were undetectable earlier when only partial data was available. It was a nice balance of “just enough” data to do most of the coding and the full amount available in time to verify the complete functionality. Because the team planned ahead, the bugs were fixed in time for the critical release.

—Lisa

Deciding on Workload

We, as the technical team, control our own workload. As we write tasks for each story and post them on our (real or virtual) story board, we add up the estimated hours or visually check the number of cards. How much work can we take on? In XP, we can’t exceed the number of story points we completed in the last iteration. In Scrum, we commit to a set of stories based on the actual time we think we need to complete them.

Lisa’s current team has several years of experience in their agile process and finds they sometimes waste time writing task cards for stories they may not have time to do during the iteration. They start with enough stories to keep everyone busy. As people start to free up, they pull in more stories and plan the related tasks. They might have some stories ready “on deck” to bring in as soon as they finish the initial ones. This sounds easy, but it is difficult to do until you’ve learned enough to be more confident about story sizes and team velocity, and know what your team can and cannot do in a given amount of time and in specific circumstances.