Agile Testing: A Practical Guide for Testers and Agile Teams

Our team’s customers list their stories in priority order and then draw a line between the stories that must be done before the release can occur, and the ones that could safely be put off. They call the less important stories “below the line,” and those stories may never get done.

For example, when we undertook the theme to allow retirement plan participants to borrow money from their retirement accounts, there was a “below the line” story to send emails to any participants whose loans are changing status to “pending default” or “default.” When the loan is in “default” status, the borrower must pay taxes and penalties on the balance. The email would be extremely helpful to the borrowers, but it wasn’t as important to our business as the software to request, approve and distribute loans, or process loan payments.

The email story didn’t make it into the release. It wasn’t done until more than two years later, after enough complaints from people who didn’t know their loans were going into default until it was too late.

—Lisa

Janet worked with a team whose customers were under the misplaced assumption that all of the features would get into their release and that when they were prioritizing, they were just picking which stories got done first. When the rest of the team realized the misunderstanding, they also implemented the idea of stories above and below the line. It helped to track progress as well as make the stories that were dropped below the line very visible.

Deadlines and Timelines

Many domains revolve around fixed dates on the calendar. Retail businesses make most of their profit during the holiday season. An Internet retail site is smart to have all new features implemented by October 1. Implementing a new feature close to the peak buying period is risky. Lisa’s company’s customers must complete government-required tasks during certain periods of the year. When it’s too late for a feature to get released this year, it often gets put off for the next year, because more urgent priorities must be addressed. Regulatory changes have specific timelines, and organizations have no choice about the timeline.

Janet’s Story

While working on this book, I was planning a release with my team at WestJet. We had several possible stories and worked with the customers to decide what the release would look like. We had one regulatory change that was light work for the programmers, but heavy for the testers. It needed to be in production by a certain date, so the other stories we were considering for the release took that into consideration.

We decided to create a small maintenance release with just that one major feature, along with a few bugs from the backlog so the release of the regulatory change would not be jeopardized. While the testers completed their testing, the rest of the team started some of elaboration stories for the next release.

An alternative plan could have been that the programmers chip in and help test and fit in more features. However, the whole team decided that this plan would work the best with the least amount of risk.

—Janet

Focus on Value

It’s rather easy for a team to start discussing a complex story and lose sight of what value the features actually deliver. Release planning is the time to start asking for examples and use cases of how the features will be used, and what value they’ll provide. Drawing flowcharts or sample calculations on the whiteboard can help pinpoint the core functionality.

Lisa’s Story

Our product owner wrote a story to provide a warning if an employer overrides the date a participant becomes eligible to contribute to a retirement account after the participant has already made contributions.

The warning needed to be incorporated into the legacy UI code, which didn’t easily accommodate it. The team discussed how it might be implemented, but every option was fairly costly. Not only would coding be tricky, but a lot of time was needed to test it adequately and update existing automated tests. This feature wouldn’t provide much value to the business, just a bit of help to the end users. The release was already pretty close to the limit on features.

One of the programmers suggested providing a report of participants who met the criteria so the plan administrators could simply call the employers who may need to make corrections. The report story was much smaller than the warning story, could easily fit into the release, and was acceptable to the customer.

—Lisa

There is no guarantee that these initial “guesstimates” at what will be in a given release will hold up over time. That is why customers needs to understand their priorities, take checkpoints at the end of every iteration, and reevaluate the priorities of remaining stories.

System-Wide Impact