Agile Testing: A Practical Guide for Testers and Agile Teams

Writing detailed test cases that communicate desired behavior effectively requires both art and science. Poorly expressed scenarios and poorly designed test cases can create more confusion than they resolve. Experiment so that you can find the right level of detail and the right test design for each story. Let’s look at some strategies to help you use tools successfully to write useful business-facing tests.

Build Tests Incrementally

After we have defined our high-level acceptance tests so that the programmer knows what to start coding, we can start elaborating on the rest of the story tests. We can work closely with the programmer to ensure we automate the best possible way.

When a programmer starts working on the programming tasks for a story, start writing detailed tests. For those of us who enjoy testing, it’s tempting to go for the biggest “smells” right away, the areas where we think the code might be fragile. Resist the temptation. Make sure the most obvious use case is working first. Write a simple, happy path automated test to show the code accomplishes the most basic task that it should. After that test passes, you can start getting more creative. Writing the business-facing tests is an iterative process.

Lisa’s Story

I start writing executable business-facing tests that support the team by writing a simple FitNesse test based on examples that the product owner provides. I show this to the programmer working on the code. He can make suggestions for changes right then, or he might modify the test himself as appropriate when he’s ready to automate it. Discussing the test often leads the programmer to realize he missed or misunderstood a requirement. We might need another three-way conversation with the customer. The programmer updates the code accordingly. We can also show the test to the product owner to make sure we captured the behavior correctly.

After the simple test passes, I write more tests, covering more business rules. I write some more complex tests, run them, and the programmer updates the code or tests as needed. The story is filling out to deliver all of the desired value.

—Lisa

Chapter 18, “Coding and Testing,” goes into more detail about how testers and programmers work together to test and code.

Confine each test to one business rule or condition. At some point you can automate or manually perform more complex scenarios, but start by covering each condition with a simple test. If you’ve followed our recommended thin slice or steel thread pattern, the first set of tests should prove the first thin slice end-to-end. As your automated tests pass, add them to the regression suite that runs in a frequent build process.

Keep the Tests Passing

After a test passes, it shouldn’t fail unless the requirements were changed. If that happens, the test should be updated before the code is altered. Of course, if a test was forgotten as part of a requirement change, we expect it to fail. It did its job as change detector. At this time, the test will likely need to change to get it passing.

Whenever a test fails in a continuous integration and build process, the team’s highest priority (other than a critical production problem) should be to get the build passing again. Don’t comment out the failing test and fix it later; that’s the road to perdition. Soon you’ll have dozens of commented-out tests and a lot of technical debt. Everyone on the team should stop what they’re doing and make sure the build goes “green” again. Determine if a bug has been introduced, or if the test simply needs to be updated to accommodate intentionally changed behavior. Fix the problem, check it in, and make sure all of the tests pass.

Lisa’s Story

Early on in our agile efforts, my team wasn’t fixing broken tests fast enough. I wrote “Tests are not temporary!” on the whiteboard to remind everyone that once a test passes, it needs to keep passing. A few days later, the words “but testers are!” had been added to get back at me. We did get much better at keeping our builds “green” after that.

—Lisa

One passing test leads to another. Keep your tests current and maintainable with refactoring. Extend them to cover other test cases. The various combinations and scenarios might or might not become part of the regression suite after they pass. We want our regression suite to run in a timely manner, and having too many tests for edge cases would slow it down.

Use Appropriate Test Design Patterns

When designing tests, look at different patterns and choose the ones that work for you. Keep them as simple as you can. Before you can design tests, you have to identify the ones you need. Pierre Veragen coined the term test genesis patterns to note the patterns that help you think of tests. Examples and use cases feed into our test genesis patterns.

Build/Operate/Check