Agile Testing: A Practical Guide for Testers and Agile Teams

Our UI regression test automation has grown 500% since April [of 2007]. This allows us to focus the attention of real human beings on more interesting testing.

Chris went on to explain, “Now that we have a lot of automation, we have the leisure to really think about what human tests need doing. For any testing that isn’t trivial, we have just about institutionalized a test-idea brainstorming session before beginning execution.” Usually, Chris and his teammates pair either two testers or one tester and a developer. Sometimes a tester generates ideas and gets them reviewed, via a mindmap, a wiki page, or a list in the release notes. Chris observed, “We almost always come up with good test ideas by pairing that wouldn’t have been found by either individual independently.”

Referring to their frequent releases of significant features, Chris says, “Thanks to the good test automation, we have the time to invest in making certain that the whole product is attractive and functional for real people. Without the automation, testing this product would be both boring and stupid. As it is, we testers have significant and interesting work to do for each release.”

We agree with Chris that the most exciting part of test automation is the way it expands our ability to improve the product through innovative exploratory testing.

Projects succeed when good people are free to do their best work. Automating tests appropriately makes that happen. Automated regression tests that detect changes to existing functionality and provide immediate feedback are a primary component of this.

Automated Regression Tests Provide a Safety Net

Most practitioners who’ve been in the software business for a few years know the feeling of dread when they’re faced with fixing a bug or implementing a new feature in poorly designed code that isn’t covered by automated tests. Squeeze one end of the balloon here and another part of it bulges out. Will it break?

Knowing the code has sufficient coverage by automated regression tests gives a great feeling of confidence. Sure, a change might produce an unexpected effect, but we’ll know about it within a matter of minutes if it’s at the unit level, or hours if at a higher functional level. Making the change test-first means thinking through the changed behavior before writing the code and writing a test to verify it, which adds to that confidence.

Janet’s Story

I recently had a conversation with one of the testers on my team who questioned the value of automated tests. My first answer was “It’s a safety net” for the team. However, he challenged that premise. Don’t we just become reliant on the tests rather than fixing the root cause of the problem?

It made me think a bit more about my answer. He was right in one sense; if we become complacent about our testing challenges and depend solely on automated tests to find our issues, and then just fix them enough for the test to pass, we do ourselves a disservice.

However, if we use the tests to identify problem areas and fix them the right way or refactor as needed, then we are using the safety net of automation in the right way. Automation is critical to the success of an agile project, especially as the application grows in size.

—Janet

When they don’t have an automated suite of tests acting as a safety net, the programmers may start viewing the testers themselves as a safety net. It’s easy to imagine that Joe Programmer’s thought process goes like this: “I ought to go back and add some automated unit tests for formatEmployeeInfo, but I know Susie Tester is going to check every page where it’s used manually. She’ll see if anything is off, so I’d just be duplicating her effort.”

It’s nice that a programmer would think so highly of the tester’s talents, but Joe is headed down a slippery slope. If he doesn’t automate these unit tests, which other tests might he skip? Susie is going to be awfully busy eyeballing all those pages.

Teams that have good coverage from automated regression tests can make changes to the code fearlessly. They don’t have to wonder, “If I change this formatEmployeeInfo module, will I break something in the user interface?” The tests will tell them right away whether or not they broke anything. They can go lots faster than teams relying exclusively on manual testing.

Automated Tests Give Feedback, Early and Often