Agile Testing: A Practical Guide for Testers and Agile Teams

The testing pyramid is a good place to start looking at how test automation can help an agile team. Programmers tend to focus on the bottom of the pyramid, and they need plenty of time and training to get over the “hump of pain” and get to the point where TDD is natural and quick. In traditional teams, testers usually have no choice but to automate tests at the GUI level. The whole-team approach used by agile teams means that testers pair with programmers and help them get better at writing tests, which in turn solidifies that brick foundation layer of the pyramid. Because tests drive development, the whole team is always designing for maximum testability, and the pyramid can grow to the right shape.

Programmers pair with testers to automate functional-level tests, filling out the middle layer. For example, a tester and customer may prepare a 400-row spreadsheet of test cases for a web services application. The programmer can help figure out a way to automate those tests. Different team members may have expertise in areas such as generating test data or using tools such as Excel macros, and all that knowledge spreads around the team. Working together, the team finds the best combinations of tools, test cases, and test data.

Involving the programmers in finding cost-effective ways to automate the top-level GUI tests has multiple benefits. These efforts may give programmers a better understanding of the system’s “big picture,” and testers can learn how to create more pliable, less straw-like GUI tests.

The more a team can work together and share knowledge, the stronger the team, the application, and the tests will become. The Big Bad Wolf won’t stand a chance. Let’s start by looking at what kind of tests we can automate and then at what we shouldn’t even try.

What Can We Automate?

Most types of testing you can think of benefit from automation. Manual unit tests don’t go far toward preventing regression failures, because performing a suite of manual tests before every check-in just isn’t practical. You can’t design code test-first through manual unit tests either. When programmers can’t run tests quickly at the touch of a button, they may not be motivated enough to run tests at all. We could manually test that different units of code work together correctly, but automated component tests are a much more effective safety net.

Manual exploratory testing is an effective way to find functional defects, but if we don’t have enough automated business-facing regression tests, we probably spend all of our time madly trying to keep up with manual regression testing. Let’s talk about all of the different kinds of testing that can be done well with automation.

To run automated tests, you need some kind of automated framework that allows programmers to check in code often, run tests on that code, and create deployable files. Let’s consider this first.

Continuous Integration, Builds, and Deploys

Any tedious or repetitive task involved in developing software is a candidate for automation. We’ve talked about the importance of an automated build process. You can’t build your automated test pyramid without this. Your team needs the immediate feedback from the unit-level tests to stay on track. Getting automated build emails listing every change checked in is a big help to testers because they know when a build is ready to test without having to bother the programmers.

See Chapter 7, “Technology-Facing Tests that Support the Team,” for examples of build automation tools.

Peril: Waiting for Tuesday’s Build

In a traditional environment, it is normal for testers to wait for a stable build, even if that means waiting until next Tuesday. In an agile environment, if testers don’t keep up with the developers, the stories get tested late in the game. If the developers don’t get the feedback, such as suggestions and bugs, the testers can lose credibility with the developers. Bugs won’t be discovered until the developers are already on another story and do not want to be interrupted to fix them until later.

Bugs pile up, and automation suffers because it can’t be completed. Velocity is affected because a story cannot be marked “done” until it is tested. This makes it harder to plan the next iteration. At the end of the release cycle, your story testing runs into the end game and you may not have a successful release. At the very least, you will have a stressful release.