Agile Testing: A Practical Guide for Testers and Agile Teams

We know it’s not easy to capture metrics. For example, trying to capture the time it takes to write and maintain automated tests versus the time it takes to run the same regression tests manually is almost impossible. Similarly, trying to capture how much it costs to fix defects within minutes of introducing them versus how much it costs to find and fix problems after the end of the iteration is also quite difficult. Many teams don’t make the effort to track this information. Without numbers showing that automating requires less effort and provides more value, it’s harder for teams to convince management that an investment in automation is worthwhile. A lack of metrics that demonstrate automation’s return on investment also makes it harder to change a team’s habits.

Code that’s Always in Flux

Automating tests through the user interface is tricky, because UIs tend to change frequently during development. That’s one reason that simple record and playback techniques are rarely a good choice for an agile project.

If the team is struggling to produce a good design on the underlying business logic and database access, and major rework is done frequently, it might be hard to keep up even with tests automated behind the GUI at the API level. If little thought is given to testing while designing the system, it might be difficult and expensive to find a way to automate tests. The programmers and testers need to work together to get a testable application.

Although the actual code and implementation, like the GUI, tends to change frequently in agile development, the intent of code rarely changes. Organizing test code by the application’s intent, rather than by its implementation, allows you to keep up with development.

In Chapter 14, “An Agile Test Automation Strategy,” we’ll look at ways to organize automated tests.

Legacy Code

In our experience, it’s much easier to get traction on automation if you’re writing brand new code in an architecture designed with testing in mind. Writing tests for existing code that has few or no tests is a daunting task at best. It seems virtually impossible to a team new to agile and new to test automation.

It is sometimes a Catch-22. You want to automate tests so you can refactor some of the legacy code, but the legacy code isn’t designed for testability, so it is hard to automate tests even at the unit level.

If your team faces this type of challenge and doesn’t budget plenty of time to brainstorm about how to tackle it, it’ll be tough to start automating tests effectively. Chapter 14 gives strategies to address these issues.

Fear

Test automation is scary to those who’ve never mastered it, and even to some who have. Programmers may be good at writing production code, but they might not be very experienced at writing automated tests. Testers may not have a strong programming background, and they don’t trust their potential test automation skills.

Non-programming testers have often gotten the message that they have nothing to offer in the agile world. We believe otherwise. No individual tester should need to worry about how to do automation. It’s a team problem, and there are usually plenty of programmers on the team who can help. The trick is to embrace learning new ideas. Take one day at a time.

Old Habits

When iterations don’t proceed smoothly and the team can’t complete all of the programming and testing tasks by the end of an iteration, team members may panic. We’ve observed that when people go into panic mode, they fall into comfortable old habits, even if those habits never produced good results.

So we may say, “We are supposed to deliver on February 1. If we want to meet that date, we don’t have time to automate any tests. We’ll have to do whatever manual tests can be done in that amount of time and hope for the best. We can always automate the tests later.”

This is the road to perdition. Some manual tests can get done, but maybe not the important manual exploratory tests that would have found the bug that cost the company hundreds of thousands of dollars in lost sales. Then, because we didn’t finish with our test automation tasks, those tasks carry over to the next iteration, reducing the amount of business value we can deliver. As iterations proceed, the situation continues to deteriorate.

Can We Overcome These Barriers?