Agile Testing: A Practical Guide for Testers and Agile Teams

Implementing a new test automation tool usually requires some experimentation to get a good balance of testable code and well-designed test scripts. Involving the whole team makes this much easier. Watir is one example of a GUI test tool that we’ve found is well suited to agile projects. Let’s look at a couple more, Selenium and Canoo WebTest.

Selenium. Selenium is another open source tool, actually a suite of tools, for testing web applications. The tests can be written as HTML tables or coded in a number of popular programming languages, and can be run directly in most modern web browsers. A Firefox plug-in called “Selenium IDE” provides a way to learn the tool quickly. A recorder is provided to help create the tests, including writing assertions. Tests can be written in several different common programming and scripting languages, including Java, C#, and Ruby.

See Chapter 14, “An Agile Test Automation Strategy,” for an example of using Selenium RC to create a domain-specific test automation framework.

Canoo WebTest. In WebTest scripts, tests are specified as “steps” in XML files, simulating a user’s actions through a web UI. Here’s an example of how a WebTest script might invoke a page and verify the results:

Rather than driving an actual browser, as Selenium and Watir do, WebTest simulates the desired browser using HtmlUnit. The advantage of specifying tests as opposed to coding test scripts, is because there’s no logic in them, you don’t have to test the test.

Lisa’s Story

My team chose WebTest to automate smoke tests for our legacy application for several reasons. Because the scripts are written in XML, the programmers on the team were comfortable using the tool. It uses Ant to run the tests, so integrating it into the continuous build process was simple. It’s easy to learn, and the tests can be designed in a modular fashion, so they’re fairly easy to maintain. WebTest supports testing PDF files, emails, and Excel files, all of which are widely used in our application.

Being accustomed to powerful commercial test tools, I was skeptical of the concept of specifying tests, as opposed to programming them. I was amazed at how effective the simple tests were at catching regression bugs. It’s possible to put logic into the tests using Groovy or other scripting languages, but we’ve only found the need in a few cases.

Writing a few tests per iteration, I automated smoke tests for all of the critical areas of our application in eight months. These simple tests find regression bugs regularly. We refactor the tests frequently, so they are relatively easy to maintain. Our ROI on these tests has been tremendous.

—Lisa

Selenium, WebTest, and Watir are just three examples of the many open source tools available for GUI testing as of the time we wrote this book. Many teams write their own test automation frameworks. Let’s look at an example in the next section.

“Home-Brewed” Test Automation Tools

Bret Pettichord [2004] coined the term “home-brewed” for the tools agile teams create to meet their own unique testing needs. This allows even more customization than an open source tool. The goal of these tools is usually to provide a way for nontechnical customer team members and testers to write tests that are actually executable by the automated tool. Home-brewed tools are tailored to the exact needs of the project. They can be designed to minimize the total cost of ownership. They’re often built on top of existing open source tools.

Janet has been involved in a few projects that have used Ruby and Watir to create a full framework for functional testing. These frameworks allowed customers to specify tests that were then turned into a functional regression suite.

No test tool guarantees success. In fact, the history of test automation is littered with failed attempts. Having the whole team think about the best tools to use is a big help, but no matter what tool you use, you need a smart approach to writing tests. We’ll discuss that in the next section.

PAS Functional Testing

This next story is about one project Janet worked on that enjoyed success with home-brewed test automation.

PAS is a production accounting application for the oil and gas industry. Using gross meter readings and contract agreements, it must calculate ownership of the various products down to a very precise level (i.e., the components in gas). There are literally thousands of interactions between the combinations available in configuring the system and the actual outputs visible to a user. Given the large number of interactions, PAS has employed many complementary strategies for testing.

Joseph King, one of the initial programmers and agile coach for the team, tells us the story of how they accomplished their functional testing.