Agile Testing: A Practical Guide for Testers and Agile Teams

See the bibliography for a full discussion by Elisabeth Hendrickson on this subject.

If you have people already expert in a vendor tool, and a use for a tool that might be used only by a subset of the development team or a team separate from development, a vendor tool could make lots of sense. Lisa’s first two XP teams used a vendor tool with some degree of success.

As of this writing, better functional test tools and IDEs are emerging. These facilitate test maintenance tasks with features such as global search/replace. Twist is an example of a tool implemented as a collection of plug-ins to the Eclipse IDE, so it can take advantage of powerful editing and refactoring features.

Agile-Friendly Tools

Elisabeth Hendrickson [2008] lists some characteristics of effective agile test automation tools. These tools should:

Support starting the test automation effort immediately, using a test-first approach

Separate the essence of the test from the implementation details

Support and encourage good programming practices for the code portion of the test automation

Support writing test automation code using real languages, with real IDEs

Foster collaboration

Part III, “Using Agile Testing Quadrants,” and particularly Chapter 9, “Toolkit for Business-Facing Tests that Support the Team,” contain examples of test automation tools that work well on agile projects.

Implementing Automation

While you’re evaluating tools, think about how quickly your top priority automation need must be addressed. Where will you get the support to help implement it? What training does the team need, and how much time will be available to devote to it? How quickly do you have to ramp up on this tool?

Keep all of these constraints in mind when you’re looking at tools. You might have to settle for a less robust tool than you really want in order to get vital automation going in the short term. Remember that nothing’s permanent. You can build your automation effort step-by-step. Many teams experience unsuccessful attempts before finding the right combination of tools, skills, and infrastructure.

Selenium at Work

Joe Yakich, a software engineer with test automation experience, describes how a team he worked with implemented a test automation effort with Selenium, an open source test automation tool.

The software company I worked for—let’s call it XYZ Corp—had a problem. The product, an enterprise-level web-based application, was a powerful, mature offering. Development projects were managed using Agile and Scrum, and a talented stable of engineers churned out new features quickly. The company was growing steadily.

So, what was the problem? XYZ was facing a future where software testing efforts might not be able to keep pace with the development effort. Software quality issues might slow adoption of the product or—worse yet—cause existing customers to look elsewhere.

Test automation seemed like an obvious way to mitigate these risks, and XYZ was fully aware of it. In fact, they had attempted to create a test automation suite twice before, and failed.

The third time, XYZ chose to use Selenium RC, driven by the Ruby programming language. Selenium RC—the RC is for “Remote Control”—is a tool for test automation. Selenium RC consists of a server component and client libraries. The Java server component acts as an HTTP proxy, making the Selenium Core JavaScript appear to originate from the webserver of the application under test (AUT). The server can start and stop browser sessions (supported browsers include nearly all modern browsers, including Internet Explorer, Firefox, and Safari) and interpret commands to interact with elements such as buttons, links, and input fields. The client libraries allow test scripts to be written in Java, .NET, Perl, Python, and Ruby.

Our team chose Ruby because it’s a purely object-oriented, dynamic, interpreted language with a syntax that is elegant, expressive, and tersely powerful. Most importantly, Ruby is an ideal tool for the creation of a Domain Specific Language (DSL). Ruby is malleable enough for the programmer to first choose the structure and syntax of the DSL and then craft an implementation, as opposed to a more rigid language that might impose constraints on that freedom. One of our goals was to create an automation framework—a DSL—hiding complex detail. We wanted to be able to say things like

editor.save

in our tests instead of

s.click("//table[@class='edit']/tbody/tr[0]//img[@src='save.gif']")

Not only is the former more readable, it’s also far more maintainable. The XPath expression in the latter can be put in a library method to be called as needed. Using a DSL that employs the nouns and verbs of the application allows an engineer writing a test to focus on the test, not the underlying complexity of interacting with on-screen controls.