Agile Testing: A Practical Guide for Testers and Agile Teams

Ruby Test::Unit. One project Janet was on used Ruby’s unit testing framework, Test::Unit, to test web services, with great success. In fact, the team was able to test early to give the programmers immediate feedback, which helped with the final design.

See the “System Test” example in Chapter 12, “Summary of Testing Quadrants,” to see how Janet’s team used Ruby Test::Unit to test web services.

soapUI.Another tool suggested for testing web services is soapUI. It has a steep learning curve but can be used for performance and load testing. Because it can loop though rows in an Excel spreadsheet or text file, it can be used for data-driven testing.

Tests that work at the layers below the presentation layer are well suited for writing and automating customer tests that guide coding. Some practitioners haven’t gotten the value they expected from story test-driven development. Brian Marick [2008] hypothesized that an application built with programmer test-driven development, example-heavy business-facing design that relies heavily on whiteboard discussions, a small set of automated sanity tests, and lots of exploratory testing could be a less expensive and equally effective approach. Whichever approach you take, if you’re testing an application with a user interface, you’ll need some automation at the GUI level.

Tools for Testing through the GUI

Wait a minute. How can we use GUI tests to drive development, because the GUI won’t be ready until the story is complete? It sound counterintuitive, but automated GUI tests are important to help us while we’re developing new functionality. Test frameworks can be used to specify test cases for a GUI tool before the code is written. In addition, you can automate GUI tests before coding is finished, either by using HTML mock-ups or by developing an end-to-end bare-bones slice through all of the screens that simply navigates but doesn’t provide all of the functionality yet. Even if you’re not using a lot of automated story tests to drive development, manual exploratory testing that helps us learn about the functionality and provides immediate feedback gets pretty tedious and slow without any assistance from automation. Let’s look at the types of GUI test tools that help drive development using business-facing tests.

A Tool Selection Rationale

David Reed, a test automation engineer, and his team went with soapUI Pro to automate testing for their web services. Here are some reasons he gave for choosing this particular tool.

• It has an open source version, so you can try it out it for free. You can learn it, kick the tires, expand stuff, and learn its strengths and weaknesses.

• It was easy to figure out what requests to make for what service.

• The assertions provided for verifying the results from requests are great and expandable. One really helpful one is verifying that the response comes back in an acceptable amount of time, raising an error if it doesn’t.

• The Pro version takes a lot of the hassle out of designing XPath queries to verify results. It also adds some nice touches for retrieving database data.

• It’s expandable with Groovy, a Java-based scripting language. (They’re working on a Java application, so it pays to have Java-friendly tools.)

• Developers can use it without sneering at it as a “test tool.”

• It’s easily integrated with our continuous integration environment.

• It has a feature to check code coverage.

• The price is right.

Record/Playback Tools

Record/playback tools are appealing because you can usually learn how to record a script and play it back quickly, and you can create lots of scripts in a short time. However, they have drawbacks. Early GUI test tools recorded mouse movements using X-Y screen coordinates. Scripts using those tools might also be sensitive to changes in screen resolution, color depth, and even where the window is placed on the screen.

Most modern GUI test tools use objects to recognize the controls in a graphical application, like buttons, menus, and text input widgets, so they can refer to them symbolically rather than with raw screen coordinates. This makes the application much more testable, because it’s more robust standing up to changes. A button might move to a different part of the screen, but the test can still find it based on its object name.

Even with improved object recognition, scripts created with record/playback are usually brittle and expensive to maintain. Recording can be a good way to start creating a script. Testers or programmers who know the tool’s scripting language can refactor the recorded script into an object-oriented model that’s easier to use and maintain. Historically, record/playback tools used proprietary scripting languages, which programmers aren’t interested in learning. It’s also more difficult to change the design patterns used in the tests.