Agile Testing: A Practical Guide for Testers and Agile Teams

Tools shouldn’t replace human interaction; they should enhance the experience. Tools can provide testers with more power to find the hard-to-reproduce bugs that often get filed away because no one can get a handle on them. Exploratory testing is unconventional, so why shouldn’t the tools be as well? Think about low-effort, high-value ways that tools can be incorporated into your testing.

Computers are good at doing repetitive tasks and performing calculations. These are two areas where they are much better than humans, so let’s use them for those tasks. Because testing needs to keep pace with coding, any time advantage we can gain is a bonus.

See the bibliography for references to Jonathan Kohl’s writings on using human and automation power together for optimal testing.

In the next few sections, we’ll look at some areas where automation can leverage exploratory testing. The ones we cover are test setup, test data generation, monitoring, simulators, and emulators.

Test Setup

Let’s think about what we do when we test. We’ve just found a bug, but not one that is easily reproducible. We’re pretty sure it happens as a result of interactions between components. We go back to the beginning and try one scenario after another. Soon we’ve spent the whole day just trying to reproduce this one bug.

Ask yourself how you can make this easier. We’ve found that one of the most time-consuming tasks is the test setup and getting to the right starting point for your actual test. If you use session-based testing, then you already know how much time you spend setting up the test, because you have been tracking that particular time waster. This is an excellent opportunity for some automation.

The tools used for business-facing tests that support the team described in Chapter 9 are also valuable for manual exploratory testing. Automated functional test scripts can be run to set up data and scenarios to launch exploratory testing sessions. Tests configured to accept runtime parameters are particularly powerful for setting up a starting point for evaluating the product.

Lisa’s Story

Our Watir test scripts all accept a number of runtime parameters. When I need a retirement plan with a specific set of options, and specific types of participants, I can kick off a Watir script or two with some variables set on the command line. When the scripts stop, I have a browser session with all of the data I need for testing already set up. This is so fast that I can test permutations I’d never get to using all-manual keystrokes.

—Lisa

The test scripts you use for functional regression testing and for guiding development aren’t the only tools that help take the tedium out of manual exploratory testing. There are other tools to help set up test data as well as to help you evaluate the outputs of your testing sessions.

Whatever tool you are using, think about how it can be adapted to run the scenario over and over with different inputs plugged in. Janet has also successfully used Ruby with Watir to set up tests to run multiple times to help identify bugs. Tools that drive the browser or UI in much the same way that an end user would makes your testing more reliable because you can play it back on your monitor and watch for anything that might not look as it should during the setup. When you get to the place where the test actually starts, you can then use your excellent testing abilities to track down the source of the bug.

Test Data Generation

PerlClip is an example of a tool that you can use to test a text field with different kinds of inputs. James Bach provides it free of charge on his website, www.satisfice.com, and it can be very helpful in validating fields. For example, if you have a field that will accept a maximum input of 200 characters, testing this field and its boundaries manually would be very tedious. Use PerlClip to create a string, put it in your automation library, and have your automation tool call the string to test the value.

Monitoring Tools

Tools like the Unix/Linux command tail -f, or James Bach’s LogWatch, can help monitor log files for error conditions. IDEs also provide log analysis tools. Many error messages are never displayed on the screen, so if you’re testing via the GUI, you never see them. Get familiar with tools like these, because they can make your testing more effective and efficient. If you are not sure where your system logs warnings and errors, ask your developers. They probably have lots of ideas about how you can monitor the system.

Simulators