Agile Testing: A Practical Guide for Testers and Agile Teams

Simulators are tools used to create data that represent key characteristics and behavior of real data for the system under test. If you do not have access to real data for your system, simulated data will sometimes work almost as well. The other advantage of using a simulator is for pumping data into a system over time. It can be used to help generate error conditions that are difficult to create under normal circumstances and can reduce time in boundary testing.

See the “System Test” example in Chapter 12, “Summary of Testing Quadrants,” to see how a simulator was critical to the testing the whole system.

Setting up data and test scenarios is half of the picture. You also need to have a way to watch the outcomes of your testing. Let’s consider some tools for that purpose.

Emulators

An emulator duplicates the functionality of a system so that it behaves like the system under test. There are many reasons to use an emulator. When you need to test code that interfaces with other systems or devices, emulators are invaluable.

Two Examples of Emulators

WestJet, a Canadian airline company, provides the capability for guests to use their mobile devices to check in at airports that support the feature. When testing this application, it is better for both the programmers and the testers to test various devices as early as possible. To make this feasible, they use downloadable emulators to test the Web Check-in application quickly and often during an iteration. Real devices, which are expensive to use, can then be used sparingly to verify already tested functionality.

The team also created another type of emulator to help test against the legacy system being interfaced with. The programmers on the legacy system have different priorities and delivery schedules, and a backlog of requests. To prevent this from holding up new development, the programmers on the web application have created a type of emulator for the API into the legacy system that returns predetermined values for specific API calls. They develop against this emulator, and when the real changes are available, they test and make any modifications then. This change in process has enabled them to move ahead much more quickly than was previously possible. It has proved to be a simple but very powerful tool.

Emulators are one tool that helps to keep testing and coding moving together hand-in-hand. Using them is one way for testing to keep up with development in short iterations. As you plan your releases and iterations, think about the types of tools that might help with creating production-like test scenarios. See if you can use the tools you’re already using for automating tests to drive development as aids to exploratory testing.

Driving development with tests is critical to any project’s success. However, we humans won’t always get all of the requirements for desired system behavior entirely correct. Our business experts themselves can miss important aspects of functionality or interaction with other parts of the system when they provide examples of how a feature should work. We have to use techniques to help both the customer and developer teams learn more about the system so they can keep improving the product.

Summary

A large part of the testing effort is spent critiquing the product from a business perspective. This chapter gave you some ideas about the types of tests you can do to make your testing efforts more effective.

Demonstrate software to stakeholders in order to get early feedback that will help direct building the right stuff.

Use scenarios and workflows to test the whole system from end to end.

Use exploratory testing to supplement automation and to take advantage of human intellect and perceptions.

Without usability in mind when testing and coding, applications can become shelfware. Always be aware of how the system is being used.

Testing behind the GUI is the most effective way of getting at the application functionality. Do some research to see how you can approach your application.

Incorporate all kinds of tests to make a good regression suite.

Don’t forget about testing documentation and reports.

Automation tools can perform tedious and repetitive tasks, such as data and test scenario setup, and free up more time for important manual exploratory testing.

Tools you’re already using to automate functional tests might also be useful to leverage exploratory tests.

Monitoring, resource usage, and log analysis tools built into operating systems and IDEs help testers appraise the application’s behavior.

Simulators and emulators enable exploratory testing even when you can’t duplicate the exact production environment.

Even when tests are used to drive development, requirements for desired behavior or interaction with other systems can be missed or misunderstood. Quadrant 3 activities help teams keep adding value to the product.