Agile Testing: A Practical Guide for Testers and Agile Teams

Session-based testing is a form of exploratory testing, but it is time-boxed and a bit more structured. I learned about session-based testing from Jonathan Bach and found it gave me the structure I needed to do exploratory testing well. I use the same skills as I do for a jigsaw puzzle: I look for patterns in color or shapes or perhaps something that just doesn’t look right, an anomaly. My thought process can take those patterns and make sense of them, using heuristics I have developed to help me solve a puzzle.

—Janet

Like solving the jigsaw puzzle by putting together the outside pieces first, we can use session-based testing to give us the framework in which we work. In session-based testing, we create a mission or a charter and then time-box our session so we can focus on what’s important. Too often as testers, we can go off track and end up chasing a bug that might or might not be important to what we are currently testing.

Sessions are divided into three kinds of tasks: test design and execution, bug investigation and reporting, and session setup. We measure the time we spend on setup versus actual test execution so that we know where we spend the most time. We can capture results in a consistent manner so that we can report back to the team.

For more information on session-based testing, check the bibliography for work by Jonathan Bach.

Automation and Exploratory Testing

We can combine exploratory testing with test automation as well. Jonathan Kohl, in his article “Man and Machine” [2007], talks about interactive test automation to assist exploratory testing. Use automation to do test set up, data generation, repetitive tasks, or to progress along a workflow to the place you want to start. Then you start using your testing skills and experience to find the really “good” bugs, the insidious ones that otherwise escape attention. You can also use an automated test suite to explore. Just modify it a bit, watch the results as it runs, modify it again, and watch what happens.

An Exploratory Tester

With exploratory testing, each tester has a different approach to a problem, and has a unique style of working. However, there are certain attributes that make for a good exploratory tester. A good tester:

Is systematic, but pursues “smells” (anomalies, pieces that aren’t consistent)

Learns to recognize problems through the use of Oracles (principle or mechanism by which we recognize a problem)

Chooses a theme or role or mission statement to focus testing

Time-boxes sessions and side trips

Thinks about what the expert or novice user would do

Explores together with domain experts

Checks out similar or competitive applications

Exploratory testing helps us learn about the behavior of an application. Testers generally know a lot about the application they’re testing. How do they judge whether the application is usable by users who are less technical or not familiar with it? Usability testing is vital for many software systems. We’ll talk about that in the next section.

Usability Testing

There are two types of usability testing. The first type is the kind that is done up front by the user experience folks, using tools such as wire frames to help drive programming. Those types of tests belong in Quadrant 2. In this section, we’re talking about the kind of usability testing that critiques the product. We use tools such as personas and our intuition to help us look at the product with the end user in mind.

User Needs and Persona Testing

Let’s look at an online shopping example. We think about who will use the site. Will it be people who have shopped online before, or will it be brand new users who have no idea how to proceed? We’re guessing it will be a mixture of both, as well as others. Take the time to ask your marketing group to get the demographics of the end users. The numbers might help you plan your testing.

One approach to using personas is for your team to invent several different users of your application representing different experience levels and needs. For our Internet retail application, we might have the following personas:

Nancy Newbie, a senior citizen who is new to Internet shopping and nervous about identity theft

Hudson Hacker, who looks for ways to cheat the checkout page

Enrico Executive, who does all his shopping online and ships gifts to all his clients worldwide

Betty Bargain, who’s looking for great deals

Debbie Ditherer, who has a hard time deciding what items she really wants to order

We might hang photos representing these different personas and their biographies in our work area so that we always keep them in mind. We can test the same scenario as each persona in turn and see what different experiences they might encounter.