Agile Testing: A Practical Guide for Testers and Agile Teams

Chapter 11 Critiquing the Product Using Technology-Facing Tests

This chapter is focused on the bottom right corner of our testing quadrant. We’ve looked at driving development with both business-facing and technology-facing tests. After the code is written, we are no longer driving the development but are looking at ways to critique the product. In the previous chapter, we examined ways to critique from a business point of view. Now we look at ways to critique from a technology-facing point of view. These tests are an important means of evaluating whether our product delivers the right business value.

Introduction to Quadrant 4

Individual stories are pieces of the puzzle, but there’s more to an application than that. The technology-facing tests that critique the product are more concerned with the nonfunctional requirements than the functional ones. We worry about deficiencies in the product from a technical point of view. Rather than using the business domain language, we describe requirements using a programming domain vocabulary. This is the province of Quadrant 4 (see Figure 11-1).

Figure 11-1 Quadrant 4 tests

Nonfunctional requirements include configuration issues, security, performance, memory management, the “ilities” (e.g., reliability, interoperability, and scalability), recovery, and even data conversion. Not all projects are concerned about all of these issues, but it is a good idea to have a checklist to make sure the team thinks about them and asks the customer how important each one is.

Our customer should think about all of the quality attributes and factors that are important and make informed trade-offs. However, many customers focus on the business side of the application and don’t understand the criticality of many nonfunctional requirements in their role of helping to define the level of quality needed for the product. They might assume that the development team will just take care of issues such as performance, reliability, and security.

We believe that the development team has a responsibility to explain the consequences of not addressing these nonfunctional or cross-functional requirements. We’re really all part of one product team that wants to deliver good value, and these technology-oriented factors might expose make-or-break issues.

Many of these nonfunctional and cross-functional issues are deemed low-risk for many applications and so are not added to the test plan. However, when you are planning your project, you should think about the risks in each of these areas, address them in your test plan, and include the tools and resources needed for testing them in your project plan.

Lisa’s Story

In the past, I’ve been asked by specialists in areas such as performance and security testing why they didn’t hear much about “ility” testing at agile conferences or in publications about agile development. Like Janet, I’ve always seen these areas of testing as critical, so this wasn’t my perception. But as I thought about it, I had to agree that this wasn’t a much-discussed topic at the time (although that’s changed recently).

Why would agile discussions not include such important considerations as load testing? My theory is that it’s because agile development is driven by customers, from user stories. Customers simply assume that software will be designed to properly accommodate the potential load, at a reasonable rate of performance. It doesn’t always occur to them to verbalize those concerns. If not asked to address them, programmers may or may not think to prioritize them. I believe that one area where testers have contributed greatly to agile teams is in bringing up questions such as, “How many concurrent users should the application support?” and “What’s the average response time required?”

—Lisa

Because the types of testing in this quadrant are so diverse, we’ll give examples of tools that might be helpful as we go along instead of a separate toolkit section. Tools, whether homegrown or acquired, are essential to succeed with Quadrant 4 testing efforts. Still, the people doing the work count, so let’s consider who on an agile team can perform these tests.

Who Does It?