Agile Testing: A Practical Guide for Testers and Agile Teams

Regardless of whether or not your team brings in additional resources for these types of tests, your team is still responsible for making sure the minimum testing is done. The information these tests provide may result in new stories and tasks in areas such as changing the architecture for better scalability or implementing a system-wide security solution. Be sure to complete the feedback loop from tests that critique the product to tests that drive changes that will improve the nonfunctional aspects of the product.

Just because this is the fourth out of four agile testing quadrants doesn’t mean these tests come last. Your team needs to think about when to do performance, security, and “ility” tests so that you ensure your product delivers the right business value.

When Do You Do It?

As with functional testing, the sooner technology-facing tests that support the team are completed, the cheaper it is to fix any issues that are found. However, many of the cross-functional tests are expensive and hard to do in small chunks.

Technical stories can be written to address specific requirements, such as: “As user Abby, I need to retrieve report X in less than 20 seconds so that I can make a decision quickly.” This story is about performance and requires specialized tests to be written, and it can be done along with the story to code the report, or in a later iteration.

Consider a separate row on your story board for tasks needed by the product as a whole. Lisa’s team uses this area to put cards such as “Evaluate load test tools” or “Establish a performance test baseline.” Janet has successfully used different colored cards to show that the story is meant for one of the expert roles borrowed from other areas of the organization.

Some performance tests might need to wait until much of the application is built if you are trying to baseline full end-to-end workflows. If performance and reliability are a top priority, you need to find a way to test those early in the project. Prioritize stories so that a steel thread or thin slice is complete early. You should be able to create a performance test that can be run and continue to run as you add more and more functionality to the workflow. This may enable you to catch performance issues early and redesign the system architecture for improvements. For many applications, correct functionality is irrelevant without the necessary performance.

The time to think about your nonfunctional tests is during release or theme planning. Plan to start early, tackling small increments as needed. For each iteration, see what tasks your team needs in order to determine whether the code design is reliable, scalable, usable, and secure. In the next section, we’ll look at some different types of Quadrant 4 tests.

Performance Testing from the Start

Ken De Souza, a software developer/tester at NCR [2008], responded to a question on the agile-testing mailing list about when to do stress and performance testing in an agile project with an explanation of how he approaches performance testing.

I’d suggest designing your performance tests from the start. We build data from the first iteration, and we run a simple performance test to make sure it all holds together. This is more to see that the functionality of the performance scripts holds together.

I used JMeter because I can hook FTP, SOAP, HTTP, RegEx, and so on, all from a few threads, with just one instance running. I can test out my calls right from the start (or at least have the infrastructure in place to do it).

My eventual goal is that when the product is close to releasing, I don’t have to nurse the performance test; I just have to crank up the threads and let go. All my metrics and tasks have already been tested out for months, so I’m fairly certain that anyone can run my performance test.

Performance testing can be approached using agile principles to build the tools and test components incrementally. As with software features, focus on getting the performance information you need, one small chunk at a time.

“ility” Testing

If we could just focus on the desired behavior and functionality of the application, life would be so simple. Unfortunately, we have to be concerned with qualities such as security, maintainability, interoperability, compatibility, reliability, and installability. Let’s take a look at some of these “ilities.”

Security