Agile Testing: A Practical Guide for Testers and Agile Teams

All of the agile literature talks about teams being generalists; anyone should be able to pick up a task and do it. We know that isn’t always practical, but the idea is to be able to share the knowledge so that people don’t become silos of information.

However, there are many tasks that need specialized knowledge. A good example is security testing. We’re not talking about security within an application, such as who has access rights to administer it. Because that type of security is really part of the functional requirements and will be covered by regular stories, verifying that it works falls within the first three quadrants. We’re talking about probing for external security flaws and knowing the types of vulnerabilities in systems that hackers exploit. That is a specialized skill set.

Performance testing can be done by testers and programmers collaborating and building simple tools for their specific needs. Some organizations purchase load-testing tools that require team members who specialize in that tool to build the scripts and analyze and interpret the results. It can be difficult for a software development organization, especially a small one, to have enough resources to duplicate an accurate production-level load for a test, so external providers of performance testing may be needed.

Larger organizations may have groups such as database experts that your team can use to help with data conversion, security groups that will help you identify risks to your application, or a production support team that can help you test recovery or failover. Build a close relationship with these specialists. You’ll need to work together as a virtual team to gather the information you need about your product.

Chapter 15, “Tester Activities in Release or Theme Planning,” explains how to plan to work with external teams.

The more diverse the skill sets are in your team, the less likely you are to need outside consultants to help you. Identify the resources you need for each project. Many teams find that a good technical tester or toolsmith can take on many of these tasks. If someone already on the team can learn whatever specialized knowledge is required, great; otherwise, bring in the expertise you need.

Skills within the Team

Jason Holzer, Product Owner for Property Testing (performance, security, stability, and reliability) at Ultimate Software, tells us that a good programmer can write a multithreaded engine to call a function concurrently and test performance. Jason feels that agile teams do have the skills to do their own performance testing; they just may not realize it.

Performance testing does require a controlled, dedicated environment. Some specialized tools are needed, such as a profiler to measure code performance. But, in Jason’s view, performance, stability, scalability, and reliability (PSR) tests can, and should, be done at the unit level. There’s a mind-set that holds that these tests are too complex and require specialists when in fact the teams do possess the necessary skills.

Jason finds that awareness of the “PSR” aspects of code needs to be part of the team’s culture.

If stakeholders place a high priority on performance, stability, scalability, and the like, Jason recommends that the team talk about ways to verify these aspects of the application. When teams understand the priority of qualities such as performance and reliability, they figure out how to improve their code to ensure them. They don’t need to depend on an outside, specialized team. Jason explains his viewpoint.

The potential resistance I see today to this plan is that someone believes that programmers don’t know how to PSR test and that there will need to be a great deal of training. In my opinion, a more accurate statement is that programmers are not aware that PSR testing is a high priority and a key to quality. I don’t think it has anything to do with knowing how to PSR test. PSR testing is a combination of math, science, analysis, programming, and problem solving. I am willing to bet that if you conducted a competition at any software development organization where you asked every team to implement a tree search algorithm, and the team with the fastest algorithm would win, that every team will do PSR testing and provide PSR metrics without teaching them anything new.

PSR testing is really just telling me “How fast?” (performance), “How long?” (stability), “How often?” (reliability), and “How much?” (scalability). So, as long as the awareness is there and the organization is seriously asking those questions with everything they develop, then PSR testing is successfully integrated into a team.

Take a second look at the skills that your team already possesses, and brainstorm about the types of “ility” testing that can be done with the resources you already have. If you need outside teams, plan for that in your release and iteration planning.