Agile Testing: A Practical Guide for Testers and Agile Teams

It could be that your next release is just an extension of your last, and you will just carry on creating your examples, automating your story tests, and doing the rest of the testing as you’ve been doing. You are one of the lucky ones. For those of you who are starting a brand new project with no previous processes in place, now is the time to consider what testing you will need. We don’t mean you have to decide how to test each story, but look at the big picture and think about the quadrants. Will you need to plan for a special UAT, or will the iteration demos be enough? It is important to raise these issues early so the team can plan for them.

Infrastructure

While you are doing your test planning, you need to consider your infrastructure. Infrastructure can mean your continuous integration setup, test environments, and test database. It can mean how you promote your builds to your test environments. It might mean your test lab, if you have one, or having a separate server to run all your automation tests. These are generally pieces of infrastructure that need some lead time to get in place. This is the time to make a plan.

Lisa’s Story

Some types of testing might require extra effort. My team had a tool to do performance testing and some scripts, but we lacked a production-style environment where we could control all of the variables that might affect performance. For example, the test database was shared by testers, programmers, and two build processes. Slower performance might simply mean someone was running database-intensive tests. We used our staging environment to get a baseline, but it was missing some of the components of production. We set a six-month goal to acquire hardware and software for a proper test environment and get it set up. We wrote a task card or two each iteration to establish the environment step by step.

—Lisa

Whatever your needs are, make sure you understand them and can plan for what you need. If you don’t have the right infrastructure, then you will waste time trying to get it together and cause a bottleneck in mid-iteration.

Test Environments

As we look at the types of features in the next release, we might see the need for a whole new test environment. Think about specialized test environments you may need as well. Will you need more tools? Do you need to expand your test lab so that you can test with different browsers and operating systems? This is the time to think about all testing considerations.

If you’re planning your first release, test environments are a key consideration. You might need a story or iteration just to set up the infrastructure you need. We’ve started more than one project where the only place we could test was the development environment. We found that doesn’t work very well, because the environment is never stable enough for effective testing.

Just as programmers have their own sandboxes to work and test in, it works well if each tester has that same availability and control. We recognize that not all applications lend themselves to this, but at the very least, you need to know what build you’re testing. You also need test data that others will not walk over with their tests. If you don’t have a testing sandbox that’s under your own control, take time to plan what you need to establish for your test environments. Brainstorm with your team about how you can obtain the necessary hardware and software. It might take time, so develop a Plan B for getting something done while waiting for the infrastructure you need.

If you’re working on a large system, you may have to queue up along with other teams to get time on a test or staging environment that includes all of the various pieces of software with which yours must work. This staging environment should mimic your production system as much as possible. If your organization doesn’t have someone responsible for creating environments, your team might need extra roles dedicated to obtaining the test environments you need. These roles might involve working with other teams as well. Release planning is the time to consider all of these testing infrastructure requirements.

Test Data

Release or theme planning is also a good time to think about what test data you might need during the project.

Using test data that closely resembles real data is generally a good practice. Plan for the data you need. We’ve had the opportunity in several organizations to use a copy of production data. Real data provides a good base for different scenarios for exploratory testing. Production data may need to be “scrubbed” before it’s used for testing in order to remove any sensitive information such as identification or bank account numbers. The data needs to be altered to hide the original values but remain valid so that it doesn’t violate database restrictions. Because it takes time for database experts to port production data to a test environment, be sure they’re included in your planning.