Agile Testing: A Practical Guide for Testers and Agile Teams

Each API call has a specific function with a number of parameters that accept different inputs. Each variation will return a different result. The easy tests are simple inputs. The more complicated testing patterns occur when the parameters work together to give many possible variations. Sometimes parameters are optional, so it’s important that you understand the possibilities. Boundary conditions should be considered as well, for both the inputs and expected results. For example, use both valid and invalid strings for parameters, vary the content, and vary the length of the strings’ input.

Another way to test is to vary the order of the API calls. Changing the sequence might produce unexpected results and reveal bugs that would never be found through UI testing. You can control the tests much more easily than when using the UI.

Lisa’s Story

My team was working on a set of stories to enable retirement plan sponsors to upload payroll contribution files. We wrote FitNesse test cases to illustrate the file parsing rules, and the programmer wrote unit tests for those as well. When the coding for the parser was complete, we wanted to throw a lot more combinations of data at the parser, including some really bizarre ones, and see what happened. We could use the same fixture as we used for our tests to drive development, enter all of the crazy combinations we could think of, and see the results. We tested about 100 variations of both valid and invalid data. Figure 10-3 shows an example of just a few of the tests we tried. We found several errors in the code this way.

Figure 10-3 Sample of parsing rules test

We didn’t keep all of these tests in the regression suite because they were just a means of quickly trying every combination we could think of. We could have done these tests in a semi-automated, ad hoc manner too, not bothering to type the expected results into the result checking table, and just eyeballing the outputs to make sure they looked correct.

—Lisa

Janet’s Story

I recently worked with a web application that interfaces to a legacy system through a well-defined API. Due to the design of the legacy system and the fact that the data is hard to replicate, the team hasn’t yet found a way to automate this testing. However, we could look in the log files to verify the correct inputs were passed and the expected result was returned. Valuable exploratory testing of APIs is possible with or without benefit of automation.

—Janet

API calls can be developed early in an application life cycle, which means testing can occur early as well. Testing through an API can give confidence in the system before a UI is ever developed. Because this type of testing can be automated, you will need to work with your programmers to understand all of the parameters and the purpose of each function. If your programmers or automation team develop a test harness that is easy to use, you should be able to methodically create a suite of test cases that exercises the functionality.

Web Services

Web services are a services-based architecture that provides an external interface so that others can access the system. There might be multiple stakeholders, and you may not even know who will be using your product. Your testing will need to confirm the quality of service that the external customers expect.

Web services generally require plenty of security, stress, and reliability testing. See Chapter 12, “Critiquing the Product using Technology-Facing Tests,” for more on these types of tests.

Consider levels of service that have been promised to clients when you are creating your test plans. Make time for exploratory testing to simulate the different ways users might access the web services.

The use of web services standards also offers other implications for current testing tools. As with API calls, web services-based integration highlights the importance of validating interface points. However, we also need to consider message formats and processing, queuing times, and message response times.

Using testing tools that utilize GUI-driven automation is simply inadequate for a web services project. A domain-specific language that encapsulates implementation details “behind the scenes” works well for testing web services.

Chapter 12, “Summary of Testing Quadrants,” has an example of testing web services.

Testing Documents and Documentation