Load testing using Six Sigma Capability Charts.

The second question (Six Sigma project) should address capability and answer the basic question:

Is the software performance capable of meeting the specified requirements?

Is the performance of the software capable?


Given a stable process the question of capability (falls within the performance requirements) can be addressed. In order to determine if the performance of a given piece of software is capable we need to specify the acceptable tolerances (ranges) for the software. The performance specification (tolerances) could be response time, memory used, CPU usage or any other measurement that characterizes a desired performance outcome. It is important, as noted in Six Sigma, that only a stable process can be characterized with regard to its capability. Therefore Step 1, determine process stability, is a prerequisite for defining the software's performance capability.

As with the stability performance test it is important to define the workload and target configuration when determining the capability of software performance. For example it is possible that for a given workload the software performance is within acceptable limits (i.e. response time) on a computer with quad CPU processors but the software performance is not capable of maintaining the acceptable response times on a computer with only dual CPU processors.

As with the stability performance project type, the project type for determining the performance capability of the software is a benchmarking project type, as the software under test is to be characterized without changes that will bring about a potential improvement. It is possible that a performance improvement project (using Six Sigma DMAIC) will follow a performance capability characterization of a given piece of software.

The software performance capability project follows the following path:-

Specify the acceptable tolerances (ranges) that will define the software performance capability.


The idea of a process being capable is taken from the Statistical Process Control arena and basically involves setting thresholds (such as response time) that the process must comply with. By way of example we may want the average response time to be no worse than 12 seconds with an average of 8 seconds. We may also want to define the standard deviation (or spread) of acceptable response times although this requires a little more statistical analysis.

Note: We can execute a load test first for a given workload on a given target system configuration and then determine (after the event) what is the systems capability using a capability chart. The approach of stating the acceptable ranges up front is more of a requirements driven approach but it is a moot point when measuring the actual capability of the performance of the software (as basically it is what it is).

After the acceptable ranges are defined for the appropriate measures the project follows a similar path to the stability performance testing project type.