In helping various collegues and companies manage their performance testing efforts I’ve come across a reuseable set of common activities that have helped me explain and divide the work necessary for a successful engagement.
The following activies:
Engaging in Risk Identification
Developing a Performance-Testing Strategy
The application of the Performance-Testing Framework which produces a Performance-Testing Plan
Assessment
Modeling
Execution
Analysis
Assessment
Performing the assessment is critical to the justification of the performance-testing effort. The result of this activity forms the basis for the charter of the entire effort. There are a number of issues that need to be addressed per the pre-requisites.
Critical questions to ask concerning application performance:
• What is the real cost of system downtime?
• What is the impact on the perception of the organization’s ability to develop, test, implement, or maintain applications?
• What is the impact on the internal or external customer?
• What is the cost of correcting performance issues post-implementation?
Just as important, an assessment of the capability and capacity to execute performance-testing activities is just as important. This includes a deep dive into the measurements that can be collected, the hardware capacity of the hardware comprising the performance testing infrastructure and the network bandwidth to support the generation of transaction load and polling the respective performance counters.
The follow-up activities are:
Determining Performance Criteria
Verify the Measurements
Validate that the Metrics are Measurable
Determine Measures of Success
Modeling
The performance test lead(s) in collaboration with the business analyst(s), system analyst(s), and test lead(s) will determine the combination of test methodologies and test types that are applicable. Since this performance-testing framework does not rely on a completed application the appropriate methodology has to be determined in order to leverage this flexibility.
A performance test plan will be written to document the results of the assessment and the methodology chosen. In addition to the performance test plan, a performance test specification detailing the user and usage profiles, the scripts being developed, the traceability matrix, performance test environment diagram, SUT diagram and scheduling will accompany the plan.
User Profile
Usage Profile
Test Data
Developing Performance Test Scenarios
Performance Test Script and Scenario Traceability
Developing Performance Test Scripts
Execution
Performance Test Infrastructure Evaluation
Apply the appropriate Test Methodologies
Apply the appropriate Test Type(s)
Analysis
Sample Size, Inclusions, and Exclusions
Verify Quality of Test Execution
In Test Analysis
Post Test Analysis Deliverables
Top Down Analysis
Root Cause Analysis
Note the separation of the Test Methodology and Test Types from the Modeling activities. A single model can be used between methods(Profiling, Benchmarking) and Test Types(Load, Stress, Volume, Baseline).
No Comments so far ↓
Your comments are welcomed.