This not being a technical training as much as a training of process and planning for structuring a performance test, I was able to take away some key points, but the most important being; Performance Tests are costly(whether it be time, resources, or tools) and therefore must be planned and set up with significant thought. Here is a list of some of the pre-test planning items that should be taken into consideration.
- Ask not the question “Can we afford the cost of a true performance Test?” but instead “Can we afford the risk of not performing a true performance Test?”
- Definition of True as in the context of the above statement: An exact or very close representation of the production environment in which the product will rest.
- As related to item #1, one must realize that performance tests can be highly influenced by the environment in which it resides. Therefore a test environment should mimic the live environment as best as it can.
- Hardware. Hardware ratio should remain the same throughout the environment as if in production. (ex. If the SQL Server has 8GB of RAM, and RAID HD Setup with 10,000 RPM drives, the test environment should have this set up)
- # of Servers should remain at least in ratio to the production environment. If you have 3 Web Server, 3 SQL Servers, 3 App Servers, your test environment should at least maintain a 1:1:1 RATIO.
- Data. Data should not be a slice of production data it should be a mimic(or production) data. Content may not have to be the same, but the variety and size should be similar/exact.
- Network: Is the application going to be influenced by Network Traffic(Internet, Extranet, IntraNet). If so this traffic should be simulated in the test environment.
- Time. Will there be certain aspects of the application that will be used differently and more frequently during certain parts of the day/year. These should be taken into consideration when determining the expectations and planning of the test boundaries.
- Security. Will there be Firewalls, Throttles, or other administrative tools that will be in production that need to be taken into consideration?
- Bottlenecks – If they exist in the production environment, they must be replicated in the test environment.
- Exceptions to above rules
- Feature Testing/object level performance. Not all performance testing is concerned with End to End/Production level performance. The above rules may not always apply when testing specific load capacities of objects, functions and services. (ex. A application server may not be in control of any outside sources(especially if coming over Internet) and therefore performance may be ran to check how quickly the application server can take data in, process and return the data.)
- When should performance Testing begin?
- In the requirements phase!!! It will always be cheaper to build it in from the beginning then to build it in later. (See Chart below – Cost of Performance Testing). By starting the requirement phase and gathering performance requirements it will be easier to start the Testing process and development process of implementing performance suitable code.
Recommended Link for introduction to performance tools(open source) to start practicing with Performance Testing at no cost(for tools) .
* Training Held by Dale Perry, SQE Training