Performance testing metrics

Document Sample
Performance testing metrics Powered By Docstoc
					   More such tips – . For
    more tips on testing follow me on Twitter - @TriagedTester

Key Performance Metrics Criteria
As a performance analyst, tester, or developer, you must produce a blueprint on how to performance test the Web
application to ensure the high- level performance goals are met. If you don't create a performance test plan, you
may find out about requirements too late in the SDLC to properly test for them. Using the performance
requirements criteria in the section above, you now need to identify key metrics that will be monitored and
analyzed during the actual performance test.


The breakpoint of your Web application can be defined in various ways. Examples include a server that exceeds
predefined resource utilization targets, too many server errors, or unacceptable response times due to processing

Key metrics for the performance test include the following:

         Server errors acceptability This may seem like a moot point, and no server errors are acceptable
          because they result in a bad user experience. However, during stress testing you will probably come
          across these errors, so you should be prepared to understand why they are occurring and decide whether
          they will happen in your live production environment, with real users hitting your Web application. For
          example, often when a stress test first begins and then again when it shuts down, errors are caused by
          too much load occurring too quickly, or by uncompleted page requests. These errors are caused by your
          stress test, so you can ignore them because they are unlikely to reoccur in the production environment.
         Server utilization acceptability This is an important aspect of performance testing. By identifying this
          up front, you will be able to determine the maximum allowable level your servers should endure. When
          performance testing, this will be a key element in determining the maximum load to apply to your Web
          application. This metric can differ for each Web application and should be documented for support,
          development, test, and management teams to agree on. For example, you might ramp the Web tier up
          until we reach 75 percent CPU utilization. At this level we are serving approximately 2,000 users per
          server, which meets the concurrent user targets identified by our performance requirements. With
          documentation of these metrics, the support team can monitor the production Web servers looking for
          spikes that meet or exceed the performance requirement. The support team can begin to scale the Web
          application up or out to support the increased traffic.
         Memory leaks or other stability issues These issues often arise when running extended performance
          tests. For example, if you execute a stress test for a short period of time you may not find the memory
          leak or other stability issues that only occur after an extended period of heavy activity. Many times
          multiple tests can be executed to accomplish different goals. You may want to run a quick one-hour test to
          determine your Web application's maximum throughput, and then run a weekend-long extended stress
          test to determine if your Web application can sustain this maximum load.
         Processing delays These will occur in almost every Web application where complex business logic
          requires coding. The key is to minimize process delays to an acceptable amount of time. It's a good idea
          to know what's acceptable before performance testing, so you don't waste time escalating an issue to your
          development team that does not require fixing because it meets performance goals. Examples of
          processing delay acceptability are shown in Table 2-5 and include stored procedures taking more than 500
          milliseconds and any Web page duration (measured by the time taken field in your Web tier logs) taking
          more than one second to process. Table 2-5 shows an example of a performance metric acceptability
          table. Your requirements may be different; the key point is to come up with a set of requirements that
          make sense for your Web application.

Table 2-5 Performance Metrics and Acceptable Levels

Metric                                    Location                                     Acceptable Level
   More such tips – . For
    more tips on testing follow me on Twitter - @TriagedTester

CPU utilization              Performance Monitor            < 75%

Memory—available MB          Performance Monitor            > 128 MB

Memory—pages/second          Performance Monitor            <2

ASP execution time           Performance Monitor            < 1 second

DB processing delays         SQL Profiler                   < 500 milliseconds

Web tier processing delays   Time Taken field of Web logs   < 1 second

Description: Document, Guidelines, templates to help any software tester.