Performance Testing Using Load Runner _Practical Approach _

Document Sample
Performance Testing Using Load Runner _Practical Approach _ Powered By Docstoc
					Performance Testing Using Load Runner

               Load Runner
Introduction                                            5
      Definition for Performance testing               5
      Types of Performance Testing                     5
      Need for Performance testing                     5
      Automate Performance Testing                     6
      Automated Performance Testing Phases             6
Requirements collection                                 7
      Design                                           7
      Build                                            7
      Execute                                          8
      Analysis and Tuning                              8
Performance Test Planning                               10
      Load / Performance Tool Evaluation & Selection   10
      Test Process / Methodology and Test Strategy     11
      Performance Test Start/Stop Criteria             11
      Test Environmental Setup and Pre-Requisites      12
      Analysis and Report Generation                   12
      Pass / Fail / Exit Criteria                      12
Common Implementation Issues                            13
      Preparation                                      13
      Test Environment Setup                           13
      Customization                                    14
      Execution/Analysis                               15
Introduction to Load Runner                             16
      Load Runner Terminology                          17
      Load Runner Vuser Technology                     17
Creating Scripts Using Load Runner                      18
      Recording Vuser Scripts using VuGen              18
      Enhancing the scripts                            19
      Configure Run time Settings                      20

                                     Load Runner
Executing Scenario using Controller            29
      Scheduling a Scenario                   29
      Setting Up Load Generator Machines      30
      Configuring Load Generators             30
      Setting up the Monitors                 31

Analyzing Results                              33
      Summary Report                          34
      Running Vusers Graph                    35
      Hits per Second                         36
      Transaction Summary                     37
      Average Transaction Response Time       38

Tuning                                         39
      Information Captured for Tuning         39
      Performance Counters to be captured     39

                                 Load Runner
The incredible pace of change and the explosion of software complexity introduce
tremendous risk into software development process. Rigorous testing is the most
common strategy to quantify and reduce this risk to the business. The question for
developers, QA teams, and management alike is how accurately and thoroughly they
can validate the system’s performance before Go Live – without breaking the Budget.

By conducting performance testing, an organization can verify that the application(s)
meet the Business needs before Go Live. This ensures that there are no Deployment
Risks, and Quantifies the impact of changes on End-User’s experience by pinpointing
the Failed Components for Rapid Resolution.

This paper presents a complete practical approach for performance testing and
covers the best approach which includes the following:

      What is performance testing?
      Why should you automate it?
      What is the right process for performance testing?
      Who should be involved in a good performance test?
      What are the requirements need to be collected for performance Testing?
      How to plan for performance Testing?
      What are the common implementation issues faced during the performance
      How to conduct a performance testing using Load Runner?
           o   Creation of Scripts
           o   Enhancing Scripts
           o   Creation and Execution of Scenarios
           o   Analyzing Results
           o   Tuning the application.

                                     Load Runner

What is Performance Testing?

Performance Testing is aimed at verifying the system’s performance requirements
like response time, Transactional throughput and no of concurrent users.
Performance testing is used to accurately measure the End-to-End performance of a
system prior to Go Live. It will identify the loop holes in Architectural Design which
helps to tune the application.
It includes the following:
      Emulating ‘n’ number of users interacting with the system using minimal
      Measuring End-User’s Response time.
      Repeating the load consistently
      Monitoring the system components under controlled load.
      Providing robust analysis and reporting engines.

Types of Performance Testing

Load Testing:
To verify that system can handle the expected load, on deploying it under Real Time

Stress Testing:
To identify the application’s Break Point, also to measures whether the application’s
environment is configured to handle the expected or potentially unexpected high
transactional volume.

Scalability testing:
Scalability testing integrates very well with performance testing. The purpose of
scalability testing is to determine whether the application automatically scales to
meet the growing user load.

Volume testing:
To verify the stability of the system with respect to handling large amounts of data
over extended period of time.

                                  Load Runner
Need for Performance testing

   To verify the below mentioned:
      Application’s response for the intended number of users
      Application’s maximum load resisting capacity
      Application’s capacity of handling the number of transactions required by the
      Application’s stability under expected and unexpected user load
      Ensuring that users have proper response time on production

Automate Performance Testing

Automating Performance Test is a discipline that leverages people, process, and
technology to reduce the Risks of the Application. It is about applying production
workloads to pre-deployment systems while simultaneously measuring system
performance and End-User experience.
Below are the loopholes in manual Load Testing that leads to Automation.
      Difficult to measure the performance of the application accurately.
      Achieving the synchronization among the users.
      ‘n’ number of real time users are required to involve in Performance Testing
      Difficult to analyze and identify the results & bottlenecks.
      Increases the infrastructure cost.

Automated Performance Testing Phases




                            Analysis and Tuning

                                   Load Runner
Requirements Collection


This is the primary phase where team will be gathering the requirements of the
performance testing. Requirements can be Business, Technical, System and Team

Business requirements includes

   1. An    Application    Overview:        Performance    Team   should   have   good
       understanding of the Application prior to Testing

   2. Business Processes List: Team should gather complete information about
       list of key Business activities that will be performed by an End User on the

   3. Business Process Flows: Document, which contains step-by-step details of
       the Business processes identified by Business Team.

   4. Transaction List:     List of Key activities with in the Business process that
       need to be performed in unit time.

Technical requirements should be collected from the System Administrators and
Database Administrators, which include the following:

   1. Complete Architecture of the Environment

   2. How many no. of users that the system should support during normal and
       peak hours.

   3. How many no. of transactions per unit time should be processed by the

   4. Response time for all the Business Processes.

   5. What are the different system workloads that will be experienced in

   6. How the load balancing will be done during the testing

                                  Load Runner

This phase consists of automating the requirements collected during the design
phase. It includes

   1. Scripting: Record the business processes gathered in the design phase into
        Automated scripts.

   2. Transactions:        These are the points that measure the performance of a
        particular key activity with in the business process.

   3. Parameterization: Input data needs to be passed for the scripts for
        performing the Business activity on an application.

   4. Scenarios: The complete End-to-End Business Workflow, which simulates the
        Real Time Environment.

   5. Monitors: Decide which servers or machines need to be monitored.

Environment setup consists of assembling Hardware, Software and Data required for
executing a successful real load. This may involve working with the systems, DBA,
Operations and Business Teams.


Execution will be done in multiple phases. It consists of various types of testing like

Baseline Testing

This testing is done to ensure that both applications’ functionality & environment
setup for performance testing is proper. This will be done with 10 to 20 users.

Performance Tests

This includes running the scripts with Average and Maximum load with specified no
of transactions. We will be simulating the real time environment with different load
conditions using different Scenarios, Think Time, Pace Time and no. of users.

Benchmark Tests

These are designed to measure and compare the performance of each machine type,
environment or build of the application in ideal situation. These tests are run after
the system undergoes scalability testing to understand the performance impact of
different architectures.

                                    Load Runner
Analysis and Tuning

During the performance testing we will be capturing all the details related to the
system like Response time and System Resources for identifying the major
bottlenecks of the system. After the bottlenecks are identified we have to tune the
system to improve the overall performance.

Performance Optimization

This quantifies to improve the performance of the system as measured by improved
end-user response time or by reducing the overall required hardware infrastructure.

Performance Test Planning

   To make any operation successful, planning plays a major role. According to the
   80-20 theories, 80% of the time should be spent in planning and 20% in real
   time    plan   execution/implementation.     In   the   similar   procedure,   software
   performance test planning is very crucial as well.

   Any Software Performance Test Plan should have the minimum contents as
   mentioned below,

          Performance Test Strategy and scope definitions.

          Test process and methodologies to follow in different test phases.

          Test tool details (Tool Evaluation, Selection, Configuration and Add-ins)

          Test cases details including scripting, library development and script
           maintenance mechanisms to be used in every performance Test Phase.

          Resource allocations and responsibilities for Testers.

          Risk management definitions.

          Test Start /Stop criteria along with Pass/Fail criteria definitions.

          Test environment setup requirements. (Hardware Setup, Software Setup,
           Os Configurations, Third Party Utilities Setup etc.)

          Virtual Users, Load (Iterations Vs Users), Volume Load Definitions for
           Different Load/Performance Test Phases.

          Results Analysis and Reporting format definitions.

                                    Load Runner
Load / Performance Tool Evaluation & Selection

   Evaluation of tool is an important task in performance test automation where
   there are several parameters to be considered. In case the tool evaluation and
   selection is completely done by the client then we don’t need to consider this.
   While selecting any tool for Load or/and Performance testing the following things
   should be analyzed:

         Test budget Vs Available Load /Performance tools in the market mapping
          to the Budget.

         Protocols, Development & Deployment Technologies, Platforms, Middle-
          Ware tools / Servers, Third-Party Integrations with the application under
          test Vs support for these factors in the available tools with prioritization of
          the availability in the tool for the scope of expected test.

         Complexity of the tool usage Vs Availability of the tool experts along with
          the timeline requirements for the tool scripting / load scenario creation /
          tool   configuration   with   respect   to   man-hours   and   other   resource

         Tool limitations and work-around factors mapped with the current scope of

         Tool’s integration / Portability factors with other tools used for monitoring
          and Analyzing.

         On evaluation and selection of the base tool, third party monitors / tools
          to be used in integration with main Load testing tool should be defined.

Test Process / Methodology and Test Strategy
   Test process, methodology and strategy for any Performance testing will
   definitely vary from project to project based on the client’s requirement /
   company. By using a methodology a project can make sure that resources are
   applied to the problem effectively and the people involved in the work in a
   structured way.

                                   Load Runner
Performance Test Start/Stop Criteria
   In performance testing, there should be a defined Start and Stop criteria to avoid
   the ad-hoc process resulting in weird outputs. This Start / Stop criteria should be
   well defined in the performance Test Plan along with all the risk management
   The start of any Performance testing should be done for any application or any
   module of the application during the design, database structure and application’s
   development platform is finalized with no major changes at any layers of the
   application that effect directly or indirectly.
   Once the Performance test is conducted, results are collected and analyzed. The
   cycles of tuning will be carried out but definitely the end criteria should be well
   defined to stop this Performance testing. Tuning will be done based on various
   factors like product release timelines (Project Deadlines) and client requirements
   in terms of system stability, scalability and reliability under load and performance
   tolerance criterions.
   In some projects the end criteria is defined based on the client performance
   requirements defined for each section of the application. So, if the product
   reaches to the expected level then that can be considered as the end criteria for
   performance testing.

Test Environmental Setup and Pre-Requisites
   In performance testing, test environment is composed of hardware environment
   like Machine, Memory, Virtual Memory, Number of CPU’s etc., software
   environment like OS, File system settings, free space for logs and application
   configuration along with middle tier server and associated configurations,
   Database, Network etc. his setup also involves the Automation Tools installation,
   configurations, Agent Installations, setup in Load Clients, Third Party Monitors on
   Clients / Servers etc.

Analysis and Report Generation
   Once the results are ready, analysis is the most important job in the whole
   performance test. The main criteria for analysis should be to find out the
   bottlenecks and to track out the transactions / users / hits that create the highest
   level of performance reduction.

                                    Load Runner
   The main factors to be considered are:
          Request / Response Timings from client to server
          Transaction Timings Vs Users
          Transaction Timings Vs Load
          Transaction Timings Vs CPU Utilization
          Server Memory Utilization
          Database server performance using DB server monitors
          Web / Application server performance
          Paging Rate Ratios
          Load Balancer Factors

   After consolidating all the results, the statistical analysis should be done and the
   performance test report should be created such that the development, design-
   architecture, database group can get a clear idea about the application’s
   performance and all the other factors affecting it. After analysis, based on the
   scope tuning like OS tuning, Database Tuning, Memory Up-gradations, CPU
   counts in Multi-CPU machine, Code tuning, Application Server cache tuning or
   other tunings can be done. Same tests executed earlier should be repeated in the
   tuned environment to make sure that there are differences in the results before
   and after tuning. These tuning cycles can be repeated until the required
   performance is achieved.

Pass / Fail / Exit Criteria
   At the end of the Performance test execution, we can find the number of failed
   users, failed transactions or failed scripts. But the Pass / Fail criteria for a
   particular test should be well defined before execution of the test, as this
   criterion entirely depends on the scope of testing and the environment. The failed
   items may be due to any of the performance test parameters. So the failed item
   details should be included in the application’s limitations list. By doing so, the
   limitations list can give a clear picture about the performance limitations with
   respect to load parameters which in turn will give inputs for the tuning /
   enhancements for the application. Based on the test results, tuning cycles will be
   repeated but the exit criteria should be pre-defined to avoid unnecessary tuning
   life cycles.

                                   Load Runner
Common Implementation Issues

   This section describes the most common issues that occur during the load testing
   and also suggests strategies for addressing those issues.


   The biggest issue with planning is that it is not performed adequately. A good
   process is to allow three months for a load test with completion at least a month
   before “go live”. Preparation, development, execution/analysis and results
   summary each should be allocated one month. Typically the results summary can
   be performed in parallel with late-term project tasks.

   Analysis and Design:
   The first issue to be considered is “what should I test?” and the related question
   “what activity does my user community create”. In many ways this is black art of
   testing. Often the vendors, system architects, IT and business managers make
   educated guesses about the “representative user load profile”. This process
   involves a study of the system’s operation profile. This also includes an interview-
   analyze-negotiate process with user community.

   Business processes:
   High volume business processes / transactions should be built into the test.
   Choosing too few might leave gaps in the test while choosing many will expand
   the script creation time. We can effectively model the most common 80 percent
   of the transaction throughput but trying to achieve greater accuracy is difficult
   and expensive.

Test Environment Setup

Load testing requires a complete, stable and independent environment for testing.

   To get the reliable results, the physical components of the test environment must
   represent the production environment. This offers the advantage for ongoing
   testing. This is particularly useful for maintenance or system expansion.

                                  Load Runner
In addition to the hardware required for load test, the test bed must also have
fully installed and functioning software. Since Load Runner (Vuser) represents a
real time user (s), the system should successfully support all the user actions.

Organization’s network infrastructure shared by computers may create significant
amount of “background noise”. While it is probably impossible to accurately
model each and every network access, but it is good to examine the current
network utilization and understand the impact on network traffic.

Often the system under test will support the global enterprise. In this
environment tests may often need to be run at remote sites across the WAN.
WAN connectivity need to be emulated in the lab or assumptions must be made.


Customization of Load Runner is possible and has been performed in many
environments. It will take considerable engineering effort and is not an out-of-
box-solution. Specifically customization of load runner requires:
1. Load Runner expert familiar with Template programming
2. System specialist familiar with the application’s API and administration.
3. An additional one-month of project time to investigate and construct a custom
   Load Runner drive.

Functional Support:
One of the most important factors in script creation productivity is the amount of
functional support provided – access to individuals who understand the
application’s functionality. When a test team member encounters a functional
error while scripting, the business Operation doesn’t function properly. The team
member typically has to stop since he or she is not equipped with the skill to
solve the issue. At that point, script creation is temporarily halted until a
functional team member helps to resolve the issue.

                               Load Runner
  Test design Quality:
  The second factor of script development is the quality of the test design. Ideally
  the test design should specify enough information for an individual with little or
  no experience to test builds. System test documentation is an excellent source of
  this information. Often designs are incorrect or incomplete as a result, any
  omission will require functional support to complete the script development.

  Business Process Stability:
  To load/ stress a large system, at first the system’s business processes should
  function properly. It is typically not effective to attempt a load test a system that
  doesn’t even work for the user. This means that the system needs to be nearly

  System Changes:
  The last factor in script development is the frequency of system changes. For
  each system revision, test scripts must be evaluated. Each test should be
  unaffected, require simple rework, or complete reconstruction. While testing tools
  are used to minimize the effect of the system change, which will reduce the
  scripting time.

  Test Data Load:
  The system should be loaded with development test data. This data often comes
  from a legacy conversion and predecessor to the volume data for the test.


  System Changes:
  Often there is shift between the systems used for development and execution
  analysis. The first step is to execute all the business scenarios with single user.
  This typically involves working with each test to account for system changes like
  difference in data load, configuration, system version etc which may cause
  issues. It is recommended to perform a backup of the development environment
  and/or database that may be restored for execution.

                                 Load Runner
   Volume data:
   System performance may vary widely with the volume of the system data. The
   system under test should have sufficient data to model the size of the product
   database at the time of deployment. It is also good to expand the database size
   to model the system performance with more than expected data for one or two

   System Support:
   Just as the development requires functional support, execution/analysis requires
   system support because the test team may not understand the system
   architecture and technology. The purpose of the system support is to help in
   interpreting the load runner results and performance log files. While Load Runner
   will describe the facts based on which the system support will describe the reason
   and suggest remedies for the problems. These suggestions can be implemented
   and the tests will be rerun to cross verify the new implementation.

Introduction to Load Runner

   Mercury’s Load Runner is an efficient & effective tool used to conduct
   performance testing. The following reasons prove its efficiency:

   1. On - demand Production Work Loads: Load Runner is able to drive ‘n’
      number of virtual users executing business process to emulate the real time

   2. Enterprise Environment Support: Load Runner supports nearly 40
      protocols; this includes Web, J2EE, .NET, XML, SAP, Siebel, Oracle,
      PeopleSoft, wireless, Citrix, and other client-server applications.

   3. Enterprise Monitoring Support: Load Runner’s real-time performance
      monitors provide detailed metrics on all parts of the system under test. This
      includes Web servers, application servers, databases, ERP and CRM systems,
      firewalls, and load balancers. Load Runner can identify hardware limitations
      and software configuration issues that might otherwise go undetected.

   4. Diagnostics: Load Runner can be used to trace, time and troubleshoots
      individual application components under load. Users can now drill down from

                                  Load Runner
       a slow end-user transaction to the bottlenecked method or SQL statement
       that is causing the slowdown. This granularity of results data ensures that
       every load test provides development with actionable results, thus reducing
       the cost and time required to optimize J2EE, Siebel, and Oracle deployments.

   5. Infrastructure     Tuning:    Load    Runner   will   provide     a   customizable
       methodology to isolate and resolve infrastructure bottlenecks in the system
       prior to going into production. Over 70 percent of performance bottlenecks
       are due to system infrastructure issues. By detecting and resolving these in
       pre-production, Load Runner saves valuable time and money.

   6. Automatic Analysis: Load Runner’s Auto-Correlation wizard automatically
       takes all the monitoring & diagnostics data and provides the top-five causes
       of performance degradation. As a result, Load Runner has the industry’s
       deepest and broadest analysis. Converting performance-testing results into
       actionable, precise data to development dramatically reduces time to
       resolution and allows for more testing cycles. This is a critical piece in
       ensuring that a high-quality application goes into production.

   7. Ease of Use: Load Runner is built from the ground up for QA users. It
       provides a visual scripting language, Data and autocorrelation wizards and
       Active Screen technology to make scripting and running of performance tests
       as easy as possible.

Load Runner Terminology

Scenario:   A scenario defines the events that occur during each testing session.
Scenario defines and controls the number of users to emulate, the actions that they
perform and the machines on which they run their emulations.

Vusers: In the scenario, Load Runner replaces human users with virtual users or
Vusers. When you run a scenario, Vusers emulate the actions of human users
working with your application.

Vuser Scripts: The actions that a Vuser performs during the scenario are described
in a Vuser script. When you run a scenario, each Vuser executes a Vuser script.

                                   Load Runner
Transactions: To measure the performance of the server, you define transactions.
A transaction represents an action or a set of actions that you are interested in

Controller: LoadRunner Controller will be used to manage and maintain your
scenarios. Using the Controller, you control all the Vusers in a scenario from a single

Load generator: When you execute a scenario, the Load Runner Controller
distributes each Vuser in the scenario to a load generator. The load generator is the
machine that executes the Vuser script, enabling the Vuser to emulate the actions of
a human user.

Load Runner Vuser Technology

                                       Remote agent
                                       Dispatcher                Vuser

       Controller                           Agent                Vuser


                                                    Load generator

Creating Scripts using Load Runner

   We have to create the scripts by recording the steps specified in the business
   scenario document. Creating the script consists of the flowing steps.

                                  Load Runner
                             Record Vuser Script

                             Enhance Vuser Script

                             Configure Runtime

                             Run the Vuser Script in
                             Stand Alone Mode

Recording Vuser Scripts using VuGen

We can use VuGen to develop the Vuser script by recording user performing typical
business processes on a client application. Each Vuser script that you create with
VuGen can directly communicate with a server by executing calls to the server API
without relying on client software. In addition, when a user communicates directly
with the server, system resources are not used on a user interface. This lets you run
large number of Vusers on a single workstation.

   1. Open the Virtual User Generator.

   2. Select the Protocol for recording.

   3. Click on Start recording.

   4. Enter URL/Path of the application for recording.

   5. Record all the steps for the specified business scenario

Enhancing the scripts

While recording a script or after recording, you can enhance its capabilities by adding
the following functions to the script.

                                    Load Runner
   1. General functions

   2. Protocol specific functions.

   3. Standard ANSI C functions.

   4. Parameterization.

   5. Correlation.

General Vuser functions:

General Vuser functions greatly enhance the script functionality of any Vuser script.
You can use general Vuser functions to measure the server performance, control
server load and debugging code or retrieve run-time information about the Vusers
participating in the scenario.

Protocol Specific Functions:

There are several library functions that you can use to enhance your script. For
example LRS functions for Win Socket applications, LRD functions for database
applications etc.

Standard ANSI C Functions:

You can enhance your Vuser scripts by adding ANSI C functions to the script. ANSI C
functions allow you to add comments, Control flow statements and conditional
statements so on to Vuser scripts.


When you record a business process VuGen generates a script that contains actual
values used for recording. Suppose you want to perform script actions like query,
Submit using different values from those recorded values. To do this you need to
replace the recorded values with parameters. This is called as parameterization.

                                     Load Runner

Correlation is used to capture the dynamic value for a particular field at run time.

The primary reason for correlating statement is

   1. To simplify or optimize your code

   2. For Dynamic Data

   3. To accommodate unique records.

You can also enhance the scripts by

   1. Inserting transactions into a Vuser script.

   2. Inserting Rendezvous point into a Vuser script

   3. Inserting comments into the script.

   4. Sending messages output.

   5. Handling errors in Vuser Scripts.

Configuring Run time Settings

The run time settings will define the way that the script runs. These settings are
stored in the file default.cfg located in the Vuser directory. Runtime settings are
applied to Vusers when you run the scripts in VuGen or controller. We have to adjust
the run time settings based on the no. of transactions/hour. We will be calculating
this using the formula.

Transactions / hour = no. of user* 60 /

Action section Execution time + Think time + Pacing Time)

Run Logic

                                   Load Runner
Number of Iterations: Based on the number of iterations set, Load Runner repeats
all the Actions for specified number of times. If you specify scenario duration in the
Controller’s Scheduling settings, the duration setting overrides the Vuser iteration
settings. This means that if the duration is set to one hour, the Vusers will continue
to run as many iterations as possible in one hour, even if the run-time settings set to
one iteration.


The Pacing Run-Time settings let you control the time between iterations. The pace
tells the Vuser how long to wait between iterations of your actions.

                                  Load Runner
You instruct the Vusers to start each iteration using one of the following

As soon as the previous iteration ends: The new iteration begins as soon as the
previous iteration ends.

After the previous iteration ends with a fixed or random delay of …: Starts
each new iteration, with a delay of specified amount of time after the end of the
previous iteration. Specify either an exact number of seconds or a range of time.

At fixed or random intervals, every … [to …] seconds: You can specify the time
between iteration either a fixed number of seconds / range of seconds. For example,
you can specify to begin a new iteration every 30 seconds, or at a random rate
ranging from 30 to 45 seconds from the beginning of the previous iteration. Each
scheduled iteration will only begin when the previous iteration is complete. The
actual amount of time that the Vuser waits between the end of one iteration and the
start of the next one appears in the Execution Log when you run the script.

                                   Load Runner

Enable Logging
This option enables automatic logging during replay. VuGen writes log messages that
you can view in the Execution log. This option only affects automatic logging and log
messages    issued   through   lr_log_message.      Messages    sent   manually,   using
lr_message, lr_output_message, and lr_error_message, are still issued. The
Log run-time settings allow you to adjust the logging level depending on your
development stage.

Standard Log
Creates a standard log of functions and messages sent during script execution to use
for debugging. Disable this option for large load testing scenarios. If the logging level
is set to Standard, the logging mode is automatically set to JIT logging when adding
it to a scenario.

                                   Load Runner
Extended Log
Creates an extended log, including warnings and other messages. Disable this option
for large load testing scenarios. If logging is disabled or if the level is set to
Extended, adding it to a scenario does not affect the log settings. You can specify
additional information, which should be added to the extended log using the
Extended log options:

Parameter substitution

Select this option to log all parameters assigned to the script along with their values

Data returned by server

Select this option to log all of the data returned by the server.

Advanced trace

Select this option to log all of the functions and messages sent by the Vuser during
the session. This option is useful when you debug a Vuser script.

Think Time

                                   Load Runner
Vuser think time emulates the time that a real user waits between actions. For
example, when a user receives data from a server, the user may wait several
seconds to review the data before responding. This delay is known as the think time.
VuGen uses lr_think_time functions to record think time values into your Vuser

By default, when you run a Vuser script, the Vuser uses the think time values that
were recorded into the script during the recording session. VuGen allows you to use
the recorded think time, ignore it, or use a value related to the recorded time.

Ignore think time: Ignore the recorded think time option replay the script by
ignoring all lr_think_time functions.

Replay the think time: The second set of think time options let you use the
recorded think time.

       As recorded: During replay, use the argument that appears in the
       lr_think_time function. For example, lr_think_time (10) waits ten seconds.

       Multiply recorded think time by: During replay, use a multiple of the
       recorded think time. This can increase or decrease the think time applied
       during playback

       Use random percentage of the recorded think time: Use a random
       percentage of the recorded think time. You set a range for the think time
       value by specifying a range for the think time.

       Limit think time to: Limit the think time’s maximum value to the specified

                                   Load Runner
Miscellaneous Run-Time Settings
Error Handling
You can specify how a Vuser handles errors during script execution. By default, when
a Vuser detects an error, it exits. You can use the run-time settings to instruct a
Vuser to continue script execution when an error occurs. To do so, select the
Continue on Error check box in the Miscellaneous run-time settings.
You can also instruct VuGen to mark all transactions in which an lr_error_message
function was issued, as Failed. The lr_error_message function is issued through a
programmed If statement, when a certain condition is met.

Vusers support multithread environments. The primary advantage of a multithread
environment is the ability to run more Vusers per load generator. Only thread safe
protocols should be run as threads.

      To enable multithreading, click Run Vuser as a thread.

      To disable multithreading and run each Vuser as a separate process, click
       Run Vuser as a process.

                                  Load Runner
Automatic Transactions
You can instruct Load Runner to handle every step or action in a Vuser script as a
transaction. This is called using automatic transactions. Load Runner assigns the step
or action name as the name of the transaction. By default, automatic transactions
per action are enabled.

      To disable automatic transactions per action, clear the Define each action
       as a transaction check box. (enabled by default)
      To enable automatic transactions per step, check the Define each step as a
       transaction check box. (disabled by default)

Speed Simulation

Using the Speed Simulation settings, you can select a bandwidth that best emulates
the environment under test. The following options are available:

       Use maximum bandwidth: By default, bandwidth emulation is disabled and
       the Vusers run at the maximum bandwidth that is available over the network.

                                  Load Runner
          Use bandwidth: Indicate a specific bandwidth level for your Vuser to
          emulate. You can select a speed ranging from 14.4 to 512 Kbps, emulating
          analog modems, ISDN, or DSL.

          Use custom bandwidth: Indicate a bandwidth limit for your Vuser to
          emulate. Specify the bandwidth in bits, where 1 Kilobit=1024 bits.

Proxy runtime settings are used to provide the Proxy server details to the scripts.

No Proxy: For all the Vusers make the connection directly to the internet, without
using Proxy Server.

Obtain the Proxy Server settings from the default Browser: All Vusers use the
proxy settings of the browser on the machine they are running. This is the default

User Custom Proxy: All Vusers connect to the Internet through a Proxy using the
settings you specify.

                                    Load Runner
Use Proxy: Enabling this check box provides the Address of the proxy to use and
port. If the proxy requires authentication information provide this in the proxy
authentication window specified below.

Executing Scenario Using Controller

Scenario:    A scenario defines the events that occur during each testing session.
Scenario defines and controls the number of users to emulate, the actions that they
perform and the machines on which they run their emulations.
Manual Scenario: Select this method if you want to build a manual scenario. You
build a manual scenario by creating groups and specifying the script, the load
generator, and the number of Vusers included in each group.
       Use the Percentage Mode to distribute the Vusers among the scripts:
       Select this option if you want to build a manual scenario by specifying a
       number of Vusers to be distributed among the selected Vuser scripts.
Goal Oriented Scenario: Select this method to have Load Runner build a scenario
for you. In a goal-oriented scenario, you define the goals you want your test to
achieve, and Load Runner automatically builds a scenario for you, based on these
goals. This scenario is mainly used for stress testing.

Scheduling a Scenario

Ramp up: Specifies the entry criteria for the users into the server.
       Load All users Simultaneously: Will load all the users at a time after the
       scenario is started.
       Start {No. of users} after every {Time}: will load the specified number of
       users in to the server in a given time.


                                   Load Runner
       Run until Completion: Will run the scripts once. If the number of iterations
       specified inside the Runtime Settings, Script will be executed for specified
       number of iterations for each user.

       Run for {Time} after ramp up has been completed: Scripts will be run
       for specified duration after all the users were ramped up. Script will be
       repeated for ‘n’ number of times based on the duration specified. On selecting
       this option, Run Logic runtime setting specified in the script will be

       Run Indefinitely: Scripts will be running indefinitely till we stop the
       scenario. This option will be used for conducting the Volume testing of an

Ramp Down: Specifies the exit criteria for the users from the server.

       Stop all Vusers Simultaneously: Exits all the users from the server at a

       Stop {No. of users} after every {Time}: Stops a certain number of
       Vusers within a specified time frame.

Setting Up Load Generator Machines

After installing the load runner agent software on the Load Generator machine we
have to configure the Agent for connectivity with the controller machine. Follow the
below steps for configuring the Agent
1. Go to Programs -> Load Runner ->Advance Settings -> Agent Configuration. On
selecting this it will display the following window

                                    Load Runner
Select the Enable Firewall Agent option and click on Settings button. It will
display all the setting related to the agent configuration. Set the following properties
for configuring the agent.

Local Machine Key: Specify the IP Address of the controller machine to establish a
unique connection between the Controller the agent machine.

Configuring Load Generators

For configuring the load Generator click on the Generator button in Controller
design window and add the load generator machine by specifying the name of the
machine. Click on Connect button to check whether the connectivity between the
controller machine and Load generator machine can be established. If the
connectivity is failed please check the following.
   1. Login into the Load Generator machine ensure that Agent process is launched
       and running.
   2. Check the network connectivity between controller machines by running the
       command Ping machine_name at command prompt of the controller/ agent
   3. Verify the Load Generator Machine name / IP address in the load generator
   4. Verify whether the agent machine configured using Agent configuration.
   5. Verify whether any firewall exists.

Setting up the Monitors


Load Runner uses a suite of integrated performance monitors to quickly isolate
system bottlenecks with minimal impact on the system. The suite consists of
monitors for the network, application servers, Web servers and database servers.
These monitors are designed to accurately measure the performance of every single
tier, server and component of the system during the load test. By correlating this
performance data with end-user loads and response times, IT groups can easily
determine the source of bottlenecks. In addition, they can collect and manage all

system performance data from the Load Runner Controller. We can configure the
monitors in controller to capture these details.

                                   Load Runner
Configuring Load Runner Monitors.

   1. Go to Run tab of controller window.
   2. Select the corresponding monitor (Web / Application / Database server) from
       Available Graphs section
   3. Drag and drop the monitor into any of the graphs.
   4. Right click on the graph and select Add Measurements

Now it will display the following window.

   5. Click on Add(top) and enter the machine name of the Web/App/DB server
   6. Now click on the Add (Down) button and select the required Performance
       counters to be measured during the scenario execution.
   7. Click on OK button.

If controller is not able to capture the details after configuring please ensure the

                                  Load Runner
   1. Verify the connectivity between the Load Runner Controller machine and
       Server Machine.
   2. Verify whether you are Admin user or you have sufficient privileges to capture
       the performance counters from the server machine.

Analyzing Results

Once the results are ready, analysis is the most important job in the whole
performance test if the requirements of performance test are not defined. The main
criteria for analysis should be to find out the bottlenecks and we should be able to
track the transactions / users / hits that create the highest level of performance
reduction. Based on the test run, Load Runner will generate various graphs and
reports. This phase contains:


Once the load test is completed user can correlate the various graphs to analyze the
results and to identify the business scenarios causing high CPU utilization and
network traffic.


It includes observation of what is happening in each tier of the application, web
server, Application server and Database server.       We can analyze the system
resources utilized and network delays.


It includes observation and understanding of the particular query or activity which
are not meeting the response time and tuning it by creating indexes, or reordering
the queries and by modifying the hardware( if required).


Performance testers have a single, unified view of how individual tiers, components
and SQL statements impact the overall performance of a business process under load
conditions. With diagnostics, performance testers using Load Runner can see all
components touched by an end – user transaction. More over, users can also see

                                  Load Runner
how long each component takes and how many times it is called. With this
information corresponding team can focus on resources to improve the end-user
experience by targeting the most significant area of the web application and
database server bottlenecks.

Summary Report

The summary Report provides the information about the scenario execution. It
provides the information like Running Vusers, Throughput, Hits Per second, HTTP
Responses per Second, Transaction Summary and Average Transaction Response
time. 90 percent column indicates the maximum response time for ninety percent of
the Transactions.

                                Load Runner
Running Vusers Graph

Running Vusers graph displays the number of users that executed the scripts and
their status during the execution. This graph is useful to determine the load on the
server at any given moment.

Hits per Second

The Hits per Second graph shows the number of HTTP requests made by Vusers to
the Web servers during the scenario execution. This graph helps you to evaluate the
amount of load Vusers generate in terms of number of hits.

                                 Load Runner
Load Runner
Transaction Summary

Transaction summary graph summarizes the number of transactions in the scenario
that failed, Passed, stopped and ended with errors.

                                  Load Runner
Average Transaction Response Time

This graph displays the average time taken to perform transactions during each
second of the scenario run.

                               Load Runner

Information Captured for Tuning

For analyzing the results and for identifying the bottlenecks capture the following
information during the Scenario execution.
   1. Run the SQL Server Trace file during the Scenario Run.
   2. Capture the Performance counters during the run into the Load Runner
       controller (if you have access to LR Monitors) otherwise use the windows
       performance monitor to capture the same.
   3. Run the Diagnosis tools, if you find any Hardware issues.

Performance Counters to be captured

   1. CPU Utilization
   2. Avg. Disk Queue Length
   3. Network Interface
   4. System
   5. Server
   6. Process
   7. Physical Disk
   8. Logical Disk
   9. Memory

Few Tips on Resources utilization of the Servers

Parameter                  Tips
                           None of the CPUs should have utilization of more than
Utilization                80 percent.
Avg. Disk Queue Length     Should not be more than 2.
Load Average               More than three processes should not wait for the CPU.
                           Any non-DB, system processes utilizing CPU intensively
CPU Intensive Processes    warrant attention.
Swapping                   Minimal to no swapped-out processes
Paging                     Minimize page faults

                                  Load Runner
Data Buffer Hit Ratio       More than 70 percent
                            Disk should have less than 50-70 I/Os per second. Also,
High Disk Activity          the disks should be less than 60 percent busy.
                            Minimal to no disk request queue. A queue of 2-4
Long Disk Queues            warrants attention.

Un-indexed Tables           All the tables should have at least one Primary Key.

Measurement Calculations:
The less time per transaction, the greater a system's throughput, as expressed by
the formula derived from Little's Law:
Server Thruput = # of real simultaneous users / ( Average Response Time + Think
Time )

So when no think time is used, the number of virtual users can be calculated by this
formula suggested by Menasce & Almeida:

# of real simultaneous users * [ Response Time / ( Response Time + Think Time ) ]

For example, to simulate 1,000 real users using 2 second response time and 20
second think times:

1,000 x [ 2 / ( 2 + 20 ) ] = 91

System Load

is measured by a rate -- the total number of tasks performed during a certain
period (an hour, day, week, month, quarter, year, etc.). Example: 300 transactions
per second.

There can be several levels of load:

          o   Peak number per hour or per second is typically used for system
          o   Normal load levels are used to plan on-going support requirements.
          o   Low load levels are used to plan the percentage of resources which
              can be taken off-line for repairs and upgrades.

                                  Load Runner
Measures of Variation

      The average measures the central tendency in a group of numbers. For
      example, the average of 10, 20, and 30 is 20. This is calculated by adding all
      three numbers together (10 + 20 + 30 = 60) and dividing this sum by the
      number of items added together (in this example, three).

      Standard Deviation

      Standard deviation is a statistical measure of the extent of variation in a set
      of data. The standard deviation tells you the extent of the spread of
      individual items around the average.

      The number in the 90% column is shown after a run is completed in the
      Analysis program, not in the Controller) because it is calculated by sorting
      all items by value obtained during the run, then identifying the value
      associated the item at the top 10% of all items.

                                 Load Runner

Shared By: