Extending Open Source solution for Performance testing of Web (http\https)
(Nupur Shrivastava, Nitika Aggarwal, Sakshi Gupta, Happy Himanshu Gupta)
(Under the Supervision of Mr. Vikas Saxena, Sr. Lecturer)
Jaypee Institute of Information Technology
In this research paper we examine the need for load testing and highlight the shortcomings of open source
load testing solutions, vis-à-vis their commercial counterparts and present our own approach to design
plug-ins over open source tools that can bring them a step closer to COTS. Our proposed solution enhances
the capabilities of Open Source testing tools by providing an in-depth performance monitoring of web,
application and database servers. This helps in determining bottleneck of the application and enables the
user to proactively correct potential problems before they become critical. We also propose here, the
implementation of IP spoofing and bandwidth simulation, which would be the add-on features of this tool.
KEYWORDS: performance, monitoring, load testing, analysis, OpenSTA, WAN emulation,
bottleneck, Analyzer, Profiler, Graphs, Overlays, Load Runner
1. INTRODUCTION 2. EXISTING TECHNOLOGIES
For any web application, the end to end response
time for a user is a critical issue. As the number 2.1 CURRENT STATE OF ART
of users accessing the same web application
increases, the performance of the application 2.1.1 Commercial tools
decreases due to slowdown . Load testing The most popular commercial tools available for
refers to checking the performance of a system Load testing and diagnostics are
under multi-users workload. It is a very critical a) Mercury LoadRunner
part for any web application before it goes live. b) IBM Rational Performance tester
There are many open source load testing c) IBM Rational Robot
solutions available today, but they do not match d) Borland Silk Performer
up completely to their commercial counterparts.
Commercial tools on the other hand prove to be 2.1.2 Popular Open source Testing Solutions
very expensive option for small scale industries. For the purpose of this research paper, we list two
The challenges of successfully conducting an of the most popular tools for performance testing
online business are as follows: of HTTP\HTTPS applications
1) high user volumes a) Apache JMeter
2) low response times for web requests b) OpenSTA
3) the overall reliability of the service
Monitoring the performance requires an efficient 2.2 SHORTCOMINGS
analysis of the various parameters associated Open source tools lag from their commercial
with performance measures. Our major concern counterparts in the following
lies with the determination of the bottleneck in
the complete n-tier application that comprises of a) Effective Integrated Monitoring and
a client, a network and different servers involved analysis capability
(web, application, database). Thus, a detailed b) User network speed simulation
analysis for every performance parameter from c) IP Spoofing capability
all the components involved under Load
condition is required for diagnosing the complete
application being monitored.
3. PROPOSED APPROACH OF THE the real time variations in the performance
SOLUTION: parameters and locate the areas that have been the
cause for erroneous or unusual results.
The key requirements for the DASHBOARD are:
3.1EFFECTIVE INTEGRATED a) Display of simultaneous graphs with
MONITORING MODEL feature of graph overlays.
b) Maintenance of a PROFILER which
allows user to reconfigure the saved
sessions. Informing user of any
abnormality in the system and giving him
the option to execute necessary measure.
c) In real environment, the values lying out
of bounds are taken care off by the
ANALYZER, which then diagnoses the
fed data (input) and tries to point out the
cause of the anomaly. We will have a
facility at the Dashboard console that can
send system alerts through emails etc.
Since the LC(s) have to be deployed on different
platforms, we decided to use a platform
independent language like PERL for the designing
of Listening components as well as Central
Component. For communication and transfer of
Figure3.1.1: Integrated Monitoring Model
data values between the CC and LC, we have used
Socket Programming in Perl. For designing of the
Graphical User Interface and the Dashboard, we
First, OPENSTA tool is used to generate virtual have used VB.net.
user load on the machines (servers) to be
monitored. For the collection of the performance 3.1.1 DASHBOARD
parameters on different server machines like IIS,
SQL, Weblogic, ORACLE etc., we design 220.127.116.11 USER INTERFACE:
lightweight listening components (LC) and
deploy them on different Servers (Web, The User Interface (UI), designed using
Application, Database) with different OS VB.NET, provides a console on which the user
environments. could design and set a session for monitoring and
can see the results in form of REAL TIME
The key requirements of the LC(s) are: GRAPHS. These real time graphs are obtained
a) They should be lightweight and should corresponding to each machine participating in the
not consume large memory. monitoring session as well as from the machines
b) They should continuously gather which are used for generating the load using
Configurable Data Points OPENSTA. The UI runs the scripts placed at
(parameters) on the server machine by backend prepared using PERL scripting language.
polling the logs or using the APIs of the
server and the OS at a configurable Following are the steps of interaction between CC
polling interval and relay it back to a and LC:
central request component (CC). 1. The CC broadcasts START request, and the
c) They should run in daemon mode. LC(s) respond back by sending their system
The central request component (CC) which acts 2. The CC requests the selected LC(s) to send their
as a Performance Data Collector Server list of parameters.
connects to the installed LC(s), the network and 3. The CC sends the configured list of parameters
the load generator tool (OPENSTA), and stores to be monitored to each LC, including the
the data for every synchronized pulse into a configured test to be run on OPENSTA
database. Dashboard, a term which defines the machines.
console at CC side, keeps on fetching the data 4. CC broadcast a “START MONITORING”
from the database and helps the user to visualize message to all selected LC(s) at the same instant.
The user can end the session whenever he wants facility of saved profiler, by recreating a new
to by sending a “STOP” broadcast. session with modifications to an old session.
The REAL TIME GRAPHS are displayed by 18.104.22.168 ANALYZER:
plotting the parameters of the machines being
monitored along with the parameters provided by The ANALYZER analyzes the complete
the OPENSTA machines such as virtual users monitoring session to come out with the results of
and response time. the performance measures and generate a report.
It provides us with an ‘integrated monitoring The saved session dump is loaded back to the
console’ (at CC) to view correlation of different database and graphs for the monitored machines
parameters of different machines are displayed. Along with the features as in Real
time graphs, we have also incorporated the
The key features of real time graphs are: following additional features:
Add Graph: displays the real time variation of a
single parameter from any of the machines being Show Data: displays the accurate value of the
monitored in the session. parameters at any instant of time (also has the
Add Parameter: Overlays the parameter to see option to select the RANGE OF VALUES).
the relative variation and correlation of two or Exporting Data to Excel: exports the data directly
more parameter values with time. to the excel sheet.
Remove Parameter: Removes the display of Making User Defined Graphs: provides an option
any undesired parameter from the running to see a particular range of data at any instant of
graphs. time (by specifying the RANGE as well as
Delete Graph: Deletes a graph from the graph INTERVAL for the data on the X-axis).
window. Report Generation: prepares a report and exports
Changing Legend Properties: Changes the the graphs to Microsoft word.
legend properties (color, width, style, scale). The parameters on a graph have been scaled with
respect to the maximum valued parameter. The
The graphs are scaled on a scale of 100, and data is also interpolated before being plotted on the
the data is interpolated for missing values. graph surface.
Figure 22.214.171.124: The UI with the real time graphs Figure 126.96.36.199: The Analyzer to analyze a saved session
188.8.131.52 SAVED PROFILER:
After the completion of every monitoring 3.2 BANDWIDTH SIMULATION MODEL
session, the user can save the current session. Till now open source load testing tools do the
A dump of the current session in .sql format is monitoring without considering the bandwidth
then created at user specified location. availability and latency at the client’s side. The
If at any time, user has to create a monitoring performance of applications largely depends on the
session with the same configurations, then availability of bandwidth and latency at client’s
instead of making an entirely new session and side. For instance, different clients might be
configuring it again, the user can avail this working at different bandwidths (33kbps, 56 kbps,
1Mbps so on…). The server might be sending 3.3 IP SPOOFING
data at a particular bandwidth say 100Mbps, and A major problem for load testing team is that any
might have a response time of sending a page genuine attempt to load the server with web-
equal to, say, 26 seconds. However, due to requests can be perceived as DOS attack by
different availability of bandwidth the time taken firewall and blocked thus failing to load the server.
in displaying the page will be different for To bypass this problem, load generation tool
different clients. should be able to send requests spoofing their
Considering this fact, it would be necessary for source IPs so that these requests can bypass
real-time to simulate a network bandwidth defense mechanism and can actually reach server.
against every task-group in OPENSTA. For Open source load testing tools such as OpenSTA
network simulation we have readily available do not have an inbuilt mechanism to achieve IP
tool such as WAN Emulator tool . spoofing.
To achieve this real time monitoring on network
performance measures we need to map the load
generated from OPENSTA to the network traffic
on WAN emulator. For this we can design an
interface to be placed between OPENSTA and
WAN emulator that would call OPENSTA APIs
to start the load test and simultaneously call the
APIs of WAN Emulator to generate appropriate
traffic so that required bandwidth on the network
can be achieved.
Figure 3.3: IP Packet Header
4. COMPARATIVE STUDY
The chart below shows comparative study of our
tool with the leading commercial tool, LOAD
RUNNER and the most popular open source tool,
Figure 3.2: The WAN emulation model
Item LoadRunner OpenSTA Our Tool
HTTP/HTTPS Protocols √ √ √
Other Protocols √ × ×
Commercial Tool √ × ×
Monitoring on Windows based servers √ √ √
Monitoring on other platform(Linux) based servers √ × √
Profiler (i.e. reconfiguring previous sessions) √ × √
Analysis of simple charts and graphs √ √ √
Overlaying graphs capability (Sophisticated & large range of √ × √
charts and graphs with overlay facilities)
IP Spoofing √ × √
WAN/LAN emulation √ × √
Effective Integrated Monitoring and analysis capability √ × √
Extensibility × √ √
(the user can modify the code according to his needs)
Borland Silk Performer
5. CONCLUSION: www.borland.com/us/products/silk/silkperformer/
The approach mentioned in this paper, is a step index.html
towards enhancing the existing Open-source load
testing technologies, which can be used by small  hp invent
industries which cannot afford high-priced Mercury LoadRunner
commercial tools. This Integrated Monitoring Tool www.mercury.com/us/products/loadrunner/
can help draw better analysis than the existing open
source tools, as it provides the user with some of the  IBM
key features found in the commercial tools only. Our Rational Performance Tester
tool helps to maximize the Return on Investment http://www-
(ROI), resulting in more satisfied users, increased 306.ibm.com/software/awdtools/tester/performance
availability of business opportunities and reducing /index.html
risk of losing valuable customers.
However, the various demerits of OPENSTA at load  hp invent
generation and script recording level have not been http://download.mercury.com/cgi-
combated. Enhancements can be made by bin/portal/download/index.jsp
implementing protocols other than HTTP/ HTTPs.
REFERENCES: Free WAN Emulator
OpenSTA, the free performance testing tool, versus  SecurityFocus
the big-guns IP Spoofing: an introduction
 ObjectWeb  CresTech
An Overview Of Load Test Tools Performance Engineering For Web Applications
Features of Rational Robot  The Apache Jakarta Project
www.ibm.com/software/awdtools/tester/robot/feature Apache JMeter
 OpenSTA Developer Home