Extending Open Source solution for Performance testing of Web by yaofenjin


									     Extending Open Source solution for Performance testing of Web (http\https)
           (Nupur Shrivastava, Nitika Aggarwal, Sakshi Gupta, Happy Himanshu Gupta)
                    (Under the Supervision of Mr. Vikas Saxena, Sr. Lecturer)
                           Jaypee Institute of Information Technology
                                           Noida, India

 In this research paper we examine the need for load testing and highlight the shortcomings of open source
load testing solutions, vis-à-vis their commercial counterparts and present our own approach to design
plug-ins over open source tools that can bring them a step closer to COTS. Our proposed solution enhances
the capabilities of Open Source testing tools by providing an in-depth performance monitoring of web,
application and database servers. This helps in determining bottleneck of the application and enables the
user to proactively correct potential problems before they become critical. We also propose here, the
implementation of IP spoofing and bandwidth simulation, which would be the add-on features of this tool.

KEYWORDS: performance, monitoring, load testing, analysis, OpenSTA, WAN emulation,
bottleneck, Analyzer, Profiler, Graphs, Overlays, Load Runner

           1. INTRODUCTION                                 2. EXISTING TECHNOLOGIES
For any web application, the end to end response
time for a user is a critical issue. As the number     2.1 CURRENT STATE OF ART
of users accessing the same web application
increases, the performance of the application          2.1.1 Commercial tools
decreases due to slowdown [7]. Load testing            The most popular commercial tools available for
refers to checking the performance of a system         Load testing and diagnostics are
under multi-users workload. It is a very critical           a) Mercury LoadRunner
part for any web application before it goes live.           b) IBM Rational Performance tester
There are many open source load testing                     c) IBM Rational Robot
solutions available today, but they do not match            d) Borland Silk Performer
up completely to their commercial counterparts.
Commercial tools on the other hand prove to be         2.1.2 Popular Open source Testing Solutions
very expensive option for small scale industries.      For the purpose of this research paper, we list two
The challenges of successfully conducting an           of the most popular tools for performance testing
online business are as follows:                        of HTTP\HTTPS applications
 1) high user volumes                                       a) Apache JMeter
 2) low response times for web requests                     b) OpenSTA
 3) the overall reliability of the service

Monitoring the performance requires an efficient       2.2 SHORTCOMINGS
analysis of the various parameters associated          Open source tools lag from their commercial
with performance measures. Our major concern           counterparts in the following
lies with the determination of the bottleneck in
the complete n-tier application that comprises of          a) Effective Integrated Monitoring and
a client, a network and different servers involved            analysis capability
(web, application, database). Thus, a detailed             b) User network speed simulation
analysis for every performance parameter from              c) IP Spoofing capability
all the components involved under Load
condition is required for diagnosing the complete
application being monitored.

3. PROPOSED APPROACH OF THE                         the real time variations in the performance
SOLUTION:                                           parameters and locate the areas that have been the
                                                    cause for erroneous or unusual results.
                                                    The key requirements for the DASHBOARD are:
3.1EFFECTIVE INTEGRATED                                 a) Display of simultaneous graphs with
MONITORING MODEL                                             feature of graph overlays.
                                                        b) Maintenance of a PROFILER which
                                                             allows user to reconfigure the saved
                                                             sessions. Informing user of any
                                                             abnormality in the system and giving him
                                                             the option to execute necessary measure.
                                                        c) In real environment, the values lying out
                                                             of bounds are taken care off by the
                                                             ANALYZER, which then diagnoses the
                                                             fed data (input) and tries to point out the
                                                             cause of the anomaly. We will have a
                                                             facility at the Dashboard console that can
                                                             send system alerts through emails etc.

                                                    Since the LC(s) have to be deployed on different
                                                    platforms, we decided to use a platform
                                                    independent language like PERL for the designing
                                                    of Listening components as well as Central
                                                    Component. For communication and transfer of
Figure3.1.1: Integrated Monitoring Model
                                                    data values between the CC and LC, we have used
                                                    Socket Programming in Perl. For designing of the
                                                    Graphical User Interface and the Dashboard, we
First, OPENSTA tool is used to generate virtual     have used VB.net.
user load on the machines (servers) to be
monitored. For the collection of the performance    3.1.1 DASHBOARD
parameters on different server machines like IIS,
SQL, Weblogic, ORACLE etc., we design      USER INTERFACE:
lightweight listening components (LC) and
deploy them on different Servers (Web,              The User Interface (UI), designed using
Application, Database) with different OS            VB.NET, provides a console on which the user
environments.                                       could design and set a session for monitoring and
                                                    can see the results in form of REAL TIME
The key requirements of the LC(s) are:              GRAPHS. These real time graphs are obtained
    a) They should be lightweight and should        corresponding to each machine participating in the
        not consume large memory.                   monitoring session as well as from the machines
    b) They should continuously gather              which are used for generating the load using
        Configurable          Data        Points    OPENSTA. The UI runs the scripts placed at
        (parameters) on the server machine by       backend prepared using PERL scripting language.
        polling the logs or using the APIs of the
        server and the OS at a configurable         Following are the steps of interaction between CC
        polling interval and relay it back to a     and LC:
        central request component (CC).             1. The CC broadcasts START request, and the
    c) They should run in daemon mode.              LC(s) respond back by sending their system
The central request component (CC) which acts       2. The CC requests the selected LC(s) to send their
as a Performance Data Collector Server              list of parameters.
connects to the installed LC(s), the network and    3. The CC sends the configured list of parameters
the load generator tool (OPENSTA), and stores       to be monitored to each LC, including the
the data for every synchronized pulse into a        configured test         to be run on OPENSTA
database. Dashboard, a term which defines the       machines.
console at CC side, keeps on fetching the data      4. CC broadcast a “START MONITORING”
from the database and helps the user to visualize   message to all selected LC(s) at the same instant.

The user can end the session whenever he wants      facility of saved profiler, by recreating a new
to by sending a “STOP” broadcast.                   session with modifications to an old session.

The REAL TIME GRAPHS are displayed by      ANALYZER:
plotting the parameters of the machines being
monitored along with the parameters provided by     The ANALYZER analyzes the complete
the OPENSTA machines such as virtual users          monitoring session to come out with the results of
and response time.                                  the performance measures and generate a report.
It provides us with an ‘integrated monitoring       The saved session dump is loaded back to the
console’ (at CC) to view correlation of different   database and graphs for the monitored machines
parameters of different machines                    are displayed. Along with the features as in Real
                                                    time graphs, we have also incorporated the
The key features of real time graphs are:           following additional features:
Add Graph: displays the real time variation of a
single parameter from any of the machines being     Show Data: displays the accurate value of the
monitored in the session.                           parameters at any instant of time (also has the
Add Parameter: Overlays the parameter to see        option to select the RANGE OF VALUES).
the relative variation and correlation of two or    Exporting Data to Excel: exports the data directly
more parameter values with time.                    to the excel sheet.
Remove Parameter: Removes the display of            Making User Defined Graphs: provides an option
any undesired parameter from the running            to see a particular range of data at any instant of
graphs.                                             time (by specifying the RANGE as well as
Delete Graph: Deletes a graph from the graph        INTERVAL for the data on the X-axis).
window.                                             Report Generation: prepares a report and exports
Changing Legend Properties: Changes the             the graphs to Microsoft word.
legend properties (color, width, style, scale).     The parameters on a graph have been scaled with
                                                    respect to the maximum valued parameter. The
The graphs are scaled on a scale of 100, and        data is also interpolated before being plotted on the
the data is interpolated for missing values.        graph surface.

Figure The UI with the real time graphs    Figure The Analyzer to analyze a saved session SAVED PROFILER:
After the completion of every monitoring            3.2 BANDWIDTH SIMULATION MODEL
session, the user can save the current session.     Till now open source load testing tools do the
A dump of the current session in .sql format is     monitoring without considering the bandwidth
then created at user specified location.            availability and latency at the client’s side. The
If at any time, user has to create a monitoring     performance of applications largely depends on the
session with the same configurations, then          availability of bandwidth and latency at client’s
instead of making an entirely new session and       side. For instance, different clients might be
configuring it again, the user can avail this       working at different bandwidths (33kbps, 56 kbps,

       1Mbps so on…). The server might be sending            3.3 IP SPOOFING
       data at a particular bandwidth say 100Mbps, and      A major problem for load testing team is that any
       might have a response time of sending a page         genuine attempt to load the server with web-
       equal to, say, 26 seconds. However, due to           requests can be perceived as DOS attack by
       different availability of bandwidth the time taken   firewall and blocked thus failing to load the server.
       in displaying the page will be different for         To bypass this problem, load generation tool
       different clients.                                   should be able to send requests spoofing their
       Considering this fact, it would be necessary for     source IPs so that these requests can bypass
       real-time to simulate a network bandwidth            defense mechanism and can actually reach server.
       against every task-group in OPENSTA. For             Open source load testing tools such as OpenSTA
       network simulation we have readily available         do not have an inbuilt mechanism to achieve IP
       tool such as WAN Emulator tool [8].                  spoofing.
       To achieve this real time monitoring on network
       performance measures we need to map the load
       generated from OPENSTA to the network traffic
       on WAN emulator. For this we can design an
       interface to be placed between OPENSTA and
       WAN emulator that would call OPENSTA APIs
       to start the load test and simultaneously call the
       APIs of WAN Emulator to generate appropriate
       traffic so that required bandwidth on the network
       can be achieved.

                                                            Figure 3.3: IP Packet Header

                                                            4. COMPARATIVE STUDY
                                                            The chart below shows comparative study of our
                                                            tool with the leading commercial tool, LOAD
                                                            RUNNER and the most popular open source tool,

       Figure 3.2: The WAN emulation model

                              Item                             LoadRunner           OpenSTA      Our Tool
                  HTTP/HTTPS Protocols                                 √                   √          √
                        Other Protocols                                √                   ×          ×
                       Commercial Tool                                 √                   ×          ×
           Monitoring on Windows based servers                         √                   √          √

    Monitoring on other platform(Linux) based servers                  √                   ×          √

        Profiler (i.e. reconfiguring previous sessions)                √                   ×          √
            Analysis of simple charts and graphs                       √                   √          √

Overlaying graphs capability (Sophisticated & large range of           √                   ×          √
         charts and graphs with overlay facilities)

                          IP Spoofing                                  √                   ×          √

                 WAN/LAN emulation                                 √               ×            √
Effective Integrated Monitoring and analysis capability            √               ×            √

                     Extensibility                                 ×               √            √
 (the user can modify the code according to his needs)

                                                            [4] Borland
                                                            Borland Silk Performer
     5. CONCLUSION:                                         www.borland.com/us/products/silk/silkperformer/
     The approach mentioned in this paper, is a step index.html
     towards enhancing the existing Open-source load
     testing technologies, which can be used by small [5] hp invent
     industries which cannot afford high-priced Mercury LoadRunner
     commercial tools. This Integrated Monitoring Tool www.mercury.com/us/products/loadrunner/
     can help draw better analysis than the existing open
     source tools, as it provides the user with some of the [6] IBM
     key features found in the commercial tools only. Our Rational Performance Tester
     tool helps to maximize the Return on Investment http://www-
     (ROI), resulting in more satisfied users, increased 306.ibm.com/software/awdtools/tester/performance
     availability of business opportunities and reducing /index.html
     risk of losing valuable customers.
     However, the various demerits of OPENSTA at load [7] hp invent
     generation and script recording level have not been http://download.mercury.com/cgi-
     combated. Enhancements can be made by bin/portal/download/index.jsp
     implementing protocols other than HTTP/ HTTPs.
                                                          [8] MyLoadTest
     REFERENCES:                                          Free WAN Emulator
     OpenSTA, the free performance testing tool, versus   [9] SecurityFocus
     the big-guns                                         IP Spoofing: an introduction
     http://www.testingreflections.com/node/view/361       http://www.securityfocus.com/infocus/1674

     [2] ObjectWeb                                        [10] CresTech
     An Overview Of Load Test Tools                       Performance Engineering For Web Applications
     http://clif.objectweb.org/load_tools_overview.pdf    http://www.crestech.in/performance_testingweb.ht
     [3] IBM
     Features of Rational Robot                           [11] The Apache Jakarta Project
     www.ibm.com/software/awdtools/tester/robot/feature   Apache JMeter
     s/index.html                                         http://jakarta.apache.org/jmeter/

                                                          [12] OpenSTA Developer Home


To top