Software Performance Engineering _SPE_

							Software Performance
 Engineering (SPE)
      CMPT 318
       Fall 2006
      SFU Surrey
               Introduction

• Software must be designed before it is
  implemented
• So far, the performance techniques we’ve
  seen can only be applied to implemented
  software
   E.g. timing and profiling only work on
    implemented software
                  Introduction
• Many applications have performance goals
     Websites
     Embedded systems
     Real-time software
• Waiting until the software is implemented could
    be too late to do anything about performance
•   Thus, for some projects it is important that
    performance be built-in from the start
    Examples of Performance
           Failures
• The NASA Flight Operations Satellite was
  delayed 8-months due to slow scheduling
  software
• A live Victoria’s Secret webcast was so
  popular (due to Super Bowl advertising)
  that 5% of viewers couldn’t see the
  webcast
    Examples of Performance
           Failures
• A Fortune 100 company tried to create a
 new distributed order management system
 based on a number of legacy systems, but
 found that there were performance
 problems that no amount of extra hardware
 or system tuning could overcome
   Why Performance Problems
             Occur
• No consideration given to performance in
  requirements/design phase
    “Just buy faster hardware and more computers.”
• Knee-jerk “fix-it later” mentality
    “Premature optimization is the root of all evil.” --
     Donald Knuth
    They key word is “premature” --- the quote does not
     discourage optimization, just premature optimization
    In performance critical systems, optimization should
     be taken into consideration from the start
      Three Performance Myths

1. It is not possible to do anything about
      performance until you have something to
      execute.
  •     Performance models can be used before a
        system is implemented to estimate its
        performance and find bottlenecks
      Three Performance Myths

2. Managing performance takes too much time.
  •   Only performance-critical system need to do
      extensive performance management.
  •   If the risk of project failure due to low performance is
      high, then only then is performance management
      needed.
       Three Performance Myths
3. Performance models are complex and
      expensive to construct.
  •     Simple and inexpensive performance models can be
        constructed.
  •     Not necessary to create a detailed performance
        simulation.
        Performance Issues

• The approach we will look at is called
  Software Performance Engineering (SPE)
• SPE is concerned with two major
  performance issues
   Responsiveness/throughput
   Scaling
           Responsiveness

• Response time is the time required to
 respond to a request
   Could be a single transaction
   Or the time to complete a longer task
• Throughput of the system
• Typically defined with respect to user
           Responsiveness
• Objective response time
• Subjective response time
   Does the user feel that the system is
    responsive?
    • E.g. outdoor ATM user in Hawaii may tolerate a 30-
      second response time, but not an outdoor ATM
      user in winter in Toronto
    • E.g. putting input forms on-screen first may make
      users feel the system is more responsive, even if
      the true response time is unchanged
                     Scalability

• Scalability is how the system performs
 when demand for it increases
     Response Time




                                   knee


                      Throughput
                Scalability

• Before the knee: linear scaling
• After the knee: exponential scaling
• If the knee is before your target load
  requirements, you have a performance
  problem
             SPE Process

• SPE is a methodology
• Thus it comes with various rules,
 instructions, and patterns for users of it to
 follow
   Plus tools and advice you can purchase
    SPE Modeling Strategies

• SPE supports three kinds of models
   Simple
   Best- and Worst- Case
   Adapt to precision
             Simple Model

• Start with the simplest possible model that
  identifies performance problems
• Good for rapid feedback in early design
  phases
      Best- and Worst- Case

• Use best- and worst-case estimates of
  resource requirements to establish bounds
  on expected performance
• Useful when there is much uncertainty
• If worst-case estimates are satisfactory,
  then you can move on
         Adapt to Precision

• Match model detail to knowledge of
  software processing
• As a project progresses, you can add more
  detail to your performance models
                SPE Process
1.   Assess performance risk
2.   Identify critical use cases
3.   Select key performance scenarios
4.   Establish performance objectives
5.   Construct performance model(s)
6.   Add software/hardware resource requirements
7.   Evaluate model
8.   If the results are feasible, modify the software,
     re-create scenarios, and goto step 5;
     otherwise, revise performance objectives
     Example: ATM Machine
• Suppose you are creating the software for
  an ATM machine
• The ATM accepts a bank card, requests a
  PIN, and the lets the user withdraw cash,
  deposit cash, or see their balance; the
  customer may perform multiple
  transactions per session.
• The ATM communicates with a host
  computer to verify the PIN and transaction
     Example: ATM Machine

• Step 1: Assess performance risk
• The risk is small since only one customer
  uses it at a time
(the performance risk of the host computer is
  much higher; but we are not worrying
  about the host here)
      Example: ATM Machine

• Step 2: Identify critical use cases
• Three uses cases
   Operator transaction (e.g. change currency
    file)
   Command function (e.g. reset ATM)
   User transaction
• Clearly, the user transaction is the critical
  one for performance
      Example: ATM Machine
• Step 3: select key performance
  scenarios
• An error-free customer transaction is the
  key scenario
• In a single transaction, the user could
  perform multiple actions
• Need to get (or estimate) probabilities that
  various actions will be performed
     Example: ATM Machine
• Step 4: establish performance
  objectives
• How long is a reasonable ATM session?
• SPE prefers to consider the entire end-to-
  end transaction, as opposed to the
  response time of individual functions
• Lets say that 30s is a reasonable time
Example: ATM Machine

      getCardInfo

        getPIN
                    UML activity
             n        graph


        process
      transaction


       terminate
      transaction
  Example: ATM Machine

getTransaction
                              UML activity
                  process       graph
                 withdrawal

                  process
                  deposit

                  process
                  balance
        Example: ATM Machine
• Step 5: hardware/software requirements
• Software resources include
     Number of screens
     Number of host interactions
     Number of log entries
     Delays due to things like the cash dispenser and
      receipt printer
• For each process on the execution graph,
  required amount of each of these resources is
  listed
         Example: ATM Machine
• Hardware resources includes
      CPUs
      Disks
      Displays
      Network cards
• Now we can create a software-by-hardware matrix, and
  estimate the time each part of a transaction will take
    This can then be used to “solve” the model, i.e. determine if the
     model predicts high-enough performance
    This particular model is very simple --- but then the performance
     risk is small, and so doesn’t need an elaborate model