Traffic characterization and analysis by eon20304

VIEWS: 17 PAGES: 49

									    Traffic characterization and
              analysis

               Nicolas Larrieu, Philippe Owezarski

                                      LAAS-CNRS
                                    Toulouse, France
                                 {nlarrieu, owe}@laas.fr


JTR’04, Montpellier, France, 4-6 octobre 2004              1
  Outline

   Introduction to Internet monitoring
       Needs, principles and applications

   State of the art on active and passive technics

   Internet traffic characterisitics and related
    performance problems
       Traffic mutation: P2P impact
       Treats for the Internet: oscillations, LRD…

   How to use monitoring results?
       Application to traffic engineering: a new congestion
        control mechanism




JTR’04, Montpellier, France, 4-6 octobre 2004                  2
  Part 1

   Introduction to Internet monitoring
       Needs, principles and applications

   State of the art on active and passive technics

   Internet traffic characterisitics and related
    performance problems
       Traffic mutation: P2P impact
       Treats for the Internet: oscillations, LRD…

   How to use monitoring results?
       Application to traffic engineering: a new congestion
        control mechanism




JTR’04, Montpellier, France, 4-6 octobre 2004                  3
  Introduction

 Deals  with both monitoring results and
    effects on network design, research and
    management

 Topic            under the spotlight

 Definition:                  « science of measurements »

 To   apply to networks in general, to the
    Internet in particular


JTR’04, Montpellier, France, 4-6 octobre 2004                4
  Monitoring and networks…

 Explosionof the Internet (in size)
 New services
 QoS
…


 Increasingcomplexity of the Internet
 No more control of the global Internet




JTR’04, Montpellier, France, 4-6 octobre 2004   5
  Monitoring and networks…

«       Defeat » statement
      For instance on QoS aspects
 Bad          knowledge of the traffic

 Because  of the current Internet
  complexity, its evolution requires a good
  knowledge of the « real » traffic
 « Real » traffic exists and is full of
  information


JTR’04, Montpellier, France, 4-6 octobre 2004   6
      Example of a provisioning problem




JTR’04, Montpellier, France, 4-6 octobre 2004   7
 What to use for network monitoring?

 Administration                          / operation tools based on
    SNMP
      Topology of networks / configuration
      Some statistics measurements
        • Granularity is too coarse: min = 5 s (but can
          be 1 hour, 1 day, 1 week or whatever)
        • Measured parameters are more or less the
          amount of traffic sent and received




JTR’04, Montpellier, France, 4-6 octobre 2004                          8
Some examples of SNMP results (1)


                  RAP  RENATER interconnection




                                       Per hour trace

                                                        Input traffic
                                                        Output traffic



JTR’04, Montpellier, France, 4-6 octobre 2004                           9
Some examples of SNMP results (2)




                                     Per Month trace




                                      Per Week trace
                                                       Input traffic
                                                       Output traffic


JTR’04, Montpellier, France, 4-6 octobre 2004                          10
 Problems for monitoring networks

 Impossible  to monitor traffic dynamics
  (second order values as variability auto-
  covariance for instance)
 Impossible to monitor traffic QoS (user
  point of view – goodput)
 Impossible to get a (formal) traffic model




JTR’04, Montpellier, France, 4-6 octobre 2004   11
 Example on network provisioning

 Common                 beliefs tell us traffic is Poisson:
      E[X]=l
      V[X]=l
      Provisioning should be 2l
 Actually,                provisioning has to be at least 1:3
    (i.e. 3l)
      RENATER 1:3
      Sprint 1:3
      WorldCom 1:5
      AT&T 1:10


JTR’04, Montpellier, France, 4-6 octobre 2004                    12
 Questions on the example

 How  explaining this over-provisioning
  requirement ?
 How to predict the traffic that will be
  supported by a new network to design ?




                                                ?
JTR’04, Montpellier, France, 4-6 octobre 2004       13
  Part 2

   Introduction to Internet monitoring
       Needs, principles and applications

   State of the art on active and passive technics

   Internet traffic characterisitics and related
    performance problems
       Traffic mutation: P2P impact
       Treats for the Internet: oscillations, LRD…

   How to use monitoring results?
       Application to traffic engineering: a new congestion
        control mechanism




JTR’04, Montpellier, France, 4-6 octobre 2004                  14
  Active measurements

 Principles
      Consists in sending packets on a network and
       observing results (Delay, RTT, Throughput,
       etc.)
      User point of view
      Best solution to evaluate the service you can
       get from the network you’re connected to
 Drawbacks
      Probe packets change the state of the network
        IETF IPPM WG is working on the definition
         of probing scenarios minimizing the effects
         on the network state


JTR’04, Montpellier, France, 4-6 octobre 2004          15
  Some active measurement tools

 Ping
 Traceroute
 MGEN
 RIPE            equipments
 Etc.


 Importance    of clock synchronization: most
    of the time GPS is required



JTR’04, Montpellier, France, 4-6 octobre 2004    16
  Passive measurements

 Principles
      Capture packets (or headers)
      Not intrusive at all
      Carrier / ISP point of view
      Best solution for a carrier to measure traffic
 Drawbacks
      Sampling issues
      Difficult to get a user point of view
      Technical limits (speed of components,
       capacity)


JTR’04, Montpellier, France, 4-6 octobre 2004           17
  On line vs. Off line measurements

 On         line
      Packets are analyzed in real-time
      Analysis on very long periods
      But complexity of analysis is quite limited
 Off          line
      Packets are stored on hard drives / SAN for
       later analysis
      Possibilities of analysis are endless
      Possibility of correlating several traces
      But amount of stored data is really huge (small
       periods only)


JTR’04, Montpellier, France, 4-6 octobre 2004            18
  Some passive measurement tools

 TSTAT
 LIBCAP
 Tcpdump
 Tcptrace
 QoSMOS
 IPANEMA
 CISCO’sNetflow
 DAG cards
 Etc.


JTR’04, Montpellier, France, 4-6 octobre 2004   19
  Part 3

   Introduction to Internet monitoring
       Needs, principles and applications

   State of the art on active and passive technics

   Internet traffic characterisitics and related
    performance problems
       Traffic mutation: P2P impact
       Treats for the Internet: oscillations, LRD…

   How to use monitoring results?
       Application to traffic engineering: a new congestion
        control mechanism




JTR’04, Montpellier, France, 4-6 octobre 2004                  20
  Current Internet traffic

 New    applications with various and changing
    requirements in terms of QoS appear

 New   P2P applications make traffic
    characteristics change




JTR’04, Montpellier, France, 4-6 octobre 2004     21
                        Internet traffic evolution (May 2000)

                                           Main TCP applications throughputs (SPRINT)
                       30000


                       25000
                                                                                                                 Other
                                                                                                                 Telnet
Throughput (kbits/s)




                       20000
                                                                                                                 RealAudio
                                                                                                                 MediaPlayer
                       15000
                                                                                                                 Quake
                                                                                                                 NNTP
                       10000                                                                                     SMTP
                                                                                                                 FTP
                        5000                                                                                     HTTPS
                                                                                                                 HTTP
                           0
                           10:00 11:00 12:00 13:00 14:00 15:00 16:00 17:00 18:00 19:00 20:00 21:00 22:00 23:00
                                                                    Time




                  JTR’04, Montpellier, France, 4-6 octobre 2004                                                                22
                         Internet traffic evolution                                    (August 2000)



                                                Main TCP applications throughputs (SPRINT)
                       40000
                       35000
                                                                                                 Other
Throughput (kbits/s)




                       30000                                                                     Telnet
                       25000                                                                     RealAudio
                                                                                                 MediaPlayer
                       20000
                                                                                                 Quake
                       15000                                                                     Napster
                       10000                                                                     NNTP
                                                                                                 SMTP
                        5000
                                                                                                 FTP
                             0                                                                   HTTPS
                                                                                                 HTTP
                        11 00
                        12 00
                        13 00
                        14 00
                        15 00
                        16 00
                        17 00
                        18 00
                        19 00
                        20 00
                        21 00
                        22 00
                        23 00
                         0: 0
                         1: 0
                         2: 0
                         3: 0
                         4: 0
                         5: 0
                         6: 0
                         7: 0
                         8: 0
                         9: 0
                            00
                            0
                            0
                            0
                            0
                            0
                            0
                            0
                            0
                            0
                          :0
                          :
                          :
                          :
                          :
                          :
                          :
                          :
                          :
                          :
                          :
                          :
                          :
                          :
                        10




                                                                       Time


                       JTR’04, Montpellier, France, 4-6 octobre 2004                                           23
                       Internet traffic evolution (May 2003)

    Main TCP applications throughputs (Renater)
Throughput (kbits/s)




                                                    Time



          JTR’04, Montpellier, France, 4-6 octobre 2004        24
               TCP flow size

               90


               80


               70


               60
Percentage




               50


               40


               30


               20


               10


                0
                                 0-10                        10-100           >100

                                                 Number of packets per flow

             JTR’04, Montpellier, France, 4-6 octobre 2004                           25
               TCP flow size vs. total bandwidth

               90

                                      « Mice vs. elephants »
               80                                                              flows
               70                                                              bytes

               60
Percentage




               50


               40


               30


               20


               10


                0
                                 0-10                        10-100           >100

                                                 Number of packets per flow

             JTR’04, Montpellier, France, 4-6 octobre 2004                             26
  Impact of P2P on traffic

 Thousands  of mice
 A large number of elephants


 Change flow size distribution




JTR’04, Montpellier, France, 4-6 octobre 2004   27
      Flow size distribution

%

                                                         Exponential
100                                                      August 2000
 90                                                      May 2003
 80
 70
 60
 50
 40
 30
 20
 10

                      20                 40         60   80          100
                                                              Nb of packets / flow
    JTR’04, Montpellier, France, 4-6 octobre 2004                                    28
  Traffic oscillation issues




JTR’04, Montpellier, France, 4-6 octobre 2004   29
  Illustration: LRD and losses




JTR’04, Montpellier, France, 4-6 octobre 2004   30
  Illustration: LRD and losses




JTR’04, Montpellier, France, 4-6 octobre 2004   31
  Illustration: LRD and losses




JTR’04, Montpellier, France, 4-6 octobre 2004   32
  Illustration: LRD and losses




JTR’04, Montpellier, France, 4-6 octobre 2004   33
  Illustration: LRD and losses




JTR’04, Montpellier, France, 4-6 octobre 2004   34
  Wavelett analysis of the traffic
  Range




                                                Time


JTR’04, Montpellier, France, 4-6 octobre 2004          35
  Log-log scale variance diagram


                                                2   different power laws

                                                2  different rating for
                                                scale invariance

                                                High    scales
                                                     H › 0.5
                                                     LRD characteristics




JTR’04, Montpellier, France, 4-6 octobre 2004                               36
  Oscillations persistence characterization




                                                H = 0.741




JTR’04, Montpellier, France, 4-6 octobre 2004               37
  Traffic characteristics bibliography

 Actual   Internet traffic has self-similarity
    and long range dependancy properties

 Due          to:
      Heavy-tailed distribution of flow size
      TCP-like congestion control mechanism
      Routers
      Human and application behaviors




JTR’04, Montpellier, France, 4-6 octobre 2004     38
  Self-similarity




JTR’04, Montpellier, France, 4-6 octobre 2004   39
  Illustration of self-similarity

 Worst              case with a highly self-similar
    traffic
      E[X] = l
      V[X]  O(l2)

 Example:
      If E[X] = 100 Mbps  Provisioning for a
       reliable network with low delays is in the order
       of 10 Gbps
      Reach the technological limits of networks with
       an average traffic of few hundreds of Mbps



JTR’04, Montpellier, France, 4-6 octobre 2004             40
  Conclusion on traffic evolution

 Actual             Internet traffic is not Poisson

 Well,   it is probably not self-similar at all
    scales?

 But   it is really LRD (and this is a major
    issue!)




JTR’04, Montpellier, France, 4-6 octobre 2004          41
  Part 4

   Introduction to Internet monitoring
       Needs, principles and applications

   State of the art on active and passive technics

   Internet traffic characterisitics and related
    performance problems
       Traffic mutation: P2P impact
       Treats for the Internet: oscillations, LRD…

   How to use monitoring results?
       Application to traffic engineering: a new congestion
        control mechanism




JTR’04, Montpellier, France, 4-6 octobre 2004                  42
  Measurement Based…

   Monitoring/measurements become the corner
    stone of research in networking
   Many topics are addressed:
       New protocols and architectures for:
         • Traffic characterization and modeling
         • Multi-domains QoS guaranty
         • Service and network utilization optimization
         • Network or VPN or CoS provisioning
         • QoS routing
         • Network security, etc.
       Techniques and mechanisms for:
         • Pricing, etc.



JTR’04, Montpellier, France, 4-6 octobre 2004             43
     Multiple causes for Internet oscillations


1.     TCP like congestion control mechanisms
       (Slow Start and Congestion Avoidance mechanisms / Closed
       control loop)
2. Increase of transmitted file size
3. Increase of network capacities (and over-
   provisioning)

 Increase of oscillations
       Amplitude                                 traffic oscillations limit
       Range                                      network performance
                                                « High variability » paper of
                                                  Willinger (IEEE ToN 96)
JTR’04, Montpellier, France, 4-6 octobre 2004                                   44
  Smoothing flow behavior

Disturbances are mainly due to elephants
 We need to increase elephant flows
  regularity

 We are going to use the TFRC mechanism
 to transmit elephant flows

 TFRC    proposes a smooth sending rate with
    very soft increases and decreases


JTR’04, Montpellier, France, 4-6 octobre 2004   45
       TFRC impact on flow QoS: throughput analysis




              Protocol                        Average troughput (kB)   Throughput σ (kB)
TCP New Reno (NR): real case                         82.335                157.959
TCP NR & TFRC: simulated case                        77.707                102.176


     JTR’04, Montpellier, France, 4-6 octobre 2004                                         46
  TFRC impact on flow QoS: LRD analysis

           Real traffic                         Simulated traffic




JTR’04, Montpellier, France, 4-6 octobre 2004                       47
  Conclusion

 Monitoring   give us very important
    information on traffic characteristics and
    related performance problems

 Traffic  engineering can be really improve
    with monitoring results

   Lots of network research fields can be
    investigated and improved
          QoS, security, pricing, networking…



JTR’04, Montpellier, France, 4-6 octobre 2004    48
                   Thank you for your attention

                                    Any questions?

                        Contact: nlarrieu@laas.fr




JTR’04, Montpellier, France, 4-6 octobre 2004        49

								
To top