Docstoc

multi-servers-load-direction-youtube

Document Sample
multi-servers-load-direction-youtube Powered By Docstoc
					CS435/535: Internet-Scale
      Applications
           Lecture 16

  Case Study: Akamai Streaming;
     YouTube; P2P Streaming

            3/4/2010
                                  1
Admin.
 Project team forming and proposal
  -   Form a team of 2 students
  -   One page proposal due: Friday, March 5

  -   Arrange a time to talk to the instructors before the
      break
       - Richard Yang’s Office hours:
           - Today 3-5 pm (AKW 307)
           - Friday 3-4 pm (AKW 307)

       - We should plan for weekly meetings after the break




                                                              2
Recap: Akamai

 Use DNS for redirection
    High-level DNS utilizes
     locations of requesting
     clients to direct to (close)
     low-level DNS
    Low-level DNS load balancing



 Load balancing considers load of servers and state
    Key requirements: load balancing; good hit ratio with
     limited memory (not wide-spread of content)
    Technique: Consistent hashing when assigning publishers
     (balls) to servers (bins); greedy assignment


                                                               3
Akamai Streaming Architecture

                                   A content publisher (e.g., a
                                   radio or a TV station) encodes
                                   streams and transfer them
                                   to entry points

                                  Group a set of streams (e.g.,
                                  some popular some not) into a
                                  bucket called a portset. A set of
                                  reflectors will distribute a given
                                  portset.

                                 When a user watches a stream
                                 from an edge server, the
                                 server subscribes to a reflector

Compare with Web architecture.
                                                                    4
  Akamai Streaming: Resource Naming

   Each unique stream is encoded by an URL
    called Akamai Resource Locator (ARL)
                                Customer#   Live
                                (NBA)       Media
                    Stream ID               service
          portset
Windows
media
player


  mms://a1897.l3072828839.c30728.g.lm.akamaistream.net
  /D/1897/30728/v0001/reflector:28839



                                                         5
Akamai Streaming Load Direction

 From ARL to edge server
  o   Similar to web direction


 From edge server to reflector
   o if (stream is active) then
        forward to client
     else if (VoD) then
        fetch from original server
     else
        using Akamai DNS to query
            portset+region code

                                     6
Streaming Redirection Interval




- 40% use 30 sec
- 10% does not have any redirection (default edge server
  cluster in Boston 72.246.103.0/24 and 72.247.145.0/24)

                                                           7
Overlapping of Servers




                         8
Testing Akamai Streaming Load Balancing




(a) Add 7 probing machines to the same edge server
(b) Observe slow down
(c) Notice that Akamai removed the edge server from DNS;
   probing machines stop
                                                           9
Outline
 Admin.
 Case studies
   o Akamai
   o YouTube




                 10
      http://video.google.com/videoplay?docid=-6304964351441328559#

You Tube
 02/2005: Founded by Chad Hurley, Steve
  Chen and Jawed Karim, who were all early
  employees of PayPal.
 10/2005: First round of funding ($11.5 M)
 03/2006: 30 M video views/day
 07/2006: 100 M video views/day
 11/2006: acquired by Google
 10/2009: Chad Hurley announced in a blog
  that YouTube serving well over 1 B video
  views/day (avg = 11,574 video views /sec )
                                                                      11
Pre-Google Team Size
 2 Sysadmins
 2 Scalability software architects
 2 feature developers
 2 network engineers
 1 DBA
 0 chefs




                                      12
YouTube Design Flow
 while (true)
 {
   identify_and_fix_bottlenecks();
   drink();
   sleep();
   notice_new_bottleneck();
 }




                                     13
YouTube Major Components
 Web servers


 Video servers


 Thumbnail servers


 Database servers
  o Will cover the social networking/database
    bottleneck/consistency issues later in the
    course

                                                 14
Outline
 Admin.
 Case studies
   o Akamai
   o YouTube
     • Web servers




                     15
YouTube: Web Servers
 Components                                       NetScaler
   Netscaler load balancer; Apache;
    Python App Servers; Databases
                                                    Apache
 Python
   Web code (CPU) is not               Web
     bottleneck
                                        servers     Python
       JIT to C to speedup
                                                   App Server
       C extensions
       Pre-generate HTML responses
   Development speed more
                                       Databases
     important




                                                                16
Outline
 Admin.
 Case studies
   o Akamai
   o YouTube
     • Web servers
     • Video servers




                       17
YouTube: Video Rates

                                                               YouTube
                                                               uses
                                                               Flash Video




See “Statistics and Social Network of YouTube Videos”, 2008.
                                                                         18
YouTube: Video Popularity




See “Statistics and Social Network of YouTube Videos”, 2008.
                                                               19
YouTube: Video Popularity



                                                      How to design
                                                      a system to handle
                                                      highly skewed
                                                      distribution?




See “Statistics and Social Network of YouTube Videos”, 2008.
                                                                       20
YouTube: Video Server Architecture

 Tiered architecture
  o   CDN servers (for popular videos)
       • Low delay; mostly in-memory operation
  o   YouTube servers (not popular 1-20 per day)
                                                    CDN
                                 Most popular
                                                   YouTube
                                                    Colo 1
                              Request

                                          Others
                                                   YouTube
                                                    Colo N
                                                             21
YouTube Redirection Architecture

  YouTube servers




                                   22
YouTube Video Servers

 Each video hosted by a mini-cluster
  consisting of multiple machines
 Video servers use the lighttpd web server
  for video transmission:
   Apache had too much overhead (used in the first few
    months and then dropped)

   Async io: uses epoll to wait on multiple fds


   Switched from single process to multiple process
    configuration to handle more connections



                                                          23
Thumbnail Servers
 Thumbnails are served by a few machines
 Problems running thumbnail servers
   o A high number of requests/sec as web pages
     can display 60 thumbnails on page
   o Serving a lot of small objects implies
      • lots of disk seeks and problems with file systems
        inode and page caches
      • may ran into per directory file limit

      • Solution: storage switched to Google BigTable (we will
        cover this later)


                                                                 24
Thumbnail Server Software
Architecture
 Design 1: Squid in front of Apache
  o   Problems
       • Squid worked for a while, but as load increased
         performance eventually decreased: Went from 300
         requests/second to 20
       • under high loads Apache performed badly, changed to
         lighttpd

 Design 2: lighttpd by default (By default
  lighttpd uses a single thread)
  o   Problem: often stalled due to I/O

 Design 3: switched to multiple processes
  contending on shared accept
  o   Problems: high contention overhead/individual
      caches
                                                               25
Thumbnails Server: lighttpd/aio




                                  26
Discussion: Scalability of Traditional
Content Distribution Architecture
                                             app.
                                             server


                                            C0



                    client 1



                      client 2
                                                      client n

                                 client 3

 Assume Akamai has 1Tbps, how many streams (@512 kbps)
  can it support?

                                                                 27
Major Objective of P2P


              Internet




  Share the resources (storage
  and bandwidth) of
  individual clients to improve
  scalability/robustness
                                  28
Peer-to-Peer Computing
- 40-70% of total traffic in many networks
- upset the music industry, drawn college students, web
  developers, recording artists and universities into court




                          Source: ipoque Internet study 2008/2009
A Key Question We Study
 How to utilize the capacities of
  participating clients?

 Although BitTorrent is the poster child for
  P2P, we will take live streaming as an
  example Internet application because it is
  more interesting

 Problem definition: how to stream a live
  event at rate R to all clients in a channel
  with a small number of servers (sources)
                                                30
Solution 1
 Build a chain
  o   server generates chunk 1 , sends to client 1
  o   Server sends chunk 2 to client 1; client 1 sends
      chunk 1 to client 2,
  o   …
        server




  Obvious problems?



                                                         31
Solution 2: Building a Relay Tree

                    server




      client 1




 Problem of building a tree?
                                    32
Ideal Scalability

 Assume                                     server
  o   need to
      achieve same
      rate to all                           C0
      clients
  o   only uplinks   client 1
                                 C1
                                                 Cn
      can be                      C2
      bottlenecks
                                       C3

 What is ideal
                     client 2
                                                      client n
  scalability                   client 3
  (upper bound)?

                                                                 33
The Scalability Problem

   Maximum                                 server

    throughput
                                           C0
    R = min{C0,
    (C0+Ci)/n}                 C1
                    client 1                    Cn
                                 C2
                                      C3
   The bound is    client 2
    theoretically                                    client n

    approachable               client 3



                                                                34
Theoretical Capacity:
upload is bottleneck
                                 R = min{C0, (C0+Ci)/n}
 Assume    c0 > (C0+Ci)/n                             c0

 Tree i:                           ci /(n-1)

  server  client i: ci /(n-1)
  client i  other n-1 clients      ci
                                                                   cn

                                                  c1         c2

                                                        C0
 Tree 0:
  server has remaining
                                          cm /n
                                                                   Cn
   cm = c0 – (c1 + c2 + … cn)/(n-1)      C1

  send to client i: cm/n                               C2     Ci
                                                                        35
What is our Status?
 From a chain to a tree to multiple trees
  o   An example multi-tree system is splitstream
 But multi-tree systems are not typically
  used in large real systems. Why?


                                - Clients come and go
                                    (churn): maintaining
                                    the trees is too
                                    expensive

                                - Client is receiving from
                                    O(n) nodes
                                                             36
Key Design Issues
 Efficiency                                                     servers
   o   Efficient utilization of client
       capacities, in particular under
       flash crowd
                                                                C0

 Scalability and robustness
   o   Resistant to churn and                        C1
                                         client 1                    Cn
       failures                                       C2
                                                           C3
 (for some systems) Incentive:
                                         client 2
  clients are willing                                                      client n
  to upload
                                                    client 3
   o   70% of Gnutella users share
       no files,
   o   nearly 50% of all responses
       are returned by the top 1% of
       sharing hosts                                                              37
Next Solution: Data-Oriented Swarming

 Started by Coolstreaming/DONet
  o   Motivated by BitTorrent


 Adopted by many very large-scale real
  systems
  o   PPLive (20 million active users; can have up to 2
      million concurrent users during a major event)
  o   Other examples include UUSee; PPStream


 Adobe flash 10.1 starts to support P2P

                                                          38
Basic Architecture
 Each node connects to a subset (~30-100)
  of other nodes (typically do not need to be
  a strongly structured topology such as a
  tree)
 Chop video data into chunks (say 1 sec.
  video data)
 Nodes exchange chunk availability using a
  data structure called BufferMap
 Nodes request/push data with neighbors


                                                39
 Locally Exchanging BufferMap



             0       1   0     1

         BufferMap           BufferMap
Peer 1                                   Peer 2




                                                  40
Pull/Push Data Exchange
 pull: a sees what b has             b
  but are missing at a,
  then a selects a chunk
  and requests the chunk
                                          requests/
                                          push
  from b
                                                        requests/
                                                        push
 push: a sees what b is
  missing but available         requests/              a
  at a, then a selects a        push
  chunk and pushes the        requests/
                                                      requests/
                                                      push
  chunk to b                  push

 Typically, pull is easier
  to implement; push needs
  more coordination
                                                                    41
Architectural Components of Data
Oriented Load Distribution System
                  Coordination
     Topology      by Tracker
      (Peer)
    Management      Local
                   Topology
                   Evolution


                   Request
                  Scheduling
       Chunk
       (Data)
     Scheduling    Response
                  Scheduling
                                    42
Architectural Instance:
BitTorrent
                   Coordination   Random
      Topology      by Tracker
       (Peer)
     Management      Local
                                  Peer exchange
                    Topology
                    Evolution


                    Request       Rarest block
                   Scheduling
       Chunk
       (Data)
     Scheduling                   Equal bw share;
                    Response      Tit-for-tat (top 4)
                   Scheduling     + 1 opportunistic
                                                   43
Question for Break: How to
design for streaming?




                             44
Backup Slides




                45
    Make Can Miss Intersections
                          V3
     V1
               I            Cluster interfaces that have similar
                                   routing performance



S                                                                  D
     Helps in reuse of measurements without loss
      of accuracy
     Fewer links to be measured
Popular Content: CDN
 Most popular content is moved to a CDN
 (content delivery network):
  o   CDNs replicate content in multiple places.
      There's a better chance of content being
      closer to the user, with fewer hops, and
      content will run over a more friendly network.

  o   CDN machines mostly serve out of memory
      because the content is so popular there's little
      thrashing of content into and out of memory.



                                                         47
Server Diversity for Yahoo


             Majority of PL nodes
             see between 10 and 50
             Akamai edge-servers



                                     Nodes far away
                                     from Akamai
                                     hot-spots




     Good overlay-to-CDN
48   mapping candidates
    Saul Steinberg; View of World from 9th Ave

Van Jacobson: “you want to use a hierarchical identifier space so that   49
you can aggregate out detail as you get far away.”
YouTube CDN vs YouTube
 CDNs serve from memory
 Random disks blocks are being accessed.
  - Caching doesn't do a lot of good in this scenario, so
  spending money on more cache may not make sense. This is a
  very interesting point. If you have a long tail product caching
  won't always be your performance savior.
  - Tune RAID controller and pay attention to other lower
  level issues to help.
  - Tune memory on each machine so there's not too much and
  not too little.




                                                                    50
Thumbnail Server Design
 Initially, same web server for both Web
  page and Thumbnails, same server pool




                                            51

				
DOCUMENT INFO
Shared By:
Categories:
Stats:
views:13
posted:8/18/2010
language:English
pages:51