client-server-multi-servers

					CS435/535: Internet-Scale
      Applications
          Lecture 11

       Multiple Servers

         2/16/2010
                            1
Admin.
r Assignment 2
   m Please use classv2 forum for discussions
   m Office hours today:
       • 3-4 (Mike @306);
       • 3:30-4:30 pm (Richard @307);
       • 4:30-5:30 (Ye @302)
r Thursday:
   m 10:30 am: Systems colloquium; Harsha V. Madhyastha
   m Class: Guest lecture by Harsha V. Madhyastha, on cluster
     configuration for data center application
r Assignment 3
r Great feedback on class pace and structure !


                                                                2
Recap: Operational Laws
r Utilization law: U = XS
r Forced flow law: Xi = Vi X
r Bottleneck device: largest Di = Vi Si
r Little’s Law: Qi = Xi Ri
  m   E.g., if start a thread pool of 75 threads, and
      each request takes on average 100 ms as
      response time, what is the arrival rate before
      running out of threads?

r Bottleneck analysis:       X ( N )  min{ Dmax , DNZ }
                                             1
                                                    
                           R( N )  max{ D, NDmax  Z }
                                                           3
Summary: Story So Far
r Avoid blocking (so that we can reach bottleneck
  throughput)
   m   Introduce threads
r Limit unlimited thread overhead
   m Thread pool, async io

r Shared variables
   m synchronization (lock, synchronized)

r Avoid busy-wait
   m Wait/notify; FSM

r Extensibility/robustness
   m Language support/Design for interfaces

r System modeling
   m Queueing analysis, operational analysis
                                                    4
Outline
r Recap
r Basic client/server request/reply
r Single machine concurrency
r Multiple machines
  m   Basic issues




                                      5
Why Multiple Machines?

r Scalability
  m Scaling beyond single machine capability
        • There is a fundamental limit on what a single server
          can
            – process (CPU/bw/disk throughput)
            – store (disk/memory)


   m   Scaling beyond geographical location capability
        • There is a limit on the speed of light
        • Network detour and delay further increase the delay



                                                                 6
Why Multiple Machines?

r Redundancy and fault tolerance


  m Administration/Maintenance     (e.g.,
   incremental upgrade)

  m Redundancy   (e.g., to handle failures)




                                              7
Why Multiple Machines?

r System/software Architecture
   m Resources may be naturally distributed at different
     machines (e.g., run a single copy of a database server due
     to single license; access to resource from third party)

   m   Security (e.g., front end, business logic, and database)




                                                                  8
Problems Caused by Multiple Servers

r Scalability
   -> Load direction


r Redundancy and fault tolerance
  -> Consistency




                                      9
Outline
r Recap
r Basic client/server request/reply
r Single machine concurrency
r Multiple machines
  m   Basic issues
  m   Load direction




                                      10
Load Direction: Overview

                       Site A
                                   Site B



                                   ?
- Global direction: select a
                                Internet
server site for each request
- Local direction: select a
specific server at the chosen
site
                                           Client
    Load Direction: Basic Architecture
r    Four components
          Server state                      Site A
      m
          monitoring
                                                        Site B
           • Load (incl. failed or
             not); what requests it
             can serve
      m   Path properties from
          each client to each                           ?
          server
           • E.g., Bw, delay, loss,                  Internet
             network cost
      m   Server selection
           •   Alg. to choose site(s) and
               server(s)
      m   Server direction
          mechanism
           •   Inform/direct a client to
                                                                Client
               chosen server(s)

    Objective: optimize user experience and minimize network cost.
                                                                         12
Load Direction

             Server           Path property
              state             to clients


                                                  Notify
                        Server                    client
Specific
request of            selection               about selection
a client              algorithm                 (Direction
                                               mechanism)



                                                                13
Basic Direction Mechanisms
r Explicit
   m Mirror/server listing: client is given a list of candidate
     servers
   m A special case is DNS indirection/rotation
        • DNS is a service to map from domain name (e.g., mail.yale.edu) to IP
          address(es): map <service type, name> to IP address
        • We can use DNS to notify clients of selected servers
            – E.g., Akamai and many email and Internet sites


r Implicit
   m IP anycast
        • Same IP address shared by multiple servers and announced at
          different parts of the Internet. Network directs different clients
          to different servers (e.g., LimeWire)
   m   Load balancer (smart switch) indirection
   m   Reverse proxy
                                                                                 14
    Direction Mechanisms are Often Combined

                               DNS indirection

            IP1                             IP2                    IPn


 Cluster1           Cluster2     Cluster2
in US East        in US West    in Europe

  Load              Load          Load
 balancer          balancer      balancer




                                    proxy
                                               Load
                                              balancer
                                                         servers
                                                                         15
 Example: wikipedia architecture




http://wikitech.wikimedia.org/images/8/81/Bergsma_-_Wikimedia_architecture_-_2007.pdf
                                                                                        16
DNS Indirection and Rotation
                                              157.166.226.25


                             router

                                              157.166.226.26
                IP address
                of cnn.com
                                              157.166.255.18
     157.166.226.25
     157.166.226.26


   IP address
   of cnn.com
                                DNS server
         157.166.226.26         for cnn.com
         157.166.226.25
                                                        17
 Example: Amazon Elastic Load Balancing

 r   Use the elb-create-lb command
     to create an Elastic Load Balancer.
 r   Use the elb-register-instances
     -with-lb command to register the
     Amazon EC2 instances that you
     want to load balance with the
     Elastic Load Balancer.
 r   Elastic Load Balancing automatically
     checks the health of your load
     balancing Amazon EC2 instances.
     You can optionally customize the
     health checks by using the
     elb-configure-healthcheck command.
 r   Traffic to the DNS name provided by
     the Elastic Load Balancer is automatically
     distributed across your load balanced, healthy
     Amazon EC2 instances.
http://aws.amazon.com/documentation/elasticloadbalancing/   18
    Details: Step 1
    1. Call CreateLoadBalancer with the following parameters:
        m   AvailabilityZones = us-east-1a
        m   Listeners
             •   Protocol = HTTP
             •   InstancePort = 8080
             •   LoadBalancerPort = 80
             •   LoadBalancerName = MyLoadBalancer
        The operation returns the DNS name of your LoadBalancer. You can then
          map that to any other domain name (such as www.mywebsite.com) using
          a CNAME or some other technique.

            PROMPT> elb-create-lb MyLoadBalancer --headers --
            listener "lb-port=80,instance-port=8080,protocol=HTTP" -
            -availability-zones us-east-1a

            Result:
            DNS-NAME DNS-NAME
            DNS-NAME MyLoadBalancer-2111276808.us-east-1.elb.amazonaws.com

http://docs.amazonwebservices.com/ElasticLoadBalancing/latest/DeveloperGuide/
                                                                                19
Details: Step 2
2. Call ConfigureHealthCheck with the following parameters:
   m   LoadBalancerName = MyLoadBalancer
   m   Target = http:8080/ping
        •    NoteMake sure your instances respond to /ping on port 8080 with an HTTP 200 status code.
   m   Interval = 30
   m   Timeout = 3
   m   HealthyThreshold = 2
   m   UnhealthyThreshold = 2

   PROMPT> elb-configure-healthcheck MyLoadBalancer --headers --target
     "HTTP:8080/ping" --interval 30 --timeout 3 --unhealthy-threshold 2 --
     healthy-threshold 2

   Result:
   HEALTH-CHECK TARGET INTERVAL TIMEOUT HEALTHY-THRESHOLD UNHEALTHY-THRESHOLD
   HEALTH-CHECK HTTP:8080/ping 30 3 2 2




                                                                                                        20
Details: Step 3
3. Call RegisterInstancesWithLoadBalancer with the following parameters:
     m LoadBalancerName = MyLoadBalancer
     m Instances = [ i-4f8cf126, i-0bb7ca62 ]




    PROMPT> elb-register-instances-with-lb MyLoadBalancer --headers --
      instances i-4f8cf126,i-0bb7ca62

    Result:
    INSTANCE INSTANCE-ID
    INSTANCE i-4f8cf126
    INSTANCE i-0bb7ca62




                                                                           21
Discussion
r Advantages and disadvantages of using
 DNS




                                          22
IP Anycast
r On the Internet, anycast is usually implemented
  by BGP, the protocol that computes the routing
  path to a destination IP address
r BGP allows a node to announce that it is the origin
  of a destination address
r If a node learns multiple paths (from different
  neighbors) to a destination, the node selects one
  of them (e.g., shortest path length)
r If multiple locations announce that they are the
  origin of the same dest. IP, BGP routers at
  different locations may choose different servers
  (with same IP)

                                                        23
BGP Basics
r BGP is a path vector protocol
  m   Each BGP node tells some of its neighbors its
      path to a destination address
  m   a path is a sequence of Autonomous Systems, e.g.,
        • gateway X sends to neighbor N its path to dest. d:
           path (X;IPd) = X,Y1,Y2,Y3
  m   if N selects path(X; IPd) advertised by X, then:
            path (N; IPd) = N, path (X; IPd)
                                                               IPd

                                                N        X




                                                                     24
BGP: Example
                  Compare W      AS B
                   and BE;
     AS F
                  choose one
     f                                    b    AS E
                                                      d
                                                e


                AS A

                               AS G
 AS W                                 g
            w                                 AS C
 d
Examples using IP Anycast
r DNS servers
  m   http://root-servers.org/
r Google Public DNS
  m   The service has IP address (8.8.8.8 or 8.8.4.4).
  m   Google deploys multiple servers with the same
      address, e.g., 8.8.8.8
  m   Google announces 8.8.8.8 at multiple locations
      on the Internet
  m   Clients with dest. 8.8.8.8 reach different
      Google DNS servers


                                                         26
Discussion
r Advantages and disadvantages of using IP
 Anycast




                                             27
Clustering with VIP: Basic Idea
r Clients get a single service IP address,
  called virtual IP address (VIP)
r A virtual server (also referred to as load
  balancer, vserver or smart switch) listens
  at VIP address and port
r A virtual server is bound to a number of
  physical servers running in a server farm
r A client sends a request to the virtual
  server, which in turn selects a physical
  server in the server farm and directs this
  request to the selected physical server
                                               28
VIP Clustering

                                               Goals
                                               server load balancing
           L4: TCP
                                               failure detection
          L7: HTTP
             SSL                               access control filtering
             etc.                              priorities/QoS
                                               request locality
          virtual IP                           transparent caching
                        smart
          addresses
Clients    (VIPs)
                       switch
                                               What to switch/filter on?
                                server array   L3 source IP and/or VIP
                                               L4 (TCP) ports etc.
                                               L7 URLs and/or cookies
                                               L7 SSL session IDs
Big Picture




              30
 Load Balancer: Basic Structure
                                             RIP1

                                             Server1

                                      VIP    RIP2
Client                                  LB   Server2

                                             RIP3
                                             Server3




   Problems of the basic structure?



                                                       31

				
DOCUMENT INFO