Lecture 1 Course Introduction and Overview

Document Sample
Lecture 1 Course Introduction and Overview Powered By Docstoc

• Goal: Communication between computers
• Eventual Goal: treat collection of computers as
  if one big computer, distributed resource
• Theme: Different computers must agree on
  many things
   – Overriding importance of standards and protocols
   – Error tolerance critical as well

• Issues:
   –   direct (point-to-point) vs. indirect (multi-hop)
   –   topology (e.g., bus, ring, DAG)
   –   routing algorithms
   –   switching (aka multiplexing)
   –   wiring (e.g., choice of media, copper, coax, fiber)
• What really matters:
   –   latency
   –   bandwidth
   –   cost
   –   reliability
                Interconnections (Networks)
• Examples (see Figure 7.19, page 633):
      – Wide Area Network (ATM): 100-1000s nodes; ~ 5,000 kilometers
      – Local Area Networks (Ethernet): 10-1000 nodes; ~ 1-2 kilometers
      – System/Storage Area Networks (FC-AL): 10-100s nodes;
                                 ~ 0.025 to 0.1 kilometers per link
end systems,

                          Interconnection Network
       SAN: Storage vs. System
• Storage Area Network (SAN): A block I/O
  oriented network between application
  servers and storage
   – Fibre Channel is an example
• Usually high bandwidth requirements, and
  less concerned about latency
   – in 2001: 1 Gbit bandwidth and millisecond latency OK
• Commonly a dedicated network
  (that is, not connected to another network)
• May need to work gracefully when saturated
• Given larger block size, may have higher bit
  error rate (BER) requirement than LAN
                SAN vs. NAS

•   Storage area network
•   Network-attached storage
•   Storage virtualization
•   Continuous data protection
       SAN: Storage vs. System
• System Area Network (SAN): A network
  aimed at connecting computers
   – Myrinet is an example
• Aimed at High Bandwidth AND Low Latency.
   – in 2001: > 1 Gbit bandwidth and ~ 10 microsecond
• May offer in order delivery of packets
• Given larger block size, may have higher bit
  error rate (BER) requirement than LAN
       More Network Background

• Connection of 2 or more networks:
• 3 cultures for 3 classes of networks
   – WAN: telecommunications, Internet
   – LAN: PC, workstations, servers cost
   – SAN: Clusters, RAID boxes: latency (System A.N.) or
     bandwidth (Storage A.N.)
• Motivate the interconnection complexity
                 ABCs of Networks

• Starting Point: Send bits between 2 computers

•   Queue (FIFO) on each end
•   Information sent called a “message”
•   Can send both ways (“Full Duplex”)
•   Rules for communication? “protocol”
     – Inside a computer:
         » Loads/Stores: Request (Address) & Response (Data)
         » Need Request & Response signaling
              A Simple Example

• What is the format of mesage?
   – Fixed? Number bytes?

      1 bit                 32 bits
0: Please send data from Address
1: Packet contains data corresponding to request
• Header/Trailer: information to deliver a message
• Payload: data in message (1 word above)
         Questions About Simple Example

• What if more than 2 computers want to
   – Need computer “address field” (destination) in packet
• What if packet is garbled in transit?
   – Add “error detection field” in packet (e.g., Cyclic Redundancy Chk)
• What if packet is lost?
   – More “elaborate protocols” to detect loss
     (e.g., NAK, ARQ, time outs)
• What if multiple processes/machine?
   – Queue per process to provide protection
• Simple questions such as these lead to more complex
  protocols and packet formats => complexity
           A Simple Example Revisted

     • What is the format of packet?
        – Fixed? Number bytes?
                                 Address/Data   CRC

           2 bits                  32 bits      4 bits

00: Request—Please send data from Address
01: Reply—Packet contains data corresponding to request
10: Acknowledge request
11: Acknowledge reply
    Software to Send and Receive

• SW Send steps
  1: Application copies data to OS buffer
  2: OS calculates checksum, starts timer
  3: OS sends data to network interface HW and says start
• SW Receive steps
  3: OS copies data from network interface HW to OS buffer
  2: OS calculates checksum, if matches send ACK; if not,
    deletes message (sender resends when timer expires)
  1: If OK, OS copies data to user address space and signals
    application to continue
• Sequence of steps for SW: protocol
  – Example similar to UDP/IP protocol in UNIX
      Low-Latency Message Passing

•   Reducing data copying
•   Interrupt coalescing
•   Decreasing context switch
•   More efficient DMA transactions
•   Wither TCP offload engine?
        Network Performance Measures

• Overhead: latency of interface vs. Latency: network
           Universal Performance Metrics

            Sender        Transmission time
Sender     Overhead       (size ÷ bandwidth)

                        Time of   Transmission time    Receiver
                         Flight   (size ÷ bandwidth)   Overhead
                             Transport Latency

                                  Total Latency

Total Latency = Sender Overhead + Time of Flight +
                Message Size ÷ BW + Receiver Overhead
Includes header/trailer in BW calculation?
             Total Latency Example
• 1000 Mbit/sec., sending overhead of 80 µsec &
  receiving overhead of 100 µsec.
• a 10000 byte message (including the header), allows
  10000 bytes in a single message
• 2 situations: distance 100 m vs. 1000 km
• Speed of light ~ 300,000 km/sec
• Latency0.01km = 80 + 0.01km / (50% x 300,000)
                  + 10000 x 8 / 1000 + 100 = 260 µsec
• Latency0.5km = 80 + 0.5km / (50% x 300,000)
                  + 10000 x 8 / 1000 + 100 = 263 µsec
• Latency1000km = 80 + 1000 km / (50% x 300,000)
                  + 10000 x 8 / 1000 + 100 = 6931
• Long time of flight => complex WAN protocol
            Universal Metrics

• Apply recursively to all levels of system
• inside a chip, between chips on a board,
  between computers in a cluster, …
• Look at WAN v. LAN v. SAN
          Simplified Latency Model

• Total Latency - Overhead + Message Size / BW

• Overhead = Sender Overhead + Time of Flight +
                Receiver Overhead

• Example: show what happens as vary
  – Overhead: 1, 25, 500 µsec
  – BW: 10,100, 1000 Mbit/sec (factors of 10)
  – Message Size: 16 Bytes to 4 MB (factors of 4)
• If overhead 500 µsec,
      how big a message > 10 Mb/s?
                                                  Overhead, BW, Size
            Delivered BW
                                                                       o1,                      o25,                 o500,
                                                                  bw1000                     bw1000                bw1000
  Effective Bandwidth (Mbit/sec)

                                                  bw100              bw100                       bw100
                                           bw10            o25,
                                                           bw10                      o500,


                                       0                                                                                                  Msg Size









•How big are                                                      Message Size (bytes)
real messages?
                            Sizes of Message for NFS
Cummulative %

                 80%                Msgs
                 60%                Bytes                                     Why?
                        0   1024   2048    3072   4096   5120   6144   7168   8192

                                  Packet size
                • 95% Msgs, 30% bytes for packets ~ 200 bytes
                • > 50% data transfered in packets = 8KB
               Impact of Overhead on Delivered
               1000.00                                       1
Delivered BW

                100.00                                       10

                 10.00                                       100

                   1.00                                      1000

                   0.10                                  MinTime




                              Peak BW (MB/sec)
 • BW model: Time = overhead + msg size/peak BW
         Interconnect Issues

• Performance Measures
• Network Media
                       Network Media
Twisted Pair:
                                        Copper, 1mm think, twisted to avoid
                                        attenna effect (telephone)
Coaxial Cable:                          "Cat 5" is 4 twisted pairs in bundle
                  Plastic Covering

                         Insulator            Used by cable companies:
                            Copper core
                                              high BW, good noise
                   Braided outer conductor immunity
                                 Buffer                        Light: 3 parts
                                    Cladding                   are cable, light
Fiber Optics                          Total internal           source, light
                                        reflection             detector.
 Transmitter                                     Receiver
  – L.E.D                                         – Photodiode Note fiber is
  – Laser Diode
        light                                                  need 2 for full
        source                                Silica core      duplex

• Multimode fiber: ~ 62.5 micron diameter vs. the 1.3
  micron wavelength of infrared light. Since wider it
  has more dispersion problems, limiting its length at
  1000 Mbits/s for 0.1 km, and 1-3 km at 100 Mbits/s.
  Uses LED as light
• Single mode fiber: "single wavelength" fiber (8-9
  microns) uses laser diodes, 1-5 Gbits/s for 100s kms
   – Less reliable and more expensive, and restrictions on bending
   – Cost, bandwidth, and distance of single-mode fiber affected
     by power of the light source, the sensitivity of the light
     detector, and the attenuation rate (loss of optical signal
     strength as light passes through the fiber) per kilometer of
     the fiber cable.
   – Typically glass fiber, since has better characteristics than
     the less expensive plastic fiber
   Wave Division Multiplexing Fiber

• Send N independent streams on single fiber!
• Just use different wavelengths to send and
  demultiplex at receiver
• WDM in 2000: 40 Gbit/s using 8 wavelengths
• Plan to go to 80 wavelengths => 400 Gbit/s!
• A figure of merit: BW* max distance
• 10X/4 years, or 1.8X per year
                Compare Media
• Assume 40 2.5" disks, each 25 GB, Move 1 km
• Compare Cat 5 (100 Mbit/s), Multimode fiber (1000
  Mbit/s), single mode (2500 Mbit/s), and car
• Cat 5: 1000 x 1024 x 8 Mb / 100 Mb/s = 23 hrs
• MM: 1000 x 1024 x 8 Mb / 1000 Mb/s = 2.3 hrs
• SM:    1000 x 1024 x 8 Mb / 2500 Mb/s = 0.9 hrs
• Car: 5 min + 1 km / 50 kph + 10 min = 0.25 hrs
• Car of disks = high BW media
          Interconnect Issues

• Performance Measures
• Network Media
• Connecting Multiple Computers
            Connecting Multiple Computers
• Shared Media vs. Switched:
  pairs communicate at same time:
  “point-to-point” connections
• Aggregate BW in switched
  network is many times shared
  – point-to-point faster since no
    arbitration, simpler interface
• Arbitration in Shared network?
  – Central arbiter for LAN?
  – Listen to check if being used (“Carrier
  – Listen to check if collision
    (“Collision Detection”)                   (A. K. A. data switching
  – Random resend to avoid repeated           interchanges, multistage
    collisions; not fair arbitration;         interconnection networks,
  – OK if low utilization                     interface message processors)
                  Main Issues

•   Addressing
•   Routing
•   Congestion control
•   Flow control
   Connection-Based vs. Connectionless

• Telephone: operator sets up connection between
  the caller and the receiver
   – Once the connection is established, conversation can continue for
• Share transmission lines over long distances by
  using switches to multiplex several conversations on
  the same lines
   – “Time division multiplexing” divide B/W transmission line into a
     fixed number of slots, with each slot assigned to a conversation
• Problem: lines busy based on number of
  conversations, not amount of information sent
• Advantage: reserved bandwidth
 Connection-Based vs. Connectionless

• Connectionless: every package of
  information must have an address =>
  – Each package is routed to its destination by looking at
    its address
  – Analogy, the postal system (sending a letter)
  – also called “Statistical multiplexing”
  – Note: “Split phase buses” are sending packets
               Routing Messages
• Shared Media
  – Broadcast to everyone
• Switched Media needs real routing. Options:
  – Source-based routing: message specifies path to the
    destination (changes of direction)
  – Virtual Circuit: circuit established from source to
    destination, message picks the circuit to follow
  – Destination-based routing: message specifies
    destination, switch must pick the path
      » deterministic: always follow same path
      » adaptive: pick different paths to avoid congestion,
      » Randomized routing: pick between several good
        paths to balance network load
         Deterministic Routing Examples
• mesh: dimension-order routing
  – (x1, y1) -> (x2, y2)
  – first x = x2 - x1,
  – then y = y2 - y1,
• hypercube: edge-cube routing
  – X = xox1x2 . . .xn -> Y = yoy1y2 . . .yn
  – R = X xor Y                                 010          110
  – Traverse dimensions of differing
    address in order
• tree: common ancestor                              011
• Deadlock free?                                               100


                                                       001         101
     Store and Forward vs. Cut-Through
• Store-and-forward policy: each switch waits for
  the full packet to arrive in switch before sending to
  the next switch (good for WAN)
• Cut-through routing or worm hole routing: switch
  examines the header, decides where to send the
  message, and then starts forwarding it immediately
   – In worm hole routing, when head of message is blocked, message
     stays strung out over the network, potentially blocking other
     messages (needs only buffer the piece of the packet that is sent
     between switches).
   – Cut through routing lets the tail continue when head is blocked,
     and putting the whole message into a single switch. (Requires a
     buffer large enough to hold the largest packet).
   Cut-Through vs. Store and Forward
• Advantage
  – Latency reduces from function of:

    number of intermediate switches X by the size of the packet


    time for 1st part of the packet to negotiate the switches
    + the packet size ÷ interconnect BW
                       Congestion Control
• Packet switched networks do not reserve bandwidth;
  this leads to contention (connection based limits input)
• Solution: prevent packets from entering until
  contention is reduced
  (e.g., freeway on-ramp metering lights)
• Options:
   – Packet discarding: If packet arrives at switch and no room in buffer,
     packet is discarded (e.g., UDP)
   – Flow control: between pairs of receivers and senders;
     use feedback to tell sender when allowed to send next packet
       » Back-pressure: separate wires to tell to stop
       » Window: give original sender right to send N packets before
         getting permission to send more; overlaps latency of
         interconnection with overhead to send & receive packet (e.g.,
         TCP), adjustable window
   – Choke packets: aka “rate-based”; Each packet received by busy
     switch in warning state sent back to the source via choke packet.
     Source reduces traffic to that destination by a fixed % (e.g., ATM)
          Protocols: HW/SW Interface

• Internetworking: allows computers on independent
  and incompatible networks to communicate reliably
  and efficiently;
   – Enabling technologies: SW standards that allow reliable
     communications without reliable networks
   – Hierarchy of SW layers, giving each layer responsibility for
     portion of overall communications task, called
     protocol families or protocol suites
• Transmission Control Protocol/Internet Protocol
   – This protocol family is the basis of the Internet
   – IP makes best effort to deliver; TCP guarantees delivery
   – TCP/IP used even when communicating locally: NFS uses IP even
     though communicating across homogeneous LAN
              Connecting Networks
• Bridges: connect LANs together, passing traffic
  from one side to another depending on the addresses
  in the packet.
   – operate at the Ethernet protocol level
   – usually simpler and cheaper than routers
• Routers or Gateways: these devices connect LANs to
  WANs or WANs to WANs and resolve incompatible
   – Generally slower than bridges, they operate at the
     internetworking protocol (IP) level
   – Routers divide the interconnect into separate smaller subnets,
     which simplifies manageability and improves security
• Cisco is major supplier;
  basically special purpose computers
               Virtual LAN

• Layer2 technology that tries to achieve
  what Layer3 routers can do: limit broadcast
• Distributed spanning tree protocol (802.1D)
• Per-tree spanning tree
• VLAN to emulate ATM
• Transparent reliable multicast
• IGMP snooping
                   Wireless Networks
• Media can be air as well as glass or copper
• Radio wave is electromagnetic wave propagated by an
• Radio waves are modulated: sound signal superimposed on
  stronger radio wave which carries sound signal, called
  carrier signal
• Radio waves have a wavelength or frequency: measure either
  length of wave
  or number of waves per second (MHz):
  long waves => low frequencies,
  short waves => high frequencies
• Tuning to different frequencies => radio receiver pick up a
   – FM radio stations transmit on band of 88 MHz to 108 MHz using
     frequency modulations (FM) to record the sound signal
                Issues in Wireless

• Wireless often => mobile => network must rearrange
  itself dynamically
• Subject to jamming and eavesdropping
   – No physical tape
   – Cannot detect interception
• Power
   – devices tend to be battery powered
   – antennas radiate power to communicate and little of it reaches
     the receiver
• As a result, raw bit error rates are typically a
  thousand to a million times higher than copper wire
  Reliability of Wires Transmission
• bit error rate (BER) of wireless link
  determined by received signal power, noise
  due to interference caused by the receiver
  hardware, interference from other sources,
  and characteristics of the channel
  – Path loss: power to overcome interference
  – Shadow fading: blocked by objects (walls, buildings)
  – Multipath fading: interference between multiple version
    of signals arriving different times
  – Interference: reuse of frequency or from adjacent
         2 Wireless Architectures

• Base-station architectures
   – Connected by land lines for longer distance
     communication, and the mobile units communicate only
     with a single local base station
   – More reliable since 1-hop from land lines
   – Example: cell phones
• Peer-to-peer architectures
   – Allow mobile units to communicate with each other, and
     messages hop from one unit to the next until delivered
     to the desired unit
   – More reconfigurable
         Unified P2P Architecture

• Completely distributed system: don’t even
  know who to talk to ?
• Advantages: scalability, fault tolerance, and
• Examples
   – KaZaA
   – Routing protocol for wired networks
   – Routing protocol for wireless networks
              Cellular Telephony
• Exploit exponential path loss to reuse same frequency at
  spatially separated locations, thereby greatly increasing
  customers served
• Divide region into nonoverlaping hexagonal cells (2-10 mi.
  diameter) which use different frequencies if nearby, reusing
  a frequency when cells far apart so that mutual interference
• Intersection of three hexagonal cells is a base station with
  transmitters and antennas
• Handset selects a cell based on signal strength and then
  picks an unused radio channel
• To properly bill for cellular calls, each cellular phone handset
  has an electronic serial number
          Cellular Telephony II
• Orginal analog design frequencies set for each
  direction: pair called a channel
   – 869.04 to 893.97 MHz, called the forward path
   – 824.04 MHz to 848.97 MHz, called the reverse path
   – Cells might have had between 4 and 80 channels
• Several digital successors:
   – Code division multiple access (CDMA) uses a wider radio
     frequency band
   – time division multiple access (TDMA)
   – global system for mobile communication (GSM)
   – International Mobile Telephony 2000 (IMT-2000) which is
     based primarily on two competing versions of CDMA and one
     TDMA, called Third Generation (3G)
        Wireless Networking vs.

• The name of the game is wireless
  communications: modulation, MIMO,
• Networking part: routing, transport
  protocol, handoff, security
 Practical Issues for Inteconnection

• Connectivity: max number of machines
  affects complexity of network and protocols
  since protocols must target largest size
• Connection Network Interface to computer
  – Where in bus hierarchy? Memory bus? Fast I/O bus?
    Slow I/O bus? (Ethernet to Fast I/O bus, Inifiband to
    Memory bus since it is the Fast I/O bus)
  – SW Interface: does software need to flush caches for
    consistency of sends or receives?
  – Programmed I/O vs. DMA? Is NIC in uncachable
    address space?
 Practical Issues for Inteconnection

• Standardization advantages:
   – low cost (components used repeatedly)
   – stability (many suppliers to chose from)
• Standardization disadvantages:
   – Time for committees to agree
   – When to standardize?
       » Before anything built? => Committee does design?
       » Too early suppresses innovation
• Reliability (vs. availability) of interconnect
                   Practical Issues
Interconnection    SAN         LAN        WAN
Example            Inifiband   Ethernet   ATM
Standard           Yes         Yes        Yes
Fault Tolerance?   Yes         Yes        Yes
Hot Insert?        Yes         Yes        Yes

• Standards: required for WAN, LAN, and likely SAN!
• Fault Tolerance: Can nodes fail and still deliver
  messages to other nodes?
• Hot Insert: If the interconnection can survive a
  failure, can it also continue operation while a new
  node is added to the interconnection?
    Cross-Cutting Issues for Networking

• Efficient Interface to Memory Hierarchy vs. to
   – SPEC ratings => fast to memory hierarchy
   – Writes go via write buffer, reads via L1 and
     L2 caches
• Example: 40 MHz SPARCStation(SS)-2 vs 50
  MHz SS-20, no L2$ vs 50 MHz SS-20 with L2$
  I/O bus latency; different generations
• SS-2: combined memory, I/O bus => 200 ns
• SS-20, no L2$: 2 busses +300ns => 500ns
• SS-20, w L2$: cache miss+500ns => 1000ns
        Crosscutting: Smart Switch vs.
        Smart Network Interface Card

              Less Intelligent   More Intelligent
              Small Ethernet     Large Ethernet
    Switch       Myrinet
                  Ethernet            Myrinet
      NIC    Infiniband Target    Inifiband Host
              Channel Adapter    Channel Adapter

•Inexpensive NIC => Ethernet standard in all computers
•Inexpensive switch => Ethernet used in home networks

• LAN switches => high network bandwidth and scaling
  was available from off the shelf components
• 2001 Cluster = collection of independent computers
  using switched network to provide a common service
• Many mainframe applications run more "loosely
  coupled" machines than shared memory machines
  (next chapter/week)
   – databases, file servers, Web servers, simulations, and
     multiprogramming/batch processing
   – Often need to be highly available, requiring error tolerance and
   – Often need to scale
                 Cluster Drawbacks
• Cost of administering a cluster of N machines
  ~ administering N independent machines
  vs. cost of administering a shared address space N
  processors multiprocessor
  ~ administering 1 big machine
• Clusters usually connected using I/O bus, whereas
  multiprocessors usually connected on memory bus
• Cluster of N machines has N independent memories
  and N copies of OS, but a shared address multi-
  processor allows 1 program to use almost all memory
   – DRAM prices has made memory costs so low that this
     multiprocessor advantage is much less important in 2001
                Cluster Advantages
• Error isolation: separate address space limits contamination of
• Repair: Easier to replace a machine without bringing down the
  system than in an shared memory multiprocessor
• Scale: easier to expand the system without bringing down the
  application that runs on top of the cluster
• Cost: Large scale machine has low volume => fewer machines to
  spread development costs vs. leverage high volume off-the-shelf
  switches and computers
• Amazon, AOL, Google, Hotmail, Inktomi, WebTV, and Yahoo rely
  on clusters of PCs to provide services used by millions of people
  every day
    Addressing Cluster Weaknesses

• Network performance: SAN, especially
  Inifiband, may tie cluster closer to memory
• Maintenance: separate of long term storage
  and computation
• Computation maintenance:
   – Clones of identical PCs
   – 3 steps: reboot, reinstall OS, recycle
   – At $1000/PC, cheaper to discard than to figure out
     what is wrong and repair it?
• Storage maintenance:
   – If separate storage servers or file servers, cluster is
     no worse?
    Clusters and TPC Benchmarks

• “Shared Nothing” database (not memory,
  not disks) is a match to cluster
• 2/2001: Top 10 TPC performance 6/10 are
  clusters (4 / top 5)
     Putting it all together: Google
• Google: search engine that scales at growth Internet
  growth rates
• Search engines: 24x7 availability
• Google 12/2000: 70M queries per day, or AVERAGE
  of 800 queries/sec all day
• Response time goal: < 1/2 sec for search
• Google crawls WWW and puts up new index every 4
• Stores local copy of text of pages of WWW (snippet
  as well as cached copy of page)
• 3 collocation sites (2 CA + 1 Virginia)
• 6000 PCs, 12000 disks: almost 1 petabyte!
  Hardware Infrastructure
• VME rack 19 in. wide, 6 feet
  tall, 30 inches deep
• Per side: 40 1 Rack Unit (RU)
  PCs +1 HP Ethernet switch (4
  RU): Each blade can contain 8
  100-Mbit/s EN or a single 1-
  Gbit Ethernet interface
• Front+back => 80 PCs +
  2 EN switches/rack
• Each rack connects to 2 128
  1-Gbit/s EN switches
• Dec 2000: 40 racks at most
  recent site
                       Google PCs
• 2 IDE drives, 256 MB of SDRAM, modest Intel
  microprocessor, a PC mother-board, 1 power supply and
  a few fans.
• Each PC runs the Linix operating system
• Buy over time, so upgrade components:
  populated between March and November 2000
   – microprocessors: 533 MHz Celeron to an 800 MHz Pentium III,
   – disks: capacity between 40 and 80 GB, speed 5400 to 7200 RPM
   – bus speed is either 100 or 133 MH
   – Cost: ~ $1300 to $1700 per PC
• PC operates at about 55 Watts
• Rack => 4500 Watts , 60 amps
• For 6000 PCs, 12000s, 200 EN switches
• ~ 20 PCs will need to be rebooted/day
• ~ 2 PCs/day hardware failure, or 2%-3% / year
   –   5% due to problems with motherboard, power supply, and connectors
   –   30% DRAM: bits change + errors in transmission (100 MHz)
   –   30% Disks fail
   –   30% Disks go very slow (10%-3% expected BW)
• 200 EN switches, 2-3 fail in 2 years
• 6 Foundry switches: none failed, but 2-3 of 96 blades of
  switches have failed (16 blades/switch)
• Collocation site reliability:
   – 1 power failure,1 network outage per year per site
   – Bathtub for occupancy
     Google Performance: Serving

• How big is a page returned by Google?
• Average bandwidth to serve searches
   70,000,000/day x 16,750 B x 8 bits/B
                24 x 60 x 60
       =9,378,880 Mbits/86,400 secs
                 = 108 Mbit/s
    Google Performance: Crawling

• How big is a text of a WWW page? ~4000B
• 1 Billion pages searched
• Assume 7 days to crawl
• Average bandwidth to crawl
1,000,000,000/pages x 4000 B x 8 bits/B
                24 x 60 x 60 x 7
       =32,000,000 Mbits/604,800 secs
                  = 59 Mbit/s
 Google Performance: Replicating Index

• How big is Google index? ~5 TB
• Assume 7 days to replicate to 2 sites,
  implies BW to send + BW to receive
• Average bandwidth to replicate new index
       2 x 2 x 5,000,000 MB x 8 bits/B
               24 x 60 x 60 x 7
      =160,000,000 Mbits/604,800 secs
                 = 260 Mbit/s
                   Co-location Sites
• Allow scalable space, power, cooling and network
  bandwidth plus provide physical security
• charge about $500 to $750 per Mbit/sec/month
   – if your continuous use measures 1- 2 Gbits/second
to $1500 to $2000 per Mbit/sec/month
   – if your continuous use measures 1-10 Mbits/second
• Rack space: costs $800 -$1200/month, and drops by 20%
  if > 75 to 100 racks (1 20 amp circuit)
   – Each additional 20 amp circuit per rack costs another $200 to $400
     per month
• PG&E: 12 megawatts of power, 100,000 sq. ft./building,
  10 sq. ft./rack => 1000 watts/rack
         Google Performance: Total

• Serving pages: 108 Mbit/sec/month
• Crawling: 59 Mbit/sec/week, 15 Mbit/s/month
• Replicating: 260 Mbit/sec/week, 65 Mb/s/month
• Total: roughly 200 Mbit/sec/month
• Google’s Collocation sites have OC48
  (2488 Mbit/sec) link to Internet
• Bandwidth cost per month?
  ~$150,000 to $200,000
• 1/2 BW grows at 20%/month
              Google Costs
• Collocation costs: 40 racks @ $1000 per
  month + $500 per month for extra circuits
= ~$60,000 per site, * 3 sites
~$180,000 for space
• Machine costs:
• Rack = $2k + 80 * $1500/pc + 2 * $1500/EN
  = ~$125k
• 40 racks + 2 Foundry switches @$100,000
  = ~$5M
• 3 sites = $15M
• Cost today is $10,000 to $15,000 per TB
  Comparing Storage Costs: 1/2001

• Google site, including 3200 processors and
  0.8 TB of DRAM, 500 TB (40 racks)
  $10k - $15k/ TB
• Compaq Cluster with 192 processors,
  0.2 TB of DRAM, 45 TB of SCSI Disks
  (17+ racks) $115k/TB (TPC-C)
• HP 9000 Superdome: 48 processors,
  0.25 TB DRAM, 19 TB of SCSI disk =
  (23+ racks) $360k/TB (TPC-C)
 Putting It All Together: Cell Phones

• 1999 280M handsets
  sold; 2001 500M
• Radio steps/components:
  –   Antenna
  –   Amplifier
  –   Mixer
  –   Filter
  –   Demodulator
  –   Decoder
Putting It All Together: Cell Phones

• about 10 chips in 2000, which should shrink,
  but likely separate MPU and DSP
• Emphasis on energy efficiency
   From “How Stuff Works” on cell phones: www.howstuffworks.com
         Cell phone steps (protocol)

1. Find a cell
   •   Scans full BW to find stronger signal every 7 secs
2. Local switching office registers call
   •   records phone number, cell phone serial number,
       assigns channel
   •   sends special tone to phone, which cell acks if correct
   •   Cell times out after 5 sec if doesn't get supervisory
3. Communicate at 9600 b/s digitally (modem)
   •   Old style: message repeated 5 times
   •   AMPS had 2 power levels depending on distance (0.6W
       and 3W)
     Frequency Division Multiple Access

• FDMA separates the
  spectrum into distinct
  voice channels by
  splitting it into uniform
  chunks of bandwidth
• !st generation analog

        From “How Stuff Works” on cell phones: www.howstuffworks.com
          Time Division Multiple Access
• a narrow band that is 30 kHz
  wide and 6.7 ms long is split
  time-wise into 3 time slots.
• Each conversation gets the
  radio for 1/3 of time.
• Possible because voice data
  converted to digital
  information is compressed so
• Therefore, TDMA has 3
  times capacity of analog
• GSM implements TDMA in a
  somewhat different and
  incompatible way from US
  (IS-136); also encrypts the

         From “How Stuff Works” on cell phones: www.howstuffworks.com
           Code Division Multiple Access
• CDMA, after digitizing data,
  spreads it out over the entire
  bandwidth it has available.
• Multiple calls are overlaid
  over each other on the
  channel, with each assigned a
  unique sequence code.
• CDMA is a form of spread
  spectrum; All the users
  transmit in the same wide-
  band chunk of spectrum.
• Each user's signal is spread
  over the entire bandwidth by
  a unique spreading code.
  same unique code is used to
  recover the signal.

                               From “How Stuff Works” on cell phones: www.howstuffworks.com
              Single-Chip PC

• What constitutes a PC?
• Can they all be packaged into one chip?
   100 million transistors
• $100 Notebook computer