Learning Center
Plans & pricing Sign in
Sign Out


VIEWS: 46 PAGES: 113

  • pg 1
									Chapter 2: Application layer
 2.1 Principles of        2.6 P2P file sharing
  network applications     2.7 Socket
 2.2 Web and HTTP          programming with TCP
 2.3 FTP                  2.8 Socket
 2.4 Electronic Mail       programming with UDP
      SMTP, POP3, IMAP    2.9 Building a Web
 2.5 DNS                   server

                                       2: Application Layer   1
Chapter 2: Application Layer
Our goals:                 learn about protocols
 conceptual,               by examining popular
  implementation            application-level
  aspects of network        protocols
  application protocols         HTTP
    transport-layer            FTP
     service models             SMTP / POP3 / IMAP
    client-server

     paradigm              programming network
      peer-to-peer         applications
       paradigm                 socket API

                                           2: Application Layer   2
Some network apps
 E-mail                Internet telephone
 Web                   Real-time video
 Instant messaging      conference
 Remote login          Massive parallel
 P2P file sharing
 Multi-user network
 Streaming stored
  video clips

                                    2: Application Layer   3
Creating a network app
Write programs that               applicatio

      run on different end
                                  network

      systems and
                                  data link

     communicate over a
     e.g., Web: Web server
      software communicates
      with browser software
No software written for                        applicatio

  devices in network core
                                                   n                      transport
                                               transport                   network
      Network core devices do
                                                network                   data link
                                              data link                   physical
      not function at app layer

     This design allows for
      rapid app development
                                                            2: Application Layer       4
Chapter 2: Application layer
 2.1 Principles of        2.6 P2P file sharing
  network applications     2.7 Socket
 2.2 Web and HTTP          programming with TCP
 2.3 FTP                  2.8 Socket
 2.4 Electronic Mail       programming with UDP
      SMTP, POP3, IMAP    2.9 Building a Web
 2.5 DNS                   server

                                       2: Application Layer   5
Application architectures
 Client-server
 Peer-to-peer (P2P)
 Hybrid of client-server and P2P

                                    2: Application Layer   6
Client-server archicture
                      always-on host
                      permanent IP address
                      server farms for
                       communicate with
                      may be intermittently
                      may have dynamic IP
                      do not communicate
                       directly with each
                               2: Application Layer   7
Pure P2P architecture
 no always on server
 arbitrary end systems
  directly communicate
 peers are
  connected and change
  IP addresses
 example: Gnutella

Highly scalable

But difficult to manage   2: Application Layer   8
Hybrid of client-server and
   File transfer P2P
   File search centralized:
      • Peers register content at central server
      • Peers query same central server to locate content
Instant messaging
   Chatting between two users is P2P
   Presence detection/location centralized:
      • User registers its IP address with central server
        when it comes online
      • User contacts central server to find IP addresses
        of buddies

                                             2: Application Layer   9
Processes communicating
Process: program running    Client process: process
  within a host.               that initiates
 within same host, two
  processes communicate     Server process: process
  using inter-process          that waits to be
  communication (defined       contacted
  by OS).
 processes in different    Note: applications with
  hosts communicate by       P2P architectures have
  exchanging messages        client processes &
                             server processes

                                      2: Application Layer   10
 process sends/receives
  messages to/from its             host or
                                                                             host or
 socket analogous to door                       controlled by
                                                 app developer
     sending process shoves       process                                   process
      message out door             socket                                    socket
     sending process relies on   TCP with                              TCP with
      transport infrastructure    buffers,            Internet          buffers,
      on other side of door       variables                             variables

      which brings message to
      socket at receiving                     controlled
      process                                 by OS

 API: (1) choice of transport protocol; (2) ability
  to fix a few parameters (lots more on this later)
                                                      2: Application Layer      11
 Addressing processes:
 For a process to receive
  messages, it must have        Identifier includes
  an identifier                  both the IP address
 Every host has a unique        and port numbers
  32-bit IP address              associated with the
 Q: does the IP address
                                 process on the host.
  of the host on which          Example port
  the process runs               numbers:
  suffice for identifying            HTTP server: 80
  the process?                       Mail server: 25
 Answer: No, many              More on this later
  processes can be running
  on same host
  Analogy: CUHK and Office #
                                             2: Application Layer   12
App-layer protocol defines
 Types of messages          Public-domain protocols:
  exchanged, eg, request      defined in RFCs
  & response messages         allows for
 Syntax of message            interoperability
  types: what fields in       eg, HTTP, SMTP
  messages & how fields      Proprietary protocols:
  are delineated              eg, KaZaA
 Semantics of the fields,   GPL protocols:
  ie, meaning of              eg, RMSS, RMSS+
  information in fields
 Rules for when and how
  processes send & respond
  to messages
                                       2: Application Layer   13
What transport service does an app need?
 Data loss                   Bandwidth
  some apps (e.g., audio)    some apps (e.g.,
   can tolerate some loss      multimedia) require
  other apps (e.g., file      minimum amount of
   transfer, telnet, E-        bandwidth to be
   banking) require 100%
   reliable data transfer
                              other apps (“elastic
  Timing                       apps”) make use of
   some apps (e.g.,           whatever bandwidth
    Internet telephony,        they get
    interactive games)
    require low delay to
    be “effective”
                                      2: Application Layer   14
   Transport service requirements of common apps

          Application Data loss           Bandwidth          Time Sensitive

          file transfer   no loss  elastic                   no
                 e-mail   no loss  elastic                   no
     Web documents        no loss  elastic                   no
real-time audio/video              audio: 5kbps-1Mbps
                          loss-tolerant                      yes, 100’s msec
  stored audio/video loss-tolerant same as above             yes, few secs
   interactive games loss-tolerant few kbps up               yes, 100’s msec
  instant messaging no loss        elastic                   yes and no

                                                      2: Application Layer   15
Internet transport protocols services
TCP service:                       UDP service:
   connection-oriented: setup      unreliable data transfer
    required between client and      between sending and
    server processes                 receiving process
   reliable transport between      does not provide:
    sending and receiving            connection setup,
    process                          reliability, flow control,
   flow control: sender won’t       congestion control,
    overwhelm receiver               timing, or bandwidth
   congestion control: throttle
    sender when network
    overloaded                     Q: why bother? Why is
   does not providing: timing,      there a UDP?
    minimum bandwidth
                                                 2: Application Layer   16
Internet apps: application, transport protocols

                         Application           Underlying
          Application    layer protocol        transport protocol

                e-mail   SMTP [RFC 2821]       TCP
     remote terminal     Telnet [RFC 854]      TCP
               access    HTTP [RFC 2616]       TCP
                  Web    FTP [RFC 959]         TCP
         file transfer   proprietary           TCP or UDP
streaming multimedia     (e.g. RealNetworks,
                         RMSS, RMSS+)
   Internet telephony    proprietary           typically UDP
                         (e.g., Dialpad)

                                                 2: Application Layer   17
Chapter 2 outline
 2.1 Principles of app       2.7 Socket
  layer protocols              programming with TCP
      clients and servers    2.8 Socket
      app requirements        programming with UDP
 2.2 Web and HTTP            2.8 Content
 2.3 FTP                      distribution
 2.4 Electronic Mail             Content distribution

 2.5 DNS
 2.6 P2P file sharing

                                             2: Application Layer   18
Web and HTTP:            1st application protocol

First some jargon
 Web page consists of objects
 Object can be HTML file, JPEG image, Java
  applet, audio file,…
 Web page consists of base HTML-file which
  includes several referenced objects
 Each object is addressable by a URL
 Example URL:

         host name            path name

                                    2: Application Layer   19
HTTP overview

HTTP: hypertext
  transfer protocol
 Web’s application layer      PC running
  protocol                      Explorer

 client/server model
    client: browser that
      requests, receives,                                   Server
      “displays” Web objects                                running
                                                          Apache Web
    server: Web server
      sends objects in
      response to requests
                                Mac running
 HTTP 1.0: RFC 1945             Navigator
 HTTP 1.1: RFC 2068

                                              2: Application Layer   20
HTTP overview (continued)
Uses TCP:                         HTTP is “stateless”
 client initiates TCP             server maintains no
  connection (creates socket)       information about past
                                    client requests (we can use
  to server, port 80 (well-         cookies for implementing state)
 server accepts TCP            Protocols that maintain
  connection from client           “state” are complex!
 HTTP messages                  past history (state) must
  (application-layer protocol      be maintained
  messages) exchanged
                                 if server/client crashes,
  between browser (HTTP
  client) and Web server           their views of “state”
  (HTTP server)                    may be inconsistent, must
 TCP connection closed            be reconciled
                                    (no no for Internet!!)

                                                  2: Application Layer   21
HTTP connections
Nonpersistent HTTP        Persistent HTTP
 At most one object is    Multiple objects can
  sent over a TCP           be sent over single
  connection.               TCP connection
 HTTP/1.0 uses             between client and
  nonpersistent HTTP        server.
                           HTTP/1.1 uses
                            persistent connections
                            in default mode

                                       2: Application Layer   22
   Non-persistent HTTP                                (contains text,
                                                     references to 10
Suppose user enters URL
                                                        jpeg images)
            CLIENT                                Server
    1a. HTTP client initiates TCP
        connection to HTTP server
        (process) at
                                      1b. HTTP server at host
                                waiting on port
                                         for TCP connection at port
                                         80. “accepts” connection,
                                         notifying client
    2. HTTP client sends HTTP
        request message (containing
        URL) into TCP connection      3. HTTP server receives
        socket. Message indicates       request message, forms
        that client wants object        response message containing
        index.html                      requested object, and sends
                                        message into its socket
                                           (we don’t send jpeg here)
                                                    2: Application Layer   23
  Nonpersistent HTTP (cont.)

                                        4. HTTP server closes TCP
     5. HTTP client receives
          response message containing
          html file, displays html.
          Parsing html file, finds 10
          referenced jpeg objects
time 6.   Steps 1-5 repeated for
          each of 10 jpeg objects

                                                   2: Application Layer   24
 Response time modeling
Definition of RRT: time to
  send a small packet to
  travel from client to
  server and back.           initiate TCP
Response time:                       RTT
 one RTT to initiate TCP        request
  connection                     file
                                                                    time to
 one RTT for HTTP                                                  transmit
  request and first few             file

  bytes of HTTP response

  to return                                 time                time
 file transmission time
total = 2RTT+transmit time
                                                   2: Application Layer   25
Persistent HTTP

Nonpersistent HTTP issues:       Persistent without pipelining:
 requires 2 RTTs per object      client issues new request
 OS must work and allocate         only when previous
   host resources for each          response has been
   TCP connection                   received
 but browsers often open         one RTT for each
   parallel TCP connections to      referenced object
   fetch referenced objects      Persistent with pipelining:
Persistent HTTP                   default in HTTP/1.1
 server leaves connection        client sends requests as
   open after sending               soon as it encounters a
   response                         referenced object
 subsequent HTTP messages        as little as one RTT for
   between same client/server       all the referenced objects
   are sent over connection
                                                 2: Application Layer   26
  HTTP request message

   two types of HTTP messages:           request, response
   HTTP request message:
     ASCII (human-readable format)

  request line
 (GET, POST,         GET /somedir/page.html HTTP/1.1
HEAD commands)       Host:
                     User-agent: Mozilla/4.0
             header Connection: close
               lines Accept-language:fr

 Carriage return,
     line feed      (extra carriage return, line feed)
  indicates end
    of message
                                                   2: Application Layer   27
    HTTP request message: general format
         (don’t memorize the packet format!!!)

End of

                                                 2: Application Layer   28
Uploading form input
Post method:
 Web page often
  includes form input     URL method:
 Input is uploaded to     Uses GET method
  server in entity body    Input is uploaded in
                            URL field of request

                                      2: Application Layer   29
Method types
HTTP/1.0                     HTTP/1.1
 GET                         GET, POST, HEAD
 POST                        PUT
 HEAD                             uploads file in entity
                                    body to path specified
     asks server to leave
                                    in URL field
      requested object out
      of response (why???)    DELETE
                                 deletes file specified
                                  in the URL field

                                             2: Application Layer   30
HTTP response message
  status line
 status code        HTTP/1.1 200 OK
status phrase)      Connection close
                    Date: Thu, 06 Aug 1998 12:00:15 GMT
                    Server: Apache/1.3.0 (Unix)
                    Last-Modified: Mon, 22 Jun 1998 …...
                    Content-Length: 6821
                    Content-Type: text/html

data, e.g.,         data data data data data ...
 HTML file

                                         2: Application Layer   31
HTTP response status codes
In first line in server->client response message.
A few sample codes:
200 OK
      request succeeded, requested object later in this message
301 Moved Permanently
      requested object moved, new location specified later in this
       message (Location:)
400 Bad Request                                           Both server
      request message not understood by server           and client
404 Not Found                                             the syntax!!
      requested document not found on this server
505 HTTP Version Not Supported
                                                2: Application Layer   32
    Trying out HTTP (client side) for yourself
   1. Telnet to your favorite Web server:

telnet 80
                       Opens TCP connection to port 80
                       (default HTTP server port) at
                       Anything typed is sent
                         to port 80 at

  2. Type in a GET HTTP request:
         GET /Index.html HTTP/1.1     By typing this in (hit carriage
                                      return twice), you send
                                      this minimal (but complete)
                                      GET request to HTTP server

  3. Look at response message sent by HTTP server!

                                                     2: Application Layer   33
     User-server interaction: authorization
Authorization : control access to
  server content                   client                     server
 authorization credentials:            usual http request msg
  typically name, password               401: authorization req.
 stateless: client must present         WWW authenticate:
  authorization in each request
    authorization: header line in      usual http request msg
     each request                       + Authorization: <cred>
    if no authorization: header,
                                        usual http response msg
     server refuses access,
        WWW authenticate:                  usual http request msg
                                           + Authorization: <cred>
        header line in response
Browser caches name & password (in clear   usual http response msg
text) so that user does not have to
repeatedly enter it.                                    2: Application Layer   34
Cookies: keeping “state”                 (just a bit pattern)

Many major Web sites
  use cookies                Example:
Four components:                  Susan access Internet
  1) cookie header line in         always from same PC
    the HTTP response             She visits a specific
    message (from server)          e-commerce site for
  2) cookie header line in         first time
    HTTP request message          When initial HTTP
    (from client or                requests arrives at
    browser)                       site, site creates a
  3) cookie file kept on           unique ID and creates
    user’s host and                an entry in backend
    managed by user’s              database for ID
  4) back-end database at
    Web site                                2: Application Layer   35
   Cookies: keeping “state” (cont.)

                   client                        server
     Cookie file        usual http request msg      server
                         usual http response +    creates ID
  ebay: 8734            Set-cookie: 1678         1678 for user

     Cookie file
                        usual http request msg
 amazon: 1678               cookie: 1678           cookie-
 ebay: 8734                                        specific
                       usual http response msg      action
one week later:
                        usual http request msg
     Cookie file                                    cookie-
                            cookie: 1678
  amazon: 1678                                     spectific
  ebay: 8734
                       usual http response msg       action

                                                       2: Application Layer   36
Cookies (continued)
What cookies can bring:   Cookies and privacy:
 authorization            cookies permit sites
 shopping carts            to learn a lot about
 recommendations           you
 user session state       you may supply name
  (Web e-mail)              and e-mail to sites
                           search engines use
                            redirection & cookies
                            to learn yet more
                           advertising
                            companies obtain
                            info across sites
                                    2: Application Layer   37
 Web caches (proxy server)
 Goal: satisfy client request without involving origin server
 user sets browser: Web
  accesses via cache                                              origin
 browser sends all HTTP
  requests to cache                           Proxy
       object in cache: cache                server
        returns object
       else cache requests
        object from origin
        server, then returns
        object to client

a) Replacement algorithms.                                        origin
b) Short vs long documents
c) Security !!
                                                 2: Application Layer   38
More about Web caching
 Cache acts as both client       Why Web caching?
  and server                       Reduce response time for
 Cache can do up-to-date           client request.
  check using If-
                                   Reduce traffic on an
  modified-since HTTP
                                    institution’s access link.
                                   Internet dense with
      Issue: should cache take
       risk and deliver cached
                                    caches enables “poor”
       object without checking?     content providers to
      Heuristics are used.
                                    effectively deliver content
 Typically cache is
  installed by ISP
  (university, company,
  residential ISP)

                                                 2: Application Layer   39
Caching example (1)
 average object size =
  100,000 bits                                    public
 avg. request rate from
  institution’s browser to origin
  serves = 15/sec
                                                       1.5 Mbps
 delay from institutional
                                                       access link
  router to any origin server
  and back to router = 2 sec
                                                            10 Mbps LAN
 utilization on LAN = 15%
 utilization on access link = 100%
 total delay  = Internet delay +                             institutional
  access delay + LAN delay                                       cache
 = 2 sec + minutes + milliseconds
       (due to high utilization)                        2: Application Layer   40
Caching example (2)
Possible solution
 increase bandwidth of access
   link to, say, 10 Mbps (this                   public
   implies that you pay for the

                                                      10 Mbps
                                                      access link
 utilization on LAN = 15%
 utilization on access link = 15%     network
                                                           10 Mbps LAN
 Total delay   = Internet delay
  + access delay + LAN delay
 = 2 sec + msecs + msecs
 often a costly upgrade

                                                       2: Application Layer   41
Caching example (3)
Install cache                                                        origin
 suppose hit rate is .4                                           servers
Consequence                                    Internet

 40% requests will be satisfied
  almost immediately
 60% requests satisfied by                          1.5 Mbps
  origin server                                      access link
 utilization of access link        institutional
  reduced to 60%, resulting in        network
                                                          10 Mbps LAN
  negligible delays (say 10
 total delay    = Internet delay
  + access delay + LAN delay                                institutional
 = .6*2 sec + .6*.01 secs +                                    cache
  milliseconds < 1.3 secs

                                                      2: Application Layer   42
  Conditional GET: client-side caching

 Goal: don’t send object if    client                           server
  client has up-to-date                HTTP request msg
  cached version                     If-modified-since:
 client: specify date of                  <date>                     not
  cached copy in HTTP                                               modified
  request                                  HTTP response
   If-modified-since:                    304 Not Modified
 server: response contains
                                         HTTP request msg
  no object if cached copy is            If-modified-since:
  up-to-date:                                  <date>                object
   HTTP/1.0 304 Not                                                  modified
     Modified                              HTTP response
                                          HTTP/1.0 200 OK
                                                       2: Application Layer   43
Chapter 2 outline
 2.1 Principles of app       2.7 Socket
  layer protocols              programming with TCP
      clients and servers    2.8 Socket
      app requirements        programming with UDP
 2.2 Web and HTTP            2.9 Content
 2.3 FTP                      distribution
 2.4 Electronic Mail             Content distribution

 2.5 DNS
 2.6 P2P file sharing

                                             2: Application Layer   44
FTP: the file transfer protocol

                   FTP                   file transfer
                           FTP                            FTP
                   user   client                         server
     at host                local file                            remote file
                            system                                system

   transfer file to/from remote host
   client/server model
        client: side that initiates transfer (either to/from
        server: remote host
   ftp: RFC 959
   ftp server: port 21

                                                         2: Application Layer   45
    FTP: separate control, data connections
 FTP client contacts FTP                   TCP control connection
    server at port 21,                             port 21
    specifying TCP as transport
    protocol (why TCP?)                     TCP data connection
   Client obtains authorization    FTP          port 20       FTP
    over control connection        client                     server
   Client browses remote           Server opens a second TCP
    directory by sending             data connection to transfer
    commands over control            another file.
    connection.                     Control connection: “out of
   When server receives a           band” (e.g., ls, dir, cd,…etc)
    command for a file              FTP server maintains
    transfer, the server opens       “state”: current directory,
    a TCP data connection to         earlier authentication (unlike
    client                           http which doesn’t maintain
   After transferring one file,     state)
    server closes connection.
                                                    2: Application Layer   46
FTP commands, responses

Sample commands:                Sample return codes
 sent as ASCII text over        status code and phrase
  control channel                   (as in HTTP)
 USER username                    331 Username OK,
 PASS password                     password required
 LIST return list of file in
                                   125 data connection
                                    already open;
  current directory
                                    transfer starting
 RETR filename retrieves          425 Can’t open data
  (gets) file                       connection
 STOR filename stores             452 Error writing
  (puts) file onto remote           file
  host (ftp happened BEFORE

                                              2: Application Layer   47
Chapter 2 outline
 2.1 Principles of app       2.7 Socket
  layer protocols              programming with TCP
      clients and servers    2.8 Socket
      app requirements        programming with UDP
 2.2 Web and HTTP            2.9 Content
 2.3 FTP                      distribution
 2.4 Electronic Mail             Content distribution

 2.5 DNS
 2.6 P2P file sharing

                                             2: Application Layer   48
  Electronic Mail                                                 outgoing
                                                             message queue

Three major components:                                        user mailbox
 user agents                                agent
 mail servers                       mail
 simple mail transfer protocol:    server
  SMTP                                        SMTP         mail
                                                          server       user
User Agent
 a.k.a. “mail reader”
                                   SMTP                               agent

 composing, editing, reading                 SMTP
  mail messages                      mail                           user
                                    server                         agent
 e.g., Eudora, Outlook, elm,
  pine, Netscape Messenger                     user
 outgoing, incoming messages                 agent
  stored on server                    user

                                                      2: Application Layer    49
Electronic Mail: mail servers
Mail Servers                              agent
 mailbox contains incoming       mail
  messages for user              server
 message queue of outgoing
  (to be sent) mail messages                               mail
                                                          server          user
 SMTP protocol between
  mail servers to send email    SMTP                                     agent

  messages                                 SMTP
    client: sending mail         mail                              user
      server                     server
    “server”: receiving mail
      server                               agent

                                                  2: Application Layer     50
Electronic Mail: SMTP [RFC 2821]

 uses TCP to reliably transfer email message from
  client to server, port 25
 direct transfer: sending server to receiving server
 three phases of transfer
    handshaking (greeting)
    transfer of messages
    closure
 command/response interaction
    commands: ASCII text
    response: status code and phrase

 messages must be in 7-bit ASCII (problem
  when we send multimedia files)

                                               2: Application Layer   51
Scenario: Alice sends message to Bob
 1) Alice uses pine to           4) SMTP client sends Alice’s
    compose message and “to”        message over the TCP                 connection
 2) Alice’s pine sends message   5) Bob’s mail server places
    to her mail server;             the message in Bob’s
    message placed in
    message queue                   mailbox
 3) Client side of SMTP opens    6) Bob invokes his user
    TCP connection with Bob’s       agent to read message
    mail server

        1                            mail
                                    server             user
       user        server
              2                                       agent
      agent          3                        6
                             4         5

                                                  2: Application Layer   52
Sample SMTP interaction                      (s=server; c=client)

S:   220
C:   HELO (handshaking phase)
S:   250 Hello, pleased to meet you
C:   MAIL FROM: <>
S:   250 Sender ok
C:   RCPT TO: <>
S:   250 ... Recipient ok
C:   DATA (this is a command!! About to send data)
S:   354 Enter mail, end with "." on a line by itself
C:   Do you like ketchup?
C:     How about pickles?
C:   . (this is to signal the end of a message)
S:   250 Message accepted for delivery
C:   QUIT (a command; client can go back to send another message)
S:   221 closing connection

                                               2: Application Layer   53
Try SMTP interaction for yourself:

 telnet servername 25
 see 220 reply from server
  QUIT commands
above lets you send email without using email
  client (reader)

Word of warning: don’t spoof mail of other

                                       2: Application Layer   54
SMTP: final words
 SMTP uses persistent         Comparison with HTTP:
                                HTTP: pull model
 SMTP requires message
  (header & body) to be in      SMTP: push model
  7-bit ASCII (done in          both have ASCII
  1983)!!                        command/response
 SMTP server uses               interaction, status codes
  CRLF.CRLF to determine
  end of message                HTTP: each object
 Therefore, message has         encapsulated in its own
  to be encoded (usually         response msg (e.g, gif,
  into either base-64 or         jpeg)
  quoted printable). Discuss    SMTP: multiple objects
  later.                         sent in a multipart msg

                                              2: Application Layer   55
Mail message format                    Define what is a
                                       message. In other
                                       words, the data
SMTP: protocol for exchanging          format
  email messages
RFC 822: standard for text             header
  message format:                                                  line
 header lines, e.g.,
      To:
      From: (????????)                body
      Subject:
   different from SMTP
 body
      the “message”, ASCII
       characters only
      This doesn’t work for
       multimedia (e.g, video/audio)
                                           2: Application Layer     56
   Message format: multimedia extensions
    MIME: multimedia mail extension, RFC 2045 (audio),
     2056 (video)
    additional lines in msg header declare MIME content
       MIME version          To:
                             Subject: Picture of yummy crepe.
        method used          MIME-Version: 1.0
     to encode data          Content-Transfer-Encoding: base64
                             Content-Type: image/jpeg
     multimedia data
      type, subtype,         base64 encoded data .....
parameter declaration        .........................
                             ......base64 encoded data
        encoded data

                                                 2: Application Layer   57
MIME types
Content-Type: type/subtype; parameters

Text                        Video
 example subtypes:          example subtypes: mpeg,
  plain, html                 quicktime

Image                       Application
 example subtypes: jpeg,    other data that must be
  gif                         processed by reader
                              before “viewable”
Audio                        example subtypes:
 exampe subtypes: basic
                              msword, octet-stream
                              (pdf format)
  (8-bit mu-law encoded),
  32kadpcm (32 kbps
                                          2: Application Layer   58
Multipart Type
 Subject: Picture of yummy crepe.
 MIME-Version: 1.0
 Content-Type: multipart/mixed; boundary=StartOfNextPart

 --StartOfNextPart                                         (delimiter)
 Dear Bob, Please find a picture of a crepe.
 Content-Transfer-Encoding: base64
 Content-Type: image/jpeg
 base64 encoded data .....
 ......base64 encoded data
 Do you want the reciple?

  Sequentially put all different message types in one email.

                                                   2: Application Layer   59
Mail access protocols
                SMTP         SMTP               access      user
        agent                                  protocol    agent

                 sender’s mail   receiver’s mail
                    server           server
 SMTP: delivery/storage to receiver’s server
 Mail access protocol: retrieval from server
      POP: Post Office Protocol [RFC 1939]
        • authorization (agent <-->server) and download
          (provide userid and password)
      IMAP: Internet Mail Access Protocol [RFC 1730]
        • more features (more complex)
        • manipulation of stored msgs on server
      HTTP: Hotmail , Yahoo! Mail, etc.
                                                     2: Application Layer   60
                              (before this, we need to open
                              a TCP connection to port 110)
POP3 protocol                   S:   +OK POP3 server ready
                                C:   user bob
authorization phase             S:   +OK
                                C:   pass hungry
 client commands:              S:   +OK user successfully logged     on
    user: declare username
                                C:   list
    pass: password             S:   1 498
                                                2 messages
 server responses              S:   2 912
                                S:   .
    +OK
                                C:   retr 1
    -ERR                       S:   <message 1 contents>
transaction phase, client:      S:   .
                                C:   dele 1     Delete message 1
 list: list message            C:   retr 2     On server
  numbers                       S:   <message 2 contents>
 retr: retrieve message by     S:   .
  number                        C:   dele 2
                                C:   quit
 dele: delete                  S:   +OK POP3 server signing off
 quit
                                              2: Application Layer   61
POP3 (more) and IMAP
More about POP3           IMAP
 Previous example uses    Keep all messages in
  “download and delete”     one place: the server
  mode.                    Allows user to
 Bob cannot re-read        organize messages in
  e-mail if he changes      folders
  client                   IMAP keeps user
 “Download-and-            state across sessions:
  keep”: copies of              names of folders and
  messages on                    mappings between
  different clients              message IDs and
                                 folder name
 POP3 is stateless
  across sessions
                                          2: Application Layer   62
Chapter 2 outline
 2.1 Principles of app       2.7 Socket
  layer protocols              programming with TCP
      clients and servers    2.8 Socket
      app requirements        programming with UDP
 2.2 Web and HTTP            2.9 Content
 2.3 FTP                      distribution
 2.4 Electronic Mail             Content distribution

 2.5 DNS
 2.6 P2P file sharing

                                             2: Application Layer   63
    How to decide a name?

 DNS: Domain Name System
People: many identifiers:           Domain Name System:
      HKID#, name, passport           distributed database
       #, CUHK ID #...etc             implemented in hierarchy of
Internet hosts, routers:              many name servers
      IP address (32 bit) -         application-layer protocol: host,
       used for addressing            routers, name servers to
       datagrams                      communicate to resolve names
                                      (address/name translation)
      “name”, e.g.,
                                        note: core Internet function,
       - used by humans                  implemented as application-
                                         layer protocol
Q: map between IP                       complexity at network’s
  addresses and name ?                   “edge”
                                    Very important core function. “Edge”
 DNS has to be a global             philosophy allows extendibility. New
 infrastructure, a                  applications spread like wildfire!!
 distributed database!!                                   2: Application Layer   64
DNS services                 Why not centralize DNS?
 Hostname to IP              single point of failure
  address translation         traffic volume
 Host aliasing               distant centralized
      Canonical and alias     database
       names                  maintenance
 Mail server aliasing
 Load distribution
                             doesn’t scale!
    Replicated Web
     servers: set of IP
     addresses for one
     canonical name

                                              2: Application Layer   65
Distributed, Hierarchical Database
                          Root DNS Servers

   com DNS servers        org DNS servers     edu DNS servers

                             DNS servers     DNS serversDNS servers
DNS servers DNS servers

Client wants IP for; 1st approx:
 Client queries a root server to find com DNS
 Client queries com DNS server to get
   DNS server
 Client queries DNS server to get IP
   address for
                                                2: Application Layer   66
     DNS: Root name servers
   contacted by local name server that can not resolve name
   root name server:
               contacts authoritative name server if name mapping not
               gets mapping
               returns mapping to local name server
                                    a NSI Herndon, VA
                                    c PSInet Herndon, VA            k RIPE London
                                    d U Maryland College Park, MD   i NORDUnet Stockholm
                                    g DISA Vienna, VA
                                    h ARL Aberdeen, MD
                                    j NSI (TBD) Herndon, VA                    m WIDE Tokyo

e NASA Mt View, CA
f Internet Software C. Palo Alto,

                                                                                              13 root name
     b USC-ISI Marina del Rey, CA
                                                                                              servers worldwide
     l ICANN Marina del Rey, CA

                                                                                              2: Application Layer   67
TLD and Authoritative Servers
 Top-level domain (TLD) servers: responsible
  for com, org, net, edu, etc, and all top-level
  country domains uk, fr, ca, jp.
   Network solutions maintains servers for com TLD
   Educause for edu TLD
 Authoritative DNS servers: organization’s
  DNS servers, providing authoritative
  hostname to IP mappings for organization’s
  servers (e.g., Web and mail).
   Can   be maintained by organization or service

                                          2: Application Layer   68
Local Name Server
 Does not strictly belong to hierarchy
 Each ISP (residential ISP, company,
 university) has one.
     Also called “default name server”
 When a host makes a DNS query, query
 is sent to its local DNS server
     Acts as a proxy, forwards query into

                                          2: Application Layer   69
Example                                        root DNS

 Host at
                                           3             TLD DNS
  wants IP address                             4          server
  for                             5

                         local DNS server
                                               7         6
                                1    8

                                               authoritative DNS server
                         requesting host


                                                    2: Application Layer   70
Recursive queries                      root DNS server

recursive query:
                                  2                    3
 puts burden of name
  resolution on                        7       6
  contacted name                                               TLD DNS server
 heavy load?
                      local DNS server
iterated query:               5

 contacted server           1    8
  replies with name of
  server to contact                        authoritative DNS server
 “I don’t know this
                     requesting host
  name, but ask this
                                                       2: Application Layer   71
DNS: caching and updating records

 once (any) name server learns mapping, it
  caches mapping
    cache entries timeout (disappear) after
     some time
    TLD servers typically cached in local name
        • Thus root name servers not often visited
 update/notify mechanisms under design by
      RFC 2136

                                                  2: Application Layer   72
DNS records
DNS: distributed db storing resource records (RR)
         RR format:    (name, value, type, ttl)

 Type=A                    Type=CNAME
    name is hostname          name is alias name for some
    value is IP address        “cannonical” (the real) name
                       is really
 Type=NS
    name is domain (e.g.
                               value is cannonical name
    value is IP address of
                             Type=MX (why we need this?)
     authoritative name
                                value is name of
     server for this domain
                                 mailserver associated with
                                            2: Application Layer   73
DNS protocol, messages
DNS protocol : query and reply messages, both
 with same message format

msg header
 identification: 16 bit #
  for query, reply to
  query uses same # (why
  do we need this ID?)
 flags:
    query or reply
    recursion desired
    recursion available
    reply is authoritative

                                     2: Application Layer   74
  DNS protocol, messages

    Name, type fields
         for a query

        RRs in reponse
              to query

          records for
 authoritative servers

    additional “helpful”
info that may be used

                           2: Application Layer   75
Inserting records into DNS
 Example: just created startup “Network Utopia”
 Register name at a registrar
  (e.g., Network Solutions)
      Need to provide registrar with names and IP addresses
       of your authoritative name server (primary and
      Registrar inserts two RRs into the com TLD server:

   (,, NS)
   (,, A)

 Put in authoritative server Type A record for and Type MX record for
 How do people get the IP address of your Web
                                              2: Application Layer   76
Chapter 2 outline
 2.1 Principles of app       2.7 Socket
  layer protocols              programming with TCP
      clients and servers    2.8 Socket
      app requirements        programming with UDP
 2.2 Web and HTTP            2.9 Content
 2.3 FTP                      distribution
 2.4 Electronic Mail             Content distribution

 2.5 DNS
 2.6 P2P file sharing

                                             2: Application Layer   77
P2P file sharing           Alice chooses one of
                            the peers, Bob.
Example                    File is copied from
 Alice runs P2P client     Bob’s PC to Alice’s
  application on her        notebook: HTTP
  notebook computer        While Alice
 Intermittently            downloads, other
  connects to Internet;     users uploading from
  gets new IP address       Alice.
  for each connection      Alice’s peer is both a
 Asks for “Hey Jude”       Web client and a
 Application displays      transient Web
  other peers that have     server.
  copy of Hey Jude.       All peers are servers =
                            highly scalable!
                                      2: Application Layer   78
  P2P: centralized directory
original “Napster” design                                                Bob
1) when peer connects, it   directory server
  informs central server:                                              peers
      IP address                                  1
      content
2) Alice queries for “Hey                          1

  Jude”                                  2     1

3) Alice requests file
  from Bob


                                                       2: Application Layer    79
P2P: problems with centralized directory

 Single point of failure
 Performance bottleneck
 Copyright infringement

                              file transfer is
                            decentralized, but
                            locating content is
                            highly centralized

                                      2: Application Layer   80
Query flooding: Gnutella
 fully distributed        overlay network: graph
    no central server      edge between peer X
 public domain protocol     and Y if there’s a
 many Gnutella clients      TCP connection
  implementing protocol     all active peers and
                             edges is overlay net
                            Edge is not a physical
                            Given peer will
                             typically be
                             connected with < 10
                             overlay neighbors

                                        2: Application Layer   81
Gnutella: protocol
                              File transfer:
 Query message               HTTP
sent over existing TCP
 peers forward
Query message
 QueryHit
sent over

limited scope
                                2: Application Layer   82
Gnutella: Peer joining
1. Joining peer X must find some other peer in
   Gnutella network: use list of candidate peers
2. X sequentially attempts to make TCP with
   peers on list until connection setup with Y
3. X sends Ping message to Y; Y forwards Ping
4. All peers receiving Ping message respond with
   Pong message
5. X receives many Pong messages. It can then
   setup additional TCP connections
Peer leaving: see homework problem!

                                       2: Application Layer   83
Exploiting heterogeneity: KaZaA

 Each peer is either a
  group leader or
  assigned to a group
      TCP connection between
       peer and its group
      TCP connections between
       some pairs of group
 Group leader tracks
                                 ordinary peer

  the content in all its
                                 group-leader peer

                                 neighoring relationships

  children.                         in overlay network

                                          2: Application Layer   84
KaZaA: Querying
 Each file has a hash and a descriptor
 Client sends keyword query to its group
 Group leader responds with matches:
     For each match: metadata, hash, IP address
 If group leader forwards query to other
  group leaders, they respond with
 Client then selects files for downloading
     HTTP requests using hash as identifier sent
      to peers holding desired file
                                        2: Application Layer   85
Kazaa tricks
 Limitations on simultaneous uploads
 Request queuing
 Incentive priorities
 Parallel downloading

                                  2: Application Layer   86
Chapter 2 outline
 2.1 Principles of app       2.7 Socket
  layer protocols              programming with TCP
      clients and servers    2.8 Socket
      app requirements        programming with UDP
 2.2 Web and HTTP            2.9 Content
 2.3 FTP                      distribution
 2.4 Electronic Mail             Content distribution

 2.5 DNS
 2.6 P2P file sharing

                                             2: Application Layer   87
Socket programming
Goal: learn how to build client/server application that
  communicate using sockets

Socket API                        socket
 introduced in BSD4.1 UNIX,
                                       a host-local,
  1981                             application-created,
 explicitly created, used,       OS-controlled interface
  released by apps                  (a “door”) into which
 client/server paradigm           application process can
 two types of transport                both send and
  service via socket API:        receive messages to/from
                                     another application
    unreliable datagram
    reliable, byte stream-

                                           2: Application Layer   88
Socket-programming using TCP
 Socket: a door between application process and end-
   end-transport protocol (UCP or TCP)
 TCP service: reliable transfer of bytes from one
   process to another

                                                    controlled by
controlled by                          process      application
  application   process
                 socket                 socket
                TCP with               TCP with     controlled by
controlled by
                                       buffers,     operating
   operating    buffers,    internet                system
      system    variables              variables

                host or                host or
                server                 server

                                             2: Application Layer   89
Socket programming with TCP
Client must contact server       When contacted by client,
 server process must first       server TCP creates new
   be running                     socket for server process to
 server must have created        communicate with client
   socket (door) that               allows server to talk with
   welcomes client’s contact         multiple clients
                                    source port numbers
Client contacts server by:
                                     used to distinguish
 creating client-local TCP
                                     clients (more in Chap 3)
 specifying IP address, port    application viewpoint
   number of server process
                                  TCP provides reliable, in-order
 When client creates
                                    transfer of bytes (“pipe”)
   socket: client TCP
                                    between client and server
   establishes connection to
   server TCP
                                                2: Application Layer   90
Stream jargon
 A stream is a sequence of
  characters that flow into
  or out of a process.
 An input stream is
  attached to some input
  source for the process, eg,
  keyboard or socket.
 An output stream is
  attached to an output
  source, eg, monitor or

                                2: Application Layer   91
Socket programming with TCP
                                                keyboard            monitor
Example client-server app:
1) client reads line from
   standard input (inFromUser

   stream) , sends to server via   Client
   socket (outToServer             Process
2) server reads line from socket
3) server converts line to
   uppercase, sends back to

                                         output                                        input
                                         stream                                       stream

4) client reads, prints modified
   line from socket                              client TCP

   (inFromServer stream)                           socket                                 TCP

                                               to network           from network

                                                 2: Application Layer                             92
Client/server socket interaction: TCP
Server (running on hostid)                    Client
     create socket,
     port=x, for
     incoming request:
     welcomeSocket =
                             TCP             create socket,
     wait for incoming
     connection request connection   setup   connect to hostid, port=x
     connectionSocket =                      clientSocket =
     welcomeSocket.accept()                         Socket()

                                               send request using
     read request from                         clientSocket

      write reply to
      connectionSocket                         read reply from
      connectionSocket                          close
                                                               2: Application Layer   93
    Example: Java client (TCP)
                     class TCPClient {

                       public static void main(String argv[]) throws Exception
                         String sentence;
                         String modifiedSentence;
      input stream        BufferedReader inFromUser =
                           new BufferedReader(new InputStreamReader(;
    client socket,        Socket clientSocket = new Socket("hostname", 6789);
 connect to server
            Create        DataOutputStream outToServer =
     output stream         new DataOutputStream(clientSocket.getOutputStream());
attached to socket
                                                               2: Application Layer   94
  Example: Java client (TCP), cont.

            Create            BufferedReader inFromServer =
      input stream             new BufferedReader(new
attached to socket             InputStreamReader(clientSocket.getInputStream()));

                              sentence = inFromUser.readLine();
          Send line
          to server           outToServer.writeBytes(sentence + '\n');

          Read line           modifiedSentence = inFromServer.readLine();
       from server
                              System.out.println("FROM SERVER: " + modifiedSentence);


                                                                2: Application Layer   95
   Example: Java server (TCP)

                      class TCPServer {

                       public static void main(String argv[]) throws Exception
                          String clientSentence;
           Create         String capitalizedSentence;
 welcoming socket
                          ServerSocket welcomeSocket = new ServerSocket(6789);
     at port 6789
                          while(true) {
Wait, on welcoming
socket for contact            Socket connectionSocket = welcomeSocket.accept();
          by client
                             BufferedReader inFromClient =
     Create input             new BufferedReader(new
stream, attached              InputStreamReader(connectionSocket.getInputStream()));
        to socket

                                                                  2: Application Layer   96
  Example: Java server (TCP), cont

   Create output
stream, attached           DataOutputStream outToClient =
       to socket            new DataOutputStream(connectionSocket.getOutputStream());
    Read in line
    from socket            clientSentence = inFromClient.readLine();

                           capitalizedSentence = clientSentence.toUpperCase() + '\n';
  Write out line
      to socket
              }                  End of while loop,
                                 loop back and wait for
                                 another client connection

                                                                 2: Application Layer   97
Chapter 2 outline
 2.1 Principles of app       2.7 Socket
  layer protocols              programming with TCP
      clients and servers    2.8 Socket
      app requirements        programming with UDP
 2.2 Web and HTTP            2.9 Content
 2.3 FTP                      distribution
 2.4 Electronic Mail             Content distribution

 2.5 DNS
 2.6 P2P file sharing

                                             2: Application Layer   98
Socket programming with UDP

UDP: no “connection” between
  client and server
 no handshaking
 sender explicitly attaches   application viewpoint
  IP address and port of
  destination to each packet   UDP provides unreliable transfer
                               of groups of bytes (“datagrams”)
 server must extract IP
                                   between client and server
  address, port of sender
  from received packet
UDP: transmitted data may be
  received out of order, or

                                              2: Application Layer   99
Client/server socket interaction: UDP
Server (running on hostid)   Client

     create socket,          create socket,
     port=x, for             clientSocket =
     incoming request:       DatagramSocket()
     serverSocket =
                             Create, address (hostid, port=x,
                             send datagram request
                             using clientSocket
      read request from

      write reply to
      specifying client        read reply from
      host address,            clientSocket
      port number              close

                                          2: Application Layer   100
Example: Java client (UDP)
                              keyboard           monitor


                                                                                Input: receives
                                                                                packet (TCP
    Output: sends                                                               received “byte
    packet (TCP sent                                                            stream”)

    “byte stream”)         UDP                                       UDP
                          packet                                    packet

                                   client UDP
                                     socket                             UDP

                              to network         from network

                                                                                     2: Application Layer   101
    Example: Java client (UDP)

                      class UDPClient {
                         public static void main(String args[]) throws Exception
      input stream        BufferedReader inFromUser =
                           new BufferedReader(new InputStreamReader(;
      client socket       DatagramSocket clientSocket = new DatagramSocket();
                          InetAddress IPAddress = InetAddress.getByName("hostname");
   hostname to IP
address using DNS         byte[] sendData = new byte[1024];
                          byte[] receiveData = new byte[1024];

                          String sentence = inFromUser.readLine();
                          sendData = sentence.getBytes();
                                                                       2: Application Layer   102
     Example: Java client (UDP), cont.
    Create datagram
  with data-to-send,     DatagramPacket sendPacket =
length, IP addr, port     new DatagramPacket(sendData, sendData.length, IPAddress, 9876);

    Send datagram        clientSocket.send(sendPacket);
         to server
                         DatagramPacket receivePacket =
                          new DatagramPacket(receiveData, receiveData.length);
    Read datagram
      from server
                         String modifiedSentence =
                            new String(receivePacket.getData());

                         System.out.println("FROM SERVER:" + modifiedSentence);

                                                                   2: Application Layer   103
  Example: Java server (UDP)

                     class UDPServer {
                      public static void main(String args[]) throws Exception
          Create        {
 datagram socket
                         DatagramSocket serverSocket = new DatagramSocket(9876);
    at port 9876
                         byte[] receiveData = new byte[1024];
                         byte[] sendData = new byte[1024];

  Create space for
                           DatagramPacket receivePacket =
received datagram
                            new DatagramPacket(receiveData, receiveData.length);
           Receive          serverSocket.receive(receivePacket);
                                                                 2: Application Layer   104
   Example: Java server (UDP), cont
                            String sentence = new String(receivePacket.getData());
      Get IP addr
                            InetAddress IPAddress = receivePacket.getAddress();
       port #, of
           sender           int port = receivePacket.getPort();

                                   String capitalizedSentence = sentence.toUpperCase();

                            sendData = capitalizedSentence.getBytes();
Create datagram
                            DatagramPacket sendPacket =
to send to client            new DatagramPacket(sendData, sendData.length, IPAddress,
      Write out
       datagram             serverSocket.send(sendPacket);
       to socket        }
                }                    End of while loop,
                                     loop back and wait for
                                     another datagram
                                                                         2: Application Layer   105
Building a simple Web server
 handles one HTTP             after creating server,
    request                     you can request file
   accepts the request         using a browser (eg IE
   parses header
                               see text for details
   obtains requested file
    from server’s file
   creates HTTP response
       header lines + file
 sends response to client

                                          2: Application Layer   106
  Socket programming: references
C-language tutorial (audio/slides):
 “Unix Network Programming” (J. Kurose),

 “All About Sockets” (Sun tutorial),
 “Socket Programming in Java: a tutorial,”

                                        2: Application Layer   107
Chapter 2 outline
 2.1 Principles of app       2.7 Socket
  layer protocols              programming with TCP
      clients and servers    2.8 Socket
      app requirements        programming with UDP
 2.2 Web and HTTP            2.9 Content
 2.3 FTP                      distribution
 2.4 Electronic Mail             Content distribution

 2.5 DNS
 2.6 P2P file sharing

                                             2: Application Layer   108
  Content distribution networks (CDNs)
                                             origin server
 The content providers are
                                             in North America
  the CDN customers.
Content replication
 CDN company installs
  hundreds of CDN servers
                                           CDN distribution node
  throughout Internet
    in lower-tier ISPs, close
     to users
 CDN replicates its
  customers’ content in CDN
  servers. When provider
  updates content, CDN           CDN server                      CDN server
  updates servers                in S. America   CDN server      in Asia
                                                 in Europe

                                                     2: Application Layer   109
                                            HTTP request for

CDN example

                           1       Origin server

                           2                DNS query for
                               CDNs authoritative
                           3      DNS server

                                            HTTP request for
origin server                      CDN server      CDN company
                                      
 distributes HTML                                  distributes gif files
 Replaces:                                         uses its authoritative                DNS server to route
    with                                             redirect requests

                                                                   2: Application Layer   110
More about CDNs
routing requests             not just Web pages
 CDN creates a               streaming stored
  “map”, indicating            audio/video
  distances from leaf         streaming real-time
  ISPs and CDN nodes           audio/video
 when query arrives at           CDN nodes create
  authoritative DNS                application-layer
  server:                          overlay network
      server determines
      ISP from which query   If you thought of this around
      originates             1997 or 1998, you are a rich
     uses “map” to          person by now !!!!!!
      determine best CDN
                                            2: Application Layer   111
 Chapter 2: Summary
 Our study of network apps now complete!
 Application architectures             specific protocols:
    client-server                         HTTP
    P2P                                   FTP
    hybrid                                SMTP, POP, IMAP
 application service                      DNS
  requirements:                         socket programming
      reliability, bandwidth, delay
 Internet transport service
      connection-oriented,
       reliable: TCP
      unreliable, datagrams: UDP

                                                 2: Application Layer   112
Chapter 2: Summary
Most importantly: learned about protocols

 typical request/reply
                                  control vs. data msgs
  message exchange:
                                        in-band, out-of-band
      client requests info or
                                    centralized vs.
      server responds with
       data, status code
                                    stateless vs. stateful
                                    reliable vs. unreliable msg
 message formats:                   transfer
   headers: fields giving
                                    “complexity at network
    info about data                  edge”
   data: info being

                                                 2: Application Layer   113

To top