Applications The

Document Sample
Applications The Powered By Docstoc
					                                                                   The application layer
                The application layer                                                                                                    Clients, servers, peers
                                                                   The TCP/IP application layer contains protocols that enable
                                                                   applications to communicate.                                          Computers connected to the Internet are end-systems or hosts
                 Olof Hagsand, NADA/KTH
                                        The TCP/IP application layer roughly maps to three OSI layers:        (they “host” application programs running on them). Hosts are
                                                                                                                                         traditionally divided into clients and servers - the difference
                                                                    • Session: session establishment, dialog control, synchronization
                       January 20, 2005                                                                                                  nowadays unclear.
                                                                    • Presentation: syntax and semantics of data: higher level data
                                                                                                                                         But from a program point of view, it is easier:
                                                                                                                                          • Client program - requests a service.
                                                                    • Application: application-specific information and protocols
                                                                                                                                          • Server program - provides a service.
                                                                   From its UNIX implementation roots, the definition of the the
                                                                   application layer is: everything that is implemented in user space!    • Peer - bot a client and a server program.
                                                                   (not in the UNIX kernel).

                                                                                                    1                                                                     2

The Socket Interface
                                                                   Protocol message formats
The socket interface is used for programming applications with a
network component.                                                 When you transfer information from one host to another, they need
                                                                                                                                         Approach 1: Binary fixed fields
                                                                   to to understand each other’s data. (Presentation layer)
Sometimes called BSD sockets - it was first implemented in C in
BSD.                                                               Protocol messages are designed in different ways, some issues:         Most common in the underlying layers of the TCP/IP stack.
Variants exist for most programming languages.                      • Performance - compact data for faster transmittal, easy to         Examples: DNS, RIP, OSPFv2, BGP, RTP
Winsock is almost the same but not quite!                             parse by a computer.                                               Predefines exactly what information is to be where in the message.
Other programming interfaces include:                               • Readability - Easy to read by humans: debugging, surveillance,     The semantics is hard-coded into the application.
 • Streams                                                                                                                               And its binary
                                                                    • Common character sets - different languages, coding.
 • Remote Procedure Calls (RPC)
                                                                    • Alignment and byte ordering - Different CPU characteristics.
The sockets API is a de facto standard for network programming.

                                3                                                                   4                                                                     5
                                                                                                                                        Pros & Cons
                                                                   Example: DNS                                                         When you feel the urge to design a [...]               complex binary
Binary fixed fields (cont)
                                                                                                                                        application protocol, it is generally wise to lie down until
                                                                   The DNS header, taken from RFC 1035.
                                                                                                                                        the feeling passes
Requires common alignment (ie on 16, 32 or 64 -bit boundaries)                                              1 1 1 1 1 1
                                                                                                                                        Eric Raymond:   ‘‘The Art of UNIX Programming’’
                                                                             0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5
Requires byte-swapping: How the CPU loads its registers from               +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
memory.                                                                    |                      ID                       |
                                                                           +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+            Pros:
Two variants:                                                              |QR|   Opcode |AA|TC|RD|RA|    Z    |   RCODE   |
                                                                           +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+               • Compact: Efficient computer processing
 • Little endian (eg Intel): LSB (Least Significant Byte) first              |                    QDCOUNT                    |
                                                                                                                                           • Fixed syntax and simple semantics
 • Big endian (eg Motorola): MSB (Most Significant Byte) first               |                    ANCOUNT                    |
                                                                           +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+            Cons:
                                                                           |                    NSCOUNT                    |
Network byte order is big endian → You need to byte-swap on i386                                                                           • Not extendable
PCs.                                                                       |                    ARCOUNT                    |
                                                                                                                                           • Not human readable.
                                                                                                                                           • Byte order, alignment problems

                                  6                                                                     7                                                                                 8

                                                                   TLV - (Type, Length, Value)                                          Example: DHCP
                                                                                                                                        A vendor extension field taken from RFC 2132.
                                                                   Binary format usually used as an extensible part of a protocol.
Approach 2: Tree based
                                                                    • Type: contains a predefined code, indicating what kind of data     3.5. Router Option
Data structured hierarchical - recursive structure.                   the value field contains.                                              The router option specifies a list of IP addresses for routers on the
Both binary and textual variants.                                                                                                           client’s subnet. Routers SHOULD be listed in order of preference.
                                                                    • Length: Contains the size (in bytes) of the value field.
More or less formal specification defining the data-types E.g. XML                                                                            The code for the router option is 3. The minimum length for the
                                                                    • Value: Contains the payload.                                          router option is 4 octets, and the length MUST always be a multiple
DTD.                                                                                                                                        of 4.
                                                                   Examples: IS-IS and OSPFv3, DHCP, and IP options.
Examples: TLV, ASN.1, XML.                                                                                                                   Code   Len         Address 1               Address 2
                                                                   TLVs can be recursive (value field contains new TLVs).                    +-----+-----+-----+-----+-----+-----+-----+-----+--
                                                                                                                                            | 3 | n | a1 | a2 | a3 | a4 | a1 | a2 | ...
                                                                   But there is no notion of specification - must be added externally.       +-----+-----+-----+-----+-----+-----+-----+-----+--

                                  9                                                                    10                                                                                 11
                                                                      A tiny part of an SNMP definition:                                XML
Abstract Syntax Notation # 1                                          PDU ::=
                                                                        SEQUENCE {                                                     Plain-text markup language: simple syntax, easy to parse.
                                                                          request-id Integer32,
A general way to define data types.                                        error-status INTEGER (                                       Definition declared externally by XML Schema or DTD.
ASN.1 is as powerful as a typed programming language.                       tooBig(1),                                                 Well suited for complex data formats with recursive and nested
In ASN.1 the type information is inherent in the data - no external         noSuchName(2),                                             structures.
specification necessary.                                                     readOnly(4),                                               Cons mainly its textual nature: parsing can be inefficient.
Used frequently in ISO protocols, but also to a certain extent in             ...                                                      <?xml version="1.0" encoding="ISO-8859-1"?>
                                                                                                                                       <!DOCTYPE note SYSTEM "InternalNote.dtd">
TCP/IP protocols.                                                                                                                      <note>
                                                                            ),                                                           <to>Eva</to>
Some examples: SNMP, UMTS, LDAP, NFSv4 and many security                                                                                 <from>Phil</from>
                                                                            error-index INTEGER (0..max-bindings),
protocols.                                                                  variable-bindings VarBindList                                <heading>Reminder</heading>
                                                                        }                                                                <body>Remember to go to the store!</body>

                                 12                                                                           13                                                             14

                                                                      RFC 822 based text protocols                     (con’t)

                                                                                                                                       RFC 822 based text protocols                  (con’t)

                                                                      For example:
Approach 3: RFC 822 formats                                            name = elements crlf               a rule                       Another example; In RFC 2048, the HTTP URL is defined as:

Classical Internet format described by BNF (Backus-Naur Form) -        crlf = %d13.d10                    characters to end a line      http URL      =     ‘‘http:’’      ‘‘//’’ host [ ‘‘:’’   port ] [abs path
derived from context-free grammars.                                    ‘‘literal’’                        a string, case insensitive    host          =     A legal Internet host domain name or IP
Several RFCs describes the actual syntax description: RFC 822,         element1 / element2                an alternative                                    address (in dotted-decimal form) as defined by
RFC 2068, RFC 2234, now called ABNF - Augmented BNF.                   (element1 element2)                a strict sequence                                 Section 2.1 of RFC 1123
RFC 822 is syntax-heavy: keywords are introduced for parsing,          DIGIT = %x30-39                    a range of characters         port          =     *DIGIT
requires specific parsers.                                              <a>*<b>element                     element repetition            abs path      =     ‘‘/’’ rel path
                                                                       [foo bar]                          optional elements             rel path      =     [ path ] [ ‘‘;’’ params ] [ ‘‘?’’         query ]
                                                                      . . . and more . . .

                                 15                                                                           16                                                             17
                                                                                                                                                                                                                               TELNET - Terminal Network
                                                                    Specific applications/protocols
                                                                                                                                                                                                                               (TCP port 23, text) Virtual Terminal local terminal appears to be
                                                                         • telnet                                                                                                                                              a terminal on a remote system
Pros & Cons
                                                                         • http                                                                                                                                                It is a nice tool to test other text-based protocols (HTTP, SMTP,
 • Pros:                                                                                                                                                                                                                       FTP, etc)
                                                                         • tftp
    – Easy to extend and flexible.
                                                                         • ftp                                                                                                                                                 Good example of interactive application
    – Human readable (easy to debug)
                                                                         • smtp                                                                                                                                                 • Tinygrams leading to silly window syndrome:
 • Cons:
                                                                         • snmp                                                                                                                                                 • Nagle’s algorithm
    – Not compact.
                                                                         • rtp                                                                                                                                                  • Delayed ack, etc
    – Syntax-heavy: may require complex parsers.
                                                                         • sip                                                                                                                                                 Control: simple options (control bytes have first bit set)
                                                                                                                                                                                                                               TELNET is security challenged: use TELNET with Kerberos or
                                                                         • Others: Instant Messaging, Peer-to-peer, Distributed gaming.

                                18                                                                                         19                                                                                                                                   20

                                                                    HTTP - example
                                                                    ->    GET /stuff/blah.html HTTP/1.1
                                                                    ->    Host:

HTTP                                                                ->
                                                                          User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.5) Gecko/20031214 Firebird/0.7
                                                                          Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,video/x-mng,image/png,image/jpeg,image/gif;q=0.2,*/*;q=0.1
                                                                    ->    Accept-Language: en-us,en;q=0.5

(TCP port 80, RFC 2616, ABNF data)
                                                                          Accept-Encoding: gzip,deflate
                                                                          Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
                                                                                                                                                                                                                               Some HTTP commands
                                                                    ->    Keep-Alive: 300

The Hypertext Transfer Protocol is the main protocol used to        ->    Connection: keep-alive
                                                                                                                                                                                                                                • GET http url: Download an http resource.
                                                                    <-    HTTP/1.1 200 OK
download resources from the world wide web.                         <-    Date: Tue, 27 Jan 2004 20:18:28 GMT                                                                                                                   • POST http url: Upload data to an http resource.
                                                                    <-    Server: Apache/1.3.27 (Unix) (Gentoo/Linux) PHP/4.3.4
Simplest form: a requestor establishes a TCP connection to the      <-    Last-Modified: Tue, 27 Jan 2004 19:53:47 GMT
                                                                    <-    ETag: "bb4047-2c-4016c1cb"                                                                                                                            • PUT http url: Write an http resource.
web server on port 80 and sends a string describing what resource   <-    Accept-Ranges: bytes
                                                                    <-    Content-Length: 44
it wants, and receives the resource in reply.                       <-    Keep-Alive: timeout=15, max=100                                                                                                                       • DELETE http url: Delete an http resource.
                                                                    <-    Connection: Keep-Alive
                                                                    <-    Content-Type: text/html
The most modern version today is HTTP/1.1.                          <-
                                                                    <-    <html>
                                                                    <-      <b>
                                                                    <-        Hello there
                                                                    <-      </b>
                                                                    <-    </html>

                                21                                                                                         22                                                                                                                                   23
                                                                     HTTP 1/1 persistent connections                                       TFTP - Trivial File Transfer Protocol
                                                                                                                                           (Text-based, UDP port 69, RFC 1350)
                                                                     In HTTP 1/0, all HTTP requests generated a new TCP connection.
                                                                                                                                           Very simple protocol to transfer files.
                                                                     But most html documents contain sub-parts → one TCP
Some HTTP status codes                (con’t)
                                                                                                                                           Character coding: netascii(like telnet) or binary.
                                                                     connection for each sub-request.
Some examples:                                                       But TCP congestion control is made for longer connections → they      Stop-and-go protocol: send datagram, wait for ack.
                                                                     can adapt to congestion in the network.                               Small implementations: typically on boot PROMS for small devices
 • 200 Ok
                                                                                                                                           and diskless clients.
 • 404 Not found                                                     When http traffic grew when the web exploded, these small flows
                                                                     were said to kill the Internet!                                       Five message types:
 • 301 Moved Permanently                                                                                                                    • RRQ - Read ReQuest
                                                                     HTTP 1/1 supports persistent connections: keep the TCP
 • 500 Internal Server Error                                         connection during the complete session: send all requests on the       • WRQ - Write ReQuest
                                                                     same TCP connection.                                                   • DATA
                                                                     Now, these longer TCP connection can perform congestion control        • ACK
                                                                     algorithm in a proper way.
                                                                                                                                            • ERROR

                                 24                                                                  25                                                                     26

FTP - File Transfer Protocol                                                                                                               Some FTP commands
                                                                     FTP modes
(Text-based, TCP ports 20 and 21, RFC 959)                                                                                                 Examples of FTP control commands (sent on control channel):
                                                                     FTP can run in two modes- active mode and passive mode. This
FTP is a more elaborate file transfer protocol.                       refers to whether the ftp server will start the data connection or     • CWD <arg> Change working directory
FTP conducts its sessions in clear text.                             not.                                                                   • RMD <arg> Remove directory
FTP uses two TCP connections:                                         • active: The server will start the TCP session for the data          • PWD Print working directory
                                                                        connection, thereby connecting to the client to a port and IP
 • The control connection - exchange commands and their replies.                                                                            • TYPE [I|A|E|L <arg>] Set the data transfer type
                                                                        specified by the client. (May not work if client is behind NAT)
   TCP session initiated by the client to the server on port 21.
                                                                      • passive: The server will not start a TCP session. Instead, the      • RETR <arg> Download a file.
 • The data connection - to transfer data in a specified mode and
                                                                        client will create a TCP session to the server, to a port and IP    • STOR <arg> Upload a file.
   type. Data transferred may be a part of a file, an entire file or
                                                                        specified by the server.
   a number of files.                                                                                                                        • LIST Download the current working directory’s content list.

                                 27                                                                  28                                                                     29
Some FTP status codes                                                                SMTP - Simple Mail Transfer Protocol

As in HTTP, FTP has a variety of status codes:                                       (Text-based, TCP port 25, RFC 2821)                                  SMTP (cont)
 • 1xx Positive Preliminary reply                                                    SMTP the protocol to transfer email from hosts to mail servers and
   The requested action is being initiated; expect another reply before proceeding                                                                        Electronic mail is different from the previous protocols in its
   with a new command.                                                               between mail servers.
                                                                                                                                                          delayed delivery in several steps:
 • 2xx Positive Completion reply                                                     Terminology:
   The requested action has been successfully completed. A new request may be                                                                              • Spooling from sending host to first MTA.
                                                                                      • User Agent(UA) - end-hosts.
 • 3xx Positive Intermediate reply                                                                                                                         • Relaying by intermediate MTAs.
   The command has been accepted but the requested action is waiting for further      • Mail Transfer Agent (MTA) - mail servers.
   information before being completed.                                                                                                                     • Downloading of email by receiving host using other protocols:
 • 4xx Transient Negative Completion reply                                                                                                                   POPv3 (Post Office Protocol) or IMAPv3 (Interactive Mail
   The command was not accepted and the requested action did not take place,
   but the error condition is temporary and the action may be requested again.
                                                                                     Addressing: <mailbox>@<domain name>                                     Access Protocol)
 • 5xx Permanent Negative Completion reply
                                                                                     This results in a a DNS MX request for <domain name>, giving a
   The command was not accepted and the requested action did not take place.         name of the MTA to transfer the message to.

                                       30                                                                             31                                                                   32

                                                                                     MIME - Multipurpose Internet Mail Extensions
SMTP syntax
                                                                                     Classical email messages must be written in US-ASCII (7-bit).        So how does it work?
Like HTTP and FTP, SMTP has special commands and status
codes.                                                                               What does this imply?
                                                                                                                                                          Related header fields:
 • HELO <hostname>:                                                                  MIME aims at redefining the format of messages to allow for:
                                                                                                                                                           • Content-Type - what kind of data the content carries.
 • MAIL FROM <email address>: Sender email address                                    • textual message bodies in character sets other than US-ASCII,        Some examples: text/plain, text/html, audio, video,
                                                                                      • an extensible set of different formats for non-textual message        application/pdf, extension-token, and multipart.
 • RCPT TO <email address>: Recipient email address
                                                                                        bodies,                                                            • Content-Transfer-Encoding - how data is encoded.
 • DATA: Tells the email server that data follows.
                                                                                      • multi-part message bodies, and                                       Some examples: 7bit, 8bit, binary, quoted-printable,
 • QUIT: Immediately close the connection.                                                                                                                   base64,...
                                                                                      • textual header information in character sets other than
The status codes are similar to that of HTTP and FTP.                                   US-ASCII

                                       33                                                                             34                                                                   35
SNMP Simple Network Management Protocol

(TCP, ASN.1)
 • It is complex to build internetworks and we need to manage
   them.                                                            Real-time multimedia                                               Signaling
    – Monitoring                                                    Time-sensitive, interactive applications: (eg, telephony).         So, RTP can be used to transfer time-sensitive data streams.
    – Debugging                                                     Use RTP- Real-Time Protocol.
                                                                                                                                       But what about signaling: how to set up sessions:
    – Control routers and other network devices                     Limited time-sensitivity: Streaming protocols.
                                                                    Use RSTP (Real-Time Streaming Protocol)                              • SIP - Session Initization Protocol
 • SNMP - Internet management
                                                                    Non-time sensitive: Transfer the data using file transfer.            • H.323
    – No special control messages use TCP/IP itself
    – Management is on TCP/IP application level
    – Same protocol is used for all managed devices
    – If IP does not work correctly,...

                                 36                                                                      37                                                                  38

                                                                    SIP Example
                                                                    From RFC 3261:
SIP                                                                      softphone
                                                                                                                           SIP Phone
                                                                                                                                       SIP message example
(TCP or UDP port 5060, ABNF)                                                |    INVITE F1   |                 |               |
                                                                                                                                       INVITE SIP/2.0
                                                                            |--------------->|     INVITE F2   |               |
                                                                                                                                       Via: SIP/2.0/UDP;branch=z9hG4bK0e4415ea
Terminology is similar to SMTP, but is a synchronous protocol (no           | 100 Trying F3 |--------------->|     INVITE F4   |
                                                                                                                                       From: "6534" <>;tag=000e38a3b7e8001d597d1d53-1bfa7620
                                                                            |<---------------| 100 Trying F5 |--------------->|
delays).                                                                    |                |<-------------- | 180 Ringing F6 |
                                                                                                                                       To: <>
                                                                            |                | 180 Ringing F7 |<---------------|       Call-ID: 000e38a3-b7e8001e-34c94c48-72c83866@
SIP uses URI’s (Uniform Resource Identifiers) as addresses:                  | 180 Ringing F8 |<---------------|     200 OK F9 |        Date: Mon, 03 Jan 2005 14:16:06 GMT
                                                                            |<---------------|     200 OK F10 |<---------------|       CSeq: 101 INVITE
<>                                                                                                                      User-Agent: CSCO/6
                                                                            |    200 OK F11 |<---------------|                 |
<>                                                        |<---------------|                 |               |       Contact: <sip:6534@>
                                                                            |                        ACK F12                   |       Expires: 180
SIP uses transactions, usually three-way (as TCP connections).              |------------------------------------------------->|       Content-Type: application/sdp
                                                                            |                    Media Session                 |       Content-Length: 251
Example:                                                                                                                               Accept: application/sdp
INVITE → 200 OK → ACK                                                       |                        BYE F13                   |
                                                                            |                      200 OK F14                  |

                                 39                                                                      40                                                                  41
IM - Instant Messaging
                                                                     Peer-to-peer file-sharing applications
On-line messaging and presence information using a central server                                                                            Peer-to-peer file-sharing applications (cont)
and many connected clients.                                          Example of content-distribition (file-sharing) using peer-to-peer
                                                                     techniques.                                                             Some are completely decentralized (GnuTella), encrypts data
Some systems: AOL IM/ICQ, MSN Messenger, Yahoo Messenger
An IM system typically has the following features:                   Build overlays – virtual networks on top of physical network.
                                                                     Overlay links are TCP/UDP connections.                                  BitTorrent, for example, works closely with HTTP - splitting up an
 • Buddy list                                                                                                                                HTTP transfer in slices, distributing the download from one
                                                                     Usually, actual data transfer is direct between hosts (peer-to-peer),
 • Chat, Images, Sounds, File-sharing                                                                                                        originator to many clients working in unison.
                                                                     often using HTTP.
                                                                                                                                             Many rely on distributed hash lookup functions to make fast
 • Real-time talk and video                                          Some have central registry (index of where files are) (Napster).
                                                                                                                                             queries and lookup of data.
Most protocols are proprietary. But SIP has messaging extensions     Others (eg KaZaa) have distributed registry: some nodes with good
                                                                                                                                             Some of the routing problems are similar to real (physical) routing,
(SIMPLE).                                                            network connections, no NAT, and large resources turn into
                                                                                                                                             but on a higher level.
A special feature is to serve many small messages in a short time,   supernodes. All clients connect to a supernode.
and to manage presence information.

                                 42                                                                    43                                                                     44

                                                                     Skype                                                                   Distributed games

Detour: NAT traversal                                                (Encrypted, TCP/UDP)                                                    Some of the best-known distributed games are interactive and
                                                                     Skype is a VoIP tool using peer-to-peer techniques for name-lookup.     real-time: Doom, Quake, Counter-Strike, Half-life, etc.
Nowadays, most hosts are behind NAT (Network Address                                                                                         Some issues are:
                                                                     Skype is a completely closed system - no open interfaces, not even
Translation) boxes
                                                                     which RFCs are implemented → No interoperation possible.                 • Low latency: “low pingers” win fights. Usually small UDP
NATs translate global IP addresses to local, and extends the         You could say this violates the Internet spirit.                           packets.
address space using TCP/UDP ports.
                                                                      • Uses high compression: iLBC coding (≈ 10x compression of              • Textures and geometric information preloaded: only deltas
One peer behind NAT: possible to initiate connection from behind        audio data)                                                             distributed.
a NAT.
                                                                      • Name lookup using same infrastructure as KaZaa: nodes and             • Movement of 3D graphics may use “dead reckoning”: no need
Both peers behind NATs: difficult to communicate directly.                supernodes.                                                             to send updates on all geometric movements: use motion
Solution: For UDP, exploit some regularities of NATs (reuse of                                                                                  equations instead.
                                                                      • NAT traversal techniques using UDP, TCP or “bounce”
same ports, etc). Or use a non-NAT peer as “protocol bouncer”.          connections via supernodes.                                           • All communication via central server, synchronizes and resolves
                                                                      • End-to-end RSA encryption                                               events (who wins a fight).

                                 45                                                                    46                                                                     47

Shared By:
Tags: Gradutes
Description: Educational