Declarative Networking Language_ Execution_ Optimizations

Document Sample
Declarative Networking Language_ Execution_ Optimizations Powered By Docstoc
					The Design and Implementation
of Declarative Networks
Boon Thau Loo
University of Pennsylvania, University of California-Berkeley*
*This dissertation was completed as a graduate student at the
University of California- Berkeley
Declarative Networking
 A declarative framework for networks:
     Network protocols are declaratively specified using a
      database query language
     Distributed query engine executes specifications to
      implement network protocols
 Success of database research:
     70‟s – today: Database research has revolutionized
      data management
     Today: Similar opportunity to revolutionize the
      Internet architecture
Motivation
  Internet faces many challenges today:
     Unwanted, harmful traffic
     Complexity/fragility in Internet routing
     Proliferation of new applications and services
  Efforts at improving the Internet:
     Evolutionary: App-level “Overlay” networks
     Revolutionary: Clean-slate designs
        NSF GENI initiative, FIND program

     Opportunity: Software tools that can
  significantly accelerate network innovation
A Declarative Network
                           messages

        Dataflow                                 Dataflow


       messages
                                  Dataflow         messages




          Dataflow
                                                      Dataflow
                                                      Distributed
                                                    recursive query
                       Dataflow


 Traditional Networks                        Declarative Networks
    Network State                              Distributed database
    Network protocol                         Recursive Query Execution
   Network messages                             Distributed Dataflow
The Case for Declarative Networking
 Ease of programming:
    Compact and high-level representation of protocols
    Orders of magnitude reduction in code size
       Declarative Chord DHT is 48 lines instead of 10,000.
    Easy customization
 Safety:
    Queries are “sandboxed” within query processor
    Potential for static analysis of safety
 What about efficiency?
    No fundamental overhead when executing standard routing
     protocols
    Application of well-studied query optimizations
    Note: Same question was asked of relational databases in the
     70‟s.
Main Contributions of Dissertation
  Declarative Routing [SIGCOMM ‟05]:
      Extensible Routers: balance of flexibility, efficiency
       and safety
  Declarative Overlays [SOSP ‟05]:
      Rapid prototyping of new overlay networks
  Database Fundamentals [SIGMOD „06]:
      Network specific query language and semantics
      Distributed recursive query execution strategies
      Query optimizations, classical and new
A Breadth of Use Cases
 Example implementations to date:
     Textbook routing protocols
     Chord DHT
     Narada mesh for end-system multicast
     Distributed Gnutella/Web crawlers
     Pastry DHT
     Replication protocols
     Lamport/Chandy snapshots
     Paxos distributed consensus
     Overlays for host mobility
     Sensor network protocols
 P2 declarative networking system
     http://p2.cs.berkeley.edu
Outline
  Background
  The Connection: Routing as a Query
     Execution Model
     Path-Vector Protocol Example
        Query specification  protocol implementation
  Query Processing
  Beyond routing: Declarative Overlays
  Ongoing work @ Penn
Traditional Router

                             Routing
                             Protocol


                   Neighbor Table Forwarding
 Control Plane        updates    Table updates

Forwarding Plane
                     Neighbor Table   Forwarding Table   Packets
                                                                      Routing
                                                                   Infrastructure
                                                         Packets


                   Traditional Router
Declarative Router
                      Query Engine
    Declarative              Routing
     Queries                 Protocol
                         Input             Output
                        Tables             Tables



                   Neighbor Table Forwarding
 Control Plane        updates    Table updates

Forwarding Plane
                     Neighbor Table   Forwarding Table
                                                         Packets
                                                                      Routing
                                                                   Infrastructure
                                                         Packets


                   Traditional Router
                   Declarative Router
All-Pairs Reachability
  R1: reachable(S,D)  link(S,D)
  R2: reachable(S,D)  link(S,Z), reachable(Z,D)


  “For all nodes S,D,
  link(a,b) – “there is a link from node a to node b”
         If there is a link from S to D, then S can reach D”.
  reachable(a,b) – “node a can reach node b”




  Input: link(source, destination)
  Output: reachable(source, destination)
All-Pairs Reachability
   R1: reachable(S,D)  link(S,D)
   R2: reachable(S,D)  link(S,Z), reachable(Z,D)

 “For all nodes S,D and Z,
           If there is a link from S to Z, AND Z can reach D, then S
          can reach D”.




   Input: link(source, destination)
   Output: reachable(source, destination)
Towards Network Datalog
  Specify tuple placement
     Value-based partitioning of tables
  Tuples to be combined are co-located
     Rule rewrite ensures body is always single-site
  All communication is among neighbors
     No multihop routing during basic rule execution
     Link-restricted rules: Enforced via simple syntactic
      restrictions
 Network Datalog                                  Location
                                                Specifier “@S”
 R1: reachable(@S,D)  link(@S,D)
 R2: reachable(@S,D)  link(@S,Z), reachable(@Z,D)
 Query: reachable(@a,N)
        reachable(@M,N)              All-Pairs Reachability


                  link               link               link         link
                 @S    D           @S    D             @S      D    @S    D
Input table:
                 @a    b            @b   c             @c      b    @d      c
                                    @b   a             @c      d


                   a                 b                   c            d
                reachable         reachable          reachable     reachable
Output table:    @S      D          @S   D             @S      D    @S      D
                 @a      b   Query: reachable(@a,N) @c
                                     @b a                      a    @d      a
                 @a      c          @b      c          @c      b    @d      b
                 @a      d          @b   d             @c      d    @d      c
Path Vector in Network Datalog
R1: path(@S,D,P)  link(@S,D), P=(S,D).
R2: path(@S,D,P)  link(@Z,S), path(@Z,D,P2), P=SP2.
Query: path(@S,D,P)                       Add S to front of P2


   Input: link(@source, destination)
   Query output: path(@source, destination, pathVector)
   Query Execution
  R1: path(@S,D,P)  link(@S,D), P=(S,D).
  R2: path(@S,D,P)  link(@Z,S), path(@Z,D,P2), P=SP2.
  Query: path(@a,d,P)

                      link            link                     link               link
    Neighbor         @S     D        @S       D            @S       D             @S     D
    table:           @a     b        @b       c            @c         b           @d     c
                                     @b       a            @c         d


                     a                   b                     c                   d

                     path                path                  path
                @S    D         P   @S    D       P       @S    D          P
Forwarding
table:                                                    @c    d         [c,d]
   Query Execution
  R1: path(@S,D,P)  link(@S,D), P=(S,D).

  R2: path(@S,D,P)  link(@Z,S), path(@Z,D,P2), P=SP2.
  Query: path(@a,d,P)            Matching variable Z = “Join”

                      link                link                          link           link
    Neighbor                             @S        D
    table:Communication   patterns are identical to @S
              @S D                                                  @S       D                D
              @a b         @b c         @c b        @d                                        c
          those in the actual path vector protocol
                           @b a         @c d


                     a                        b                         c              d
                       path(@a,d,[a,b,c,d])        path(@b,d,[b,c,d])
                     path                     path                      path
                @S    D      PP         @S     D        PP        @S     D       P
Forwarding
table:          @a    d     [a,b,c,d]   @b     d       [b,c,d]    @c     d     [c,d]
Other Routing Examples
  Best-Path Routing
  Distance Vector
  Dynamic Source Routing (Wireless)
  Policy Decisions
  QoS-based Routing
  Link-state
  Multicast Overlays (Single-Source & CBT)
Outline
  Background
  The Connection: Routing as a Query
  Query Processing
  Beyond routing: Declarative Overlays
  Sampling of ongoing work
Recursive Query Evaluation
    Semi-naïve evaluation:
        Iterations (rounds) of synchronous computation
        Results from iteration ith used in (i+1)th
                           10                                          9
                            9
                                   3-hop           7
                            8                                          5
                            7                          4   1       2       10
                            6                  8               0
                            5
                                   2-hop
                            4                                  3
                            3
                            2                                  6
                                   1-hop
                            1

Link Table            Path Table                           Network
                Problem: Unpredictable delays and failures
Pipelined Semi-naïve (PSN)
    Fully-asynchronous evaluation:
        Computed tuples in any iteration pipelined to next iteration
        Natural for network protocols

                            10
                                                                       9
                             9                     7
                                                                       5
                             6                         4           2       10
                                                           1
                             3
                             8                 8
                                           Relaxation of 0
                             5              semi-naïve 3
                            2
                            7                                  6
                            4
                            1

Link Table             Path Table                          Network
Pipelined Evaluation
  Challenges:
     Does PSN produce the correct answer?
     Is PSN bandwidth efficient?
        I.e. does it make the minimum number of
        inferences?
  Proofs for
        p(x,z) :- p1(x,y), p2(y,z), …, pn(y,z), q(z,w)
                         recursive w.r.t. p

     Basic technique: local timestamps
Execution Plan                                     Strands




                   UDP
                    Rx
                            lookup



  Network                                                            Network




                                                          Round
                                                          Robin
                            lookup



     In                                                                Out
                   CC
                   Rx




                                                          CC
                                                          Tx
  Messages          Queue
                                                                      Messages




                                                             Queue
                    Demux




                                                          UDP
                                                           Tx
                                     link   path    ...

                                       Local Tables

                               Single Node
  Nodes in execution plan (“operators”):
      Network operators (send/recv, cc, retry, rate limitation)
      Relational operators (selects, projects, joins, aggregates)
      Flow operators (mux, demux, queues)
Localization Rewrite
  Rules may have body predicates at different locations:
  R2: path(@S,D,P)  link(@S,Z), path(@Z,D,P2), P=SP2.

                        Matching variable Z = “Join”

 Rewritten rules:
  R2a: linkD(S,@D)  link(@S,D)

  R2b: path(@S,D,P)  linkD(S,@Z), path(@Z,D,P2), P=SP2.

                         Matching variable Z = “Join”
Localized Rule Compilation
 R2b: path(@S,D,P)  linkD(S,@Z), path(@Z,D,P2), P=SP2.

                                 Execution Plan

                    path      Join         Project
                                                        Send to




                                                                  Network Out
                             path.Z =     path(S,D,P)
       Network In




                              linkD.Z
                                                        path.S


                              linkD


                    linkD     Join         Project
                                          path(S,D,P)
                                                        Send to
                             linkD.Z =
                               path.Z
                                                        path.S


                               path
Optimizations
  Traditional: evaluate in the NW context
     Aggregate Selections
     Magic Sets rewrite          PV/DV (Wired) 
                                   DSR (Wireless)
     Predicate Reordering
  New: motivated by NW context
     Multi-query optimizations:
        Query Results caching
        Opportunistic message sharing
     Cost-based optimizations
        Neighborhood density function   Hybridized protocols:
                                         Zone Routing Protocol
        Hybrid rewrites
Beyond Routing: Declarative Overlays
 Language extensions to support
 events and soft-state predicates
 Chord Routing, including:
   Multiple successors

   Stabilization

   Optimized finger maintenance
   Failure detection

 48 rules
 11 table definitions
 MIT-Chord: x100 more code
 Another example:
     Narada mesh in 22 rules

                                    10 pt font
Outline
  Background
  The Connection: Routing as a Query
  Query Processing
  Beyond routing: Declarative Overlays
  Ongoing work @ Penn
Ongoing Work @ Penn
 Declarative secure networking
    Difficult to design/implement/reason about secure networks
    Network Datalog + logic-based security languages [NetDB ‟07]
    Authenticated path vector protocol, DNSSEC, secure DHTs,…
    Moving forward:
       Exploit fine-grained control over networks and security policies


 Data-centric querying and routing in heterogeneous
 networks
    Internet: Wired infrastructure with wireless clouds at the edges
    Flexible network support for mobility [ACM MobiArch ‟07]
       Declarative queries for addressing and naming mobile hosts
       Session-aware customizable QoS routing
    Moving forward:
       Declarative wireless ad-hoc networks
       Cost-based query optimizations to adapt protocols
Summary
 Declarative networking:
    Declarative Routing:
       Extensible routing infrastructure
    Declarative Overlays
       Rapid prototyping overlay networks
    Database fundamentals
       Query language
       New distributed query execution strategies and
        optimizations
       Semantics in dynamic networks
    P2 declarative networking system
     (http://p2.cs.berkeley.edu)
Many Thanks…
 Advisors: Joseph M. Hellerstein, Ion Stoica
 Collaborators:
     UC Berkeley: Tyson Condie, Ryan Huebsch
     Intel Research: David Gay, Petros Maniatis, Timothy Roscoe
     Yahoo! Research: Minos Garofalakis, Raghu Ramakrishnan
     Rice University: Atul Singh
     Many others…

				
DOCUMENT INFO