Open Source Routing, Firewalls and Traffic Shaping

							Open Source Routing, Firewalls
and Traffic Shaping




Russell Sutherland
Computing and Networking Services
University of Toronto
russell.sutherland@utoronto.ca
Reference URLs for Tree Huggers
 This presentation
   – http://madhaus.cns.utoronto.ca/~russ/canheit2004/
 Routing
   – http://www.quagga.net/
   – http://www.xorp.net/
   – http://latrc.org/
 Traffic Shaping
   – Linux   http://tcng.sourceforge.net/
   – FreeBSD http://info.iet.unipi.it/~luigi/ip_dummynet/
 Packet Filtering
   – Linux (iptables) http://www.netfilter.org/
   – FreeBSD (ipfw) http://www.freebsd.org/
   – OpenBSD (pf)     http://www.benzedrine.cx/pf.html
Routing Chronology

 1984      BSD 4.2 ships with routed (RIPv1)
 1986      Fuzz Ball PDP-11 NSFNet Routers
 1988      Age of dedicated routing machines
  – Cisco, Proteon, Wellfleet, ACC
 1992      Gated Consortium Formed
 1996      GNU Zebra
 2002      Quagga, XORP
Quagga Routing Architecture

 Modular Design
 One process per protocol
  – bgpd, ospfd, ripd
 One main controlling process
  – zebra
 Extensible
Quagga Architecture Diagram


   ospfd        ripd         bgpd




                zebra




      Unix Kernel Routing Table
Quagga Routing Protocols

 RIPv1, RIPv2, RIPng
 OSPFv2, OSPFv3
 BGP-4, BGP+
 BGP route server and reflector
 IPv6
 Supported RFCs
  – 1058 RIPv1, 2453 RIPv2, 2080 RIPng
  – 2328 OSPFv2, 2740 OSPF for Ipv6
  – 1771 BGPv4, 1965, 1997, 2545 BGPv6, 2796 BGP
    Route Reflection, 2858 Multiprotocol extensions,
    2842 Capabilities Advertisement
Quagga Supported Platforms

 GNU Linux
  – Debian, RedHat, SuSE, Slackware
  – Kernels 2.2.x - 2.4.x
 FreeBSD
  – versions 4.x and 5.x
 OpenBSD
  – version 3.x
 NetBSD
  – version 1.4
 Solaris
  – 2.6 and version 7
Hardware Requirements

 CPU              Intel 2.0 – 3.0 Ghz
 Memory           512MB
 Disks            18GB
  – RAID-1 (optional)
  – SCSI or IDE
 Ethernet Interfaces
  – 2 x 10/100 Intel, 2 x 10/100/100 Broadcom
 Redundancy
  – hot spare serves as backup to N production units
Scottish Economics

 Router Prices
  – Cisco Mid-size
      7204VxR, Catalyst 3550
      $15k – $32k
  – Extreme
      Alpine 3800
      $31k - $38k
  – Foundry
      BigIron 4000
      $16k
  – Intel 2.x Ghz server
      Dell 2650, IBM x335
      $2.5k - $3.5k
             Network Topology

                        Traffic Shaper
                                                    Cogent A [100Mbps]
            McL
                           Skye

                                                    Cogent B [100Mbps]
                           Mull
 Internal

                           Jura
1. UofT A                                           C4 [1000Mbps]
2. UofT B
3. ResNet
                           Bute

                  Touchdown Network      External
                  1000 Mbps
Network Routing Policy

 Three classes of traffic (based on src IP)
   – ResNet
   – UofT A
   – UofT B
 ResNet
   – to (via TS) Skye to Cogent A
   – No C4 transit !!!
 UofT A
   – to C4 if dst IP == C4 otherwise via Skye to Cog A
 UofT B
   – to C4 if dst IP == C4 otherwise via Mull to CogB
Network Packet Filtering Policies

 Drop all packets with
  – spoofed (non UofT) source IP addresses
  – non-routable destination addresses
      0.0.0.0/8, 10.0.0.0/8, 127.0.0.0/8
      169.254.0.0/16, 172.16.0.0/12, 192.168/16
      etc.
  – nasty tcp/udp M$ worm ports (Blaster, Welchia,
    etc.)
      67, 68, 69, 135, 137, 139
      161, 162
      445, 593, 707, 1433, 1434, 3127, 4444
  – non-assigned UofT subnets
 Allow everything else
Network Traffic Shaping Policies

 All traffic from a local Redhat ftp site to the
  outside world gets a 50 kbps pipe
 Peer to peer traffic to and from UofT A&B
  gets a 256 kpbs full duplex pipe
   – KaZaa        1214
   – eDonkey      466[12]
   – BitTorrent   6881-6889
 ResNet traffic gets conditioned by a
  dedicated Traffic Shaper (Packeteer)
 Everything else flows freely
Routing Protocols and
Configuration
 Jura
  – runs OSPF on int. intf. with other UofT routers
  – runs BGP on external interface with C4 peer
  – contains all UofT and C4 specific routes
 Mull
  –   runs OSPF on int. intf. with other UofT routers
  –   runs BGP on external interface with Cogent B peer
  –   advertises UofTB routes
  –   defaults points to Cogent B
 Skye
  – same setup as Mull but with Cogent A
  – advertises UofTA and ResNet routes
Quagga Routing Configuration

 Command line interface similar to Cisco IOS
C4# conf t
C4(config)# interface eth2
C4(config-if)# description dummy interface
C4(config-if)# ip address 10.1.2.3/24
C4(config-if)# exit
C4(config)# exit
C4#

C4# conf t
C4(config)# router   bgp 328
C4(config-router)#   bgp router-id 10.1.1.10
C4(config-router)#   network 10.1.1.0/24
C4(config-router)#   redistribute static
C4(config-router)#   neighbor 10.1.1.1 remote-as 999
C4(config-router)#   exit
C4(config)# exit
C4#
      Quagga Operation

# show ip route
Codes: K - kernel route, C – connected, S – static, O -OSPF
B – BGP, > - selected route, * FIB route
S>* 0.0.0.0/0 [10/0] via 128.100.96.194, disc0
B>* 6.1.0.0/16 [20/0] via 205.211.94.97, yk0, 01w4d03h
B>* 6.2.0.0/22 [20/0] via 205.211.94.97, yk0, 01w4d03h
B>* 6.3.0.0/18 [20/0] via 205.211.94.97, yk0, 01w4d03h
# show bgp neighbors
BGP neighbor is 205.211.94.97, remote AS 549, local AS 239, external link
BGP version 4, remote router ID 205.211.94.253
BGP state = Established, up for 01w4d22h
FreeBSD ipfw Packet Filtering

 Native packet filtering interface
 Implemented as a multifunction user
  command
 The packet passed to the firewall is
  compared against each of the rules in the
  firewall ruleset.
    When a match is found, the action
    corresponding to the matching rule is
    performed and the search terminates.
 General syntax
    – ipfw [rule number] action [log] body
  ipfw examples
 Drop all www traffic from a network
   – ipfw add deny tcp from 12.12.12.0/24 to www.ubc.ca 80
 Drop all telnet traffic from a bad host
   – ipfw add deny tcp from bad.host.com to my.host.com 23
 Throw away RFC 1918 networks
   – ipfw add deny all from 10.0.0.0/8 to any in via fxp0
   – ipfw add deny all from 172.16.0.0/12 to any in via fxp0
   – ipfw add deny all from 192.168.0.0/16 to any in via fxp0
 Allow ssh
   – ipfw add allow tcp from any to any 22 in via fxp0 setup
     keep-state
ipfw actions

 allow | accept | pass | permit
   – Allow packets that match rule. The search ends.
 deny | drop
   – Discard packets that match rule. The search ends.
 fwd | forward ipaddr[,port]
   – Change the next-hop on matching pckts to ipaddr
 pipe N
   – Pass packet to a dummynet(4) for bandwidth
     limitation. [ conditionally end or continue ]
 count
   – Update counters for all packets that match rule.
     The search continues with the next rule
Traffic Control Concepts I

 Set of mechanisms to condition net traffic
 Examples
  – raise priority of some kinds of traffic
  – limit the rate at which traffic is sent
  – block undesirable traffic (same as packet filtering)
 TC is done at the network interface
  – ingress (traffic entering an interface)
      limited set of functions (classifying, dropping)
  – egress (traffic leaving an interface)
      full range of functions available
      queueing
     Traffic Control Concepts II




Classification   Queueing   Scheduling
Traffic Control Concepts III

 Classification
   – looks at packet content and assigns each to one
     or more classes.
 Queueing
   – stuffs incoming packets into storage silos based
     on class
 Scheduling
   – transmitting packets in queues based upon
     priority
 Queueing and Scheduling are often
  combined into queuing disciplines
Traffic Control Concepts IV

 Common Queueing Disciplines
  – simple drop tail (FIFO)
      stores and emits packets in order which they arrive
  – Random Early Detection (RED)
      starts dropping packets already before reaching
       maximum queue size
  – Token Bucket Filter (TBF)
      shapers that emits packets at a fixed rate
  – Priority Scheduler (PQ)
      emits packets in higher priority classes before
       packets in lower priority classes
  – Weighted Fair Queueing (WFQ)
      assigns an independent queue for each flow
      a weight can be defined for each queue
FreeBSD Dummynet Features

 Integrated with ipfw to classify packets
 Can be used equally well on egress/ingress
 Abstractions/features
   – pipes
       fixed bandwidth channels
       variable queue size, delays, random packet loss
   – queues
       queues of packets
       weighted
       share bandwidth of pipe they are associated with
        proportionally to their weight
 WF2Q+ used for queuing discipline
Dummynet Examples

 Limit WWW traffic to 100Mbps
       ipfw pipe 1 config bw 100Mbit/s
       ipfw add pipe 1 ip from any to any dst-port 80
 Prefer ssh to telnet traffic
         ipfw   pipe 2 config bw 256kbit/s
         ipfw   queue 1 config pipe 2 weight 7
         ipfw   queue 2 config pipe 2 weight 3
         ipfw   add queue 1 ip from any to any dst-port 22
         ipfw   add queue 2 ip from any to any dst-port 23
 Rate limit each network host's upload rate
       ipfw pipe 3 config mask src-ip 0x000000ff bw
        16kbit/s queue 8Kbytes
       ipfw add pipe 3 ip from 12.18.123.0/24 to any out
        via xl0
Routing Policy Using ipfw

 All ResNet traffic forwarded directly to Skye
   – ipfw add fwd $skye from $resnet to any in recv $uoft_if
 Block spoofed packets
   – ipfw add allow all from $uoftnet to any in recv $uoft_if
   – ipfw add deny in recv $uoft_if
 Block bad packets (M$ worms etc.)
   – for i in 67-69 135-139 161 162 445 593 707 4444
        do
            – ipfw add deny udp from any to any $i
            – ipfw add deny tcp from any to any $i
        done
 C4 traffic follows specific routes from BGP
Routing Policy Using ipfw Cont.

 Block all traffic to non-defined UofT addrs
   – ipfw add deny all from any to $uoftnet out xmit $def_if
 Partition UofT A/B traffic to Skye/Mull
   – add fwd $skye all from $uoftA to any out xmit $def_if
   – add fwd $mull all from $uoftB to any out xmit $def_if
 Traffic Shaping
   – limit RH ftp server
       ipfw pipe 1 config bw 50Kbit/s
       ipfw add pipe 1 ip from $rhftp to any in recv $uoftif
   – limit peer to peer
       ipfw pipe 2 config bw 256 Kbit/s
       ipfw add pipe 2 ip from $uoftA to any dst_port
        1214,4661,4662
Linux Packet Filtering: iptables

 Similar to ipfw in functionality and use
 User based command line interface
 Syntax
   – iptables rule-action table name conditions action
 Very rich set of conditions and actions
 Extensible modular actions
 More complicated in concept than ipfw or pf
 hierarchy: tables -> chains -> rules
 three default tables with default policies
   – filter, nat, mangle
 Linux iptables Anatomy Ingress
                    Network Interface

                       Contrack
                       mangle
                       IMQ
                                          PREROUTING
                       nat


                       QOS Ingress


                 INPUT ROUTING and RPDB
INPUT                                          FORWARD

        mangle                              mangle
        filter                              filter


 LOCAL PROCESSES                        REMOTE IP ADDR
Linux iptables Anatomy Egress
LOCAL PROCESSES


OUTPUT ROUTING
                                      REMOTE IP ADDR

   contrack
   mangle         OUTPUT
   nat
   filter



                       mangle
                       nat              POSTROUTING
                       IMQ


                     QOS Egress


                  Network Interface
       iptables examples
 Drop all www traffic from a network
       iptables -A FORWARD -p tcp –dport 80 -s 12.12.12.0/24
        -d www.ubc.ca -j DROP
 Drop all telnet traffic from a bad host
       iptables -A INPUT -p tcp -s bad.host.com -d my.host.com
        –-dport 23 -j DROP
 Throw away RFC 1918 networks from inside
       iptables -A FORWARD -s 10.0.0.0/8 -i eth0 -j DROP
       iptables -A INPUT      -s 10.0.0.0/8 -i eth0 -j DROP
       iptables -t mangle -A PREROUTING -s 172.16.0.0/12 -i eth0
        -j DROP
 Allow ssh and keep state
       iptables -A FORWARD -p tcp –dport 22 -i fxp0 -m state     -–
        state NEW,ESTABLISHED -j ACCEPT
Linux Routing – Multiple Tables

 Multiple routing/forwarding tables
 Three fixed prefined tables
   – local
   – main
   – default
 Each table is assigned a priority number
   –     0     local
   – 32766     main
   – 32767     default
 match is sought starting with highest
  priority tables (local -> main -> default)
 Linux Routing Policy Database


Traditional Routing   Routing Policy Database
                      (RPDB)

Destination IP Address Destination IP Address
Type of Service        Source IP Address
                       Type of Service
                       Iptables FW mark
Linux Traffic Control: tc

 Uses queueing disciplines for managing
  bandwidth
 Largely concerned with data being sent
  rather than received.
 Classless queueing disciplines
  – reschedule, drop or delay
  – applied to the bulk interface
  – pfifo_fast
      default, can't be changed
  – TBF (Token Bucket Filter)
      passes traffic up to a fixed rate
      drops the rest
      allows short burst in excess of fixed rate
tc: Classless qdiscs

 SFQ (Stocastic Fair Queueing)
  – Traffic split into large number of FIFO queues, one
    per flow
  – Traffic gets sent/serviced in a round robin fashion,
    giving each flow a chance to sent its data.
  – Leads to fair behaviour
  – prevents one flow from hogging all the
    bandwidth
  – only really useful when the link is full
 RED (Random Early Detection)
  – drops packets statistically before queues are
    full
  – leads to a congested link to slow more
    gracefully
tc: Classful qdiscs

 Used when different types of traffic need
  different treatment.
 CBQ (Class Based Queueing)
  – very complicated to set up and tune
 PRIO
  – classify and traffic into a number of bands each
    with its own priority.
 u32
  – used as the tool to classify the traffic into sub
    queues
  – based on actual offset of information in the IP
    header
   Linux: tcng

 tc syntax is very complicated both in setting
  up the qdisc's and classification
         tc qdisc add dev eth0 root handle 1:0 prio
         tc qdisc add dev eth0 parent 1:0 protocol ip u32 match
          ip protocol 6 ff match tcp dst 50 ffff classid 1:1
         tc qdisc add dev eth0 parent 1:3 handle 30: sfq
         tc filter add dev eth0 parent 1:0 protocol ip prio 1 u32
          match ip sport 80 0xffff flowid 1:3
 tcng was created as a higher level tool
   –   simple to configure
   –   more natural language to set up classes and qdisc
   –   compiles to tc or “C”
   –   comes with a simulator
      tcng: Example Input
dev “eth0” {
egress {
         class (<$high>) if tcp_port == 80;
         class (<$low>) if 1;
         prio {
                  $high = class {
                            tbf(limit 10kB, rate 20kbps,
                                burst 2kB, mtu 1500B);
                  $low = class {
                            fifo(limit 30kB)
                  }
         }
}
}
           tcng: Example Output
tc qdisc add dev eth0 handle 1:0 root dsmark indices 4 default_index 0
tc qdisc add dev eth0 handle 2:0 parent 1:0 prio
tc qdisc add dev eth0 handle 3:0 parent 2:1 tbf burst 2048 limit 10240 mtu 1500 rate 2500bps
tc qdisc add dev eth0 handle 4:0 parent 2:2 bfifo limit 30720
tc filter add dev eth0 parent 2:0 protocol all prio 1 tcindex mask 0x3 shift 0
tc filter add dev eth0 parent 2:0 protocol all prio 1 handle 2 tcindex classid 2:2
tc filter add dev eth0 parent 2:0 protocol all prio 1 handle 1 tcindex classid 2:1
tc filter add dev eth0 parent 1:0 protocol all prio 1 handle 1:0:0 u32 divisor 1
tc filter add dev eth0 parent 1:0 protocol all prio 1 u32 match u8 0x6 0xff at 9 offset at 0 mask 0f00
shift 6 eat link 1:0:0
tc filter add dev eth0 parent 1:0 protocol all prio 1 handle 1:0:1 u32 ht 1:0:0 match u16 0x50 0xffff at 2
classid 1:1
tc filter add dev eth0 parent 1:0 protocol all prio 1 u32 match u32 0x0 0x0 at 0 classid 1:2
Routing Policy Using Linux

 Routing Tables
   –   0:       from   all lookup local
   –   100:     from   142.151.0.0/16 lookup resnet
   –   1000:    from   all lookup main
   –   2000:    from   142.150.0.0/16 lookup uoftA
   –   32767:   from   all lookup default
 resnet contains a single default to syke
 uoftA contains a default to skye
 default contains a default to mull
 main contains all the C4 routes
            Linux Traffic Shaping Policy
dev eth1 {
   egress {
class ( <$rhftp> ) if ip_src == 128.100.17.10;
class ( <$p2p> ) if ( (tcp_dport == 1214 ||
tcp_dport == 4661 || tcp_dport == 4662) &&
ip_src:16 == 128.100.0.0 );
class ( <$high> ) if 1 ;

        htb () {
           class ( rate 100Mbps , ceil 100Mbps ) {
              $rhftp = class ( rate 50kbps, ceil 75kbps );
              $p2p = class ( rate 256kbps, ceil 325kbps );
              $high = class ( rate 90Mbps, ceil 100Mbps );
           }
        }
    }
}
OpenBSD packet filtering

 pf runs as the native packet filtering engine
 similar in syntax to ipfw
 traffic shaping (ALTQ) integrated with pf
 BSD only supports one main routing table
 pf (like ipfw) supports a forwarding action
  to explicitly forward a packet
 URLs
   – www.openbsd.org
   – www.csl.sony.co.jp/person/kjc/software.html
   – www.benzedrene.cx/pf.html
Results and Conclusions

 OSS Routers in service for > 18 months
 Scaled easily from 1 to 3 machines
 Currently running
  – FreeBSD 4.x, 5.x, dummynet, ipfw
 Will be moving to Linux in next 3 months
 Standard network monitoring via SNMP
 CPU running < 40%
 OSS is a viable option for policy based
  routing and shaping at the edge