Manage Traffic with Iproute

LINUX WORLD Expert Linux How to manage traffic with iproute2 This month, David Coulson looks at using your Linux box to manage your bandwidth n the last issue, we looked at using iproute2 to maintain our network configuration and to route different types of traffic over various connections to maximise the use of the bandwidth we had available. This is great if you’ve got more than one place for the traffic to leave, but what if you’ve just got one line and want to make sure everything is fine? Whether you have a 56K modem or some variety of DSL, making sure that specific traffic types, such as your mail or news, are not slowed down by ftp downloads is important. This is particularly important if the former occurs in high volumes, or the latter is a frequent problem out of your control. This is a well recognised problem amongst, for example, students who share an internet connection, without anyone having much in the way of regard for the ability of others who use the internet at the same time. With only one way for the traffic to enter the internet, the only way we can maintain a level of usability is to limit the rate at which the traffic leaves, or enters our network. We can limit incoming ftp traffic, so that there is 16K per second left over for SMTP, or other traffic. We can set it up so that NNTP can ‘borrow’ bandwidth from ftp, should it be available, so if something needs to 172/PCPlus.co.uk #181 ,There are many methods of traffic queueing available, and a large selection of them are available in the Linux kernel. ,Traffic queueing Notwithstanding the unpleasant looking abbreviations, the scheduling modules from Linux are fairly easy to configure and maintain – once you’re used to the syntax... Before we can start setting up our gateway, we need to enable a few options in the kernel and compile the user-space maintenance tools. Within ‘Networking options’ in the kernel configuration, there is a sub-menu headed ‘QoS and/or fair queueing’. First, we need to select ‘y’ for ‘QoS and/or fair queueing’ and either ‘y’ or ‘m’ for ‘CBQ packet scheduler’. Not only do we need a method of creating queue classes – which CBQ does – but it’s quite handy to be able to queue packets within a class, which is the whole point of this. CBQ can’t queue packets at all, so we use the Stochastic Fair Queueing (SFQ) module. There are a huge number of schedulers available for Linux, but a large proportion of them are for very distinct traffic types and SFQ is a good choice for basic traffic because it uses very little CPU time. We also need to select some methods for matching packets that will be queued, within the ‘Packet classifier API’ section. You’ll want to select almost all of them, either as modules or compiled into the kernel, although the only ones we will be looking at are ‘Firewall basec classifier’ and ‘u32 classifier’. Once you’ve decided what you want to be available, it’s a case of recompiling the kernel and rebooting. Even if you’ve selected the whole lot as modules, there is still some queueing code that needs to be compiled into the kernel before the modules can be used – a little like netfilter and the packet-filtering modules. Now we’ve got a kernel up-and-running, we need to build the userspace tool we use to configure our network. As with the routing, we looked at in the previous issue, we need the iproute2 package that will either come with your distribution or can be found at ftp://ftp.inr.ac.ru/ip-routing. The binary we want is named ‘tc’, and is within the iproute2/tc/ directory if you’re looking for it in the source code. A quick ‘make’ in this directory will compile your ‘tc’ binary, and you’ll then need to copy it out to /usr/sbin because there is no ‘make install’ procedure. We’re going to be dealing with a 64K ISDN connection, purely because it’s a nice number to work with when we come to divide it into parts and that we get 64K each way, full-duplex. If you’ve got a 512K up and 128K down ADSL line then it won’t work out quite as nicely but, in either case, you will have to check out your own bandwidth usage using something like iptraf so that you don’t end up with 16kbit spare because you don’t handle much email. Before we can start setting up the classes, we need to decide how much bandwidth we want to allocate to particular services. 16Kbit per second will be adequate for SMTP and NNTP to share, and an extra 16Kbit per second for SSH and telnet. The other 32Kbit can be used by http, ftp and any other traffic types leaving the network. The first thing we have to do is configure the queueing discipline of our network interface, in this case ippp0, using ‘tc’. tc qdisc add dev ippp0 root handle 10: cbq bandwidth 64Kbit avpkt 1000 I Related files on SuperDisc 1 PATH: linux download a great deal of content, it’s not limited to 16K per second. We’re going to look at how to set up traffic management within the Linux kernel, and how to manage it using the user space tools that come as part of the iproute2 package. There is, naturally, nothing stopping you setting up a similar system to establish hard limits for servers hosting content on your line, but that’s a rather unadventurous and simple use of such capabilities. Instead, we’re going to look at setting up a gateway for a selection of machines, including regular workstations and servers handling news, mail and publiclyaccessible web services. PCP David Coulson dcoulson.pcpmag.co.uk NEXT MONTH Next month we look at the new packet filtering code in Linux, iptables, and using it to secure your network EXPERT LINUX nWith ‘tc filter show dev ippp0’, we can see which filters are applied to the interface – the IP address is written in hexadecimal, though. nThe initial root class on a device is both a class and a queueing discipline, and is shown in both outputs. Looks scary, doesn’t it? Simply, we’ve assigned ‘CBQ’ as the root discipline for the interface and named it ‘10’. We’ve also told it that the interface can handle 64Kbit per second and that the average packet size is around 1,000bytes. This isn’t really a queueing class, more of a handle to attach a class to an interface, so we need to add a proper class to the device that can use all the bandwidth available, and then divide that into the smaller classes: tc class add dev ippp0 parent 10:0 classid 10:1 cbq bandwidth 64Kbit rate 64Kbit allot 1514 weight 6Kbit prio 8 maxburst 20 avpkt 1000 We then need to add a number of sub-classes to this 10:1 class, for each of our traffic types: tc class add dev ippp0 parent 10:1 classid 10:100 cbq bandwidth 64Kbit rate 32Kbit allot 1514 weight 3Kbit prio 5 maxburst 20 avpkt 1000 tc class add dev ippp0 parent 10:1 classid 10:200 cbq bandwidth 64Kbit rate 32Kbit allot 1514 weight 3Kbit prio 5 maxburst 20 avpkt 1000 bounded Although many of them are tuning parameters that are not particularly complicated, we’ve added a class to ippp0 that has the parent class ‘10:0’, which is the root discipline we defined earlier, and named it ‘10:1’. The class can handle 64Kbit per second and will enable the whole 64Kbit to be used, as we’ve set the rate to 64Kbit. The last five numbers generally stay the same, because they define the priority of the connection and the size of the packets, but the ‘weight’ value is generally defined as one-tenth of the total bandwidth available to a class, from the ‘rate’ parameter. Rather than creating three classes, we’re going to create two 32Kbit classes, then cut one in half for the two 16Kbit classes, so they can share bandwidth between themselves without getting in the way of the other traffic. Note that we’ve used ‘bounded’ on the 10:200 class, so that it won’t use anymore than 32Kbit, but the other can. If we had three classes and wanted to stop one having bandwidth ‘stolen’ from it we would use the ‘isolated’ parameter, which will www.futureforums.co.uk/pcplus ,Monitoring your traffic It’s rather difficult to manage traffic before you know exactly what is going in and out – so there are numerous tools to help you Unless you’re an extraordinary person, and can figure out how much bandwidth a connection is using from the output of tcpdump, there isn’t a very clean way to monitor individual connections on a network without resorting to a ‘behind the scenes’ method of counting packets, such as by using ipchains or something similar. IPTraf is a lovely little ncurses-based traffic monitor that you can watch and see how much bandwidth individual connections are using. It comes with a number of distributions, but since using the source code is much more fun, you can download it from http://cebu.mozcom.com/riker/iptraf, which also contains documentation and mailing list information. The installation is a little non-standard because it involves running a ‘Setup’ script as root, which compiles and installs the ‘iptraf’ binary. Frustratingly, it also installs it in /usr/local/bin, but with permissions so that only root can run it, but /usr/local/sbin, or /usr/sbin/ is a preferred location for such a binary. It’s quite handy, as it returns an obscene amount of information on connections, interfaces and such like, so you can pop it on a gateway and leave it running, and it’ll churn out bandwidth statistics for all the network interfaces on that machine. Unless you’re familiar with what each port does, it’s usually a little difficult to figure out what each connection really belongs to, particularly since it only shows the source IP and port, rather than the much more useful destination address. So, it’s a case of shelling into the source machine, assuming we can, and running ‘netstat -np’ to work out what process the particular connection belongs to. As well as returning information on individual connections, you can also get information on bandwidth usage between NICs attached to the same physical LAN, as long as the traffic is heading through the machine running iptraf. The only caveat with such a piece of software is that it’s almost impossible to use it remotely because the nIPTraf can give us statistics on individual interfaces and connections. SSH traffic is counted along with the rest of the network traffic. This causes it to update which, itself, creates more SSH traffic, so you end up with a rather unpleasant loop eating up all your bandwidth. The alternative is to set up a serial console on the gateway, and to access another machine to run minicom, so you can get all the IPTraf information remotely. If you’re more of a GUI fan and want nice little graphs and things to click on, KSnuffle does much the same as IPTraf, but it does require KDE2 and all the libraries it needs, so is not quite as appealing on a machine dedicated as a router. # 181 PCPlus.co.uk /173 LINUX WORLD ,Apache throttling Limiting traffic on anything more discrete than a per-port basis requires your service to do something a bit special nWriting a script helps you keeping track of what you’ve set up. allow it to borrow bandwidth from another class, but not allow others to take bandwidth from it. We want to split 10:100 into two 16Kbit classes, which we do with: tc class add dev ippp0 parent 10:100 classid 10:101 cbq bandwidth 64Kbit rate 16Kbit allot 1514 weight 1600 prio 5 maxburst 20 avpkt 1000 bounded tc class add dev ippp0 parent 10:100 classid 10:102 cbq bandwidth 64Kbit rate 16Kbit allot 1514 weight 1600 prio 5 maxburst 20 avpkt 1000 QWe can use mod_throttle to limit traffic on Apache. In this case limiting the volume of traffic to 10MB per hour. So, we now have three classes that we want to use to queue traffic: 10:200 for FTP/HTTP, 10:101 for SMTP/NNTP and 10:102 for SSH and telnet. We apply another queueing discipline to each of those, just as we did to ippp0 earlier, but we’re going to use SFQ rather than CBQ: tc qdisc add dev ippp0 parent 10:200 sfq quantum 1514b perturb 15 tc qdisc add dev ippp0 parent 10:101 sfq quantum 1514b perturb 15 tc qdisc add dev ippp0 parent 10:102 sfq quantum 1514b perturb 15 ernel level packet queueing only goes so far, as we often want to limit bandwidth to a higher level than IP. The most common connections that need throttling like this are ftp and HTTP, and many ftp servers, including ProFTPd, support rate limiting of connections based on the IP the client is connected to. Apache, however, relies on a thirdparty module known as ‘mod_throttle’ that can limit bandwidth usage by the httpd by checking many parameters, including directory of the fetched file, the virtual host used, client IP address and many types of authentication variables. This is a benefit if you’re allowing people K to download large files and want to ensure that smaller files don’t end up being slowed down by a number of clients using up the bandwidth with just a couple of large transfers. Mod_throttle is available from www.snert.com/Software/mod_ throttle, and comes with plenty of documentation about its usage and installation, as either a DSO or compiled into the httpd binary. You’ll want to check the documentation for the appropriate arguments to use and, when you’re done, you can watch the throttle-status page and see just how much the use of mod_throttle is improving the efficiency of your website. Now that we have our interface setup with the different queueing disciplines, we can allocate different traffic to each of them. If we want to apply the ‘SMTP/NNTP’ class – which is 10:101 to all traffic coming from or to 10.1.1.5 – we would use: tc filter add dev ippp0 parent 10:0 protocol ip prio 100 u32 match ip src 10.1.1.5 flowid 10:101 ,A simple traffic shaper If CBQ, SFQ, iproute2 and tc are all a little too much, there is an alternative... %The ‘shaper’ network device is an alternative to CBQ, but it lacks many of the features. As we did before with iproute2 for traffic, we’ve set a priority of 100, so that it will get caught before any other rules, assuming they have a priority of less than 100. We might assign different rules to 10.1.1.0/24, but want our limitations to 10.1.1.5 to apply first, and apply the filter for 10.1.1.0/24 the priority of 50. As 10:101 is for all SMTP/NNTP traffic, we also want to apply it to items heading to, or from, port 119 or 25, and we use iptables to mark our packets: iptables -t mangle -i ippp0 -A PREROUTING -m multiport --sport 25,119 -j MARK --set-mark 1 iptables -t mangle -i ippp0 -A POSTROUTING -m multiport --dport 25,119 -j MARK --set-mark 1 tc filter add dev ippp0 parent 10:0 protocol ip prio 90 u32 match fw classid 1 flowid 10:101 “There are a huge number of schedulers available for Linux, but a large proportion of them are for very distinct traffic types” 174/PCPlus.co.uk #181 Those of us who’ve had a good poke around the Kernel configuration menu will have come across the ‘Traffic Shaper’ option and will be wondering what exactly it does. The shaper.o module uses the ‘shapecfg’ user-space tool to assign a ‘shaper’ interface to an existing network device, which has a fixed bandwidth, and you can route traffic over it. This is handy for those times when you need to stop a machine using loads of bandwidth or if you want to reduce the traffic on a network. However, given the static nature of the bandwidth utilisation, it’s really not that useful on a gateway that handles lots of different traffic types. You either end up with a set of different connections fighting for bandwidth, like you have without any shaping, or with a few connections using very little bandwidth, overall not using much of the bandwidth online. It is far simpler to set up than CBQ and SFQ, but don’t let that fool you into thinking that it’s comparable to it.