Docstoc

Detecting Transparent Network Devices

Document Sample
Detecting Transparent Network Devices Powered By Docstoc
					                       Detecting Transparent Network Devices




                 THE UNIVERSITY OF NEW SOUTH WALES

                SCHOOL OF ELECTRICAL ENGINEERING AND
                        TELECOMMUNICATIONS



                Detecting Transparent Network Devices
                                     TM 30



                              Albert Adi-Wijaya

                     Bachelor of Engineering (Electrical)


                                November 2005

                          Supervisor: Dr Tim Moors




Page 1 of 103                                                  Albert Adi-Wijaya
                                           Detecting Transparent Network Devices



Table of Contents
1.   Abstract ....................................................................................................................................8
2.   Acknowledgements ..................................................................................................................9
3.   Introduction ............................................................................................................................10
  3.1.    Background ....................................................................................................................10
  3.2.    Problem Statement .........................................................................................................10
  3.3.    Motivation ......................................................................................................................10
  3.4.    What do we mean by transparent? .................................................................................11
  3.5.    How are we going to detect these devices?....................................................................13
  3.6.    Scope for this thesis project ...........................................................................................13
     3.6.1.     Key features............................................................................................................13
     3.6.2.     Key challenges .......................................................................................................13
  3.7.    Organisation of thesis.....................................................................................................14
  3.8.    What’s included in the CD-ROM?.................................................................................14
4. Related Work..........................................................................................................................15
  4.1.    Similar Thesis Projects...................................................................................................15
     4.1.1.     Improving the Dependability of Networks for End Users .....................................15
     4.1.2.     Network Service Outage Notifications ..................................................................15
     4.1.3.     Fault Report Generation .........................................................................................16
     4.1.4.     Monitoring and measuring server availability .......................................................16
  4.2.    Fault Management..........................................................................................................16
  4.3.    Performance Management..............................................................................................16
  4.4.    Existing Tools ................................................................................................................17
     4.4.1.     Ping.........................................................................................................................17
     4.4.2.     Traceroute...............................................................................................................17
     4.4.3.     LibPcap (WinPCap) ...............................................................................................17
     4.4.4.     Libnet .....................................................................................................................17
     4.4.5.     Nemesis ..................................................................................................................17
     4.4.6.     Ethereal...................................................................................................................18
     4.4.7.     Wget .......................................................................................................................18
5. Detecting Ethernet Switches (Bridges) ..................................................................................19
  5.1.    Introduction ....................................................................................................................19
  5.2.    Aim.................................................................................................................................19
  5.3.    Motivation ......................................................................................................................19
  5.4.    Literature Review...........................................................................................................21
     5.4.1.     SNMP and the Spanning Tree MIB .......................................................................21
     5.4.2.     SNMP and Forwarding Databases of Bridges........................................................23
     5.4.3.     Inferring Ethernet Topology by injecting probe packets .......................................25
  5.5.    Discussion of Existing Approaches................................................................................29
  5.6.    Are bridges really transparent? ......................................................................................29
  5.7.    A ‘new’ approach to detect bridges................................................................................30
  5.8.    Ethernet Topology Discovery using BPDUs .................................................................30
     5.8.1.     Detecting a single bridge........................................................................................31
     5.8.2.     Detecting other bridges ..........................................................................................31
     5.8.3.     Simple Topology ....................................................................................................32
     5.8.4.     Using the Root Cost Path to infer other bridges.....................................................33
Page 2 of 103                                                                                                         Albert Adi-Wijaya
                                          Detecting Transparent Network Devices

     5.8.5.    Assumptions about network ...................................................................................35
     5.8.6.    Formal Statement of Layer 2 Ethernet Topology Discovery Algorithm ...............36
     5.8.7.    Design & Implementation ......................................................................................39
     5.8.8.    Results ....................................................................................................................41
  5.9.    A modified approach......................................................................................................43
     5.9.1.    Discovering the Ethernet Topology using the Bridge Connection Theorem .........44
     5.9.2.    Path Overlap Test ...................................................................................................45
     5.9.3.    The basic idea.........................................................................................................46
     5.9.4.    Removing the endpoints.........................................................................................48
     5.9.5.    A comprehensive example .....................................................................................49
     5.9.6.    Design & Implementation ......................................................................................51
     5.9.7.    Results ....................................................................................................................52
  5.10.      Discussion ..................................................................................................................52
     5.10.1. BPDU availability on LANs ..................................................................................53
     5.10.2. Spanning Tree Vulnerabilities................................................................................53
     5.10.3. Logical Vs Physical Topology ...............................................................................54
     5.10.4. Distributed Vs Stand-alone System........................................................................54
     5.10.5. Performance Analysis ............................................................................................55
     5.10.6. Performance Analysis: Reconfiguring Spanning Tree...........................................57
  5.11.      Conclusion..................................................................................................................58
  5.12.      Alternate phrasing of Path Overlap Test ....................................................................58
6. Detecting Web Caches ...........................................................................................................60
  6.1.    Introduction ....................................................................................................................60
  6.2.    Types of Web Caches.....................................................................................................60
  6.3.    Web Proxy......................................................................................................................60
  6.4.    Transparent Interception Cache .....................................................................................61
  6.5.    Detecting Web Proxies...................................................................................................61
     6.5.1.    Web Browser Configuration Settings ....................................................................61
     6.5.2.    Protocol Analysis ...................................................................................................62
  6.6.    Detecting Transparent Interception Caches ...................................................................64
     6.6.1.    Performance Analysis ............................................................................................64
     6.6.2.    Protocol Analysis ...................................................................................................66
     6.6.3.    Web Cache Protocols .............................................................................................69
  6.7.    Design and Implementation ...........................................................................................70
     6.7.1.    TTL Experiment.....................................................................................................70
     6.7.2.    Performance Measurement Experiments ...............................................................71
     6.7.3.    Performance Measurements End User Tools .........................................................73
  6.8.    Discussion ......................................................................................................................75
     6.8.1.    Other Web Cache Performance Metrics.................................................................75
     6.8.2.    Viability of the TTL approach to detect caches .....................................................76
     6.8.3.    Performance Based Detection of Web Caches.......................................................77
  6.9.    Conclusion......................................................................................................................77
7. Network Address Translators (NATs) ...................................................................................78
  7.1.    Introduction ....................................................................................................................78
  7.2.    Aim.................................................................................................................................78
  7.3.    Motivation ......................................................................................................................78
  7.4.    ipconfig & IP address website........................................................................................78
Page 3 of 103                                                                                                       Albert Adi-Wijaya
                                          Detecting Transparent Network Devices

  7.5.    Counting the number of hosts on the Internet ................................................................81
  7.6. STUN .............................................................................................................................82
  7.7. STUNT ...........................................................................................................................84
     7.7.1.    STUNT 1 ................................................................................................................85
     7.7.2.    STUNT 2 ................................................................................................................86
     7.7.3.    NATBlaster ............................................................................................................86
     7.7.4.    Peer-to-Peer NAT...................................................................................................87
  7.8.    Packet Mangling.............................................................................................................88
  7.9.    Detecting routers and layer 3 topology discovery..........................................................88
  7.10.      Discussion ..................................................................................................................89
  7.11.      Conclusion..................................................................................................................89
8. Firewalls .................................................................................................................................90
  8.1.    Introduction ....................................................................................................................90
  8.2.    Aim.................................................................................................................................90
  8.3.    Motivation ......................................................................................................................90
  8.4.    Existing Tools to detect firewalls...................................................................................90
     8.4.1.    NMap......................................................................................................................90
     8.4.2.    Firewalking.............................................................................................................91
     8.4.3.    Layer 3 Topology Discovery..................................................................................91
  8.5.    Conclusion......................................................................................................................92
9. Summary ................................................................................................................................93
  9.1.    Bridges ...........................................................................................................................93
  9.2.    Web Caches....................................................................................................................93
  9.3.    Network Address Translators.........................................................................................93
  9.4.    Firewalls .........................................................................................................................93
10.     Conclusion..........................................................................................................................94
11.     Appendix ............................................................................................................................95
  11.1.      Appendix A: References ............................................................................................95
     11.1.1. Bridges ...................................................................................................................95
     11.1.2. Web Caches............................................................................................................95
     11.1.3. NATs ......................................................................................................................95
     11.1.4. Firewalls .................................................................................................................95
     11.1.5. General ...................................................................................................................97
  11.2.      Appendix B: Acronyms..............................................................................................98
  11.3.      Appendix C: Summary Sheet.....................................................................................99
  11.4.      Appendix D: Thesis Pointers....................................................................................101
  11.5.      Appendix E: Instructions..........................................................................................103




Page 4 of 103                                                                                                       Albert Adi-Wijaya
                                       Detecting Transparent Network Devices



Table of Figures
Figure 1: Network Devices in the OSI Reference Model ..............................................................12
Figure 2: Improving Network Dependability for the end user.......................................................15
Figure 3 - (Left: BEFORE) (Right: AFTER) .................................................................................20
Figure 4: Ethernet Switch Outage Notification..............................................................................21
Figure 5: Direct Link Analysis - Bridge MIB [Source: RFC 1493]...............................................22
Figure 6: Direct Link Analysis - Two bridges directly connected to one another .........................22
Figure 7: Phase 1 - Segmenting the network into a tree topology .................................................26
Figure 8: Phase 2 - Resulting Tree Topology ................................................................................26
Figure 9: Phase 2 - Possible Scenario I ..........................................................................................26
Figure 10: Phase 2 - Possible Scenario II.......................................................................................26
Figure 11: Hosts and bridges grouped into 'islands'.......................................................................27
Figure 12: Path Crossover Test is used to investigate connections between switches ..................28
Figure 13: Detecting a single bridge ..............................................................................................31
Figure 14: Simple Topology ..........................................................................................................32
Figure 15: Bridges which cannot be detected ................................................................................33
Figure 16: Using Root Cost Path (RCP) to discover other bridges................................................34
Figure 17: Scenario 1 (LEFT) & Scenario 2 (RIGHT) ..................................................................35
Figure 18: Scenario 1 (LEFT) & Scenario 2 (RIGHT) ..................................................................36
Figure 19: Sample Network Topology...........................................................................................37
Figure 20: 'Master' Station injecting BPDUs onto network ...........................................................37
Figure 21: Stations capturing BPDUs from the bridges.................................................................38
Figure 22: 'Master' Station collecting BPDU information from other stations..............................39
Figure 23: Design of Ethernet Topology Mapper ..........................................................................41
Figure 24: Sample Topology Output 1...........................................................................................42
Figure 25: Sample Topology Output 2...........................................................................................42
Figure 26: Sample Topology Output 3...........................................................................................43
Figure 27: Islands of hosts and bridges..........................................................................................44
Figure 28: Example Network .........................................................................................................45
Figure 29: Example Network .........................................................................................................47
Figure 30: Network - all direct links discovered............................................................................48
Figure 31: Example Network - Only direct links at the fringe of the network are discovered ......49
Figure 32: Remaining Bridge Topology ........................................................................................49
Figure 33: A comprehensive example of an Ethernet switched network.......................................50
Figure 34: Example network with some links removed.................................................................50
Figure 35: Example network with more links removed.................................................................51
Figure 36: Example network with even more links removed ........................................................51
Figure 37: Example network - ALL links have been discovered...................................................51
Figure 38: System Design of BCT implementation .......................................................................52
Figure 39: Attacker becoming the STP root and causing the spanning tree to reconfigure...........54
Figure 40: UNSW CSE Computing Help Desk Policy ..................................................................54
Figure 41: Combinations formula ..................................................................................................55
Figure 42: Select a bridge then choose two at a time from the remainder .....................................55
Figure 43: Number of combinations possible with N bridges .......................................................56
Figure 44: Experimental Setup – Reconfiguring the Spanning Tree .............................................57
Figure 45: Web Proxy Example .....................................................................................................60
Page 5 of 103                                                                                              Albert Adi-Wijaya
                                         Detecting Transparent Network Devices

Figure 46: Transparent Interception Caching Example .................................................................61
Figure 47: IE6 - Web Proxy Configuration....................................................................................62
Figure 48: Web Proxy Configuration.............................................................................................63
Figure 49: GET Request with no proxy .........................................................................................63
Figure 50: GET Request with proxy ..............................................................................................64
Figure 51: Performance Analysis – Round Trip Time ...................................................................64
Figure 52: Elapsed Time Measurement [Source KUROSE 2004].................................................65
Figure 53: Graph of Elapsed Time Measurements.........................................................................66
Figure 54: Ethereal Packet Capture from 144.135.8.152, TTL = 61 .............................................67
Figure 55: Ethereal Packet Capture from 203.26.51.42, TTL = 61 ...............................................67
Figure 56: Ethereal Packet Capture from 202.58.48.51, TTL = 61 ...............................................68
Figure 57: Ethereal Packet Capture from 65.170.56.4, TTL = 61 .................................................69
Figure 58: TTL Experiment Configuration....................................................................................70
Figure 59: iptables command to redirect HTTP traffic from client to cache .................................71
Figure 60: Web Cache Performance Analysis Experimental Setup...............................................72
Figure 61: Web Cache Detection Tool – Flowchart ......................................................................74
Figure 62: Sample Outputs of the Web Cache Detection Tool......................................................74
Figure 63: Configuration with NO proxy.......................................................................................75
Figure 64: Configuration with proxy .............................................................................................75
Figure 65 - How a public IP address is discovered ........................................................................79
Figure 66 - http://myipaddress.com/ ..............................................................................................80
Figure 67 - http://www.formyip.com/ ............................................................................................80
Figure 68 - http://www.showmyip.com/au/ ...................................................................................80
Figure 69 - http://www.mywanip.com/ ..........................................................................................80
Figure 70 - http://www.urgentclick.com/whats-my-ip-address.php ..............................................80
Figure 71 - ipconfig........................................................................................................................81
Figure 72: Counting the number of NATs by using the IP ID field...............................................81
Figure 73: Graph of IP ID fields ....................................................................................................82
Figure 74: A Typical STUN Configuration [Source: STUN RFC3489] .......................................83
Figure 75: WinSTUN Output.........................................................................................................84
Figure 76: STUNT #1 [STUNT 2004]...........................................................................................85
Figure 77: STUNT #2 [STUNT 2004]...........................................................................................86
Figure 78: NATBlaster [STUNT 2004] .........................................................................................87
Figure 79: P2PNAT [STUNT 2004] ..............................................................................................88
Figure 80: Example use of NMap ..................................................................................................91
Figure 81: Sample Output of Firewalk...........................................................................................91




Page 6 of 103                                                                                                     Albert Adi-Wijaya
                                        Detecting Transparent Network Devices



Table of Tables
Table 1: Contradictions in Bridge Connections [Source: LOWE 2001]........................................25
Table 2: Path Costs [Source: IEEE802.1D] ...................................................................................33
Table 3: Station A - hop table ........................................................................................................39
Table 4: Station B - hop table.........................................................................................................39
Table 5: Station C - hop table.........................................................................................................39
Table 6: Station D - hop table ........................................................................................................39
Table 7: # of frame/byte transmissions for Bridge Topology Algorithm ......................................56
Table 8: Experimental Results - Reconfiguring the Spanning Tree: TCP Stream Test.................57
Table 9: Experimental Results - Reconfiguring the Spanning Tree: UDP Stream Test ................58
Table 10: % reduction in elapsed time ...........................................................................................72
Table 11: % reduction in Inter-Arrival Time .................................................................................73
Table 13: Web Latency Response Times - Source: [LIU 1998] ....................................................76
Table 14: Definitions - Source [LIU 1998]....................................................................................76




Page 7 of 103                                                                                                 Albert Adi-Wijaya
                             Detecting Transparent Network Devices




1. Abstract
The Internet is a technology that is prevalent in today’s society and one on which end users are
becoming increasingly dependent upon. However, many end users are mystified when their
internet access ceases to operate or its performance is severely degraded. Furthermore, end users
are not empowered with the ability to localise where these faults are occurring and find it
particularly frustrating when it is occurring in a device they were completely unaware of. In order
to assist the end user, this thesis developed a number of tools and techniques to detect the
transparent networking devices that end users are dependent upon when accessing the Internet.
These tools and techniques are based on the identification of specific packets created by certain
devices, observing subtle changes in packet headers and also through data analysis methods.

The implementation of these tools and techniques has achieved various levels of success, with
some permitting the discovery of bridges, web caches, network address translators and firewalls,
whilst others allowing the perfect identification of them.

We developed a collaborative end user system to allow the detection/identification of bridges and
the topology they are arranged in. Furthermore, we have developed performance analysis tools to
measure the response time and inter-arrival time of packets from web caches, providing a
mechanism by which they can be detected. In addition, we have also surveyed existing methods
to detect the presence of network address translators and firewalls.

As a result, we have proven that although many networking devices are intentionally designed to
be transparent to the end user, mechanisms do exist which enable the detection of such devices.




Page 8 of 103                                                                  Albert Adi-Wijaya
                           Detecting Transparent Network Devices




2. Acknowledgements
Firstly, I would like to thank my thesis supervisor, Dr Tim Moors, for his time, effort and
assistance in the production of this thesis. His guidance and expertise has proven to be
invaluable.

I would also like to thank my family and friends for the support they have provided during the
course of this thesis.




Page 9 of 103                                                              Albert Adi-Wijaya
                             Detecting Transparent Network Devices




3. Introduction
   3.1. Background
As the popularity of the Internet proliferates, end-users are becoming increasingly dependant on
the abundance of services it provides. The public Internet is a complex interconnection of high
speed links and switches, vastly distributed amongst many different administrative domains. It is
inevitable that a system as large and complicated as the Internet be prone to errors and faults
which may deny end users access to services they require. Compounding the problem further,
many end users are not well-versed in networking technology and thus un-able to diagnose a
problem, solve it or even provide useful information when soliciting the assistance from experts.
As a result, this has prompted a need for systems that improve the dependability and reliability of
communication networks for the end user.

   3.2. Problem Statement
The goal of this thesis is to design and develop systems which allow an end user to detect the
network elements that he/she is connected to, or reliant upon for access to internet services. Quite
often, these devices will be transparent to the end user whom is un-aware that he may be
accessing web pages from a web cache rather than the intended web server, or be residing behind
a NAT and sharing an IP address with other hosts when accessing the internet, or have it’s
packets being filtered by a firewall. Being able to detect such seemingly transparent devices, will
assist the end user in diagnosing network problems, interpreting network outage notifications and
also generating network fault reports which may be passed on to IT help desks or ISPs.

   3.3. Motivation
Ethernet switches (bridges), firewalls, web caches, network address translators and the like, are
all useful networking devices which when configured and connected correctly, play an important
role in providing a reliable and efficient communications service. Such devices are designed to be
transparent to the end user, as it promotes their ease of deployment. It is natural to for the curious
reader to ponder, why an end user would wish to detect these devices, since after all, they are
intentionally designed to be transparent? There are a number of circumstances in which an end
user may wish to be empowered with the knowledge of which network devices they are reliant
upon when accessing the internet. These include:

   1. Localise and diagnose the network faults
      End users can be hindered when troubleshooting networking problems if the fault is
      occurring in a device they are unaware of. By being aware of what/which devices an end
      user is dependant upon, can facilitate the ease of localising where such faults are
      occurring.

   2. Interpret outage notification systems

Page 10 of 103                                                                    Albert Adi-Wijaya
                             Detecting Transparent Network Devices

       End users can interpret planned network outage notifications distributed by network
       administrators, by being empowered with the knowledge of which devices are to be made
       temporarily offline and which devices they are dependant upon.

   3. Generating fault reports
      End users who are not particularly well-versed in networking technology can often
      struggle when trying to describe network problems he/she is experiencing. This often
      frustrates the IT help desk operator whom is confronted with the challenge of trying to
      solve a problem which is not clearly defined. A standardised fault report which details the
      devices an end user is connected to and times that loss of connectivity was experienced
      etc. may be useful in the efficient rectification of the problem.

   4. Measuring Server Availability
      A user who is monitoring the availability of a web server may over-estimate its
      availability if it is accessed via a transparent interception cache, which may send replies
      when the origin server is unavailable. The ability to detect a web cache may assist the user
      in this situation.

   5. Protocol Tuning
      The performance of networking protocols may be enhanced if they are tuned to be aware
      of the presence or parameters of transparent devices. For example, TCP’s congestion
      control algorithm is appropriate for paths that traverse nodes that may congest (e.g.
      bridges or routers), but TCP’s Slow Start may unnecessarily slow communication
      between end-systems separated by a single hop. If end-systems could determine that the
      path does not traverse intermediate systems, even transparent bridges, then they might
      choose to disable TCP’s Slow Start. The end user will benefit from an enhanced
      performance by its networking protocols.

   3.4. What do we mean by transparent?
Before proceeding further, we should elaborate on what we mean by a networking device being
‘transparent’. It does after all seem contradictory to be able to detect a device, if it is truly
transparent. A number of interpretations of the word transparent are useful in relation to the work
of this thesis project. These include:

   1. The end system need not be specially configured to operate with the network device i.e.
      the device is essentially ‘plug and play’.
      Note: there may be some exceptions, for example some setups of proxy caches require the
      configuration of the end user’s browser to ‘point’ to the proxy cache.

   2. The device is an ‘intermediary’ node located on a path between end systems and is not
      directly addressed to, by the end user/system. Eg. An end user accessing a web server
      may have to pass through a firewall first.

   3. The device is discrete and subtle in its behaviour i.e. a device accepts a packet on an
      incoming interface before making a ‘minor’ modification to the packet and then

Page 11 of 103                                                                 Albert Adi-Wijaya
                            Detecting Transparent Network Devices

       transmitting it on an outgoing interface. E.g. a router decrementing the TTL field of an IP
       packet header it receives, before forwarding the packet towards its destination.

   4. These devices typically reside in a layer below the application layer of the OSI layer, e.g.
      the network, data-link and physical layers as depicted in Figure 1.
      The Internet itself is modular in design i.e. the OSI reference model consists of layers,
      each providing clearly defined services to the layers above, whilst abstracting details of its
      implementation. Such modularity facilitates the rapid deployment of many applications
      which need not be specially configured to interact with the various devices which reside
      in each of these layers. These devices thus appear to be transparent to the end user.

   5. The end user may not be well-versed in communication networks and thus unaware or
      completely oblivious of the network elements that he/she is dependent upon for access to
      network services.
      Note: In this case, ANY network device regardless of whether it satisfies the definitions
      above will be considered transparent.



                                                 firewall
                                                 (application
                                                 layer gateway)
         Application                                                                      web caches

        Presentation


          Session
                                                                                    firewall
                                                             router                 (packet filter)
         Transport                          NAT

          Network

                                            bridge / Ethernet switch
         Data-link
                                                                                                       hub

          Physical                      repeater         Media Converter
                                                                               SD




                                                                                                      multiplexer
                                                    5VDC.
                                                   _ __ __ 1A
                                                         +
                                                                               RX

                                                   UP LINK                     TX


                                                           LINK   PWR   LINK



                             modem          media converter                         optical
                                                                                    transmitter
                        Figure 1: Network Devices in the OSI Reference Model

As such, devices which we consider to be transparent would include routers, bridges, web caches,
firewalls, network address translators, repeaters, hubs etc. Other devices which we consider to be
NOT transparent include DNS servers (as they are directly addressed by the end system), web


Page 12 of 103                                                                                    Albert Adi-Wijaya
                             Detecting Transparent Network Devices

servers, ftp servers, email servers (as they are directly addressed by the end user and are not
‘intermediary’) etc.

   3.5. How are we going to detect these devices?
Fortunately, these devices are not entirely transparent, as they do possess certain characteristics
which may be exploited to discover their presence. Applications listen for connections on ‘well-
known’ port numbers e.g. HTTP on port 80, FTP on port 21. Other devices may alter certain
fields in packet headers as packets traverse through them, such as routers which decrement the
TTL field in IP packets.

The tools and techniques to be developed will be based on:
   • The identification of specific packets emitted only by certain devices.
   • Analysing packet headers and examining subtle changes/modifications.
   • Data analysis (performance measurements).

   3.6. Scope for this thesis project
To define a feasible scope for the thesis project, we will limit the detection of transparent
networking devices to the following:
   • Ethernet switches (bridges)
   • Web caches
   • Network Address Translators
   • Firewalls

Where possible (or ideally), we attempt to achieve a complete identification of the device. That
is, not only can we actually confirm that the end user is dependent on and connected to such a
device (i.e. confirm its presence), but can also obtain a description of its salient features e.g. IP
address, MAC address, physical location, manufacturer, what operating system is being run on it
etc.

       3.6.1.      Key features

The tools and techniques that are to be developed in this thesis will incorporate the following
features:
    • Operate from the end user’s perspective. Many of the existing systems operate from the
        perspective of a network/system administrator and not from that of an end-user.
    • Require no special privileges or knowledge by the end user.
    • Be network security friendly and not raise alarms with intrusion detection systems.

       3.6.2.      Key challenges

The key challenges that will confront the author in the successful development of a system
include:


Page 13 of 103                                                                   Albert Adi-Wijaya
                            Detecting Transparent Network Devices


   •   Network security: As packet capturing techniques are to be used, this raises security
       concerns as packet sniffing may allow a user to access information not intended for
       him/her.
   •   Firewalls, NATs, web caches and bridges have many different implementations and
       configurations. We need to develop techniques that are generic and can to apply to as
       many scenarios as possible.

   3.7. Organisation of thesis
We begin by reviewing literature and analysing existing tools related to the work investigated in
this thesis project. Next, we proceed to describe the tools and techniques developed to detect
transparent network devices individually. To do so, the thesis is divided into four sections,
namely bridges, web caches, NATs and firewalls. In each we outline our approaches to detect the
devices and discuss its performance. Finally, a discussion section and a summary will conclude
the thesis.

   3.8. What’s included in the CD-ROM?
At the back of this document you will find attached a CD-ROM which includes an electronic
copy of this thesis dissertation, the source code developed, software used, papers reviewed and
useful tutorials.




Page 14 of 103                                                                Albert Adi-Wijaya
                            Detecting Transparent Network Devices




4. Related Work
   4.1. Similar Thesis Projects
       4.1.1.     Improving the Dependability of Networks for End Users

This thesis is part of a broader topic which explores the development of systems which improve
the dependability of the internet for end users.


       Fault Localisation           Network Service                      Fault Report
         / Detection               Outage Notifications                   Generation



                      Detecting Transparent Network Devices

                     Figure 2: Improving Network Dependability for the end user

       4.1.2.     Network Service Outage Notifications

Network service outage notifications are designed to inform affected users of services that will
become temporarily unavailable in the future [TM 32]. In order for this to be successful, the
following three criteria must be satisfied:
    • Who: Only users that are affected by this particular service outage should be informed.
    • What: The particular services that are unavailable should be mentioned in these
       notifications. E.g. Internet access, Intranet access, database servers, file servers, mail
       server etc.
    • When: The exact timing of when this outage is to occur must be made known.

Many network service outage notifications however, are deficient in the following areas:
  • Many messages are broadcast to all end users, including those that remain un-affected by
      the network outage. Network administrators guilty of such a habit reduce the credibility of
      future messages as many end users may get into the habit of ignoring them.
  • Many notifications are un-comprehendible to end users lacking the technical literacy to
      extract any useful information from these seemingly useless messages. E.g. If internet
      access if unavailable does this effect email as well?
  • At times, the specific services that are affected are not mentioned. For example, though
      internet access may be unavailable, the intranet or email remains unaffected.

A system which can detect the particular network devices that an end user is connected to or is
reliant upon for certain network services, can assist the end user or network administrator to
determine relevancy of a particular network service outage for an end user.
Page 15 of 103                                                                    Albert Adi-Wijaya
                             Detecting Transparent Network Devices


       4.1.3.      Fault Report Generation

Network faults can be more efficiently solved if the network administrator can observe the
problem from the view of the end user, and be provided with information outlining the events
that occurred which lead to the network fault [TM 25]. In addition, the reporting of network
faults regarding web services can be hindered, if the provider is examining service logs to
identify the fault via the IP address/port number supplied by the user, and there is a NAT between
them. Being able to detect the transparent device will assist greatly in this situation.

       4.1.4.      Monitoring and measuring server availability

The ability to monitor and measure the availability Internet services such as HTTP (WWW) or
SMTP (email) can assist mystified end users diagnose problems experienced when accessing
such services [TM 34].

   4.2. Fault Management
Fault management plays an important role in the proactive/reactive detection, diagnosis and
recovery from network faults. There has been widespread research investigating strategies,
algorithms and architectures for the detection, localising, diagnosing and correction of network
faults.

Communication networks are susceptible to faults from time to time. Should such a network fault
occur being able to diagnose the problem and localise where the fault is occurring is essential for
the end user or network administrator to solve the problem efficiently and effectively.

Projects include the proactive detection of network faults through the use of SNMP and
observing unusual changes in the MIB variables of a device. These variations are threshold
against a set value, which if exceeded causes an alarm to be raised [THOTTAN 1998].

   4.3. Performance Management
As a network’s performance tends to deteriorate in the presence of network faults, being able to
localise, diagnose and correct these efficiently is vital to maintain a high level of QoS. Research
in this area from the end user’s perspective includes [MAHAJAN 2003], where a tool called
tulip is developed that can locate a reordering or loss of packets to within three hops (routers)
and a queuing of packets to within four hops. This is achieved through novel uses of probes to
routers, ICMP timestamp requests to routers and the ID field in IP packet headers. This system
requires the end user to have no special privileges and operate amongst many administrative
domains.

The tulip program heavily relates to the work carried out by the author in that the system
operates from the end user’s perspective whilst requiring no special privileges/knowledge from
the end user.


Page 16 of 103                                                                  Albert Adi-Wijaya
                             Detecting Transparent Network Devices


   4.4. Existing Tools
In this section of the thesis, we examine existing tools that can detect transparent devices to some
extent. We also provide a brief overview of the tools that are to be used in this thesis project.

       4.4.1.      Ping

Ping is a program which allows the reachability of a destination host to be determined
[STEVENS 1994]. This tool could aid an end user in determining connectivity to other devices
which reside in the network layer e.g. web servers, routers, ftp servers, etc. However, one
obvious disadvantage is that the end user will be required to know the particular IP addresses or
domain name of the host that to ping to in the first place, and is thus limited in it’s ability to
detect transparent devices. Ping may be more suited to monitoring the availability of a known
device, in the context of enhancing the dependability for the end user.

       4.4.2.      Traceroute

Traceroute is a tool which allows a user to determine the route that is taken by a packet from a
source host to a destination host [STEVENS 1994]. This is done by incrementing the TTL,
starting from zero, of the IP packets it sends out and observing the ICMP error messages
returned.

Such an application will aid the user in determining what routers he/she is connected to. Of
particular interest would be the routers that are on the same subnet or domain as the end user as
these would be more relevant to the end user. Also, as will be discussed in a later section of the
thesis, we show that traceroute is useful in mapping (network) layer 3 topology.

       4.4.3.      LibPcap (WinPCap)

Libpcap is a system-independent interface for user-level packet capture. This API will be used to
capture and identify packets, and also to examine for any changes in the various fields of the
packet headers [LIBPCAP]. WinPCap is a windows port of libpcap.

       4.4.4.      Libnet

Libnet is a generic API which allows the creation of packets and their injection onto a network. A
variety of protocols supported including Ethernet, IP, TCP, UDP etc. Some of the active
approaches in this thesis will involve the creation of packets and libnet will be useful in the
development of such tools [LIBNET].

       4.4.5.      Nemesis

Nemesis is a command-line network packet crafting and injection utility built on top of the
Libnet API. This provides a much simpler and user friendly mechanism to create and inject
packets [NEMESIS].

Page 17 of 103                                                                   Albert Adi-Wijaya
                            Detecting Transparent Network Devices


       4.4.6.     Ethereal

Ethereal is a GUI network protocol analyser which allows the capture and inspection of packets.
This provides a simple and graphical method to examine various fields in the packet headers
[ETHEREAL].

       4.4.7.     Wget

Wget is a useful tool to retrieve files using protocols such as HTTP, HTTPS and FTP. Its ability
to run from a command line prompt promotes it ease of use in scripts [WGET]. With regards to
the detection of web caches in this thesis project via performance measurements, the wget tool is
used.




Page 18 of 103                                                                Albert Adi-Wijaya
                             Detecting Transparent Network Devices




5. Detecting Ethernet Switches (Bridges)
   5.1. Introduction
Ethernet switches (bridges) are useful networking devices that provide connectivity to hosts
located on separate shared LAN segments. Their operation and implementation is transparent to
end systems (stations) which has facilitated their ease of deployment. This transparency does
come with its disadvantages, namely the difficulty for an end user to localise a fault occurring in
such a device as its existence is unknown to the end user.

We thus explore techniques which the end user may adopt to detect Ethernet switches (referred to
more generally as bridges from now on).

   5.2. Aim
The aim of this part of the thesis project is to develop tools and techniques which can be
deployed by end users to detect and identify bridges (e.g. their MAC Address) within their subnet
or LAN. We find that the detection of a single bridge to be quite trivial and thus broaden our
scope to explore the vibrant area of Layer 2 Ethernet Topology Discovery.

First we begin by motivating why an end user may be interested in bridges they are directly
connected to, or more broadly, the overall Ethernet topology of the subnet/LAN. Next, we review
literature to evaluate the existing approaches to Ethernet topology discovery. We then proceed to
describe our approaches and discuss their design and implementation. Finally we conclude with
the results achieved and their limitations.

   5.3. Motivation
Before proceeding, we should contemplate briefly on why an end user may be interested in being
empowered with the knowledge of which bridges he/she is connected to, and in addition,
mapping the layer 2 topology of the LAN he/she is connected to.

      “…the complexity of performing Ethernet topology discovery arises from the inherent
transparency of Ethernet bridge hardware. Endpoints are unaware of the presence of bridges in
   the network. The bridges themselves only communicate with their neighbours in the limited
       exchanges of the spanning tree protocol and that is not used in all environments…”
                                         [LOWE 2001]

Bridges are very subtle and discreet in their behaviour and hence their presence is essentially
unknown to the end user. Though this facilitates the ease of deployment of these devices onto a
network, it makes it quite troublesome for end users to localise faults in such devices. As the end
user is unaware of these devices, how could he/she ever localise a network fault occurring in such
a device?

Page 19 of 103                                                                 Albert Adi-Wijaya
                               Detecting Transparent Network Devices

Furthermore, the end user may also be interested in other bridges than that he/she is directly
dependant upon, as these may provide access to important network services (e.g. email server,
file server).

As the work carried out in this thesis falls under the category of fault/performance management
by the end user, the following reasons for mapping Ethernet topology logically arise:
    • By inspecting the topology over a period of time, pro-active fault detection may be
        possible. E.g. certain paths which were previously active suddenly being unavailable may
        indicate a fault in a device/link. In Figure 3 below, the bridge circled, may have a fault.
    • Frequently, the initial step in network fault diagnosis is to examine the topology of the
        network to determine which links, devices or computers are reachable and compare this to
        the expected state.
    • There may be other ‘important’ bridges that the end user may wish to know about other
        than the one he/she is directly connected to. E.g. the bridge that is connected to the
        gateway router.
    • For performance management, the topology can be used to help in locating areas of
        congestion. For example, finding the path between a pair of hosts which may be
        experiencing poor VoIP performance.

There are many network management systems that a network/system administrator can use to
carry out the aforementioned tasks; however we believe that they better done by the end user.
This is because it is the end user who is directly experiencing the problem and is thus positioned
better to directly view, analyse and hopefully suggest a fix to the problem.




                           bridge                                    bridge




                 station                                   station
                               Figure 3 - (Left: BEFORE) (Right: AFTER)

The figure below provides a real life example of when it would be useful for end users to be
knowledgeable about the bridge they were dependant on. The broadcast email from the School of
EET’s network administrator informing of a bridge outage makes no mention of who specifically
will be affected or what network services will be affected. This can be particularly frustrating for

Page 20 of 103                                                                  Albert Adi-Wijaya
                             Detecting Transparent Network Devices

lay users who are unsure if the problem will affect them. By being able to detect the bridge they
are connected to, an end user can more competently interpret the outage notification.

>   From: jeffl@syscon.ee.unsw.edu.au
>   [mailto:jeffl@syscon.ee.unsw.edu.au]
>   Sent: Wednesday, 30 April 2003 2:36 PM
>   To: staff@ee.unsw.edu.au; pgresearch_all@ee.unsw.edu.au
>   Subject: Network service disruptions on 149.171.36.0/23 net on Comms
>   Unit's switch eebu2s5
>
>
>   To all users:
>
>   There is a hardware problem on Comms Unit's CISCO switch
>   (eebu2s5) in cabinet 2, Rm 240. The switch is not routing!!
>   It affects some of you (24 ports on the switch)!! The problem has been
>   reported to Comms Unit.
>
>   Cheers,
>       - jeff
                             Figure 4: Ethernet Switch Outage Notification

     5.4. Literature Review
This section provides a brief overview of existing approaches to layer 2 path discovery. A
detailed examination is beyond the scope of this thesis.

There are essentially three different approaches that have been proposed thus far to discovery
Ethernet topology. These are:
   • Use of SNMP to access a bridge’s Spanning Tree MIB
   • Forwarding Databases of bridges
   • Inferring the Ethernet topology by injecting probe packets

All approaches follow a similar methodology to map the Ethernet topology. It begins by
identifying all the nodes (hosts and bridges), and then determining the direct connections between
the bridges to complete the topology discovery. We examine each of the approaches, individually
below.

        5.4.1.      SNMP and the Spanning Tree MIB

The approach outlined in [STOTT 2002] allows Layer 2 paths to be discovered by first using
SNMP to access the Spanning Tree MIBs of bridges to gather the necessary data, and then
processing the data to determine the topology of the hosts and bridges.

It consists of the following three phases:
    • Data Collection
    • Direct Link Analysis
    • Endpoint Analysis.


Page 21 of 103                                                                 Albert Adi-Wijaya
                              Detecting Transparent Network Devices

The Data Collection phase identifies hosts, switches and routers, collects the spanning tree MIBs
from the bridges and the address translation MIBs from the routers. This is achieved by first
using the ExamiNetTM discovery module which scans a network by sending probe SNMP
messages to every IP address in a given range and collecting the responses. The system OID or
other MIB variables are used to classify the device as a host, router or switch etc. The second step
is to collect the MIB tables from the switches, which contain several objects that describe the
results of the spanning tree algorithm. The third step in this phase is to collect the MIB tables
from the routers to obtain the mapping from physical addresses to IP addresses.

The next phase is called Direct Link Analysis. Its purpose is to reconstruct the spanning tree by
finding all direct links between switches. The procedure involves examining the MIB tables for
each switch and identifying the links between switches by their ports.

It applies the following simple rule to determine these direct links:

For each entry in a switch’s dot1dStpPortTable, for which the Designated Bridge is different
from the switch’s own Bridge ID, there is a direct link between the original switch and the switch
identified by the Designated Bridge.

Figure 5 below shows the relevant section of the MIB.

dot1dStpPortDesignatedBridge OBJECT-TYPE
              SYNTAX BridgeId
              ACCESS read-only
              STATUS mandatory
              DESCRIPTION
                        "The Bridge Identifier of the bridge which this
                        port considers to be the Designated Bridge for
                        this port's segment."
              REFERENCE
                        "IEEE 802.1D-1990: Section 4.5.5.6"
              ::= { dot1dStpPortEntry 8 }
                Figure 5: Direct Link Analysis - Bridge MIB [Source: RFC 1493]

                                              MIB
                                          Designated
                                          Bridge ID: Y




                           Bridge ID: X                    Bridge ID: Y




                                                                   Designated Port
              Figure 6: Direct Link Analysis - Two bridges directly connected to one another

Figure 6 above, shows an example where bridge X identifies a direct connection to bridge Y.

Page 22 of 103                                                                        Albert Adi-Wijaya
                             Detecting Transparent Network Devices

The final phase called Endpoint Analysis, completes the topology discovery by determining the
links between the bridges and the hosts.

The above is a simplified treatment of the algorithm developed by [STOTT 2002]. For a more
robust treatment of this approach, the reader is encouraged to read the paper.

        5.4.2.      SNMP and Forwarding Databases of Bridges

The method developed by [BREITBART 2000] uses SNMP to access the Forwarding Databases
(FDB) of bridges and gather the necessary information for mapping the Ethernet topology.

Essentially how this algorithm works is by first selecting a bridge and determining the bridges
directly connected to each port. The rest of the Ethernet topology is discovered by traversing
along each link discovered and repeating the process.

This algorithm is based upon the Direct Connection Theorem, which we state here below.

Let FCx denote the forwarding database for port x of bridge C. Let N denote the number of hosts
in the network.

Direct Connection Theorem: Assume that Fi x and F jy are complete. Two bridges Bi and Bj are
directly connected via the link connecting port x on Bi and port y on Bj if and only if Fi x ∩ F jy =
0 and Fi x ∪ F jy = N .

The completeness requirement of the DCT is a significant short coming, whereby each bridge is
required to have entries for ALL hosts in the network in its forwarding databases, which must be
kept up to date. This can be difficult to achieve, particularly for larger networks and thus limits
how deployable and practical the algorithm is. It has been suggested in [LOWE 2001] that the
extra traffic can be generated by end users e.g. using ping, but this is undesirable, especially
when the network is experiencing faults or severe performance degradation.

As a result, the work in [LOWE 2001] endeavors to improve this theorem, by removing the
assumption of the completeness requirement. It operates on the principle that rather than
attempting to determine if two switches are directly connected to one another, it is much ‘easier’
to determine if two switches are NOT directly connected to one another. This is because only a
simple counterexample is needed to do this. By correlating which bridges are NOT directly
connected, this leaves the result of which bridges are connected.

We provide an example below. Say we have a bridge, A, that has three ports labeled 1, 2 & 3.
Each port has an entry for host X, Y & Z respectively, associated with it in the FDB. Likewise,
we have a bridge B, with two ports, 1 & 2, with an entry for host X associated with the former
and for hosts Y and Z associated with the later in the FDB.

To determine which ports (if any) directly connect the two bridges, we check for any
contradictions between entries in the forwarding databases as illustrated in Table 1 below.
Page 23 of 103                                                                   Albert Adi-Wijaya
                          Detecting Transparent Network Devices



Ports                               Mapping                           Conflict
A B
1   1       {Z}                                                       Y and Z
                  3
                          1   {X}      {X} 1           2 { Y,Z }
                      A                           B
                  2
           {Y}
3    1      {X}                                                          Y
                  1
                          3 {Z}        {X} 1          2 { Y,Z }
                      A                           B
                  2
           {Y}
2    2      {Z}                                                          X
                  3
                          2 {Y}       { Y,Z } 2       1   {X}
                      A                           B
                  1
           {X}
2    1     {Z}                                                           Z
                  3
                          2 {Y}        {X} 1          2 { Y,Z }
                      A                           B
                  1
           {X}
1    2      {Z}                                                       NONE
                  3
                          1   {X}     { Y,Z } 2       1   {X}
                      A                           B
                  2
           {Y}




Page 24 of 103                                                     Albert Adi-Wijaya
                                   Detecting Transparent Network Devices


3     2       {Y}                                                                            X
                       2
                                   3 {Z}      { Y,Z } 2            1    {X}
                           A                                  B
                       1
              {X}
                       Table 1: Contradictions in Bridge Connections [Source: LOWE 2001]

From Table 1 above, we can see that bridges A and B are directly connected via ports 1 and 2
respectively.

This approach removes the assumption of the completeness requirement, and only requires each
bridge’s FDB contain entries for three “appropriately” placed hosts (X, Y & Z). These three
appropriately placed hosts must satisfy the Minimum Knowledge Requirement.

For a more comprehensive treatment of this method, the curious reader is encouraged to consult
[LOWE 2001].

          5.4.3.       Inferring Ethernet Topology by injecting probe packets

The approach in [BLACK 2004] discovers the Ethernet topology by inferring the presence of
bridges by repetitively sending a particular sequence of packets and observing on which hosts the
packets are received. It is a collaborative system whereby participation by a number of end
stations is required to yield accurate results. It should be emphasised that the bridges are not
identified by their MAC/IP Address or the like.

The approach consists of four phases. The first of these involves collating the hosts onto their
shared segments in a tree format. This is achieved by each host sniffing for packets on the
network and determining which other hosts’ packets can be seen. If a host can see the packets of
another host, this implies that they are on the same segment. These results are collated by a
master node, which allows a tree like structure (with respect to the master) to be determined. This
is done so, by each host observing which other hosts’ transmission to the master can be seen. We
illustrate this phase via an example.


                   Hub

                                    Switch
                   A       B
                                                        Hub            Switch
                                                                                      Hub
                                     Hub
                               G                       C      D
                                     J                                                E     F
Page 25 of 103                                                                         Albert Adi-Wijaya
                                    Detecting Transparent Network Devices

                          Figure 7: Phase 1 - Segmenting the network into a tree topology

An arbitrary host, for example host A, is selected1 as the collector. Each other host sends to the
collector, the source addresses of other hosts that it can ‘see’ by sniffing traffic. In Figure 7
above, it is clear that hosts A & B, G & J, C & D and E & F can see each other. Moreover, hosts
C & D, can also see the traffic from E & F which is being passed to host A. This allows host A to
sort the hosts into a tree, where the collector A is at the root of the tree, with hosts being placed
above the others whose transmissions to host A, can be seen.

The result of this phase is shown in below.

                                                      Segment:
                                                        A, B


                                          Segment:                Segment:
                                            G, J                    C, D


                                                                 Segment:
                                                                    E, F
                                    Figure 8: Phase 2 - Resulting Tree Topology

The second phase involves inferring the bridges that each host in each segment is connected to.
More fundamentally, it determines whether or not, two hosts on the same segment are connected
to the same bridge i.e. distinguishing between the scenarios in Figure 9 and Figure 10. This is
done via a unique sequence of packet transmissions.




                                                     A              B
                                       Figure 9: Phase 2 - Possible Scenario I




                                              A                            B
                                      Figure 10: Phase 2 - Possible Scenario II



1
    The paper makes no mention of how this is done e.g. via a voting mechanism and the result broadcast
Page 26 of 103                                                                                 Albert Adi-Wijaya
                                   Detecting Transparent Network Devices


   •   Host A: M       L1
   •   Host B: M       L2
   •   Host A: A       M

First, host A sends a probe packet to some address L1 with a spoofed source address, say M.
Secondly, host B then sends a probe packet to some address L2 with a spoofed source address M
as well. Thirdly, host A sends a probe packet with destination address M, with a source address
of A. (Note: L1 and L2 are addresses on the same port as host A and B respectively, which the
bridge(s) needs to have learnt).

Now, if host B receives the probe packet from host A, then both A and B are connected to the
SAME switch. If host B does not receive the probe packet, then hosts A and B are connected to
DIFFERENT switches.

This test is repeated for each pair of hosts on each segment.

Thus far, each host and the bridge they are directly connected to have been discovered. Phase 3,
then discovers the presence of any other switches which provide connectivity between the islands
as shown in Figure 11 below. In particular it confirms that Island 0 and Island 1 are connected via
switch B, and infers the presence of switches S and T, through a sequence of probe packet
transmissions and observing who can see them.



                                   B           A               M
                                                                        Island 0

                              Switch: B     Hub             Switch




                                           Switch: S           Switch: T
                    Switch




                   C           D                                 Hub          Switch
                                                   Switch
                   Island 1


                                                       E             F              G
                                                                   Island 2
                              Figure 11: Hosts and bridges grouped into 'islands'

Phase 4 completes the topology discovery by examining the nature of any gaps in the network,
that is, segments between switches S and T to determine the presence of any other additional
switches. They may be directly connected to one another or perhaps via another bridge. A
recursive use of their path crossing test is required to accomplish this phase.

Page 27 of 103                                                                          Albert Adi-Wijaya
                               Detecting Transparent Network Devices

The path crossing test is stated below:

Path Crossing Test: the purpose of the test is to determine if a path from A to B intersects a path
from P to Q. It goes as follows:

   1.   A sends a training packet to B (A: X B)
   2.   P sends a training packet to Q (P: N Q)
   3.   B sends a probe packet to N (B: B N)
   4.   A and P report whether or not they receive the probe

Possible outcomes are:

   •    [Result I] Only P observes the probe: since A does not observe the probe, some switch
        on the path from A to B has been trained by P. That is, there is a switch that is on the path
        from A to B and the path from P to Q. Thus the two paths AB and PQ are said to
        crossover.

   •    [Result II] Only A observes the probe: since P does not observe the probe, we know that
        the switches on the path from A to B and the switches on the path from P to Q are
        disjoint.

   •    [Result III] Both A and P observes the probe: this reveals the presence of a hub that
        duplicates the packet sent to N.

The path cross over test is applied to situations such as those depicted in the figure below.




                                               Switch: B




                                 Switch: A                     Switch: C


             Figure 12: Path Crossover Test is used to investigate connections between switches

Recursive Gap Discovery: We recursively use the path crossover test to split each gap into
smaller gaps, until we are left with direct connections between switches. This may occur is a
result I is returned when the cross over test is applied to paths A-B and A-C, which implies the
presence of another switch. On the other hand, if result II is returned, then we can infer that
switches A, B and switches A, C are directly connected.


Page 28 of 103                                                                         Albert Adi-Wijaya
                             Detecting Transparent Network Devices

For a more comprehensive treatment of this approach, the author is encouraged to consult
[BLACK 2004].

   5.5. Discussion of Existing Approaches
Each of the existing approaches are able to successfully determine the topology of Ethernet
bridges, having made a number of assumptions. How feasible and deployable these methods are,
depends on how realistic the assumptions are. We discuss them below.

The work in [BREITBART 2000], [LOWE 2001], and [STOTT 2002] rely on the use of SNMP
to access the relevant MIBs and forwarding databases of bridges. This poses to be a problem for
many reasons. Firstly, not all bridges support the use of SNMP. Secondly, many bridges may not
have SNMP enabled on them by network administrators for security concerns. Many network
administrators will also be unhappy with end users accessing the bridge MIBs in the first place.
Such bridges will not be included in the topology discovered, and thus reduces the accuracy of
the algorithm.

All of the approaches mentioned in the previous sections are limited to finding the logical
topology of the network. i.e. only the links which remain active after the spanning tree protocol
has blocked certain ports. This means other physical links remain un-detected, which may be of
concern to the end user. E.g. to find alternate paths if a links fails or is congested.

The algorithm in [BLACK 2004] does not actually identify the bridges by their address. Their
presence is only inferred. When generating fault reports or requesting assistance from network
administrators or IT helpdesks, it may be useful to provide specific information such as an
address to help in localising the fault.

An advantage that the approach in [BLACK 2004] has over the others is that it does not require
the spanning tree protocol to be supported by the switches. In fact, it is capable of discovering
hubs as well.

In addition, the work in [BREITBART 2000], [LOWE 2001], and [STOTT 2002] only requires a
single end user to map the topology. The approach in [BLACK 2004] is a collaborative and
distributive system which requires participation by as many end users as possible to map the
topology accurately.

   5.6. Are bridges really transparent?
Although bridges are designed to be transparent to the end user, there are inherent characteristics
of their behaviour which make bridges, not quite as transparent as one would assume. There are
many not so transparent features of bridges as outlined in [PERLMAN 2000], which we mention
below.

The following quantitative performance changes are likely to occur when bridges are introduced
in a network.
    1. The probability of packet loss increases.

Page 29 of 103                                                                 Albert Adi-Wijaya
                             Detecting Transparent Network Devices

   2.   The delay increases.
   3.   The packet lifetime increases.
   4.   The error rate increases.
   5.   Packet miss-ordering become possible (although unlikely)
   6.   The probability of duplicate packets increases.

There are also functional-type changes that are introduced when bridges are added onto a
network. These are:
   1. Stations cannot use the LAN maximum packet size.
      Bridges can interconnect different types of LANs which have different maximum packet
      sizes.

   2. LAN-specific information in the data-link layer may be lost.
      Some LANs have LAN-specific information fields (e.g. priority) which other LANs may
      not have, and thus cannot be carried onto.

   3. Unexpected packet format conversion may occur.

It is unlikely that the end user would be able to make any use of these properties to detect
bridges. This is because they would require data to be collected over a period of time, and then
analysed to observe any trends. Moreover, the end users have very little control over the network,
they cannot add or remove bridges from the network to observe performance/functional changes
prior to and after the addition of bridges.

In the next section, we discuss another property of bridges which we will adopt to detect bridges.

   5.7. A ‘new’ approach to detect bridges
The operation of a bridge is not completely transparent, as it does have characteristics which may
be utilised to discover its presence. The main approach used in this dissertation is that bridges
emit Bridge Protocol Data Units (BPDU) to communicate between themselves. These are the
Spanning Tree Protocol (STP) configuration messages to configure and maintain the spanning
tree topology and also Topology Change Notifications (TCN) to ensure that other bridges are
aware of a change in topology.

The salient fields of the BPDUs which are of particular concern to us include:
   • Root ID: The bridge ID of the root bridge.
   • Transmitting ID: The bridge ID of the bridge transmitting the BPDU.
   • Path Cost: The total cost of the paths from the transmitting bridge to the root bridge.

The use of BPDUs will allow the identification of bridges by their MAC address (unlike that of
[BLACK 2004]). We also follow a similar methodology whereby we first identify the nodes
(hosts and bridges) then proceed to discover the direct links between the bridges.

   5.8. Ethernet Topology Discovery using BPDUs

Page 30 of 103                                                                 Albert Adi-Wijaya
                             Detecting Transparent Network Devices

We now present our method for end users to discover the layer 2 Ethernet topology of a
subnet/LAN they are connected to. We take an incremental approach to allow the reader to
appreciate the step by step development.

       5.8.1.     Detecting a single bridge

It is quite trivial for a station (end user) to detect a connection to a bridge by simply capturing
packets on the LAN and identifying a BPDU. This allows the perfect identification of the bridge,
including details such as its MAC address. Moreover, the root bridge can also be perfectly
identified from the information in the BPDU.



                                                         root bridge




                                              bridge




                                    station
                                 Figure 13: Detecting a single bridge

This scenario is depicted in the Figure 13 above. Note that the line between the bridge and root
bridge is dotted to indicate that there may be other bridges in between the two.

The end user is now empowered with the knowledge of the bridge that he is directly dependent
upon for access to network services. He is also aware of the presence of a root bridge located
elsewhere in the network.

It should be clear to the reader that this approach is dependant on the bridge supporting the
Spanning Tree Protocol and that the bridge is configured to forward the BPDU onto ports
connected to end stations. This is discussed further in section 5.10.1.

       5.8.2.     Detecting other bridges

With the task of detecting the bridge that an end user’s station is directly connected to being
easily accomplished, we now broaden our scope to explore the possibility of discovering the
layer 2 topology from the end user’s perspective.

Since each bridge does not forward BPDUs received from other bridges, it is not possible to
detect other bridges by a single end user alone. This prompts the need for a collaborative system


Page 31 of 103                                                                 Albert Adi-Wijaya
                            Detecting Transparent Network Devices

much like that in [BLACK 2004] where the end users share information amongst each other to
determine the network topology.

Thus we require that each end user shares their information regarding which BPDU they are
connected to, with other end users in order for the topology to be discovered. Each end user can
then collate this information and attempt to infer the topology of the LAN.

       5.8.3.     Simple Topology

Now, if each end user were to share with one another, information regarding which bridge they
were connected to, we could establish the following topology.



                                              root bridge




                                              ?



                           bridge




                                                  station
                                    Figure 14: Simple Topology

This can be easily established from the BPDU received by each station which reveals the Bridge
ID of the bridge transmitting the BPDU and also the Bridge ID of the root Bridge. Stations
connected to the same bridge can easily be inferred from BPDU which have the same
transmitting Bridge ID.

However, what is missing from Figure 14 above, is that the network elements (if any) between
the bridge directly connected to the station and the root bridge, which cannot be immediately
inferred.

Another consideration is that, only bridges which have an end user station connected to it can be
successfully identified.


Page 32 of 103                                                                Albert Adi-Wijaya
                             Detecting Transparent Network Devices



                                             root bridge




                                             ?




                                                 bridge




                   server
                                                 station
                             Figure 15: Bridges which cannot be detected

The bridge circled in Figure 15 above, cannot be identified as it is not connected to a station
participating in the topology discovery exercise. This bridge may be of interest to an end user, as
it may provide connectivity to a file server, web server etc, or it may provide a redundant link (in
the blocking state) to a bridge connected to the end user.

       5.8.4.      Using the Root Cost Path to infer other bridges

In an attempt to discover the presence of switches located between the edge and the root bridge,
we may estimate these from the root path cost information available in the BPDU. The IEEE has
set default values for the cost for each port depending on the speed of the LAN, e.g. 19 for Fast
Ethernet (100 Mbps) etc.

Parameter           Link Speed          Recommended           Recommended         Range
                                        Value                 Range
Path Cost           4 Mb/s              250                   100 – 1000          1 - 65 535
Path Cost           10 Mb/s             100                   50 – 600            1 - 65 535
Path Cost           16 Mb/s             62                    40 – 400            1 - 65 535
Path Cost           100 Mb/s            19                    10 – 60             1 - 65 535
Path Cost           1 Gb/s              4                     3 – 10              1 - 65 535
Path Cost           10 Gb/s             2                     1–5                 1 - 65 535
                               Table 2: Path Costs [Source: IEEE802.1D]

Hence, if the station is connected to a Fast Ethernet LAN, and the BPDU has a root path cost of
38, we can conclude that there must be exactly one switch between the bridge on the edge and the
root bridge. Other possibilities are that there are nineteen 10 Gbps bridges, but this can be
discarded if we know the total number of bridges, which is got by noting the number of unique
BPDUs captured by the different end stations allowing some (possibly all) other scenarios to be
Page 33 of 103                                                                  Albert Adi-Wijaya
                             Detecting Transparent Network Devices

discarded. However, it is not possible to determine the bridge ID (MAC address) of the newly
discovered bridge as it’s presence is only inferred. Another example is if the BPDU had a root
cost path of 57, we can infer that there are 2 bridges between the edge and root bridge, and so on.

                                                            root bridge



                                                    19




                                                   Unknown MAC address
                                         19

                            bridge



                                        RCP: 38




                                       station
                    Figure 16: Using Root Cost Path (RCP) to discover other bridges

Of course, for this method to be successful, we require that the network administrator does not
alter the port costs from their default values. As changing the port cost is tedious and of little
benefit, we believe that network administrators will be reluctant to do so.

This approach, when correlating this information with other end users, does not allow us to
determine if it is the same or different bridges that are between the edge and root bridges. Thus
the following two scenarios depicted in Figure 17 are indistinguishable.




Page 34 of 103                                                                        Albert Adi-Wijaya
                                   Detecting Transparent Network Devices

                                root bridge                                               root bridge




        bridge                                                     bridge




        A                                       station A
                                          B                                                             B    station
                              Figure 17: Scenario 1 (LEFT) & Scenario 2 (RIGHT)

Stations A and B may be able to infer based on the root path cost that there is a bridge between
them and the root, but cannot determine if it is the same or different bridge.

         5.8.5.       Assumptions about network

It is essential that we introduce some assumptions in our topology discovery algorithm in order
for the end user to experience some sort of success in loosely determining the Ethernet topology.
Since it is end users and NOT network administrators trying to discover the topology, some
margin of error is acceptable.

The assumptions we make from this point onwards are:

    1. All port costs are the same. (This will indeed be the case if the same Ethernet
       technology is used eg. 10Mbps, 100Mbps etc AND the network administrator does not
       alter the default settings)2
    2. All bridges have at least one station connected to them.

It should be also evident that in order for our technique to progress, we must incorporate some
mechanism to allow a direct connection between two ‘known’ bridges. Consider the scenario
depicted in the figures below:




2
 It is possible to remove this assumption by just ‘estimating’ the number of hops based on port costs e.g. a port cost
of 21is equal to 19 + 2, therefore a 100Mbps port and a 10Gbps port
Page 35 of 103                                                                                  Albert Adi-Wijaya
                             Detecting Transparent Network Devices

                                   root bridge

                                                                                root bridge




 station
                                                 station




                 bridge                                                                       bridge




                          Figure 18: Scenario 1 (LEFT) & Scenario 2 (RIGHT)

After using the root cost path method to infer bridges between the root bridge and the direct
bridge, the station which is circled in the above figure will not be able to distinguish between the
two situations, using purely the root cost path alone.

           5.8.6.   Formal Statement of Layer 2 Ethernet Topology Discovery
                Algorithm

The following is a formal statement of our approach to determine the layer-2 Ethernet topology
of a switched network.

In brief: Each station captures a BPDU from the bridge they are connected to. This information is
then distributed amongst the stations using a sockets application. Finally, each station has all the
information necessary to now determine the Ethernet topology.

Aim: To determine the actual physical topology of the network. That is, we are not confining
ourselves to the logical spanning tree topology as other algorithms do.

    1. Initially the network is in a valid, configured and settled state i.e. no topology changes are
       occurring.




Page 36 of 103                                                                   Albert Adi-Wijaya
                            Detecting Transparent Network Devices




                               Figure 19: Sample Network Topology

   2. Each station commences an ARP capture process. This allows the end user to identify
      other stations that wish to take part in the Ethernet topology discovery.

   3. One station is selected as the ‘Master’ (this is done arbitrarily) and captures exactly one
      BPDU from the bridge it is connected to. Next, it starts injecting BPDUs onto the network
      with a spoofed ‘low’ bridge ID so as to ensure that it becomes the ‘root bridge’ when the
      spanning tree reconfigures.




                      Figure 20: 'Master' Station injecting BPDUs onto network



Page 37 of 103                                                                   Albert Adi-Wijaya
                                     Detecting Transparent Network Devices

    4. Whilst this is taking place, ALL stations wait a period of time to ensure that the spanning
       tree has reconfigured and stabilised. Once this period has elapsed, ALL the end stations
       will then capture exactly one BPDU from the bridge they are connected to3.

                       ROOT Bridge




                                                  BPDU                                r   e
                                                                                   ptu
                                                                                 Ca
                         Station A                                                                   Station D

                                                                                              BPDU




                                                                                              Ca
                            BPDU                                                                ptu
                                                                                                    re
                                                C
                                           ap




                                                                         BPDU
                                         ur   t
                                            e




                         Station B                                                                   Station C
                             Figure 21: Stations capturing BPDUs from the bridges

    5. The Master station then collects the captured BPDUs from all the other stations, and
       ceases emitting BPDUs.
                                                                  Blocked Link




3
 Note: This is the only BPDU that can be captured, as the BPDUs from other Bridges are not forwarded on to other
ports.
Page 38 of 103                                                                                                   Albert Adi-Wijaya
                              Detecting Transparent Network Devices

               Figure 22: 'Master' Station collecting BPDU information from other stations

   6. Steps 3 to 5 are repeated whereby each of the other end stations takes turns being the
      ‘Master’ station.

   7. Once each end station has gathered it’s BPDU information, it infers how many ‘hops’
      away each other bridge and station is, by using the Root Path Cost field in the BPDUs.

                                             Bridge # of
                                                    hops
                                             B      1
                                             C      2
                                             D      1
                                      Table 3: Station A - hop table

                                             Bridge # of
                                                    hops
                                             A      1
                                             C      1
                                             D      2
                                      Table 4: Station B - hop table

                                             Bridge # of
                                                    hops
                                             A      2
                                             B      1
                                             D      1
                                      Table 5: Station C - hop table

                                             Bridge # of
                                                    hops
                                             A      1
                                             B      2
                                             C      1
                                      Table 6: Station D - hop table

   8. Each end station then passes this hop information to each of the other end stations.

   9. Each end station is now able to determine the layer-2 topology of the network. This is
      done by determining the direct connections between each station. The direct connections
      are revealed by which bridges are exactly one hop away.

      5.8.7.       Design & Implementation

In this section, we discuss some design and implementation details of the Ethernet topology
mapper developed in this thesis project.

Page 39 of 103                                                                       Albert Adi-Wijaya
                             Detecting Transparent Network Devices

Identifying other end users

The first step in the algorithm involved identifying other stations (end users) that wish to take
part in the topology discovery exercise. There are essentially two approaches that were
considered.

   •   macof

The first approach involves spoofing MAC addresses using a tool called macof which floods the
local network with a large number of randomly generated MAC addresses. The will cause the
limited capacity of the forwarding tables in the bridge directly connected to an end user to be
filled with fake addresses. Once all of the forwarding tables in all of the bridges have been filled
with fake addresses, this will force each bridge to flood all ports with frames containing real
addresses. Thus, the end user can captures frames on the LAN that will reveal the real addresses
of other stations and keep a record of these.

One disadvantage with this approach is that the network administrator may not be pleased with
an end user spoofing MAC Addresses and forcing a switched network to become a broadcast
network. The macof tool is not equipped with the ability to detect once the forwarding tables are
completely filled. Thus the user is uncertain as to when he can stop the running of macof and
start the process of capturing for real MAC addresses.

   •   ARP Capture

An alternate approach is to capture ARP broadcasts which will reveal the real MAC addresses
and IP addresses of other stations. This approach has the advantage of being passive and not
increasing the amount of traffic on the network, whilst a disadvantage is that it may take longer
than the previous approach (for example, ARPs by default refresh every 20 minutes).

We decided that the second approach was the better option, as the extra traffic created by the
former approach would have been undesirable during a network fault or performance decrease.

Distributing the information amongst the stations

The next consideration was deciding on how to distribute the information (BPDU) collected by
an end user to other stations. There were two approaches considered:

   •   Capture then broadcast BPDU

Each station could capture a single BPDU and then broadcast the BPDU so that other stations can
capture it. This approach would increase the amount of traffic onto the network and may not be
scalable for a large number of hosts.

   •   Client/Server program

The alternative is to have a client and a server program running on each PC, where the client
sends a request to the server for the BPDU information. This technique can raise security issues
Page 40 of 103                                                                  Albert Adi-Wijaya
                              Detecting Transparent Network Devices

with the network administrator as each PC now has a server running on it. However, each station
can capture the BPDU, process it for the necessary information (e.g. root cost path etc) and then
distribute it to clients whom request it.

It was decided that the latter method was to be used to reduce the amount of network traffic and
to ease of implementation.

A sockets program was implemented in this thesis, and a design choice of using the TCP or UDP
protocol needed to be made. It was decided that the UDP protocol was to be used. This is because
the system developed may be needed is at times when the network may be severely congested
and packets are being lost. If the sockets program was running over TCP, TCP's congestion
control would cause it to back-off and stop sending messages at precisely the time when the
application is needed most. UDP also supports multi-casting which may be useful when mapping
topologies which include routers.

System Design

                                  CONTROL


                               WRITE
                                            Client
                                                            REQUEST

                                   READ                  RESPONSE
                                            Server

                                            BPDU                    Network
                                            Inject     INJECT
                   Database                                  CAPTURE
                                            Packet
                               WRITE        Capture
                                              •ARP
                                             •BPDU

                           Figure 23: Design of Ethernet Topology Mapper

Figure 23 above shows a systems level design of the code developed to map the Ethernet
topology. It consists of a Server program that handles requests for BPDU or HOP information
from other clients. Each station also has a Client program that can request BPDU or HOP
information from other servers on other stations. There is also a BPDU Injection module that
creates and periodically injects BPDUs onto the network to force the spanning tree to
reconfigure. There is also a Packet Capture program that can sniff the network for ARPs or
BPDUs. The implementation of the Database consists of plain text files for simplicity.

       5.8.8.     Results

The system developed by the author has been tested on a total of five Cisco Switches available in
EE Lab343A. The topology was successfully discovered each time. An example of some of the
outputs of the program is shown below.


Page 41 of 103                                                                Albert Adi-Wijaya
                 Detecting Transparent Network Devices




                    Figure 24: Sample Topology Output 1




                    Figure 25: Sample Topology Output 2



Page 42 of 103                                            Albert Adi-Wijaya
                            Detecting Transparent Network Devices




                               Figure 26: Sample Topology Output 3

Due to the limited number of bridges and PCs that were made available to the author, this was the
extent of ‘real-life’ testing that was possible. However, fundamentally this approach can work
with much larger networks with any number of bridges. Much larger topologies have been
mapped using thought experiments and artificially created data.

   5.9. A modified approach
Due to the limitations (that is, the dependency on port cost of BPDU) of using the Root Cost Path
of the BPDU to determine direct links between bridges, we attempt a different approach which
was inspired from the work of [BLACK 2004]. This new approach makes use of the simple
property that a bridge that looks at the source address of a MAC frame to learn which port the
address resides on.




Page 43 of 103                                                                Albert Adi-Wijaya
                             Detecting Transparent Network Devices




                                Figure 27: Islands of hosts and bridges

Thus far, we have been able to identify other hosts, and the bridges that they are connected to.
However, what is needed is some mechanism to determine direct links between the bridges.

       5.9.1.   Discovering the Ethernet Topology using the Bridge
            Connection Theorem

We now present an algorithm to determine the Ethernet topology of a switched network. We
present the following theorem:

Bridge Connection Theorem: A bridge BI is directly connected to another bridge BJ, if the path
formed does not overlap any other end to end path formed between bridge BI and any other
bridge.

Note: the “other end to end path” could refer to a direct or indirect connection.

It should be clear to the reader that the contrapositive of the above statement is NOT true, namely
bridges BI and BJ are NOT directly connected to one another if the path they form overlaps a path
formed by bridges BI and another bridge, as this can occur quite often which we will see later on.

Before proceeding further we introduce some notation:

HA: Host A (with MAC address A)
BA: Bridge A (bridge connected to Host A)

We characterise a path completely by its bridge start point and bridge end point e.g. BA-BB. Since
we assume that the network has been configured with the spanning tree protocol, there can only
be one unique path from two endpoints. It should also be noted that the paths BA-BB and BB-BA
are equivalent.


Page 44 of 103                                                                      Albert Adi-Wijaya
                               Detecting Transparent Network Devices

A transmission of a frame from a host A, with source MAC address A and destination MAC
address B is presented by:

HA: A      B

In order to implement the Bridge Connection Theorem, we require the Path Overlap Test, which
is described in the next section. It is this test that provides the underlying foundations for the
Bridge Connection Theorem.

        5.9.2.    Path Overlap Test

The path overlap test determines if two paths originating from the same bridge have some part of
their paths overlapping.


                          HB                      HA




                          BB                      BA




                           BD                          BC                BE




                          HD                       HC                    HE
                                     Figure 28: Example Network

Consider Figure 28 as shown above. The paths BB-BA and BA-BC do NOT overlap. However,
paths BA-BC and BA-BE do overlap (the segment BA-BC is common). The paths BB-BD and BB-BE
also overlap.

For this test, we need two MAC addresses reserved for the purposes of this test. We label them X
and Z.

Path Overlap Test: Two paths, BA-BB and BA-BC do NOT overlap provided that both hosts HB
and HC can receive a return frame from HA, after the following sequence of frame transmission
takes place.

Phase 1)

   1. HC: X      A
   2a. HB: Z     A
Page 45 of 103                                                                 Albert Adi-Wijaya
                             Detecting Transparent Network Devices

   2b. HB: X     Z
   3. HA: A      X

HC should receive the frame from HA, not HB if the paths do NOT overlap.

Phase 2)

   1. HB: X      A
   2a. HC: Z     A
   2b. HC: X     Z
   3. HA: A      X

HB should receive the frame from HA, not HC if the paths do NOT overlap.

Explanation of Phase 1

The purpose of step 1) is to train bridges BA & BC that a host HX is located on the same port as
HC.

The purpose of step 2a) is to ‘train’ bridge BB that a host HZ is located on the same port as HB.
The purpose of step 2b) is to train bridge BB that HX is located on the same port as HB.

The purpose of step 2a) should be obvious to the reader. It is to prevent bridge BB from
broadcasting the frame in step 2b) to all other ports, and thus informing other bridges such as BA,
that host HX is located on the same port as Host HB.

Now, in step 3) if the packet from host HA to HX does in fact return to HC, this implies that bridge
BB does not lie between bridges BA and BC. If it did, then HC would not have received the frame,
as bridges BB, BA & BC would all be trained to believe that host HX is located on the same port as
HB during step 2b). Thus HB would have received the frame from HA.

Although a correct result for phase A is a necessary condition for paths BA-BB and BA-BC to not
overlap, it is not sufficient. The correct result for phase 2 must also be returned.

Phase 2 is similar to phase 1, except that the roles of hosts HB and HC are reversed.

       5.9.3.      The basic idea

The basic idea is to select a bridge, and for all the remaining bridges, select two at a time to apply
the path overlap test on, until all possible combinations have been tried. Eventually, if it is found
that one or more particular paths do NOT overlap with any other path, we can conclude that there
is a direct link between the two bridges. This process is then repeated for each other bridge to
determine all other direct links.

We examine the case below:


Page 46 of 103                                                                    Albert Adi-Wijaya
                               Detecting Transparent Network Devices



                         A                 B               C                  D




                           A               B                C                 D
                                     Figure 29: Example Network

Select HA:

Possible paths are: BA-BB, BA-BC, BA-BD

                                  Path 1   Path 2   Overlap (Y/N)
                                  BA-BB    BA-BC          Y
                                  BA-BB    BA-BD          Y
                                  BA-BC    BA-BD          Y

Since all pairs of paths overlap, we cannot conclude anything, despite there being a direct link
between BA and BB.

Now, select HB:

Possible paths are: BB-BA, BB-BC, BB-BD

                                  Path 1   Path 2   Overlap (Y/N)
                                  BB-BA    BB-BC          N
                                  BB-BA    BB-BD          N
                                  BB-BC    BB-BD          Y

Since path BB-BA is NOT involved with any overlaps, we can conclude that BA and BB are
directly connected.

Now, select HC:

Possible paths are: BC-BA, BC-BB, BC-BD

                                  Path 1   Path 2   Overlap (Y/N)
                                  BC-BA    BC-BB          Y
                                  BC-BB    BC-BD          N
                                  BC-BA    BC-BD          N

Since path BC-BD is NOT involved with any overlaps, we can conclude that BC and BD are
directly connected.

The selection of HD is suppressed here as it yields the same results as HA.
Page 47 of 103                                                                    Albert Adi-Wijaya
                             Detecting Transparent Network Devices



Now, it is clear that the link between BB and BC was not detected. However, this can be inferred
since we know that all bridges must be connected somehow in a spanning tree. A direct link
between BB and BC is the only way possible.

This is because, if BA and BD were directly connected instead of BB and BC, then paths BD and BA
could not overlap, which contradicts our results. A similar argument also exists for direct
connections between BB and BD or BA and BC to replace BB and BC.

       5.9.4.        Removing the endpoints

It is clear that the algorithm thus far, is useful in determining the Ethernet topology of networks
such as the one shown in Figure 30 below. With bridges BA, BD, BE, BF and BG selected, none of
the direct links can be discovered. However, with bridge BB selected, the direct links BB-BA, BB-
BD and BB-BE can all be discovered. Likewise, with bridge BC selected, the direct links BC-BA,
BC-BF and BC-BG can all be discovered.

                                                    A


                            B
                                                    A                    C

                                        B                     C


                                D               E         F              G

                 D                                                                  G

                                            E                     F
                            Figure 30: Network - all direct links discovered

However, what if we have a network with a topology at that shown in Figure 31?

It is clear that the above Topology Discovery algorithm is limited to determining the direct links
between switches at the fringes of a network. Links in the middle remain undetected, and some
mechanism is needed to find them systematically.

In brief, the method we propose is to identify the endpoints and to ‘eliminate’ them from the
topology. Then, we proceed to apply Bridge Connection Theorem to the remainder network. We
provide an example below.




Page 48 of 103                                                                 Albert Adi-Wijaya
                              Detecting Transparent Network Devices



         A             B              C              D               E              F                G



         A              B              C              D              E              F                G
        Figure 31: Example Network - Only direct links at the fringe of the network are discovered

Paths BB-BA and BF-BG are the only direct links that can be identified in the above figure. The
question remains, how to systematically determine the remaining links BB-BC, BC-BD, BD-BE and
BE-BF?

We do so by identifying the endpoints, BA and BG, ‘remove’ them from the network, and apply
the algorithm again, to the remainder of the network.

To identify the two endpoints, we can do so by using the following Endpoint Bridge Proposition.

Endpoint Bridge Proposition: An endpoint bridge has ALL pairs of its paths to other bridges,
overlapping.

It is clear that in the above figure, the bridges BB, BC, BD, BE and BF do not satisfy the above
proposition. Therefore, these bridges are not endpoint bridges. For example, paths BB-BA and BB-
BC do not overlap, and thus BB is NOT an endpoint bridge.

Continuing with our example, with bridges BA and BG removed from the network, we are left
with the following figure:


                        B             C              D              E              F



                        B              C              D              E              F
                                 Figure 32: Remaining Bridge Topology

When we apply our topology discovery algorithm, we can determine the direct links BC-BB and
BE-BF.

And we then continue to apply the algorithm recursively.

       5.9.5.      A comprehensive example

We now provide a comprehensive example to display how the algorithm will work in its entirety.
We have the network as shown in the figure below.



Page 49 of 103                                                                          Albert Adi-Wijaya
                                     Detecting Transparent Network Devices



                                                                                   A
                                                   B
                                                                       A
                             C
                                                   B                           D


                 E                         C                       D
                                                                                           H

                                 E             F               G               H
                                                                                                           L
            I
                                           F                                                   L
                                                                           G
                         I                                                         K
                                               J           J
            M                                                                                  K


                                                   N                       O
           M
                                                                                   O
                                                   N
                 Figure 33: A comprehensive example of an Ethernet switched network

The direct links between the hosts and their relevant bridge, i.e. HA-BA, HB-BB … HO-BO can all
be discovered in the first phase, when each hosts captures a BPDU. We remove these links from
the network. We then apply the topology discovery algorithm as outlined above. The network
now looks like:


                                                                           A

                                                           B

                                               C                           D


                                       E               F               G           H
                     I
                                                                                                    L

                                                                                       K
                                                                   J
                     M

                                                    N             O
                             Figure 34: Example network with some links removed


Page 50 of 103                                                                                     Albert Adi-Wijaya
                              Detecting Transparent Network Devices

Here we show that the direct links BB-BA, BC-BF, BI-BM, BJ-BN, BJ-BO, BH-BK and BH-BL have
all been discovered. Now, we need to determine which bridges are endpoint bridges. We do so by
applying the endpoint test as detailed above. Having identified bridges BA, BF, BH, BN, BM, BO,
BK and BL as endpoint bridges, we remove them from the network and we are left with the
following.


                                                        B

                                               C                   D


                                        E                      G              H
                          I



                                                            J
                         Figure 35: Example network with more links removed

Again, we apply the topology discovery algorithm to determine the direct links BE-BI, BG-BJ and
BD-BH. Then we discover the endpoint bridges BI, BJ and BH. Removing the endpoint bridges, we
are left with:


                                                        B

                                             C                     D


                                    E                          G
                      Figure 36: Example network with even more links removed

After repeating the same steps as described above, we are left with:


                                                   B

                                        C                      D
                     Figure 37: Example network - ALL links have been discovered

The last step is quite trivial. Thus we have now determined all links in the network!

       5.9.6.      Design & Implementation



Page 51 of 103                                                                     Albert Adi-Wijaya
                             Detecting Transparent Network Devices


                    Host A                                              Host B
                    Packet                       1                      Packet
       Server      Capture        Client                   Server      Capture       Client
                   Program                                             Program



                                     3                 2




                                              Packet
                                 Server      Capture        Client
                                             Program
                                              Host C
                           Figure 38: System Design of BCT implementation

Figure 38 above provides a system level design of the architecture of this approach. Each host
consists of a client, server and packet capture program. The role of each component is listed
below:
    • Client
           o Sends a request to a server on another host B, to send a probe to host C.
           o Asks a server on host C, if it received a probe from host B.
    • Server
           o Upon receiving a request to send a probe to host C from host A, it sends a probe to
               host C.
           o After being asked by host A if it has received a probe from host B, it responds Yes
               or No to host A.
    • Packet Capture Program
           o Captures any probes received.

       5.9.7.      Results

So far, the testing of this approach has not been successful due to the bridges not functioning
correctly. The approach relies on the bridges learning the addresses of the frames they receive on
each port. The bridges that the author has tested do not seem to do this, which could be due to the
fault of the bridge or the incompetence of the author. It has still been not determined which it is.
A formal proof of the algorithm was also to be developed, but due to the lack of success in
implementation, this was not pursued.

   5.10. Discussion
There are a number of issues and concerns that are raised with the tools developed to discover
Ethernet topology for both approaches. They are discussed below.

Page 52 of 103                                                                   Albert Adi-Wijaya
                             Detecting Transparent Network Devices


       5.10.1.    BPDU availability on LANs

The system developed is based upon the assumption that each station can capture BPDUs from
the bridges that they are connected to. Now this in turn requires that not only the bridges support
the IEEE802.1D Spanning Tree Protocol, but furthermore, the network administrator has this
enabled on all the switches. How true this assumption is, will ultimately influence how
deployable the system is and how successful the topology will be mapped.

From what the author has been able to gather, there are indeed numerous switches/bridges that
are available on the market that support the spanning tree protocol. Provided that the
network/system administrator enables this setting, the BPDU availability should not be a
problem.

It has also been raised that it may be possible to configure switches to not forward BPDUs on
ports connected to end stations, and only forward them onto ports connected to other bridges.
This was suggested by the author’s supervisor to explain why the School of EET’s LAN network
does not appear to have any BPDUs on the LAN despite using Cisco Switches. The author has
consulted with many switch manufacturers, whom have said that such individual port
configuration is not possible on their switches. This does somewhat alleviate concerns about
BPDU availability, but at this point in time, the un-availability of BPDUs on EET’s LAN remain
a mystery.

       5.10.2.    Spanning Tree Vulnerabilities

There are security raised in [CISCO 2003] regarding the sending of BPDUs to force topology
changes in a network resulting in Denial of Service. An attacker posing as rogue ‘switch’ can
send BPDUs with a lower bridge priority to access switches to become the root switch, and
causing the spanning tree to converge. This allows the attacker to observe previously unseen
traffic as depicted in the figure below.




Page 53 of 103                                                                 Albert Adi-Wijaya
                             Detecting Transparent Network Devices




         Figure 39: Attacker becoming the STP root and causing the spanning tree to reconfigure

The Ethernet Topology Discovery by using BPDUs approach may be frowned upon by network
administrators whom are security conscious. The raises security concerns with the first approach
where end stations force the spanning tree to reconfigure by injecting BPDUs onto the LAN and
becoming the root bridge.

Many network/system administrators have implemented security policies to block access to ports
when a frame with an unknown MAC address is transmitted on it. For example, consider the
notice below from the UNSW CSE Computing Help Desk web page:

“…All of the switches through which the lab machines are connected have port
security enabled, which means that the switches will not allow any machine
other than the registered and recognised lab workstation to be connected to
the CSE network via their ports. Attempting to connect any machine other than
the lab workstation will result in the switch blocking all connections to that
port (including reconnection of the registered workstation), and has been
known to result in more serious network related problems…”
                      Figure 40: UNSW CSE Computing Help Desk Policy
                [http://www.cse.unsw.edu.au/faq/questions/laptop-connect.html]

       5.10.3.    Logical Vs Physical Topology

Though the first approach discovers the physical topology of the bridges, that is, all links
including those disabled by the spanning tree protocol, the second approach however only
discovers the logical topology after links have been blocked by the STP.

       5.10.4.    Distributed Vs Stand-alone System

Both of the Ethernet topology discovery algorithms developed in this thesis are distributed
systems where the end users must collaborate with each other in order to accurately map the
Page 54 of 103                                                                      Albert Adi-Wijaya
                              Detecting Transparent Network Devices

topology. This incurs the disadvantage of depending on as many end users as possible
participating in order to accurately map the topology. However, the existing approaches which
allow the topology to be mapped by a single stand-alone end user, requires that the user has
administrator privileges e.g. access to MIBs, SNMP etc. Thus, the approaches developed in this
thesis are more deployable.

       5.10.5.    Performance Analysis

The second approach to mapping bridge topology by training switches and sending suitable
probes, incurs the cost of transmitting ‘many’ frames onto the network. We derive a formula to
quantify the number of transmissions in order to gauge the performance of the algorithm more
accurately.
We make use of the following combinations formula:
                                                    n!
                                         n
                                           Cr =
                                                r!(n − r )!
                                   Figure 41: Combinations formula

Where n = number of different objects (in our case, bridges)
      r = number of objects taken at a time

Let N = number of bridges

Each Path Overlap Test (POT) requires 12 frame transmissions. In the first phase, four are needed
to ‘train’ the switches and an additional two are needed to probe the hosts to check if they have
received the frame or not. This is repeated for the second phase of the test and thus a total of 12
frames are transmitted for each execution of the POT.

The number of times the POT needs to be repeated depends on N, the number of bridges. A
station is selected, and for the remaining bridges, two at a time are selected. The total number of
combinations possible for N bridges is given by:




                 Figure 42: Select a bridge then choose two at a time from the remainder


Page 55 of 103                                                                        Albert Adi-Wijaya
                                Detecting Transparent Network Devices


                                 P = N −1C 2 =
                                                     (N − 1)! = (N − 1)!
                                                 2!(( N − 1) − 2)! 2(N − 3)!
                         Figure 43: Number of combinations possible with N bridges

The number of times the Bridge Connection Theorem needs to be repeated depends on the
number of reductions required, whereby the endpoints are removed. This of course is dependant
on the logical topology of the network. We examine the scenario for the worst case situation
which is when the bridges are connected in a bus topology, i.e. straight line (this is where the
number of endpoints is minimal, that is only 2 endpoints).

For an odd number of bridges, we need
                                             (N − 3)      repetitions.
                                                  2
For an even number of bridges, we require
                                                   (N − 2)   repetitions.
                                                      2

Therefore we have the following formula:

   •   Odd Number of Bridges

T = 12 ×
            (N − 1)! × (N − 3) = 3(N − 1)!
           2( N − 3)!    2       (N − 4)!
   •   Even Number of Bridges

T = 12 ×
            (N − 1)! × (N − 2) = 3(N − 2)(N − 1)!
           2( N − 3)!     2         (N − 3)!
       N (# of bridges)                # of frame transmissions                      # of bytes
               4                                   36                                    720
               5                                   72                                   1,440
               6                                  240                                   4,800
               7                                  360                                   7,200
               8                                  756                                   15,120
               9                                 1,008                                 20,160
              10                                 1,728                                 34,560
              11                                 2,160                                 43,200
              20                                18,468                                 369,360
              50                                338,688                               6,773,760
             100                               2,852,388                             57,047,760
                    Table 7: # of frame/byte transmissions for Bridge Topology Algorithm

From the table above we can see that as the size of the bridged network increases, the amount of
additional traffic generated to map the topology is increasing at an accelerating rate. However,
even for a moderately sized bridged network, ten bridges could be roughly used as an
approximate maximum. Hence, one can intuitively see that 43MB of traffic generated onto a

Page 56 of 103                                                                         Albert Adi-Wijaya
                               Detecting Transparent Network Devices

network over a period of 5-10 minutes on a LAN (0.57-1.15Mbps) which can support 100Mbps
is negligible.

       5.10.6.     Performance Analysis: Reconfiguring Spanning Tree

The approach of mapping the Ethernet topology by using BPDUs raises the issue of how will it
effect the normal flow of traffic. This is of particular concern as this approach, in contrast to the
later approach of training and probing switches, involves re-configuring the spanning tree in
order to determine the physical (not just the logical) topology.

In light of this, we conduct some experiments to measure the performance of the network whilst
its spanning tree is being reconfigured. In particular we use a tool called netperf to measure the
traffic flow (Mbps) between stations whilst the spanning tree is being reconfigured.


                                                    HA




                                               BA




                                 BC                            BB


                          HC                                                HB
                   Figure 44: Experimental Setup – Reconfiguring the Spanning Tree

The above figure shows the experimental setup and we firstly measure the traffic flow possible
when the spanning tree has stabilised to gauge the performance of the network under ‘normal’
conditions (In this case, bridge BC is the root bridge – lowest Bridge ID). We then proceed to
alter the spanning tree by each host (HA, HB & HC) taking turns in ‘emulating’ the root bridge by
inject BPDUs onto the network and measure the flow of traffic possible. The results are tabulated
below (NOTE: the highlighted cells indicate that the “From” host is emulating the root bridge).
We carry out both a TCP and a UDP Stream Test for a period of 20 seconds.

                                  Traffic Flow (Mbps)
                 From C         From B           From A       Total Network
              To A   To B   To A     To C    To B     To C       Traffic
                           No Spanning Tree Configuration
              51.96 52.14 85.33 62.42 89.47 62.41                 403.77
                          Whilst Spanning Tree is reconfiguring
              54.04 54.08 81.77 54.73 87.53 54.71                 386.86
              65.17 65.27   8.85     93.96    8.66    94.02       335.93
              79.25 79.13 10.02 94.03         9.10    93.97        365.5
           Table 8: Experimental Results - Reconfiguring the Spanning Tree: TCP Stream Test
Page 57 of 103                                                                       Albert Adi-Wijaya
                             Detecting Transparent Network Devices

We can see from the results above that traffic flow between any two hosts can actually increase
or decrease whilst the spanning tree is being re-configured. It appears that when host HC is
‘emulating’ the root bridge, the traffic flow between all stations follows a similar pattern to that
when the spanning tree is stable. However, whenever hosts HA or HB emulate the root bridge, the
traffic flow between the two of them seem to decrease. This is due to the fact that the link BA-BB
is blocked (BC was the original root bridge with the lower Bridge ID).

For the TCP Stream Test, it appears that reconfiguring the spanning tree poses little hindrance to
the deployment of this topology algorithm as even the worse case situation, allows traffic flow of
approximately 10Mbps which is still reasonable for a 100Mbps LAN and only for a few seconds.

                                  Traffic Flow (Mbps)
                 From C         From B          From A        Total Network
              To A   To B   To A     To C    To B     To C       Traffic
                           No Spanning Tree Configuration
              52.61 52.62 96.22 96.27 96.24 96.24                 490.2
                          Whilst Spanning Tree is reconfiguring
              57.34 57.34 96.19 96.16 96.22 96.21                 499.46
              53.06 53.07 96.11 96.09 96.19 96.22                 490.74
              50.99 50.14 96.19 96.21 96.09 96.12                 485.74
           Table 9: Experimental Results - Reconfiguring the Spanning Tree: UDP Stream Test

The UDP Stream Test does not appear to suffer from any impairment when subject to a spanning
tree reconfiguration, in stark contrast to the TCP Stream Test. This can perhaps be explained by
TCP’s congestion control algorithm where traffic is withheld when the network is congested.

Overall, we envisage that reconfiguring the spanning tree poses no hindrance to the deployment
of the topology algorithm.

   5.11. Conclusion
The tools and techniques developed to discover the Ethernet topology of a LAN/subnet from the
end user’s point of view has had a reasonable level of success. We have been able to remove
some of the assumptions and constraints with the existing approaches, but have had to introduce
some of our own. However, overall we have developed a system which is more deployable and
more security friendly that will be useful in fault/performance management by the end user.

   5.12. Alternate phrasing of Path Overlap Test
This is a modified version of an alternate phrasing of the Path Overlap Test as suggested by the
author’s thesis supervisor.

To determine if two paths from a Bridge BA, to bridges BB and BC overlap or not, it comes down
to discovering the order of the bridges BA, BB, BC.



Page 58 of 103                                                                     Albert Adi-Wijaya
                             Detecting Transparent Network Devices

To do this, we program two of the bridges (BB & BC) to both think that the host (station) directly
connected to them has a particular MAC address, Z. The third bridge (BA) then sends a test
frame to address Z, and depending on which of the other two bridges receives it, indicates the
order of the three bridges.

The orderings that are possible are:
   1. BA BB BC
   2. BA BC BB
   3. BB BA BC

Note: Other sequences, such as BB BC BA, are just mirror images of the orderings above.

We choose hosts HB and HC to program their bridges (BB & BC), and host HA to send the frame.
The possible outcomes are:
1. Only HB will receive the test frame from HA.
2. Only HC will receive the test frame from HA.

There is a trick to "programming the switches": the basic idea is to send a packet with source
address Z to the switch, so that it learns that Z is on a port. However, the second time we do this,
we don't want to undo the effect of the first time, so we use a source address for that second
transmission which we have programmed the switch not to forward. Hence, the programming
process is:

1. HB sends a frame with source address X to host HA
2. HC sends a frame with source address Z to any host (HA is chosen for convenience)
3. HC sends a frame with source address X to destination address Z

The Path Overlap Test is as follows:

   •   After programming the bridges BB & BC as outlined above, it should be host HB that
       receives the test frame from HA.

   •   We then reverse the roles of bridges BB & BC, and this time it should be host HC that
       receives the test frame from HA.

Provided that the above two conditions are satisfied, then the ordering of the bridges are BB BA
BC i.e. the paths BB-BA & BA-BC do NOT overlap.

If BOTH of the conditions are NOT satisfied, then one of the other orderings applies to the
bridges and the paths do overlap.




Page 59 of 103                                                                  Albert Adi-Wijaya
                            Detecting Transparent Network Devices




6. Detecting Web Caches
   6.1. Introduction
Web caches are useful networking devices which store recently accessed web objects to reduce
the latency and bandwidth of internet traffic on a network. This section of the thesis explores
tools and techniques which the end user can adopt to detect and possibly identify the web caches
that he/she is reliant upon when accessing the internet.

   6.2. Types of Web Caches
There are many different implementations of web caches that exist today. The two types that will
be considered in this dissertation are:
    • Web Proxy
    • Transparent Interception Caching

   6.3. Web Proxy
Web proxies are intermediate nodes, whereby the end system forwards a request for a web object
to the proxy, which checks if it has a copy of the object requested. If so, the proxy returns the
web object to the client. If not, it forwards on the request to the web server, receives the web
object returned, stores a copy in its cache, and forwards a copy to the end system. This is
illustrated in Figure 45.




                                  Figure 45: Web Proxy Example


Page 60 of 103                                                                Albert Adi-Wijaya
                             Detecting Transparent Network Devices

Although such a device is not completely transparent, as it requires the end user to configure their
web browser to work behind the proxy, it is still an intermediary device which is very discreet
and subtle in it behaviour, so we provide a minimal discussion in this thesis.

   6.4. Transparent Interception Cache
The second type of web cache is known as a transparent interception cache, which have become
increasingly popular, as it does not require end users to configure their browsers. This setup
requires gateway routers to divert HTTP requests to the web cache, which will then process the
request as do proxy caches.




                         Figure 46: Transparent Interception Caching Example

This second type of caching will receive a more comprehensive treatment than the former, and as
we will see shortly, detecting a transparent interception cache is much more difficult.

   6.5. Detecting Web Proxies
There are many techniques that the end user can adopt to determine if he/she is connected to a
web proxy. (Some are quite trivial!) These include:

   •   Web Browser Configuration Settings
   •   Protocol Analysis
          o TTL
          o Source IP Address
          o HTTP GET Request

       6.5.1.      Web Browser Configuration Settings

The web browser must have been configured to work behind a proxy in the first instance, so the
end user should be able to check the settings of the browser to determine if he/she reliant upon a
web proxy. For example, with Internet Explorer 6, the end user can select Tool           Internet
Options      Connections        LAN Settings to find the section which relates to proxies, as
depicted in Figure 47 below. This allows a perfect identification of the web proxy with its IP
Address.



Page 61 of 103                                                                  Albert Adi-Wijaya
                            Detecting Transparent Network Devices




                             Figure 47: IE6 - Web Proxy Configuration

       6.5.2.     Protocol Analysis

The next, not so obvious approach that an end user could use is to analyse the packets exchanged
between the end system and the web proxy. Though this approach would be more suited to
detecting transparent caches (as will be discussed in the next section), since the easiest method
was already discussed above.

   •   Source IP address

The source IP address of the HTTP packets returned from the web proxy is that of the proxy
itself, NOT the IP address of the web server. So a simple comparison of the IP address of the web
server and that returned in the HTTP packet can detect the presence of a web proxy and identify
it by its IP address.

   •   Time-To-Live

In general, the Time-To-Live (TTL) field of the IP Packet header in the HTTP response is the
same value, regardless of which web server is visited.




Page 62 of 103                                                                Albert Adi-Wijaya
                             Detecting Transparent Network Devices


                                                   Web Server

                   Web Server                                          Web Server




                                                     Web Cache




                                Figure 48: Web Proxy Configuration

If there were no proxy in place, this could NOT be possible, as each web server sets it own initial
TTL value, and the number of routers between each web server and the end station (decrementing
the TTL) is most likely to be different. So it is highly unlikely that the TTLs returned from
different web servers will be the same value.

However, since there is a web proxy in between, this acts as a server for the web client, and thus
sets its own TTL each time it returns a packet to the client. Thus the TTL returned is usually the
same value each time, regardless of the web server visited by the client.

The possible exception occurs if there are multiple routers between the web cache and the end
user, which may result in different routes being traversed and thus a different number of
decrements in the TTL. Hence the user may see some slight variations in the TTL values
returned.

   •   Hyper-Text Transfer Protocol (HTTP)

When communicating with a known web proxy, the client issues a slightly different request than
when it is communicating with the origin server. The request line of a proxy request includes the
full URL. Figure 49 shows a GET request with no proxy in place, whereas Figure 50 does have a
proxy in place. Note the http://www.nlanr.net/ address included in the request.

GET /index.html HTTP/1.1
Host: www.nlanr.net
Accept: */*
Connection: Keep-alive
                                Figure 49: GET Request with no proxy

GET http://www.nlanr.net/index.html HTTP/1.1
Page 63 of 103                                                                      Albert Adi-Wijaya
                              Detecting Transparent Network Devices

Host: www.nlanr.net
Accept: */*
Proxy-connection: Keep-alive
                          Figure 50: GET Request with proxy

   6.6. Detecting Transparent Interception Caches
We now move on to the more challenging task of detecting transparent interception web caches.
The implementation of such caching involves not just the cache itself, but a router to divert the
HTTP traffic towards it. Routers do this by inspecting for network traffic with a port number of
80.

The techniques that were investigated during the scope of this thesis project include:
   • Performance Analysis
   • Protocol Analysis
   • Inter-Cache Protocols

       6.6.1.      Performance Analysis

The use of a web cache is designed to delay the latency of accessing recently and regularly used
web objects. This is because the first time a web object is accessed; it needs to be retrieved from
the original web server. Each subsequent occasion, it can be retrieved locally from the cache by
the end user, as depicted in Figure 51 below.

                                                   Web Server

                 Web Server                                            Web Server




                                             Internet




                                                        Web Cache




                         Figure 51: Performance Analysis – Round Trip Time




Page 64 of 103                                                                  Albert Adi-Wijaya
                             Detecting Transparent Network Devices

By identifying a significant drop in the delay when accessing a web page, this allows an end user
to detect a web cache. We measure the elapsed time to download a webpage, that is, from the
time when the user initiated the request for a web page till the time that the entire transfer is
complete. This is illustrated in Figure 52 below.




                 Elapsed Time




                     Figure 52: Elapsed Time Measurement [Source KUROSE 2004]

By repeating this measurement numerous times, we should eventually see a significant drop in
the elapsed time required to download a web object, as depicted in Figure 53.

The number of measurements that need to be taken, and the decrease in elapsed time experienced,
will obviously depend on a number of factors. These include:
    • The caching policy of the web cache.
    • The cache directives in the HTTP responses.
    • Any traffic congestion on the network.
    • Other users of the same web cache

The policy of a web cache determines if a temporary copy of the web object downloaded is to be
stored in the cache. If the end user decides to download an object that is not stored in the cache
due to its caching policy then it is unlikely that a decrease in response will be seen.
The caching directives in HTTP headers of the web objects also will impact the viability of this
method. If the caching directives are not to cache, then a decrease in response time is not likely to
be seen as the object is always going to be retrieved from the distant web server. The exception
would be if the web cache is designed to ignore cache directives).
If another network user is also sharing the same web cache, he/she may have already downloaded
the same web object request by another user. The later user is then unlikely to see a decrease in
response time on repeated requests for the same object.
The level of congestion on the network can also influence results by altering the time required to
download a web object. Thus any reduction seen can be misinterpreted as a cached object.


Page 65 of 103                                                                   Albert Adi-Wijaya
                             Detecting Transparent Network Devices

In order to mitigate these hindrances, we can visit a variety of web sites and repeat measurements
for consistency.


                elapsed
                  time




                                                                           sample
                             s1     s2     s3      s4    s5      s6
                           Figure 53: Graph of Elapsed Time Measurements

       6.6.2.      Protocol Analysis

The protocol analysis techniques that were used to detect proxy caches may not necessarily
directly apply to transparent interception caches. This was investigated during the scope of this
thesis. It was found that most implementations of interception caches reset the TTL field of IP
packet headers from different web sites.

This particular property was accidentally stumbled upon by the author’s thesis supervisor (Dr
Tim Moors), whom when working on a project to improve traceroute, found that the TTL
value returned to a UNSW web client from any web server was the same value. After probing for
further information from UNSW’s IT unit, it was uncovered that it was in fact UNSW’s web
cache that was resetting this field to the same value.

The author approached the Communications Unit for further details about the particular
cache/proxy hardware/software that was being used. The UNSW Web Cache is configured so
that the border routers redirect all web traffic to the cache engine. The cache engines (also called
content engines) used are Cisco CE series 500.

The following figures are packet captures taken by Ethereal on a PC in Lab343A, when visiting
various different websites. Note that the TTL returned from visiting different websites is the
same, 61. This TTL value is interpreted as having an initial value of 64 from the Cisco Content
Engine, which has then passed through three routers prior to being received at the web cache.




Page 66 of 103                                                                  Albert Adi-Wijaya
                          Detecting Transparent Network Devices




                 Figure 54: Ethereal Packet Capture from 144.135.8.152, TTL = 61




                 Figure 55: Ethereal Packet Capture from 203.26.51.42, TTL = 61

Page 67 of 103                                                                     Albert Adi-Wijaya
                          Detecting Transparent Network Devices




                 Figure 56: Ethereal Packet Capture from 202.58.48.51, TTL = 61




Page 68 of 103                                                                    Albert Adi-Wijaya
                             Detecting Transparent Network Devices




                    Figure 57: Ethereal Packet Capture from 65.170.56.4, TTL = 61

It is also worth mentioning that the source IP addressed returned by the cache, is not the IP
address of the cache, but the IP address of the web server itself. The rationale for ‘spoofing’ the
source IP address is to enhance the transparency of interception caches.

       6.6.3.     Web Cache Protocols

The identification of packets emanating from web caches is a valid means of detecting their
presence, as well as identifying them via their IP address. There are many different types of
protocols available for the end user to observe. Web caches communicate with one another to
assist in determining alternative locations to retrieve a web object from and reduce response
times. The configuration and monitoring of web caches is also done via protocols.

Examples of web cache protocols include:

   •   Internet Cache Protocol (ICP)
       ICP is a protocol which facilitates communication between web caches. It permits the
       exchanging of queries and replies with regards to the contents in their respective caches.
       By doing so, they can determine the most appropriate location to retrieve a web object
       from in order to reduce the response time and avoid retrieving it from a ‘distant’ server.
       [Source: http://icp.ircache.net/]

   •   Hyper Text Caching Protocol (HTCP)

Page 69 of 103                                                                      Albert Adi-Wijaya
                             Detecting Transparent Network Devices

       HTCP is a protocol for discovering HTTP caches and cached data, managing sets of
       HTTP caches, and monitoring cache activity.
       [Source: http://www.networksorcery.com/enp/protocol/htcp.htm]

   •   Web Cache Co-ordination Protocol (WCCP)
       In order to perform interception caching, WCCP provides services such as routing web
       traffic from the router to the web cache, automatic failure detection and recovery as well
       as load balancing amongst multiple web caches.
       [Source: http://www.swelltech.com/support/procyonguide/ch10.html]

The identification of such packets on a network is a method of detecting the presence of a web
cache and is also a means for identifying the cache by its IP address. A disadvantage with this
approach is that the end user may not be in a position to capture such packets. We also require
that web caches support these protocols, as some implementation may not.

   6.7. Design and Implementation
This section of the thesis describes the experiments conducted by the author and the tools that
were developed to detect the presence of web caches.

       6.7.1.     TTL Experiment

The aim of this experiment is to determine whether or not the TTL is reset to a particular value by
a web cache when visiting different websites. In Lab343A of the Electrical Engineering Building,
we set up an interception proxy cache, as shown in the figure below.


                                 Internet




                          eth0: DHCP

                                            eth1: 192.168.10.254            Web
                         Linux    Router                                   Cache
                                                            192.168.10.1

                                       eth2: 192.168.20.254                Windows




                                       eth0: 192.168.20.1


                                 Web Client

                                    Linux
                                 Figure 58: TTL Experiment Configuration



Page 70 of 103                                                                       Albert Adi-Wijaya
                               Detecting Transparent Network Devices

The implementation of the router in the above figure is a Linux machine using iptables to act as
a gateway router for the client and cache, as well as redirecting HTTP requests from the client to
the cache to set up the transparent interception cache. The latter can be achieved by using the
following iptables command:

iptables -t nat -A PREROUTING -i eth2 -p tcp --dport 80 -j DNAT --to 192.168.10.1:80
                 Figure 59: iptables command to redirect HTTP traffic from client to cache

The following web caching software was used in this experiment on a Windows PC:
   • Vicromsoft RapidCache
   • RhinoSoft.com AllegroSurf
   • Squid

Because the packet goes through one router, we expect that the TTL returned is 127, since
Windows sets the TTL value to 128 initially.

We repeated the experiment with three different web caching software and the result returned is
the same, that is, the TTL is reset to the same value for different web sites.

       6.7.2.       Performance Measurement Experiments

The purpose of these experiments is to measure the decrease in response time that can be
achieved when using a web cache. Web caches are deployed at a number of levels, ranging from
those on the same LAN or caching provided by the ISP.

The specific performance metrics that we are concerned with are:
   • Elapsed Time: This is the time taken for a web object to be downloaded.
   • Inter-arrival time: Time intervals between packets arriving from the web server/cache.
   • Round Trip Time: Time interval between when the packet is first sent from the host to
       when a reply packet is received from the server/cache.

We expect that these response times will decrease when the same objects are repeatedly
requested. By conducting these studies, it will provide the basis for the development of end user
web cache detection tools based on performance metrics.




Page 71 of 103                                                                        Albert Adi-Wijaya
                              Detecting Transparent Network Devices


                                          Internet



                                                     ISP Cache




                                                     UNSW Cache




                                                     LAN
                                                     Cache




                                       LAB 343A
                    Figure 60: Web Cache Performance Analysis Experimental Setup

Experiment I: Elapsed Time

In this experiment, we download a web object repetitively until we can see a decrease in time
required to download the object. We try a variety of different object sizes, and located on
different servers around the world. With regards to the experimental setup, we first conduct the
studies with a direct connection to the ISP at the author’s home. We also perform the same
experiments at UNSW which also has a web cache. Finally, we set up a transparent web cache in
Lab343A of the EE building and perform the same experiment.

To download the web object, we use a program called wget with the caching turned off. We use
the system clock to measure the time taken for the download.

We expect that the elapsed time to decrease when an object is retrieved from a cache rather than
from a ‘distant’ web server.

The results are as follows:

                                  % reduction in Elapsed Time
         LAN Cache                      UNSW Cache                      ISP (Telstra BigPond) Cache
           88%                               79%                                Inconclusive
                                Table 10: % reduction in elapsed time

We have a substantial percentage reduction in elapsed time for both the LAN Cache and the
UNSW Cache. This is particularly true for large files (8MB to 10MB) where the download rate
was on the order of 100-200Kbps when the object was retrieved the first time from the origin
server, but was much larger 3-4Mbps when being downloaded from the cache.

The detection of an ISP cache has proven to be more difficult, with a few results suggesting that a
cache may be present, but other results returned either inconsistent (e.g. increasing elapsed time,
Page 72 of 103                                                                     Albert Adi-Wijaya
                                    Detecting Transparent Network Devices

perhaps due to other users sharing the internet connection) or no ‘significant’ reduction detected
(perhaps due to another user having already downloaded the object causing it to be stored in the
cache).

Experiment II: Inter-Arrival Time

We anticipate that the inter-arrival time of packets arriving from a web cache to be less than that
from an origin server due to higher LAN link speeds and also less variable router delays. We use
tcpdump to capture the packets whilst retrieving the object using wget. tcpdump can also print
out the time differential between receiving packets. These are then processed by a C program to
determine the average. This entire process is repeated for the same web object.

                                      % reduction in Inter-Arrival Time
             LAN Cache                          UNSW Cache              ISP (Telstra BigPond) Cache
               89%                                  83%                         Inconclusive
                                    Table 11: % reduction in Inter-Arrival Time

Again, we see similar results to that of the elapsed time, with substantial reductions for both the
LAN Cache and the UNSW Cache. The results from the ISP Cache have again proven to be
inconclusive.

           6.7.3.       Performance Measurements End User Tools

Based on the results which were derived empirically, we now describe the tool developed in this
thesis for the end user, to detect the presence of a web cache using performance measurements.
The performance metric we considered in this thesis was the elapsed time.

In brief, the elapsed time is measured by recording the system time before and after a web object
is downloaded. There were two different implementations to download the web object. The first
of these used wget. The second was to write a sockets program which sent a HTTP request. In the
former case, we also presented the option of downloading a ‘random’ page from a web site (i.e. a
web object that does not exist4, to prevent a situation where another user has already downloaded
the same web object).

The process described above is repeated until a significant decrease in response time is observed.
The purpose of the empirical measurements was to justify what constitutes a ‘significant’
decrease. The current version of software uses a reduction of 33% in order to confirm that a web
object was retrieved from the cache, which is fairly conservative in the light of the results
obtained. Further, we wait and observe a decreased performance on at least three consecutive
occasions to ensure that noisy measurements do not influence the results. We assume that this
decrease in response time is caused by the object being cached.

The process in which the tool follows is illustrated in the figure below.



4
    This simply results in a HTTP 404 error message returned.
Page 73 of 103                                                                    Albert Adi-Wijaya
                           Detecting Transparent Network Devices


                                        Get URL
                                        from file


                                        Download
                                       web object
                 Yes                                          No
                 Web Cache                                    Web Cache
                 Detected!              Measure               NOT Detected!
                                       time taken


                                           Is
                                   there a decrease
                                      in response
                                         time?
                         Figure 61: Web Cache Detection Tool – Flowchart

The following figure shows some sample outputs.

---------------
www.nus.edu.sg
137.132.12.114
---------------
(0) Time taken: 2.154663
(1) Time taken: 0.032593
(2) Time taken: 0.023682
(3) Time taken: 0.022217
Web Cache Detected!

---------------
www.nrl.com
64.224.10.182
---------------
(0) Time taken: 0.492554
(1) Time taken: 0.264404
(2) Time taken: 0.241211
(3) Time taken: 0.260132
Web Cache Detected!

-----------------------
Test Results
-----------------------
No. of tests: 15
No. of tests that did detect caches: 8
No. of tests that did NOT detect caches: 7
***Conclusion***
There is a HIGH probability of a cache being present on your LAN/Subnet.
                  Figure 62: Sample Outputs of the Web Cache Detection Tool

Some implementation details:

Page 74 of 103                                                                Albert Adi-Wijaya
                               Detecting Transparent Network Devices


   •    We repeatedly download the object to detect a decrease in elapsed time for a maximum of
        15 times before aborting.
   •    It should also be clear to the user and the reader that even though a reduction in response
        time is NOT seen, this does not imply that there is no cache present. For example, the
        cache may not have stored a local copy of the object due to its caching policy, or perhaps
        another user has already downloaded the object previously.

   6.8. Discussion
        6.8.1.      Other Web Cache Performance Metrics

The performance analysis methods developed in this thesis differs from existing approaches
which have largely focused on the hit ratio of a web cache such as in [ABRAMS 1995]. That is,
the probability that a web object requested, is already stored in the cache which ultimately
depends on the caching policy and replacement algorithm adopted by the web cache.

There have also been similar studies focusing on the performance aspects of web caching which
are peripheral to our work. The work in [LIU 1998] focused on web latency, including a study on
the effects of web proxy caching. One of it’s aims is to answer the question of “Does proxy
caching improve response time?”. Contrary to what one might intuitively think about web proxy
caching, the study finds that a stand-alone squid proxy cache does not always reduce the response
time for the workloads experimented.

The experimental setup is shown in the figures below. A log of URLs was obtained from various
proxy servers as test data for the client to request from web servers. In Figure 63, the web objects
were accessed without a proxy server connection, whilst in Figure 64, a web proxy was placed
between the client and the internet.

                    Internet                                         Internet




                                                                                 Proxy
                                                                                 Cache




                        Client                                          Client
       Figure 63: Configuration with NO proxy             Figure 64: Configuration with proxy



The results of this experiment are shown in the table below.
Page 75 of 103                                                                   Albert Adi-Wijaya
                               Detecting Transparent Network Devices



 Number of          Network          Average            Average             Ratio 1         Ratio 2
 processes         Connection       Connection          Elapsed
                                      Time               Time
      1             Ethernet           0.47               0.86               1.76             0.54
      20            Ethernet           0.81               1.43               1.96             0.56
                      Table 12: Web Latency Response Times - Source: [LIU 1998]

                                           Definitions
Ratio 1             Ratio between the connection time with a proxy to the connection time
                    without a proxy.
Ratio 2             Ratio between the elapsed time with a proxy to the elapsed time without a
                    proxy.
Connection time     Measures the duration between a request sent by a client and the first byte
                    received from the server.
Elapsed time        Measures the duration between a request sent by a client and the last byte
                    received from the server.
                                Table 13: Definitions - Source [LIU 1998]

The results indicate that proxy caching increases both the average connection time and the
elapsed time. It also indicates that an increased in traffic load (as shown by the number of
processes) also degrades the performance of the proxy caching server.

These counter-intuitive results may be explained by the fact that proxy caching can achieve hit
ratios of 30% to 50%. The first time that a web object is retrieved (i.e. no local copy stored in the
cache), the response time would be larger, since a connection needs to be setup between the client
and proxy, and also between the proxy and web server, as well as the additional processing by the
proxy itself. Now if only 30% to 50% of the web objects are stored locally, then the majority of
the web requests will be sent to the original web servers, with the burden of the additional delay
introduced by the proxy cache.

The usefulness of this work to the notion of detecting web caches is limited in that the end user
has little control of accessing the internet with or without a proxy. Our work focuses exclusively
on a scenario where the end user is behind the cache already, and attempts to discover it’s
presence by repeatedly accessing the same object and trying to observe a decrease in response
time.

Furthermore, the result that proxy caches do not decrease response times can be a serious
implication on our work, albeit our focus is on transparent interception caches (which may or
may not yield similar results).

          6.8.2.   Viability of the TTL approach to detect caches

It is still unclear whether or not the TTL is reset for all interception transparent caches. The
author’s thesis supervisor did a packet capture on NICTA’s network when accessing the Internet
and did not observe the TTL being reset when visiting different websites. Likewise, the author
also tried similarly at home with the ISP being Telstra BigPond. Although it is not known if
Page 76 of 103                                                                        Albert Adi-Wijaya
                             Detecting Transparent Network Devices

BigPond deploys a web cache, one would intuitively think they would (to reduce costs), and if
they do, the specific type of caching technology that is used.

In addition, it may be possible that other devices such as NATs, firewalls or even routers that
could be resetting the TTL, albeit routers in particular should not if they are to conform to their
specifications.

It is also possible that the TTL not being ‘spoofed’ correctly to match the incoming packet could
be just an error or design flaw in the implementation of the web cache. If this ‘flaw’ is
widespread, then this technique could still be a valid method to detect the presence of a cache, for
a while yet.

       6.8.3.      Performance Based Detection of Web Caches

The use of performance metrics such as elapsed time has the advantage of being the most generic
methods detailed in this section, but also incurs the cost of not being able to identify the cache by
its IP address and can only infer the presence of the cache. In spite of this, it has proven to be the
most successful and viable of all approaches investigated in this section of the thesis.

   6.9. Conclusion
In this section of the thesis, we have investigated a number of techniques to detect web caches, in
particular interception caches which are transparent unlike proxy caches. These techniques
included both protocol analysis methods such as the TTL field being reset, performance based
detection using the elapsed time to download a web object and also the identification inter-cache
protocols. We also carried out numerous experiments to empirically justify the tools developed
for the end user.

The detection of caches using performance measurements is intuitively what one would expect,
however the work in this thesis, confirmed this hypothesis and implemented the concept into a
tool that can be used by an end user.




Page 77 of 103                                                                    Albert Adi-Wijaya
                             Detecting Transparent Network Devices




7. Network Address Translators (NATs)
   7.1. Introduction
Network Address Translators (NATs) are networking devices which allow multiple hosts to share
the same IP address. The popularity of NATs has been spurred on by the shortage of IPv4
addresses that are available. However, the use of NATs has hindered the deployment of many
applications such as Peer-to-peer file sharing programs. This is because the end system’s public
IP address is required for such applications to operate, but the end system is only aware of its
LAN IP address.

   7.2. Aim
In this section of the thesis we only review existing tools, techniques and literature which involve
detecting the presence of NATs and discovering their salient features.

   7.3. Motivation
NATs can be troublesome devices for applications which require the public IP address of a host
in order to communicate with other hosts, e.g. peer-to-peer file sharing programs, multi-media
applications, multi-player games etc. NATs also inhibit TCP connections from taking place
between hosts where both are behinds NATs.

Thus there needs to be some mechanism to discover and characterise any NATs that an end user
is dependant upon. This allows applications to make the necessary configurations or adjustments
in order to communicate with other end systems. The detection of NATs can also enhance the
dependability of fault reporting. The end user generating the fault report can include details such
as their LAN IP address before the NAT and their public IP address after the NAT.

The use of Application Layer Gateways embedded in NATs such as MIDCOM have had limited
success in alleviating this problem. There are a number of alternate techniques that are available
to an end user to detect the presence of NATs and in this section of the thesis we survey them.

   •   ipconfig + IP address website
   •   Counting NATs
   •   STUN
   •   STUNT

   7.4. ipconfig & IP address website
There are many websites available that provide public IP address discovery services. When an
end user visits such a web site, it displays the end user’s public IP address.

Page 78 of 103                                                                  Albert Adi-Wijaya
                                    Detecting Transparent Network Devices



How these websites determine the public IP address of an end system, is by observing the source
IP address of the IP packet being received, and displaying this address on the web page that the
end user is viewing. Figure 65 below illustrates this.


                     src: 192.168.0.3 dst: 12.148.220.166          src: 138.130.82.142 dst: 12.148.220.166




                                LAN                   NAT/Router                   Internet
       192.168.0.3                                                                                 12.148.220.166



                                                                                                  Web page
                                                                                                138.130.82.142


                                  Figure 65 - How a public IP address is discovered

Unfortunately, this approach lacks the ability to determine the IP address of the NAT or any other
salient features. We are also uncertain of the number of NATs that may be present as well, since
this method will only find the NAT closest to the public internet.

This technique also incurs the problem of requiring the end user to know about these websites in
the first place. Thus ideally, we would prefer a unilateral detection of NATs where the end user
alone can detect NATs and not rely on the assistance of an additional server.

A visit to any one of the following websites will display an end system’s public IP address, which
when used in conjunction with ipconfig can infer the presence of at least one NAT, if the IP
addresses are different.

   •      http://myipaddress.com/
   •      http://www.formyip.com/
   •      http://www.showmyip.com/au/
   •      http://www.urgentclick.com/whats-my-ip-address.php
   •      http://www.mywanip.com/

The following are some sample screen shots of these websites.




Page 79 of 103                                                                                   Albert Adi-Wijaya
                             Detecting Transparent Network Devices




                                  Figure 66 - http://myipaddress.com/




          Figure 67 - http://www.formyip.com/                Figure 68 - http://www.showmyip.com/au/




         Figure 69 - http://www.mywanip.com/
                                                           Figure 70 - http://www.urgentclick.com/whats-
                                                                         my-ip-address.php



A comparison of the public IP address got by visiting one of the mentioned websites can be
compared with the IP Address obtained by typing ipconfig at a DOS prompt (or ifconfig for
Linux users). A difference between these addresses suggests that the end user is residing behind a
NAT.

Page 80 of 103                                                                      Albert Adi-Wijaya
                               Detecting Transparent Network Devices




                                            Figure 71 - ipconfig

   7.5. Counting the number of hosts on the Internet
The presence of NATs has been a hindrance for researchers who strive to estimate the number of
hosts on the internet. This is due to the fact that many hosts reside behind a NAT and share the
same IP address. Thus some mechanism is needed to determine the number of hosts which reside
behind the NAT.

[BELLOVIN 2002] approach to counting the number of hosts behind a NAT exploits the fact that
many operating system’s implementation of the identification field of the IP Packet header is in
fact a simple counter. By analysing packet traces from a NAT, the number of hosts behind it can
be discovered by determining the number of increasing sequences in the IP ID field.

                       5   4    3   2   1




                   138 137 136 135 134

                                                                     136 895 2 68 135 134 1

                       72 71 70 69 68
                                                        router/NAT




                                                                       capture these packets




                   899 898 897 896 895



                   Figure 72: Counting the number of NATs by using the IP ID field

Page 81 of 103                                                                       Albert Adi-Wijaya
                              Detecting Transparent Network Devices

The identification field in IP packet headers is used to distinguish the fragments of one
datagram from those of another. The source host of the IP datagram must set the
identification field to a value that is unique for that source-destination pair for the time that
the datagram will be active in the internet. Many operating systems such as Linux, implement
this field as a simple incremental counter i.e. 1, 2, 3, 4 etc. Subsequently, consecutive packets
emitted by a host will carry sequential IP ID fields. A string of consecutive IP ID fields will thus
represent a string of consecutive packets from a host. Thus by counting the number of strings
from a given IP address, the number of hosts behind the address can be discovered.

There are a number of limitations and assumptions to this approach. These include:
   • Not all packets from a host are sent to the Internet. Some of these packets remain on the
       LAN. Thus, there may be gaps or irregular spacings between sequences of IP ID fields.
       This can lead to the false conclusion of more hosts than there really are.
   • Different operating systems have different implementations of the IP ID field. Some OSes
       implement it as a random number generator, eg. OpenBSD, Solaris.

                           6,000




                     IP ID value




                               0
                                                       time
                                   Figure 73: Graph of IP ID fields

The graph above shows the increasing sequences of IP ID fields that suggests the presence of
four hosts behind a NAT.

By counting the number of hosts behind a NAT to detect the presence of a NAT seems to be a
valid approach. However, the constraint with this approach is that it may be difficult for an end
user to gain access to the packets emitted by a NAT router as shown in Figure 72.

   7.6. STUN
Simple Traversal of User Datagram Protocol (UDP) Through Network Address Translators
(NATs)

STUN is a protocol that allows an application on an end-system to discover the presence and
learn the type of NATs and firewalls between it and the public Internet. Furthermore, it provides

Page 82 of 103                                                                  Albert Adi-Wijaya
                            Detecting Transparent Network Devices

the ability for applications to determine the public IP addresses allocated to it by the NAT.
[STUN RFC3489]

A typical STUN configuration is as indicated in Figure 74.

                                                     STUN determines this IP address

                         192.168.0.5    10.0.0.1         10.0.0.1   138.130.80.137


                       Private                     ISP                      Public
                        LAN                        LAN                     Internet
                                   NAT 1                       NAT 2
    STUN Client                                                                       STUN Server

                                STUN discovers the presence of this NAT
                  Figure 74: A Typical STUN Configuration [Source: STUN RFC3489]

Note that STUN only detects the NAT closest to the public internet (STUN Server). Any NATs
in a private LAN, ISP LAN etc will not be detected, unless of course they are the ‘closest’ to the
STUN Server. This is depicted in Figure 74 above.

The STUN Client itself is often embedded in another application such as a Peer-2-Peer Client e.g.
Shareaza that needs to obtain a public IP address and port number that can be used to retrieve
data. The types of NATs which can be detected by the STUN protocol include full clone,
restricted cone, port restricted cone and symmetric.

STUN classifies NATs into the following four types:
  • Full Cone (all requests from the same internal IP address/port are mapped to the same
     external IP address/port. Thus any external host can send a packet to the internal host by
     sending a packet to the mapped external address.)
  • Restricted Cone (Similar to above, but an external host can send a packet to an internal
     host only if the internal host has previously sent a packet to the external host)
  • Port Restricted Cone (Similar to above, except with the added restriction of the same port
     number being used to communicate between internal and external hosts)
  • Symmetric (all requests from the same internal IP address/port to a specific destination IP
     address/port are mapped to the same external IP address/port which will be a different
     mapping each time a different destination is chosen)

WinSTUN is one such implementation of a STUN client, and following figures depicts the
typical output:




Page 83 of 103                                                                       Albert Adi-Wijaya
                             Detecting Transparent Network Devices




                                    Figure 75: WinSTUN Output

The output in Figure 75 [left] was retrieved by author on his desktop PC at home on 22/04/05 at
17:13, whilst the output in Figure 75 [right] was retrieved by the author on a PC in Lab 343c on
29/04/05 at 13:30.

larry.gloo.net is a STUN server on the public Internet (66.7.238.210). These servers can be
located anywhere, but quite often will be located on the public Internet. They are usually set up
by service providers whose clients may be located behind one or more NATs.

It should be noted that this protocol is not a cure-all for problems associated with NATs. In
addition it also operates under assumptions about a NAT’s treatment of UDP packets.

STUN also has the capability to determine the type of NAT as well, which has the benefit of
providing applications with the extra information necessary to configure their system in order to
work behind the NAT.

Although STUN is a very useful protocol for discovering the presence of NATs closest to the
public internet, which allow many applications like Skype (VoIP application), to function
correctly, what is needed for the scope of this thesis, is a mechanism to detect NATs closer to the
end user.

   7.7. STUNT
Simple Traversal of UDP Through NATs and TCP too (STUNT)

STUNT “…is a lightweight protocol that allows applications running behind a NAT to determine
external IP and port-binding properties, packet filtering rules and various timeouts associated
with TCP connections through the NAT. Knowing these parameters allows applications to
establish TCP sessions between two NAT'ed hosts…” [STUNT 2004].

STUN’s lack of support for TCP traversals through NATs has prompted research into this area.
NATs prevent hosts outside a NAT’ed network from initiating a session with a host inside the
NAT’ed network.

Page 84 of 103                                                                 Albert Adi-Wijaya
                              Detecting Transparent Network Devices

The problem with TCP connections through NATs is elaborated as follows. Say we have a
situation where Alice wishes to communicate with Bob and both reside behind NATs. Alice first
sends a SYN packet to initiate the connection with Bob. Now, Alice’s OS stack and her NAT
expect to receive a SYNACK from Bob in reply. However, the SYN from Alice is blocked by
Bob’s NAT. Thus no SYNACK is received and the TCP connection is not possible.

There have been a number of possible solutions raised to solve this problem, including:
   • STUNT #1
   • STUNT #2
   • NATBlaster
   • P2PNAT

Each of these approaches require that both ends initiate a TCP connection. The outbound SYN
packet from each host creates the necessary NAT state for its own NATs. Each approach then
reconciles the two TCP attempts into a single connection via different means. We outline each of
these individually in the ensuing sections.

       7.7.1.     STUNT 1

With this method, each endpoint sends an initial SYN with a TTL high enough to pass through
their respective NATs, yet small enough so that the packet is dropped prior to reaching the other
NAT. The endpoints can learn the initial TCP sequence number used by their OS stack by
listening for the outbound using packet sniffing. The endpoints then send their respective TCP
sequence numbers to a globally reached STUNT server. This STUNT server then spoofs a return
SYNACK to each of the NATs with the appropriate sequence number. The ACKs complete the
connection through the network as normal [STUNT2004]. This process is depicted in Figure 76
below.

The obvious problem with this approach is determining what TTL to use.

                                                  Internet

                                NAT A                               NAT B
                   Client A                                                         Client B

                                             STUN
                                             Server


                        SYN (low TTL)                                      SYN (low TTL)

                        ICMP error                                           ICMP error

                        TCP Seq #
                                                                              TCP Seq #


                                     SYNACK (spoofed)
                                                        SYNACK (spoofed)

                       ACK
                                                                                    ACK


                                     Figure 76: STUNT #1 [STUNT 2004]
Page 85 of 103                                                                                 Albert Adi-Wijaya
                             Detecting Transparent Network Devices




       7.7.2.    STUNT 2

This approach is similar to STUNT 1 except that only one end system sends the initial SYN
packet with a low TTL. The sender then aborts the connection attempt and creates a passive TCP
socket on the same address/port. The other end system then initiates a regular TCP as normal.
[STUNT 2004] This process is depicted in the Figure 77 below.

                                                Internet

                               NAT A                          NAT B
                  Client A                                             Client B

                       SYN (low TTL)

                       ICMP error

                                                                       SYN

                      SYNACK

                                                                       ACK




                                    Figure 77: STUNT #2 [STUNT 2004]

       7.7.3.    NATBlaster

With this approach, each end-station sends out a low TTL SYN packet and notes the TCP
sequence number used by the stack. After the SYN packets have been dropped by the network,
the two end-systems exchange sequence numbers. The rest of the TCP connection is set up as
normal. [STUNT 2004]




Page 86 of 103                                                                    Albert Adi-Wijaya
                                 Detecting Transparent Network Devices


                                               Internet

                              NAT A                              NAT B
                Client A                                                         Client B

                     SYN (low TTL)                                      SYN (low TTL)

                     ICMP error                                           ICMP error


                     TCP Seq #
                                                                           TCP Seq #


                                                                             SYNACK
                     SYNACK


                    ACK
                                                                                 ACK

                                   Figure 78: NATBlaster [STUNT 2004]

       7.7.4.        Peer-to-Peer NAT

Both end systems initiate a TCP connection by sending SYN packets. If the SYN packets cross
the network (i.e. both end systems can send a SYN before receiving a SYN from the other end
system), both the endpoints respond with SYNACKs to establish the connection.

If Client A’s SYN arrives at the Client B’s NAT and is dropped before Client B’s SYN has left
the NAT, the former performs a TCP simultaneous open whilst the other performs a regular open.
[STUNT 2004]




Page 87 of 103                                                                      Albert Adi-Wijaya
                            Detecting Transparent Network Devices


                                            Internet

                            NAT A                            NAT B
              Client A                                                     Client B

                   SYN
                                                                          SYN



                   SYNACK
                                                                       SYNACK


                                                                           ACK
                   ACK




                                 Figure 79: P2PNAT [STUNT 2004]

   7.8. Packet Mangling
It has been suggested in [STUNT 2004] that NATs may alter certain fields in packet headers
which may be useful in detecting such devices. These alterations include:
    • NAT changing the IP identification of packets traversing through the NAT.
    • Adding a random offset to the TCP sequence number of outgoing TCP packets for a
        connection and subtracting for incoming packets.
    • Setting the TTL field to a different value rather than decrementing it by one.

The results of this work as published at https://www.guha.cc/saikat/stunt-results.php?,
indicate that the TCP sequence numbers are altered for some NAT products. The TTL and other
fields remain un-altered.

   7.9. Detecting routers and layer 3 topology discovery
As many NAT functions are embedded inside routers, perhaps it may be useful for an end user to
detect the routers he/she is dependant upon when accessing internet services. A tool such as
traceroute discovers all the routers on the end to end path between an end user and the service
he/she is accessing.

Broadening the scope of traceroute, would be to discovery the layer3 topology of an end user’s
LAN/subnet. There may be other routers that an end user would be interested in, that provide
redundancies and alternate paths when faults/congestion occurs. There has also been a lot of work
in this area as well.


Page 88 of 103                                                                Albert Adi-Wijaya
                             Detecting Transparent Network Devices

One such approach is [STOTT 2002a] which discovers the layer-3 path between endpoints by
using SNMP to gather commonly used MIBs from routers. It includes an algorithm to find the
first hop router, followed by the path between routers to the router connected to the destination
address. As we have mentioned in previous sections of this thesis, any approach which uses
MIBs is rather intrusive and also limited to functioning with networks where SNMP is supported
and enabled.

The work in [BRAN 2001] enhances the traceroute to include network topology mapping
capabilities. It allows the exploration of corporate intranets or the “center”/core of the Internet.
This is done by selecting hosts randomly from networks announced via BGP and launching
traceroute towards it. The version of traceroute used is a more light weight process (e.g. it
aborts after 2 unsuccessful hops) in order to reduce the load of traffic on the Internet. The data is
collated and visualisation algorithms are used to display the data.

   7.10. Discussion
Detecting NATs via unilateral means whereby the end user alone determines the presence of the
NAT is quite a challenging task, if not impossible. The efforts of the author have been unable to
do so, and to the best of his knowledge, such a technique does not exist. This is because the end
user alone quite often, does not have access to the packets which have been altered by the NAT
device. The only viable method of detecting NATs is STUN, a bilateral method where both a
client and a server are required on opposite sides of the NAT. Though the bilateral approach
yields more accurate and useful results, the main disadvantage is that the end user needs to be
aware of the STUN server in the first instance.

   7.11. Conclusion
As a result of the work carried out in this section of the thesis, we recommend that if an end user
wishes to determine if he/she is connected to a NAT, or more importantly his/her public IP
address and features of the NAT, he/she should just use the STUN protocol. This is the approach
taken by many applications which do not work when there are on hosts located behind NATs. If
the application uses the TCP protocol, the newer STUNT protocol can be used.

If the end user is more interested in locating NATs closer to them, then it is recommended that
they explore mapping the Layer 3 topology. Many NAT functions are embedded inside routers,
so discovery the topology of routers will assist in the diagnosis of faults with NATs.




Page 89 of 103                                                                   Albert Adi-Wijaya
                             Detecting Transparent Network Devices




8. Firewalls
   8.1. Introduction
                   “…Transparency is the major benefit of proxy firewalls…”
                                     [ANDRESS 2002]

Firewalls are useful networking devices which when properly configured can protect a network
from malicious attacks by crackers. It is the transparency of the device which makes it so
effective in thwarting network attacks. If the device was not transparent, then crackers could
develop techniques to bypass the security measures introduced by the firewall.

There are two main types of firewalls that are considered in this thesis:
   • Packet Filters: selectively route packets between trusted and un-trusted networks,
       allowing and denying packets based on the site’s security policy. They are application
       independent and examine each packet at the network layer and are often implemented in
       routers.
   • Proxy firewalls: function at the application layer by taking requests for Internet services
       and forwards them to the actual services.

   8.2. Aim
Due to security concerns, we have decided it is in the best interests for both network
administrators and end users that firewalls remain un-detected. There have been a number of
techniques used to detect firewalls, but the author cannot guarantee that any method developed
with good intentions will not be abused by malicious internet crackers. However, for
completeness, we discuss a few approaches to detect firewalls in the following sections.

   8.3. Motivation
Despite the security concerns with detecting firewalls, there are times when an end user may
benefit from the knowledge of which firewalls he/she is dependant upon when accessing network
services. For example, if the user had difficulty accessing a web page, perhaps the packets are
being blocked by the firewall. The user may not realise this as he is unaware of the firewall in the
first place.

   8.4. Existing Tools to detect firewalls
       8.4.1.      NMap

The most simple way of detecting or identifying a firewall is to use a simple scan for open ports.
Certain firewalls can be identified by their default open ports. For example, Checkpoint’s
Firewall-1 by default listens on TCP ports 256, 257 and 258; Microsoft’s Proxy Server by default
Page 90 of 103                                                                  Albert Adi-Wijaya
                             Detecting Transparent Network Devices

listens on TCP ports 1745 and 1080. A tool such as Nmap can provide this port scanning
functionality. [Ka0ticSH 2002]

e.g. >nmap –n –vv –P0 –p 256, 257, 258, 1080, 1745 <IPaddress>
                                  Figure 80: Example use of NMap

       8.4.2.     Firewalking

Firewalking provides a traceroute-like analysis of IP Packet Responses to determine gateway
Access Control Lists (ACL) [FIRE 1998]. It is a technique in which information about a remote
network protected by a firewall can be gathered. By mapping any ‘open’ or ‘pass-through’ ports
on a gateway, it can determine if packets with various control information can pass through a
given gateway.

In order to use firewalking, two following is required:
    • The IP address of the last known gateway before the firewall
    • The IP address of a host located behind the firewall

The following figure provides a sample output for Firewalk.

zuul:#firewalk -n -P1-8 –pTCP 10.0.0.5 10.0.0.20
Firewalking through 10.0.0.5 (towards 10.0.0.20) with a maximum
of 25 hops.
Ramping up hopcounts to binding host...
probe: 1 TTL: 1 port 33434: <response from> [10.0.0.1]
probe: 2 TTL: 2 port 33434: <response from> [10.0.0.2]
probe: 3 TTL: 3 port 33434: <response from> [10.0.0.3]
probe: 4 TTL: 4 port 33434: <response from> [10.0.0.4]
probe: 5 TTL: 5 port 33434: Bound scan: 5 hops <Gateway at
5 hops> [10.0.0.5]
port 1: open
port 2: open
port 3: open
port 4: open
port 5: open
port 6: open
port 7: *
port 8: open
13 packets sent, 12 replies received
                          Figure 81: Sample Output of Firewalk

       8.4.3.     Layer 3 Topology Discovery

As mentioned previously, many firewalls (packet filters) and NATs are now commonly being
implemented in routers. It is becoming less common for NATs and firewalls to exist as stand
alone devices. Thus, if an end user was interested in discovering the firewalls he/she is dependant
on, he would perhaps be better off attempting to map the layer3 topology of routers, even though
identifying which of the routers are firewalls/NATs may not be possible.


Page 91 of 103                                                                 Albert Adi-Wijaya
                             Detecting Transparent Network Devices


   8.5. Conclusion
In this section of the thesis we have provided a limited treatment of the detection of firewalls due
to the security issues they may arise. We have however provided some existing techniques to
detect firewalls, most which are dependent on some design ‘flaw’ in their implementation. Lastly,
we have recommended that the end user should aim to determine the layer 3 topology in order to
detect NATs/firewalls.




Page 92 of 103                                                                  Albert Adi-Wijaya
                           Detecting Transparent Network Devices




9. Summary
This section summarises the contributions of the author in each section of the thesis. The
distribution of the work turned out to be 60% on bridges, 25% on web caches, 10% for NATs and
5 % for firewalls.

   9.1. Bridges

   •   Literature Review
   •   Topology Discovery Algorithm using BPDUs (including a successful implementation)
   •   Topology Discovery Algorithm by training switches and probing hosts (a partial
       implementation completed)
   •   Theoretically and experimentally measure the performance of the algorithms

   9.2. Web Caches

   •   Literature Review
   •   Experiments
           o TTL Reset Investigation
           o Performance Analysis
   •   End User Web Cache Detection Tool based on performance analysis

   9.3. Network Address Translators

   •   Literature Review
   •   Analysis of Existing Tools (STUN)

   9.4. Firewalls

   •   Literature Review




Page 93 of 103                                                             Albert Adi-Wijaya
                             Detecting Transparent Network Devices




10. Conclusion
The Internet is a technology that end users are becoming increasingly dependant upon. However
whenever a fault occurs, end users are mystified particularly if the fault is occurring in a device
that they were unaware of i.e. transparent. This thesis explored tools and techniques that an end
user could adopt in order to detect and hopefully identify the networking devices he/she is
dependant upon when accessing the Internet.

In the detection of bridges, we were able to design two algorithms for the end user (in
collaboration with other end users) to determine the layer 2 Ethernet topology of the network.
The first of these was successfully implemented and proved to be successful when tested on the
networks available to the authors. The later was only partially implemented. In both cases, we
were able to analyse theoretically and experimentally their performance.

Furthermore, we synthesised a number of strategies to detect transparent interception caches,
though methods such as performance analysis, protocol analysis (TTL) and also identification of
inter-cache protocol packets. We implemented an end user tool to detect web caches based on
performance measurement techniques. This tool has had reasonable level of success in detecting
LAN caches.

Our work with NATs and firewalls was largely limited to surveying existing tools and methods.

Possible extensions of this work would include investigating what an end user could do with the
information generated by the tools developed in this thesis. For example, by monitoring the
Ethernet topology over a period of time, what performance/fault management enhancements are
possible?

Overall, we have shown that although devices are intentionally designed to be transparent, there
are methods which can be used to detect them and situations where this detection would be useful
for end users. By doing so, we have been able to enhance the dependability of the internet for end
users.




Page 94 of 103                                                                 Albert Adi-Wijaya
                           Detecting Transparent Network Devices




11. Appendix
   11.1. Appendix A: References
      11.1.1.    Bridges

[LOWE 2001]           Lowekamp, B., O'Hallaron, D., and Gross, T. 2001. Topology discovery
                      for large ethernet networks. In Proceedings of the 2001 Conference on
                      Applications, Technologies, Architectures, and Protocols For Computer
                      Communications (San Diego, California, United States). SIGCOMM '01.
                      ACM Press, New York, NY, 237-248. DOI=
                      http://doi.acm.org/10.1145/383059.383078
[BLACK 2004]      Ethernet topology discovery without network assistance, Black, R.;
                  Donnelly, A.; Fournet, C.; Network Protocols, 2004. ICNP 2004.
                  Proceedings of the 12th IEEE International Conference on 2004
                  Page(s):328 – 339, Digital Object Identifier 10.1109/ICNP.2004.1348122
[BREITBART 2000] Topology discovery in heterogeneous IP networks, Breitbart, Y.;
                  Garofalakis, M.; Martin, C.; Rastogi, R.; Seshadri, S.; Silberschatz, A.;
                  INFOCOM 2000. Nineteenth Annual Joint Conference of the IEEE
                  Computer and Communications Societies. Proceedings. IEEE, Volume 1,
                  26-30 March 2000 Page(s):265 - 274 vol.1, Digital Object Identifier
                  10.1109/INFCOM.2000.832196
[STOTT 2002]      David Stott, Layer-2 Path Discovery Using Spanning Tree MIBs,
                  Internal Technical Report, Avaya Labs Research, Avaya Inc., March.
                  2002.
[IEEE802.1D 2004] IEEE Std 802.1DTM 2004 - IEEE Standard for Local and Metropolitan
                  Area Networks: Media Access Control Bridges
[PERLMAN 2000]    Perlman, Radia. (2000), Interconnections, Second Edition, USA, Addison
                  Wesley.
[CISCO 2003]      Deploying Layer 2 Security in Server Farms, Internal Technical
                  Report, Cisco Systems Inc, March 2003
[LOWE 2000]       Bruce Lowekamp, 2000, Discovery and Application of Network
                  Information, [online] available: http://reports-
                  archive.adm.cs.cmu.edu/anon/2000/CMU-CS-00-147.pdf [accessed:
                  02/06/05]
[RFC 1493]        E. Decker, P. Langille, A. Rijsinghani, K. McCloghrie, 1993, RFC1493
                  Definitions of Managed Objects for Bridges, [online] available:
                  http://www.faqs.org/rfcs/rfc1493.html [accessed: 14/07/05]

      11.1.2.    Web Caches

[LIU 1998]            Binzhang Liu, 1998, Characterising Web Response Time
[ABRAMS 1995]         M. Abrams, C.R. Standridge, G. Abdulla, S. Williams, 1995, Caching
                      Proxies: Limitations and Potentials, [online] available:
Page 95 of 103                                                            Albert Adi-Wijaya
                         Detecting Transparent Network Devices

                     http://ei.cs.vt.edu/~succeed/WWW4/WWW4.html [accessed:
                     21/10/05]

      11.1.3.     NATs

[BELLOVIN 2002]      Bellovin, S. M. 2002. A technique for counting natted hosts. In
                     Proceedings of the 2nd ACM SIGCOMM Workshop on internet
                     Measurment (Marseille, France, November 06 - 08, 2002). IMW '02.
                     ACM Press, New York, NY, 267-272. DOI=
                     http://doi.acm.org/10.1145/637201.637243
[GUHA 2005]          S. Guha and P. Francis. "Characterization and Measurement of TCP
                     Traversal through NATs and Firewalls," in Proceedings of Interet
                     Measurement Conference (IMC), Berkeley, CA, Oct 2005.
[STUNT 2004]         STUNT - Simple Traversal of UDP Through NATs and TCP too
                     [Online] Available:
                     http://nutss.gforge.cis.cornell.edu/pub/draft-guha-STUNT-
                     00.txt [Accessed: 03/10/05]
[STOTT 2002a]        David Stott, SNMP-based Layer-3 Path Discovery, Internal Technical
                     Report, Avaya Labs Research, Avaya Inc., March 2002
[sFLOW RFC]          InMon Corp, 2001, RFC 3176 InMon Corporatioon’s sFlow: A
                     Method for Monitoring Traffic in Switched and Routed Networks
                     [online] available: http://www.faqs.org/rfcs/rfc3176.html
                     [accessed: 17/10/05]
[sFLOW 2003]         InMon Corp, 2003, Traffic Monitoring using sFlow, [online]
                     http://www.sflow.org/sFlowOverview.pdf [accessed: 17/10/05]
[sFLOW PHAAL]        Peter Phaal, Detecting NAT Devices using sFlow, [online] available:
                     http://www.sflow.org/detectNAT/ [accessed: 17/10/05]
[STUN RFC3489]       J. Rosenberg, J. Weinberger, C. Huitema, R. Mahy, 2003, RFC 3489
                     STUN – Simple Traversal of User Datagram Protocol (UDP)
                     Through Network Address Translators (NATs) [online] available:
                     http://www.faqs.org/rfcs/rfc3489.html [accessed: 17/10/05]
[BRAN 2001]          What can you do with Traceroute? Branigan, S.; Burch, H.; Cheswick,
                     B.; Wojcik, F.; Internet Computing, IEEE Volume 5, Issue 5, Sept.-Oct.
                     2001 Page(s):96

      11.1.4.     Firewalls

[ANDRESS 2002]       Mandy Andress, 2002, Surviving Security, 1st Edition, Indianapolis,
                     SAMS
[FIRE 1998]          David Goldsmith, Michael Schiffman, 1998, Firewalking, [online]
                     available: http://www.packetfactory.net/firewalk/firewalk-
                     final.html [accessed: 26/10/05]
[Ka0ticSH 2002]      Ka0ticSH, 2002, Detection of Firewalls, and Probing networks behind
                     firewalls [online] available:
                     http://neworder.box.sk/newsread.php?newsid=2914 [accessed:
                     26/10/05]

Page 96 of 103                                                            Albert Adi-Wijaya
                           Detecting Transparent Network Devices


      11.1.5.    General

[MAHAJAN 2003]      Mahajan, R., Spring, N., Wetherall, D., and Anderson, T. 2003. User-
                    level internet path diagnosis. In Proceedings of the Nineteenth ACM
                    Symposium on Operating Systems Principles (Bolton Landing, NY, USA,
                    October 19 - 22, 2003). SOSP '03. ACM Press, New York, NY, 106-119.
                    DOI= http://doi.acm.org/10.1145/945445.945456
[THOTTAN 1998]      Adaptive thresholding for proactive network problem detection,
                    Thottan, M.; Chuanyi Ji; Systems Management, 1998. Proceedings of the
                    IEEE Third International Workshop on 22-24 April 1998 Page(s):108 –
                    116
[KUROSE 2004]       James F. Kurose, Keith W. Ross, 2004, Computer Networking – A top
                    down approach featuring the Internet, 3rd Edition, Addison Wesley
[STEVENS 1994]      W. Richard Stevens, 1994, TCP/IP Illustrated, Volume 1 The
                    Protocols, 1st Edition, Addison Wesley
[TM 34]             Monitoring and measuring server availability, [online] accessible:
                    http://comp1.ee.unsw.edu.au/%7Eeet/thesis_dB/descriptionThe
                    sisTopic_students.php?topic_id=TM34 [accessed 26/10/05]
[TM 32]             Notifying users of outages and remedial actions, [online] available:
                    http://uluru.ee.unsw.edu.au/~tim/projects/notify/2005/
                    [accessed: 26/20/05]
[TM 25]             Informative fault report wizard, [online] available:
                    http://uluru.ee.unsw.edu.au/~tim/projects/reporting/
                    [accessed: 26/10/05]
[LIBPCAP]           The libpcap project, [online] available:
                    http://sourceforge.net/projects/libpcap/ [accessed: 26/10/05]
[LIBNET]            Libnet, [online] available: http://libnet.sourceforge.net/
                    [accessed: 26/10/05]
[NEMESIS]           Nemesis, [online] available: http://nemesis.sourceforge.net/
                    [accessed: 26/10/05]
[ETHEREAL]          Ethereal, [online] available: http://www.ethereal.com/ [accessed:
                    26/10/05]
[WGET]              Wget, [online] available:
                    http://www.gnu.org/software/wget/wget.html [accessed: 26/10/05]




Page 97 of 103                                                             Albert Adi-Wijaya
                         Detecting Transparent Network Devices




   11.2. Appendix B: Acronyms
ACL      Access Control List
API      Application Programming Interface
ARP      Address Resolution Protocol
BGP      Border Gateway Protocol
BPDU     Bridge Protocol Data Unit
DCT      Direct Connection Theorem
HTCP     Hyper Text Caching Protocol
HTTP     Hyper Text Transfer Protocol
ICMP     Internet Control Message Protocol
ICP      Internet Cache Protocol
ID       Identification
IP       Internet Protocol
ISP      Internet Service Provider
IT       Information Technology
LAN      Local Area Network
MIB      Management Information Base
NAT      Network Address Translator
NICTA    National Information Communication Technology Australia
OID      Object Identifier
RTT      Round Trip Time
SNMP     Simple Network Management Protocol
STP      Spanning Tree Protocol
STUN     Simple Traversal of UDP through NATs
STUNT    Simple Traversal of UDP Through NATs and TCP too
TCN      Topology Change Notification
TCP      Transmission Control Protocol
TTL      Time To Live
VoIP     Voice Over Internet Protocol
WCCP     Web Cache Co-ordination Protocol




Page 98 of 103                                                     Albert Adi-Wijaya
                  Detecting Transparent Network Devices




   11.3. Appendix C: Summary Sheet




Page 99 of 103                                            Albert Adi-Wijaya
                              Detecting Transparent Network Devices

Thesis title: Detecting Transparent Network Devices                           Topic number: TM 30
Student Name: Albert Adi-Wijaya                                               Student ID: 3019452

A.       Problem statement
The Internet is a technology that end users are becoming increasingly dependent
upon. However, a system as large and complicated as the Internet is susceptible to
faults, which deny end users access to network services e.g. email. Compounding
the problem further, many end users are not well-versed in networking and thus
un-able to diagnose nor recover from a network fault. Many of these faults will occur
in a device that is transparent to the end user to facilitate the ease of their deployment.
However, this makes it particularly difficult to locate a fault in a device that they are
unaware of. Thus some tools are needed for an end user to detect such devices.
B.       Objective
The aim of this thesis is to develop tools and techniques which allow an end user to
detect/discover the networking devices that they are dependant upon when accessing
the Internet. We focus on transparent devices such as Ethernet switches (bridges), web
caches, network address translators and firewalls.
C.       My solution
To detect bridges/switches, we capture BPDUs and by collating them amongst end
users, it is possible to approximate the layer 2 Ethernet topology. We can also detect
bridges and their topology by ‘training’ switches and observing the destination of
suitable probe packets. To detect web caches, we can observe if the TTL field is being
reset or discover a change in performance when a web object is repeatedly requested.

D.      Contributions
1. Algorithm to discover layer 2 Ethernet topology by capturing BPDUs and collating
them amongst end users.
2. Algorithm to discover layer 2 Ethernet topology by training switches and observing
the destination of suitable probe packets.
3. Performed experiments and developed tools to measure the performance (response
time, inter-arrival arrival time) of web caches.
4. Performed experiments to investigate if web caches reset the TTL field of IP
packets.
5. Surveyed existing tools and literature to detect/discover NATs and firewalls.
E.      Suggestions for future work
As the end user is empowered with the knowledge of which networking devices he
is dependent upon, it should be investigated what can be done with this information.
E.g. fault reporting tool, interpreting network service outage notifications

While I may have benefited from discussion with other people, I certify that this thesis is entirely
my own work, except where appropriately documented acknowledgements are included.


Signature: _________________________________                           Date: __ / __ / ____


Page 100 of 103                                                                    Albert Adi-Wijaya
                    Detecting Transparent Network Devices




   11.4. Appendix D: Thesis Pointers




Page 101 of 103                                             Albert Adi-Wijaya
                            Detecting Transparent Network Devices


                                      Thesis Pointers
10          Problem Statement
10          Objective

Theory
30          IEEE 802.1D (Spanning Tree Protocol) bridges emit BPDUs.
43          Bridges learn the source address of frames and the ports they came from.
66          Transparent Interception Web Caches may reset the TTL field.
64          Caches reduce the response time and inter-arrival time.
88          Many NATs and firewalls are implemented in routers.

Method of solution
30         Ethernet Topology Discovery using BPDUs
44         Ethernet Topology Discovery by training switches and sending probes
72, 73     Web Cache response time experiment and tool
73         Web Cache inter-arrival time experiment
70         Web Cache TTL experiment

Contributions
30         Algorithm to discover Ethernet topology by capturing/sharing BPDUs
44         Algorithm to discover Ethernet topology by training switches and probing
72, 73     Experimental results on web cache performance enhancements
70         Investigation of web caches resetting TTL fields
78, 90     Survey of existing tools and literature to detect/discover NATs/firewalls.
89         Survey of layer 3 (network layer) topology discovery.

My work
30, 44      Description of algorithms to discover Ethernet topology.
70-73       Design/description of web cache experiments
78          Survey of techniques to discover NATs.

Results
41          Succinct summary/presentation of results
52          Analysis of results
76, 77      Significance of results

Conclusion
94         Statement of whether the outcomes met the objectives
94         Suggestions for future research
Literature
29, 30                                 [PERLMAN 2000]
25                                     [BLACK 2004]
75                                     [LIU 1998]
82                                     [STUN RFC3489]
84                                     [GUHA 2005]

Page 102 of 103                                                              Albert Adi-Wijaya
                             Detecting Transparent Network Devices



     11.5. Appendix E: Instructions
The following are instruction for executing the Ethernet topology discovery program which uses
the BPDU approach.

1.   >./arp

     This command runs the ARP Capture Program. It sniffs the network for ARP
     Requests/Responses and writes them to a text file. The purpose of this program is to identify
     other workstations. Press <CTRL-C> to end the program after the other stations have been
     identified.
2.   >./data_server

     This command runs the data server programs which when receives a request, responds with
     either the BPDU or topology information depending on the type of the request. It runs
     continuously throughout the entire process, and should be run in a separate terminal.
3.   >./bpdu_capture root

     This command allows a station to capture a single BPDU from the bridge it is connected to.
     The special root option ensures that the root path cost value is written as being 19, i.e. to
     create the illusion that it is directly connected to the “root” bridge.
4.   >./bpdu_inject

     The ‘master’ station which is emulating the root bridge can start injecting frames onto the
     network with this command. The bridge ID of the frame is set to a very low value so as to
     force the spanning tree to reconfigure. Wait a few seconds for the spanning tree to
     reconfigure. Now wait for each station to capture a BPDU (see step 5 below), then end the
     program by pressing <CTRL-C>.
5.   >./bpdu_capture

     Whilst step 4 is being executed, after waiting for a few seconds for the spanning tree
     protocol to reconfigure, each station can now capture exactly one BPDU from the bridge
     they are connected to with this command.
6.   >./data_client bpdu

     The master station only, runs this program, which will collect the BPDU information from
     each of the other stations and process it.
7.   >./data_client hop

     Step 3,4,5,6 should be repeated by each other station which takes turns in being the ‘master’
     station. Each station can then execute the above command to collect the topology
     information from each of the other stations.
8.   >./graph_maker

     Finally, this last command plots a graph of the layer 2 Ethernet topology of the network.


Page 103 of 103                                                                Albert Adi-Wijaya

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:10
posted:12/28/2011
language:
pages:103