# Comparison of Q Routing and Shortest Path Algorithms by nyut545e2

VIEWS: 23 PAGES: 22

• pg 1
```									Comparison of Q Routing and
Shortest Path Algorithms

Firat Tekiner, Z Ghassemlooy
Optical Communications Research Group
University of Northumbria, UK

Sheffield Hallam University, UK
overview

   Routing Algorithms

   Shortest Path Routing Algorithm
   Q Routing
   Dual Reinforcement Q Routing

   NSFNET
   PVM
   Simulation Results
   Conclusions
Routing
   Routing is the transmission of data (packets) from a source s to
its destination d on the network or internetwork.

   At every node the packet is received, stored and then routed to
the next hop until it reaches its destination
Routing Algorithms

Path Determination
 Computes the optimal path to every node in the network based
on the routing metric of the links.

Switching
 Forwards the packet to the next hop of the optimal path until it
reaches destination
Classification
   Static Versus Dynamic
   Single-Path Versus Multipath
   Flat Versus Hierarchical
   Host-Intelligent Versus Router-Intelligent

Non-Adaptive Eg. Q-learning methods, hybrid agent
based Distance Vector algorithms
Shortest Path Algorithm
Let
     V : the array of all the edges (nodes)
     D[n]: the Array of Shortest cost of the
     C[s,w]: the cost of the vertices
     s: Router Id
     w: the neighbour

Initialisation
D[w] = ∞
w ->(V-s) with minimal C[s, w]
Relax the node
D[v]= min(D[v], D[w]+c[w,v]),
where C[w,v] are the costs of the vertices,
connected to w
Greedy Algorithm- Optimises the routes to
all the nodes to a single least cost path and it
insists to take this single best path without
regard to future consequences.
Shortest Path Routing
   Computes Shortest Path using the Dijkstra Algorithm.

   Routes the packets based on least cost of the links

   Non-adaptive Routing Algorithm - routing table never change once initial routes
have been selected unless there is a route failure

Drawbacks

   Sometimes there may be congested queues in the intermediate nodes and
packets spend waiting in the queues, which delays the packets travelling to the
destination.

   For node r lying in the shortest path from s to d, it is also shortest path to r to s
and d and vice versa
Q Routing
   Reinforcement learning – all the nodes in the network learn a global
routing policy

   Builds a routing table based on the delivery times (Q values) of the
packets to every node in the network.

   Routes the packet to the destination on path with least delivery time

   For every data packet routed to the next hop, the node gets back a
learning update about the remaining delivery time estimate for the packet
to reach destination (Forward Exploration)

   Synchronised Q Information at every node in the network – balances
minimizing the number of “hops” a packet will take with the possibility of
congestion along popular routes.
Forward Exploration
     When a node x sends a packet to
node d via its neighbour y, it gets
back y’s estimated remaining trip
times to the destination d, selects
the neighbour with the smallest
delivery time

Q x ( z , s )  min Q y ( z , s )
z N ( y )

Qy ( y, s)  b (Qx ( z, s)  q y  Qy ( x, s))

ηb is the learning rate parameter
Dual Reinforcement Q Routing

   Modified Q Routing with backward exploration along with
backward exploration – 2 way learning

   Every data packet carries the Q Information- the delivery time
estimate of the sender node to its processor node (Backward
Exploration)

Backward Exploration
   When x sends a packet to node y to
gets its estimated remaining trip times,
y gets x’s estimated trip times for its link
with s.

Q x ( z , s )  min Q y ( z , s )
z N ( y )

Qy ( y, s)  b (Qx ( z, s)  q y  Qy ( x, s))

ηb is the learning rate parameter
NSFNET
PVM

   Software used for distributed computation

   Permits a heterogeneous collection of Unix and/or Windows
computers hooked together by a network to be used as a single
large parallel computer.

   Supports C, C++ , and Fortran languages

   Built in library functions

   Identifies the application as multiple tasks
Simulation

Routing algorithms were implemented using PVM in C language on
LINUX platform.

Master and slave paradigm: Every node of the NFSNET was
assigned to a PVM task or a slave process, which is spawned by an
initialising task or a master process on the same machine.
slave process
Spawns
Master Process

Single-program multiple-data Model
Tasks with similar function executed in parallel
Simulation Conditions

   NSFNET is used as the network topology

   Random uniform traffic distribution is used with varying packet creation
rates.

   For each simulation, packet generation is stopped after creating 5000
packets per node and simulation is stopped after all packets are arrived
to their destinations or detected and deleted from the network.

   A random link fails every 5 seconds for a period of 4 seconds and
traffic is directed to other neighbours.

   Hello packets were exchanged between the nodes to know whether its
neighbouring nodes are active.

   Payload is fixed –512 Kbps
Performance Parameters

    Average packet delay:
Is the average delay a packet experiences while being routed
from source to destination.

    Average throughput per packet:
Is the average number of packets being forwarded by a node
for the duration of the simulation.
Average Packet Delay Vs. System Load

1.25        OSPF
QR
1.05        DRQR
Avg. Packet Delay(sec)

0.85

0.65

0.45

0.25

0.05
0.025   0.075       0.125       0.175      0.225   0.275
300
OSPF
QR
250         DRQR
Throughput(kbits/sec)

200

150

100

50
0.025   0.075    0.125     0.175     0.225     0.275
Throughput Vs. Learning Rate

220
DRQR
219
QR
Throughput(Kbits/sec)

218

217

216

215

214

213

212
0.1    0.3          0.5            0.7   0.9
Learning Rate (sec)
Average Packet Delay vs. Learning Rate

0.68
DRQR
0.66         QR
Avg. Packet Delay(sec)

0.64

0.62

0.6

0.58

0.56

0.54
0.1    0.3           0.5           0.7   0.9
Learning Rate (sec)
Conclusions

   Q-Routing does not always guarantee to find the shortest path.

   At high system load QR display lower average packet delay compared
with DRQR and SPRA

   Low bandwidth utilization in Q Routing and packet overhead in DRQR –
can be overcome by generating learning packets only when the queues
get filled to a threshold

   Shortest path algorithms ignores the bottleneck as the network traffic
increases.

   The need for an efficient adaptive routing algorithm.
Thank you

```
To top