Shortest path Lecture

Document Sample

```					We began our study of algorithmic techniques with greedy algorithms

The most natural approach to algorithm design

It’s not hard to propose multiple possible greedy algorithms

The challenge is to prove that the greedy algorithm gives a correct solution

For some of the problems the real difficulty is not the proof

There is no greedy algorithm that works.
We need an alternative approach.           Divide & Conquer?

Reduces a running time that was unnecessarily large, but already polynomial
down to a faster running time

Example, Bubble Sort with O(n2) time to
Merge Sort/Quick Sort (n log n) time

Not strong enough to reduce exponential brute-force search down to polynomial time

More powerful technique – Dynamic programming

Basic idea is drawn from the Divide & Conquer:
Carefully decompose the problem into series of sub problems

Then build up correct solutions to larger and larger subproblems.
Dynamic Programming
• A hard topic.

• I try to provide a unified way to think it
and a fixed set of steps to follow.

• I provide analogies (little bird) to make it
hopefully more fun & easier to follow.
Optimization Problems
An important and practical class of computational problems.

For most of these, the best known algorithm runs in exponential time.

Industry would pay dearly to have faster algorithms.
Optimization Problems
Ingredients:
•Instances: The possible inputs to the problem.
•Solutions for Instance: Each instance has an exponentially large
set of solutions.
•Cost of Solution: Each solution has an easy to compute cost or
value.

Goal:
Find the valid solution for this instance with optimal cost (min or max).

The solution might not be unique.
Search Graph For Best Path

We use it because it nicely demonstrates
the concepts in a graphical way.
Search Graph For Best Path
An instance (input) consists of <G,s,t>.

G is a weighted directed layered graph
s source node
t sink node
Search Graph For Best Path
An instance (input) consists of <G,s,t>.
A solution for an instance is a path from s to t.
The cost of a solution is the sum of the weights.
The goal is to find a path with minimum total cost.

2+6+3+7=18
4+2+1+5=12
Brute Force Algorithm

Try all paths, return the best.
But there may be an exponential number of paths!
An Algorithm As
A Sequence of Decisions
I ask a question about the solution.
"Which edge should we take first?"
Some how I decide <s,v3>.
My friend ask the next question.
"Which edge do we take second?"
Some how he decides <v3,v5>.
His friend ask the next question.
"Which edge do we take third?"
Some how he decided <v5,v8>.
An Algorithm As
A Sequence of Decisions
I ask a question about the solution.
"Which edge should we take first?"
How do I decide?

The greedy algorithm?
Taking the best first edge.
Does not work!
Local vs Global Considerations

• We are able to make local observations and choices.
– Eg. Which edge out of s is cheapest?
• But it is hard to see the global consequences
– Which path is the overall cheapest?
• Sometimes a local initial sacrifice can globally
lead to a better overall solution.
An Algorithm As
A Sequence of Decisions
I ask a question about the solution.
"Which edge should we take first?"
How do I decide?
•In reality we will try
all possible first edges.
•But lets skip this part
by pretending that we have
a little bird to answer this
little question.
Little Bird & Friend Alg

I ask a question about the solution.
"Which edge should we take first?"
The bird answers <s,v1>.
My friend asks the next question.
"Which edge do we take second?"
The bird answers <v1,v4>.
But we don’t want to
worry about how our friend
solves his problem.
SubInstance for Friend
Our instance is <G,s,t>: Find best path from s to t.

Our friend is a smaller version of ourselves

We can trust him to give us a correct
answer
As long as we give him a smaller
instance
of the same problem.

What sub instance do we give him?
Little Bird & Friend Alg
The bird answers <s,v1>.
If I trust the little bird, I take step along edge <s,v1>
and ask my friend,
"Which is the best path from v1 to t?"
Friend answers <v1,v6,t>
with weight 10.
To get my solution
I tack on the bird’s edge
making the path <s,v1,v6,t>
with weight 10+3=13.
Faulty Bird
But what if we do not have a bird that we trust?
This work is not wasted, because we have found
•the best solution to our instance
from amongst those
consistent with this birds answer.
•i.e. the best path from s to t
from amongst those
starting with <s,v1>.

In reality we will try
all possible first edges,
giving …..
Faulty Bird

…the best path from amongst
those starting with <s,v1>.
Faulty Bird

… and the best path from amongst
those starting with <s,v2>.
Faulty Bird

… and the best path from amongst
those starting with <s,v3>.
Faulty Bird

… and the best path from amongst
those starting with <s,v4>.
Faulty Bird

At least one of these four paths
must be an over all best path.

I give the best of the best
as the best path.
The Recursive Structure of the Problem
The optimal path from s   to t goes through v3

The path from s to t to be optimal,
the subpath from v3 to t must optimal.
The Recursive Structure of the Problem
For a path from s to t ( s … vi …t) to be optimal,
the subpaths - from s to vi and
vi to t must be optimal.

finding such subpaths
(s to vi and vi to t)
are subInstances of the
same computational
vi
problem.
t
s
Same as Brute Force Algorithm
I try each edge out of s.
A friend tries each edge
out of these.
A friend tries each edge
out of these.

Time?

Same as the brute force algorithm
that tries each paths.
Same as Brute Force Algorithm
But there may be an exponential number of paths!

Dynamic programming is dangerously close to the edge of brute-force search!

It is cleverly working through the exponentially large set of possible set of solutions

without examining them all explicitly
Speeding Up the Time

Sometimes entire an branch can be pruned off.

A Greedy algorithm prunes off all branches
except the one that looks best.
Speeding Up the Time
Memorization
•Remembers the solutions for the subinstances
so that if ever it needs to be solved again,
the answer can be used.
•This effectively prunes off this later branch of the
classification tree.
Exponential Time Redoing Work
"Which is the best path from v5 to t?"
How many friends solve this subinstance?
Exponential Time Redoing Work
"Which is the best path from v5 to t?"
How many friends solve this subinstance?
Exponential Time Redoing Work
"Which is the best path from v5 to t?"
How many friends solve this subinstance?
Exponential Time Redoing Work
"Which is the best path from v5 to t?"
How many friends solve this subinstance?
Exponential Time Redoing Work
"Which is the best path from v5 to t?"
How many friends solve this subinstance?
Exponential Time Redoing Work
"Which is the best path from v5 to t?"
How many friends solve this subinstance?
Exponential Time Redoing Work
"Which is the best path from v5 to t?"
How many friends solve this subinstance?

Once for each
path to v5

Waist time
Having many friends
redoing work                         solving this same
subinstance
Save time by only                    is a waist of time.

doing once.
Speeding Up the Time

We allocate one friend to the job.

It is my job to find
the optimal solution to my subinstance
i.e. the best path from v5 to t
Speeding Up the Time
When I need to find the best path from
v1 to t I will ask you for the best path
from v5 to t
Speeding Up the Time

When I need to find the best path from
v1 to t I will ask you for the best path
from v5 to t
Speeding Up the Time
When I need to find the best path from
v1 to t I will ask you for the best path
from v5 to t
Speeding Up the Time

When I need to find the best path from
v1 to t I will ask you for the best path
from v5 to t
Speeding Up the Time
When I need to find the best path from
v1 to t I will ask you for the best path
from v5 to t

I will find my best path
from v5 to t and tell you.

But I hate to wait for you.

Why don’t you go first?
Speeding Up the Time

Before anyone asks me,
I will find my best path
and remember it.

Memorization:
Remember the solutions for the subInstances so that if
ever it needs to be solved again, the answer can be used.
Set of SubInstances
But what subinstance need to be solved
and in which order?
Given an instance I,

Determine the complete
set of subI ever given to
you, your friends, their
friends, …
Set of SubInstances
Guess the complete set S of subI.
“Best path from v7 to t?”     Yes
“Best path from v21 to t?”    No
v21 is not a part of our
v21
original instance.
Set of SubInstances
Guess the complete set S of subI.
“Best path from v7 to t?”     Yes
“Best path from v21 to t?”    No
“Best path from v3 to v5?”    No
All paths considered
end in t.
Set of SubInstances
Guess the complete set S of subI.
“Best path from v7 to t?”     Yes
“Best path from v21 to t?”    No
“Best path from v3 to v5?”    No
All paths considered
end in t.
Set of SubInstances
Guess the complete set S of subI.
“Best path from v7 to t?”                Yes
“Best path from v21 to t?”               No
“Best path from v3 to v5?”               No

“For all i, Best path from vi to t?”   Yes
Set of SubInstances
Guess the complete set S of subI is
“For all i, Best path from vi to t?”

Assign one friend
to each subI.
Set of SubInstances
Guess the complete set S of subI is
“For all i, Best path from vi to t?”
The set S of subI needs to:
•include our given I
Order to complete
The complete set S of subI is
“For all i , Best path from vi to t?”

In what order should they go?
•in an order such that
no friend must wait.
•from “smallest” to “largest”

For this problem,
the order relies on
the graph being “leveled.”
Order to complete
Note that the graph is levelled:
Indices of the vertices in the path from s to t
must be in increasing order (eg. s,v3,v5,v6, t)

Examples:
s, v3, v5, v6, t
s, v1, v4, v7, t
s, v2, v4, v7, t
Order to complete
The complete set S of subI is
“For all i , Best path from vi to t?”

In what order should they go?
•in an order such that
no friend must wait.
•from “smallest” to “largest”
First    Base Case Easy

Last     Instance to be solved.
Order to complete
"Which is the best path from v8 to t?"

Easy
Order to complete
"Which is the best path from v7 to t?"

Easy
Order to complete
"Which is the best path from v6 to t?"

Easy
Order to complete
"Which is the best path from v5 to t?"

Harder
Order to complete
"Which is the best path from v5 to t?"
Little bird
suggests first
edge <v5,v7>

Friend gives best
path <v7,t>.
Order to complete
"Which is the best path from v5 to t?"
Little bird
suggests first
edge <v5,v8>

Friend gives best
path <v8,t>.
Order to complete
"Which is the best path from v5 to t?"
Take best of best
Order to complete
"Which is the best path from v4 to t?"
Order to complete
"Which is the best path from v4 to t?"
Little bird
suggests first
edge <v4,v6>

Friend gives best
path <v7,t>.
Order to complete
"Which is the best path from v4 to t?"
Little bird
suggests first
edge <v4,t>

Friend gives best
path <t,t>.
Order to complete
"Which is the best path from v4 to t?"
Little bird
suggests first
edge <v4,v7>

Friend gives best
path <v7,t>.
Order to complete
"Which is the best path from v4 to t?"
Take best of best
Order to complete
"Which is the best path from v3 to t?"
Order to complete
"Which is the best path from v3 to t?"
Little bird
suggests first
edge <v3,v5>

Friend gives best
path <v5,t>.
Order to complete
"Which is the best path from v3 to t?"
Little bird
suggests first
edge <v3,v8>

Friend gives best
path <v8,t>.
Order to complete
"Which is the best path from v3 to t?"
Take best of best
Order to complete
vk
t
vi                              Let,
C[k] : cost of the best path from vk to t.
w(i, k): weight of the edge (Vi, Vk)

If we know C[k], can compute C[i]:
C[i] = w(i, k) + C[k]

k and all the vertex-indices in the path from Vk to t are greater than i.

We can compute C[i] in decreasing order of i:           C[7], C[6], C[5], …… C[1], C[s]
vk
t
vi

vi may have more than one edge (vi, vk)

vk

t
vi
vk

t
vi

0        if i = t
C[i] =
min ( w(i,k) + C[k])             if i ≠ t
for all k, (Vi, Vk) is an
edge

Store the resuts in two arrays C and P:
C[i] – cost of the best path from Vi to t
P[i] – the first vertex in the best path from Vi to t.
"Which is the best path from t to t?"

Base Case
Easy

C                                               0

0(s)   1   2   3    4   5     6     7   8       9(t)
P                                               t
"Which is the best path from v8 to t?"

Easy

C                                            5   0

0(s)   1   2   3   4   5    6        7   8       9(t)
P                                            t   t
"Which is the best path from v7 to t?"

Easy

C                                         4       5   0

0(s)   1   2    3    4    5     6        7   8       9(t)
P                                         t       t   t
"Which is the best path from v6 to t?"

Easy

C                               7     4       5   0

0(s)   1   2   3   4   5    6         7   8       9(t)
P                               t     t       t   t
"Which is the best path from v5 to t?"

Harder

C                                  7    4       5   0

0(s)   1   2    3    4    5    6       7   8       9(t)
P                                   t    t      t   t
"Which is the best path from v5 to t?"
Little bird
suggests first
edge <v5,v7>

Friend gives best
path <v7,t>.

cost = w(5,7) + C[7] = 5 + 4

C                                 7    4       5    0

0(s)   1   2   3   4    5     6        7   8        9(t)
P                                 t      t     t    t
"Which is the best path from v5 to t?"
Little bird
suggests first
edge <v5,v8>

Friend gives best
path <v8,t>.

cost = w(5, 8) + C[8] = 1 + 5

C                                    7      4       5    0

0(s)   1     2    3    4   5      6         7   8       9(t)
P                                     t     t       t   t
"Which is the best path from v5 to t?"
Take best of best

C                            6      7    4       5   0

0(s)   1   2   3    4    5      6        7   8       9(t)
P                           v8      t    t       t   t
"Which is the best path from v4 to t?"

C                            6   7    4       5   0

0(s)   1   2   3    4   5    6        7   8       9(t)
P                           v8   t    t       t   t
"Which is the best path from v4 to t?"
Little bird
suggests first
edge <v4,v6>

Friend gives best
path <v6,t>.
cost = w(4, 6) + C[6] = 3 + 7

C                             6      7    4       5    0

0(s)   1   2   3     4   5        6       7   8        9(t)
P                            v8       t   t        t   t
"Which is the best path from v4 to t?"

Little bird
suggests first
edge <v4,t>

Friend gives best
path <t,t>.
cost = w(4,t) + C[9] = 9 + 0

C                           6    7     4       5    0

0(s)   1   2   3   4   5      6        7   8        9(t)
P                          v8     t     t      t    t
"Which is the best path from v4 to t?"

Little bird
suggests first
edge <v4,v7>

Friend gives best
path <v7,t>.
cost = w(4, 7) + C[7] = 2 + 4

C                             6   7      4       5    0

0(s)   1   2   3     4   5     6         7   8       9(t)
P                            v8    t     t       t   t
"Which is the best path from v4 to t?"
Take best of best

C                       6    6     7     4       5   0

0(s)   1   2   3    4   5      6         7   8       9(t)
P                      v7   v8     t     t       t   t
"Which is the best path from v3 to t?"

C                       6    6   7   4       5   0

0(s)   1   2   3    4   5    6       7   8       9(t)
P                      v7   v8   t   t       t   t
"Which is the best path from v3 to t?"
Little bird
suggests first
edge <v3,v5>

Friend gives best
path <v5,t>.
cost = w(3, 5) + C[5] = 2 + 6

C                       6    6      7    4       5    0

0(s)   1   2   3    4    5      6        7   8       9(t)
P                      v7   v8      t    t       t   t
"Which is the best path from v3 to t?"
Little bird
suggests first
edge <v3,v8>

Friend gives best
path <v8,t>.
cost = w(3, 8) + C[8] = 7 + 5

C                       6    6      7    4       5    0

0(s)   1   2   3    4    5      6        7   8       9(t)
P                      v7   v8      t    t       t   t
"Which is the best path from v3 to t?"
Take best of best

C                   8    6    6     7    4       5   0

0(s)   1   2    3    4   5      6        7   8       9(t)
P                  v5   v7   v8     t    t       t   t
"Which is the best path from v2 to t?"

C                   8    6    6   7   4       5   0

0(s)   1   2    3    4   5    6       7   8       9(t)
P                  v5   v7   v8   t   t       t   t
"Which is the best path from v2 to t?"
Little bird
suggests first
edge <v2,v4>

Friend gives best
path <v4,t>.
cost = w(2, 4) + C[4] = 6 + 6

C                   8    6    6      7    4       5    0

0(s)   1   2    3    4   5        6       7   8        9(t)
P                  v5   v7   v8       t   t        t   t
"Which is the best path from v2 to t?"
Little bird
suggests first
edge <v2,v7>

Friend gives best
path <v7,t>.

cost = w(2, 7) + C[7] = 7 + 4

C                   8    6    6    7     4       5    0

0(s)   1   2    3    4   5      6        7   8       9(t)
P                  v5   v7   v8     t    t       t   t
"Which is the best path from v2 to t?"
Take best of best

C              11    8    6    6     7    4       5    0

0(s)   1   2     3    4   5       6       7   8       9(t)
P              v7   v5   v7   v8      t   t       t   t
"Which is the best path from v1 to t?"

C              11    8    6    6   7   4       5   0

0(s)   1   2     3    4   5    6       7   8       9(t)
P              v7   v5   v7   v8   t    t      t   t
"Which is the best path from v1 to t?"
Little bird
suggests first
edge <v1,v6>

Friend gives best
path <v6,t>.

cost = w(1, 6) + C[6] = 3 + 7

C              11    8    6    6    7     4       5    0

0(s)   1   2     3    4   5     6         7   8       9(t)
P              v7   v5   v7   v8    t     t       t   t
"Which is the best path from v1 to t?"
Little bird
suggests first
edge <v1,v4>

Friend gives best
path <v4,t>.

cost = w(1, 4) + C[4] = 5 + 6

C              11    8    6    6    7     4       5    0

0(s)   1   2     3    4   5     6         7   8       9(t)
P              v7   v5   v7   v8    t     t       t   t
"Which is the best path from v1 to t?"
Little bird
suggests first
edge <v1,v5>

Friend gives best
path <v5,t>.

cost = w(1,5) + C[5] = 4 + 6

C              11    8    6    6    7     4       5    0

0(s)   1   2     3    4   5      6        7   8        9(t)
P              v7   v5   v7   v8     t     t      t    t
"Which is the best path from v1 to t?"
Take best of best

C          10   11    8    6    6     7   4       5   0

0(s)   1    2     3    4   5      6       7   8       9(t)
P          v6   v7   v5   v7   v8     t    t      t   t
"Which is the best path from s to t?"
Original Problem

C          10   11    8    6    6   7   4       5   0

0(s)   1    2     3    4   5    6       7   8       9(t)
P          v6   v7   v5   v7   v8   t   t       t   t
"Which is the best path from s to t?"
Little bird
suggests first
edge <s,v1>

Friend gives best
path <v1,t>.

cost = w(s, 1) + C[1] = 3 + 10

C          10   11    8    6    6     7    4       5    0

0(s)   1    2     3    4   5       6       7   8        9(t)
P          v6   v7   v5   v7   v8      t    t      t    t
"Which is the best path from s to t?"
Little bird
suggests first
edge <s,v2>

Friend gives best
path <v2,t>.

cost = w(s, 2) + C[2] = 2 + 11

C          10   11    8    6    6     7    4       5    0

0(s)   1    2     3    4   5       6       7    8       9(t)
P          v6   v7   v5   v7   v8      t    t       t   t
"Which is the best path from s to t?"
Little bird
suggests first
edge <s,v3>

Friend gives best
path <v3,t>.

cost = w(s, 3) + C[3] = 4 + 8

C          10   11    8    6    6     7    4       5    0

0(s)   1    2     3    4   5       6       7    8       9(t)
P          v6   v7   v5   v7   v8      t    t       t   t
"Which is the best path from s to t?"
Take best of best

DONE

C    12    10   11    8    6    6     7   4       5   0

0(s)   1    2     3    4   5      6       7   8       9(t)
P    v3    v6   v7   v5   v7   v8     t    t      t   t
"Which is the best path from s to t?"

Constructing the
optimal path

C      12       10   11    8    6    6   7   4       5   0

0(s)       1    2     3    4   5    6       7   8       9(t)
P      v3       v6   v7   v5   v7   v8   t   t       t   t
"Which is the best path from s to t?"

Constructing the
optimal path

C      12       10   11    8    6    6   7   4       5   0

0(s)       1    2     3    4   5    6       7   8       9(t)
P      v3       v6   v7   v5   v7   v8   t   t       t   t
"Which is the best path from s to t?"

Constructing the
optimal path

C      12       10   11    8    6    6   7   4       5   0

0(s)       1    2     3    4   5    6       7   8       9(t)
P      v3       v6   v7   v5   v7   v8   t   t       t   t
"Which is the best path from s to t?"

Constructing the
optimal path

DONE

C      12       10   11    8    6    6   7   4       5   0

0(s)       1    2     3    4   5    6       7   8       9(t)
P      v3       v6   v7   v5   v7   v8   t   t       t   t
Dynamic Programming
Step 1: Characterize the structure of an optimal solution.
For a path from s to t ( s … vi …t) to be optimal, the subpath vi to t must be optimal.

Step 2: Recursively define the value of an optimal solution.
0     if i = t
C[i] =
min ( w(i,k) + C[k])                if i ≠ t
for all k, (Vi, Vk) is an
edge

Step 3. Compute the value of an optimal solution in a bottom-up fashion
compute C[i] (and P[i]) in decreasing order of i

Step 4: Construct an optimal solution.
C[0] – cost of the best path. Construct the best path from Array P
Running Time
For each vertex Vi,
can compute C[i]:
C[i] = w(i, k) + C[k]
in O(1) time.

vk
t
vi
vi may have more than one edge (vi, vk)
Running Time
vk

t
vi

0        if i = t
C[i] =
min ( w(i,k) + C[k])             if i ≠ t
for all k, (Vi, Vk) is an
edge

Let d is degree of Vi        i.e. there are d number of ks

C[i] can be computed in O(d) time
Running Time

For each i, compute C[i] takes O(d) time

where d is the maximum degree of the graph

We have to compute C[i] for all the vertices.

Suppose there are n vertices

Algorithm runs in O(n × d) time

```
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
 views: 0 posted: 12/3/2011 language: English pages: 110