Docstoc

Shortest path Lecture

Document Sample
Shortest path Lecture Powered By Docstoc
					We began our study of algorithmic techniques with greedy algorithms

The most natural approach to algorithm design

It’s not hard to propose multiple possible greedy algorithms

The challenge is to prove that the greedy algorithm gives a correct solution

For some of the problems the real difficulty is not the proof


         There is no greedy algorithm that works.
 We need an alternative approach.           Divide & Conquer?


Reduces a running time that was unnecessarily large, but already polynomial
  down to a faster running time

                    Example, Bubble Sort with O(n2) time to
                             Merge Sort/Quick Sort (n log n) time



Not strong enough to reduce exponential brute-force search down to polynomial time


 More powerful technique – Dynamic programming

  Basic idea is drawn from the Divide & Conquer:
         Carefully decompose the problem into series of sub problems

         Then build up correct solutions to larger and larger subproblems.
      Dynamic Programming
• A hard topic.

• I try to provide a unified way to think it
  and a fixed set of steps to follow.

• I provide analogies (little bird) to make it
  hopefully more fun & easier to follow.
            Optimization Problems
An important and practical class of computational problems.


For most of these, the best known algorithm runs in exponential time.


Industry would pay dearly to have faster algorithms.
              Optimization Problems
 Ingredients:
     •Instances: The possible inputs to the problem.
     •Solutions for Instance: Each instance has an exponentially large
     set of solutions.
     •Cost of Solution: Each solution has an easy to compute cost or
     value.

Goal:
Find the valid solution for this instance with optimal cost (min or max).


    The solution might not be unique.
Search Graph For Best Path

We use it because it nicely demonstrates
the concepts in a graphical way.
     Search Graph For Best Path
An instance (input) consists of <G,s,t>.

  G is a weighted directed layered graph
  s source node
  t sink node
     Search Graph For Best Path
An instance (input) consists of <G,s,t>.
A solution for an instance is a path from s to t.
The cost of a solution is the sum of the weights.
The goal is to find a path with minimum total cost.



             2+6+3+7=18
             4+2+1+5=12
         Brute Force Algorithm

Try all paths, return the best.
But there may be an exponential number of paths!
              An Algorithm As
           A Sequence of Decisions
 I ask a question about the solution.
   "Which edge should we take first?"
    Some how I decide <s,v3>.
My friend ask the next question.
   "Which edge do we take second?"
    Some how he decides <v3,v5>.
His friend ask the next question.
   "Which edge do we take third?"
    Some how he decided <v5,v8>.
             An Algorithm As
          A Sequence of Decisions
I ask a question about the solution.
  "Which edge should we take first?"
   How do I decide?


   The greedy algorithm?
     Taking the best first edge.
       Does not work!
     Local vs Global Considerations

• We are able to make local observations and choices.
   – Eg. Which edge out of s is cheapest?
• But it is hard to see the global consequences
   – Which path is the overall cheapest?
• Sometimes a local initial sacrifice can globally
  lead to a better overall solution.
             An Algorithm As
          A Sequence of Decisions
I ask a question about the solution.
  "Which edge should we take first?"
   How do I decide?
     •In reality we will try
     all possible first edges.
     •But lets skip this part
     by pretending that we have
     a little bird to answer this
     little question.
         Little Bird & Friend Alg

I ask a question about the solution.
  "Which edge should we take first?"
   The bird answers <s,v1>.
My friend asks the next question.
  "Which edge do we take second?"
   The bird answers <v1,v4>.
   But we don’t want to
 worry about how our friend
    solves his problem.
             SubInstance for Friend
Our instance is <G,s,t>: Find best path from s to t.

Our friend is a smaller version of ourselves


     We can trust him to give us a correct
     answer
     As long as we give him a smaller
     instance
          of the same problem.

     What sub instance do we give him?
         Little Bird & Friend Alg
The bird answers <s,v1>.
 If I trust the little bird, I take step along edge <s,v1>
 and ask my friend,
"Which is the best path from v1 to t?"
Friend answers <v1,v6,t>
with weight 10.
To get my solution
 I tack on the bird’s edge
 making the path <s,v1,v6,t>
  with weight 10+3=13.
                   Faulty Bird
But what if we do not have a bird that we trust?
This work is not wasted, because we have found
 •the best solution to our instance
 from amongst those
 consistent with this birds answer.
 •i.e. the best path from s to t
 from amongst those
 starting with <s,v1>.

In reality we will try
all possible first edges,
  giving …..
                Faulty Bird



…the best path from amongst
those starting with <s,v1>.
                Faulty Bird



… and the best path from amongst
those starting with <s,v2>.
                Faulty Bird



… and the best path from amongst
those starting with <s,v3>.
                Faulty Bird



… and the best path from amongst
those starting with <s,v4>.
                   Faulty Bird



At least one of these four paths
must be an over all best path.



I give the best of the best
as the best path.
The Recursive Structure of the Problem
  The optimal path from s   to t goes through v3


The path from s to t to be optimal,
the subpath from v3 to t must optimal.
The Recursive Structure of the Problem
 For a path from s to t ( s … vi …t) to be optimal,
 the subpaths - from s to vi and
                vi to t must be optimal.


                                          finding such subpaths
                                          (s to vi and vi to t)
                                          are subInstances of the
                                          same computational
                  vi
                                          problem.
                                      t
 s
Same as Brute Force Algorithm
                   I try each edge out of s.
                         A friend tries each edge
                         out of these.
                         A friend tries each edge
                         out of these.

Time?



 Same as the brute force algorithm
 that tries each paths.
       Same as Brute Force Algorithm
But there may be an exponential number of paths!




Dynamic programming is dangerously close to the edge of brute-force search!

It is cleverly working through the exponentially large set of possible set of solutions

without examining them all explicitly
        Speeding Up the Time




Sometimes entire an branch can be pruned off.

   A Greedy algorithm prunes off all branches
   except the one that looks best.
         Speeding Up the Time
Memorization
•Remembers the solutions for the subinstances
so that if ever it needs to be solved again,
the answer can be used.
•This effectively prunes off this later branch of the
classification tree.
    Exponential Time Redoing Work
"Which is the best path from v5 to t?"
How many friends solve this subinstance?
    Exponential Time Redoing Work
"Which is the best path from v5 to t?"
How many friends solve this subinstance?
    Exponential Time Redoing Work
"Which is the best path from v5 to t?"
How many friends solve this subinstance?
    Exponential Time Redoing Work
"Which is the best path from v5 to t?"
How many friends solve this subinstance?
    Exponential Time Redoing Work
"Which is the best path from v5 to t?"
How many friends solve this subinstance?
    Exponential Time Redoing Work
"Which is the best path from v5 to t?"
How many friends solve this subinstance?
          Exponential Time Redoing Work
      "Which is the best path from v5 to t?"
     How many friends solve this subinstance?

Once for each
path to v5

Waist time
                                     Having many friends
redoing work                         solving this same
                                     subinstance
Save time by only                    is a waist of time.

doing once.
   Speeding Up the Time


We allocate one friend to the job.


             It is my job to find
     the optimal solution to my subinstance
       i.e. the best path from v5 to t
     Speeding Up the Time
When I need to find the best path from
v1 to t I will ask you for the best path
from v5 to t
       Speeding Up the Time


When I need to find the best path from
v1 to t I will ask you for the best path
from v5 to t
Speeding Up the Time
When I need to find the best path from
v1 to t I will ask you for the best path
from v5 to t
 Speeding Up the Time

When I need to find the best path from
v1 to t I will ask you for the best path
from v5 to t
             Speeding Up the Time
When I need to find the best path from
v1 to t I will ask you for the best path
from v5 to t


                   I will find my best path
                    from v5 to t and tell you.



                      But I hate to wait for you.

                       Why don’t you go first?
                   Speeding Up the Time



                             Before anyone asks me,
                             I will find my best path
                                and remember it.




Memorization:
  Remember the solutions for the subInstances so that if
  ever it needs to be solved again, the answer can be used.
   Set of SubInstances
But what subinstance need to be solved
and in which order?
Given an instance I,


           Determine the complete
           set of subI ever given to
           you, your friends, their
           friends, …
   Set of SubInstances
Guess the complete set S of subI.
          “Best path from v7 to t?”     Yes
          “Best path from v21 to t?”    No
                   v21 is not a part of our
            v21
                     original instance.
    Set of SubInstances
Guess the complete set S of subI.
          “Best path from v7 to t?”     Yes
          “Best path from v21 to t?”    No
          “Best path from v3 to v5?”    No
                 All paths considered
                       end in t.
    Set of SubInstances
Guess the complete set S of subI.
          “Best path from v7 to t?”     Yes
          “Best path from v21 to t?”    No
          “Best path from v3 to v5?”    No
                 All paths considered
                       end in t.
    Set of SubInstances
Guess the complete set S of subI.
          “Best path from v7 to t?”                Yes
          “Best path from v21 to t?”               No
          “Best path from v3 to v5?”               No


            “For all i, Best path from vi to t?”   Yes
    Set of SubInstances
Guess the complete set S of subI is
      “For all i, Best path from vi to t?”

                 Assign one friend
                   to each subI.
    Set of SubInstances
Guess the complete set S of subI is
  “For all i, Best path from vi to t?”
         The set S of subI needs to:
           •include our given I
     Order to complete
The complete set S of subI is
      “For all i , Best path from vi to t?”

           In what order should they go?
             •in an order such that
             no friend must wait.
             •from “smallest” to “largest”

                  For this problem,
                 the order relies on
             the graph being “leveled.”
          Order to complete
Note that the graph is levelled:
      Indices of the vertices in the path from s to t
      must be in increasing order (eg. s,v3,v5,v6, t)




                Examples:
                                s, v3, v5, v6, t
                                s, v1, v4, v7, t
                                s, v2, v4, v7, t
     Order to complete
The complete set S of subI is
      “For all i , Best path from vi to t?”

           In what order should they go?
             •in an order such that
             no friend must wait.
             •from “smallest” to “largest”
              First    Base Case Easy

              Last     Instance to be solved.
          Order to complete
"Which is the best path from v8 to t?"

                               Easy
          Order to complete
"Which is the best path from v7 to t?"

                               Easy
          Order to complete
"Which is the best path from v6 to t?"

                               Easy
          Order to complete
"Which is the best path from v5 to t?"

                               Harder
          Order to complete
"Which is the best path from v5 to t?"
                              Little bird
                              suggests first
                              edge <v5,v7>

                              Friend gives best
                              path <v7,t>.
          Order to complete
"Which is the best path from v5 to t?"
                              Little bird
                              suggests first
                              edge <v5,v8>

                              Friend gives best
                              path <v8,t>.
          Order to complete
"Which is the best path from v5 to t?"
                              Take best of best
          Order to complete
"Which is the best path from v4 to t?"
          Order to complete
"Which is the best path from v4 to t?"
                              Little bird
                              suggests first
                              edge <v4,v6>

                              Friend gives best
                              path <v7,t>.
          Order to complete
"Which is the best path from v4 to t?"
                              Little bird
                              suggests first
                              edge <v4,t>

                              Friend gives best
                              path <t,t>.
          Order to complete
"Which is the best path from v4 to t?"
                              Little bird
                              suggests first
                              edge <v4,v7>

                              Friend gives best
                              path <v7,t>.
          Order to complete
"Which is the best path from v4 to t?"
                              Take best of best
          Order to complete
"Which is the best path from v3 to t?"
          Order to complete
"Which is the best path from v3 to t?"
                              Little bird
                              suggests first
                              edge <v3,v5>

                              Friend gives best
                              path <v5,t>.
          Order to complete
"Which is the best path from v3 to t?"
                              Little bird
                              suggests first
                              edge <v3,v8>

                              Friend gives best
                              path <v8,t>.
          Order to complete
"Which is the best path from v3 to t?"
                              Take best of best
                                Order to complete
                                vk
                                                                   t
                 vi                              Let,
                                                   C[k] : cost of the best path from vk to t.
                                                   w(i, k): weight of the edge (Vi, Vk)




If we know C[k], can compute C[i]:
                                       C[i] = w(i, k) + C[k]


k and all the vertex-indices in the path from Vk to t are greater than i.


We can compute C[i] in decreasing order of i:           C[7], C[6], C[5], …… C[1], C[s]
             vk
                                       t
vi

     vi may have more than one edge (vi, vk)

                  vk


                                       t
vi
                                      vk


                                                                    t
                   vi




               0        if i = t
C[i] =
             min ( w(i,k) + C[k])             if i ≠ t
         for all k, (Vi, Vk) is an
                   edge


                     Store the resuts in two arrays C and P:
                              C[i] – cost of the best path from Vi to t
                              P[i] – the first vertex in the best path from Vi to t.
"Which is the best path from t to t?"

                                Base Case
                                 Easy




C                                               0

    0(s)   1   2   3    4   5     6     7   8       9(t)
P                                               t
"Which is the best path from v8 to t?"

                               Easy




C                                            5   0

    0(s)   1   2   3   4   5    6        7   8       9(t)
P                                            t   t
    "Which is the best path from v7 to t?"

                                   Easy




C                                         4       5   0

     0(s)   1   2    3    4    5     6        7   8       9(t)
P                                         t       t   t
"Which is the best path from v6 to t?"

                               Easy




C                               7     4       5   0

    0(s)   1   2   3   4   5    6         7   8       9(t)
P                               t     t       t   t
    "Which is the best path from v5 to t?"

                                   Harder




C                                  7    4       5   0

     0(s)   1   2    3    4    5    6       7   8       9(t)
P                                   t    t      t   t
"Which is the best path from v5 to t?"
                                Little bird
                                suggests first
                                edge <v5,v7>

                                Friend gives best
                                path <v7,t>.

                            cost = w(5,7) + C[7] = 5 + 4


C                                 7    4       5    0

    0(s)   1   2   3   4    5     6        7   8        9(t)
P                                 t      t     t    t
           "Which is the best path from v5 to t?"
                                Little bird
                                suggests first
                                edge <v5,v8>

                                Friend gives best
                                path <v8,t>.

                                cost = w(5, 8) + C[8] = 1 + 5


C                                    7      4       5    0

    0(s)   1     2    3    4   5      6         7   8       9(t)
P                                     t     t       t   t
"Which is the best path from v5 to t?"
                                 Take best of best




C                            6      7    4       5   0

    0(s)   1   2   3    4    5      6        7   8       9(t)
P                           v8      t    t       t   t
    "Which is the best path from v4 to t?"




C                            6   7    4       5   0

    0(s)   1   2   3    4   5    6        7   8       9(t)
P                           v8   t    t       t   t
"Which is the best path from v4 to t?"
                                  Little bird
                                  suggests first
                                  edge <v4,v6>

                                  Friend gives best
                                  path <v6,t>.
                             cost = w(4, 6) + C[6] = 3 + 7


C                             6      7    4       5    0

    0(s)   1   2   3     4   5        6       7   8        9(t)
P                            v8       t   t        t   t
"Which is the best path from v4 to t?"

                            Little bird
                            suggests first
                            edge <v4,t>

                            Friend gives best
                            path <t,t>.
                            cost = w(4,t) + C[9] = 9 + 0


C                           6    7     4       5    0

    0(s)   1   2   3   4   5      6        7   8        9(t)
P                          v8     t     t      t    t
"Which is the best path from v4 to t?"

                                   Little bird
                                   suggests first
                                   edge <v4,v7>

                                   Friend gives best
                                   path <v7,t>.
                             cost = w(4, 7) + C[7] = 2 + 4


C                             6   7      4       5    0

    0(s)   1   2   3     4   5     6         7   8       9(t)
P                            v8    t     t       t   t
"Which is the best path from v4 to t?"
                                 Take best of best




C                       6    6     7     4       5   0

    0(s)   1   2   3    4   5      6         7   8       9(t)
P                      v7   v8     t     t       t   t
"Which is the best path from v3 to t?"




C                       6    6   7   4       5   0

    0(s)   1   2   3    4   5    6       7   8       9(t)
P                      v7   v8   t   t       t   t
"Which is the best path from v3 to t?"
                                 Little bird
                                 suggests first
                                 edge <v3,v5>

                                 Friend gives best
                                 path <v5,t>.
                             cost = w(3, 5) + C[5] = 2 + 6



C                       6    6      7    4       5    0

    0(s)   1   2   3    4    5      6        7   8       9(t)
P                      v7   v8      t    t       t   t
"Which is the best path from v3 to t?"
                                 Little bird
                                 suggests first
                                 edge <v3,v8>

                                 Friend gives best
                                 path <v8,t>.
                             cost = w(3, 8) + C[8] = 7 + 5



C                       6    6      7    4       5    0

    0(s)   1   2   3    4    5      6        7   8       9(t)
P                      v7   v8      t    t       t   t
"Which is the best path from v3 to t?"
                                  Take best of best




 C                   8    6    6     7    4       5   0

     0(s)   1   2    3    4   5      6        7   8       9(t)
 P                  v5   v7   v8     t    t       t   t
"Which is the best path from v2 to t?"




C                   8    6    6   7   4       5   0

    0(s)   1   2    3    4   5    6       7   8       9(t)
P                  v5   v7   v8   t   t       t   t
"Which is the best path from v2 to t?"
                                  Little bird
                                  suggests first
                                  edge <v2,v4>

                                  Friend gives best
                                  path <v4,t>.
                             cost = w(2, 4) + C[4] = 6 + 6



C                   8    6    6      7    4       5    0

    0(s)   1   2    3    4   5        6       7   8        9(t)
P                  v5   v7   v8       t   t        t   t
"Which is the best path from v2 to t?"
                                  Little bird
                                  suggests first
                                  edge <v2,v7>

                                  Friend gives best
                                  path <v7,t>.

                             cost = w(2, 7) + C[7] = 7 + 4


C                   8    6    6    7     4       5    0

    0(s)   1   2    3    4   5      6        7   8       9(t)
P                  v5   v7   v8     t    t       t   t
"Which is the best path from v2 to t?"
                                   Take best of best




C              11    8    6    6     7    4       5    0

    0(s)   1   2     3    4   5       6       7   8       9(t)
P              v7   v5   v7   v8      t   t       t   t
    "Which is the best path from v1 to t?"




C              11    8    6    6   7   4       5   0

    0(s)   1   2     3    4   5    6       7   8       9(t)
P              v7   v5   v7   v8   t    t      t   t
"Which is the best path from v1 to t?"
                                   Little bird
                                   suggests first
                                   edge <v1,v6>

                                   Friend gives best
                                   path <v6,t>.

                              cost = w(1, 6) + C[6] = 3 + 7


C              11    8    6    6    7     4       5    0

    0(s)   1   2     3    4   5     6         7   8       9(t)
P              v7   v5   v7   v8    t     t       t   t
"Which is the best path from v1 to t?"
                                   Little bird
                                   suggests first
                                   edge <v1,v4>

                                   Friend gives best
                                   path <v4,t>.

                              cost = w(1, 4) + C[4] = 5 + 6


C              11    8    6    6    7     4       5    0

    0(s)   1   2     3    4   5     6         7   8       9(t)
P              v7   v5   v7   v8    t     t       t   t
"Which is the best path from v1 to t?"
                                   Little bird
                                   suggests first
                                   edge <v1,v5>

                                   Friend gives best
                                   path <v5,t>.

                               cost = w(1,5) + C[5] = 4 + 6


C              11    8    6    6    7     4       5    0

    0(s)   1   2     3    4   5      6        7   8        9(t)
P              v7   v5   v7   v8     t     t      t    t
"Which is the best path from v1 to t?"
                                    Take best of best




C          10   11    8    6    6     7   4       5   0

    0(s)   1    2     3    4   5      6       7   8       9(t)
P          v6   v7   v5   v7   v8     t    t      t   t
"Which is the best path from s to t?"
                              Original Problem




C          10   11    8    6    6   7   4       5   0

    0(s)   1    2     3    4   5    6       7   8       9(t)
P          v6   v7   v5   v7   v8   t   t       t   t
"Which is the best path from s to t?"
                                    Little bird
                                    suggests first
                                    edge <s,v1>

                                    Friend gives best
                                    path <v1,t>.

                               cost = w(s, 1) + C[1] = 3 + 10


C          10   11    8    6    6     7    4       5    0

    0(s)   1    2     3    4   5       6       7   8        9(t)
P          v6   v7   v5   v7   v8      t    t      t    t
"Which is the best path from s to t?"
                                    Little bird
                                    suggests first
                                    edge <s,v2>

                                    Friend gives best
                                    path <v2,t>.

                                cost = w(s, 2) + C[2] = 2 + 11


 C          10   11    8    6    6     7    4       5    0

     0(s)   1    2     3    4   5       6       7    8       9(t)
 P          v6   v7   v5   v7   v8      t    t       t   t
"Which is the best path from s to t?"
                                    Little bird
                                    suggests first
                                    edge <s,v3>

                                    Friend gives best
                                    path <v3,t>.

                                cost = w(s, 3) + C[3] = 4 + 8


 C          10   11    8    6    6     7    4       5    0

     0(s)   1    2     3    4   5       6       7    8       9(t)
 P          v6   v7   v5   v7   v8      t    t       t   t
"Which is the best path from s to t?"
                                    Take best of best


                                          DONE




C    12    10   11    8    6    6     7   4       5   0

    0(s)   1    2     3    4   5      6       7   8       9(t)
P    v3    v6   v7   v5   v7   v8     t    t      t   t
    "Which is the best path from s to t?"


Constructing the
  optimal path




   C      12       10   11    8    6    6   7   4       5   0

        0(s)       1    2     3    4   5    6       7   8       9(t)
   P      v3       v6   v7   v5   v7   v8   t   t       t   t
    "Which is the best path from s to t?"


Constructing the
  optimal path




   C      12       10   11    8    6    6   7   4       5   0

        0(s)       1    2     3    4   5    6       7   8       9(t)
   P      v3       v6   v7   v5   v7   v8   t   t       t   t
    "Which is the best path from s to t?"


Constructing the
  optimal path




   C      12       10   11    8    6    6   7   4       5   0

        0(s)       1    2     3    4   5    6       7   8       9(t)
   P      v3       v6   v7   v5   v7   v8   t   t       t   t
    "Which is the best path from s to t?"


Constructing the
  optimal path




                                            DONE



   C      12       10   11    8    6    6   7   4       5   0

        0(s)       1    2     3    4   5    6       7   8       9(t)
   P      v3       v6   v7   v5   v7   v8   t   t       t   t
               Dynamic Programming
Step 1: Characterize the structure of an optimal solution.
             For a path from s to t ( s … vi …t) to be optimal, the subpath vi to t must be optimal.



Step 2: Recursively define the value of an optimal solution.
                        0     if i = t
             C[i] =
                             min ( w(i,k) + C[k])                if i ≠ t
                        for all k, (Vi, Vk) is an
                                  edge

Step 3. Compute the value of an optimal solution in a bottom-up fashion
                compute C[i] (and P[i]) in decreasing order of i

Step 4: Construct an optimal solution.
                 C[0] – cost of the best path. Construct the best path from Array P
                  Running Time
For each vertex Vi,
               can compute C[i]:
                          C[i] = w(i, k) + C[k]
                                             in O(1) time.

                       vk
                                                t
          vi
                                                  vi may have more than one edge (vi, vk)
Running Time
                                                           vk


                                                                                    t
                                       vi



                 0        if i = t
  C[i] =
               min ( w(i,k) + C[k])             if i ≠ t
           for all k, (Vi, Vk) is an
                     edge


Let d is degree of Vi        i.e. there are d number of ks

                                            C[i] can be computed in O(d) time
                    Running Time

For each i, compute C[i] takes O(d) time

                      where d is the maximum degree of the graph


We have to compute C[i] for all the vertices.


  Suppose there are n vertices



                    Algorithm runs in O(n × d) time

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:0
posted:12/3/2011
language:English
pages:110