heuristic-search

Document Sample
heuristic-search Powered By Docstoc
					        Heuristic Search

Russell and Norvig: Chapter 4


CSMSC 421 – Fall 2005
Modified Search Algorithm
1. INSERT(initial-node,FRINGE)
2. Repeat:
       If FRINGE is empty then return failure
       n  REMOVE(FRINGE)
       s  STATE(n)
       If GOAL?(s) then return path or goal state
       For every state s’ in SUCCESSORS(s)
       Create a node n’
       INSERT(n’,FRINGE)
Search Algorithms
 Blind search – BFS, DFS, uniform cost
    no notion concept of the “right direction”
    can only recognize goal once it’s achieved



 Heuristic search – we have rough idea of how
 good various states are, and use this
 knowledge to guide our search
Best-first search
 Idea: use an evaluation function f(n) for
 each node
    Estimate of desirability
 Expand most desirable unexpanded
 node
 Implementation: fringe is queue sorted
 by descreasing order of desirability
    Greedy search
    A* search
Heuristic
Webster's Revised Unabridged Dictionary (1913) (web1913)
   Heuristic \Heu*ris"tic\, a. [Greek. to discover.] Serving to discover
     or find out.
The Free On-line Dictionary of Computing (15Feb98)
   heuristic 1. <programming> A rule of thumb, simplification or
     educated guess that reduces or limits the search for solutions
     in domains that are difficult and poorly understood. Unlike
     algorithms, heuristics do not guarantee feasible solutions and
     are often used with no theoretical guarantee. 2. <algorithm>
     approximation algorithm.
From WordNet (r) 1.6
   heuristic adj 1: (computer science) relating to or using a heuristic
     rule 2: of or relating to a general formulation that serves to
     guide investigation [ant: algorithmic] n : a commonsense rule
     (or set of rules) intended to increase the probability of solving
     some problem [syn: heuristic rule, heuristic program]
Informed Search
 Add domain-specific information to select the
 best path along which to continue searching
 Define a heuristic function, h(n), that
 estimates the “goodness” of a node n.
 Specifically, h(n) = estimated cost (or
 distance) of minimal cost path from n to a
 goal state.
 The heuristic function is an estimate, based
 on domain-specific information that is
 computable from the current state
 description, of how close we are to a goal
Greedy Search


 f(N) = h(N)  greedy best-first
Robot Navigation
Robot Navigation

f(N) = h(N), with h(N) = Manhattan distance to the goal


   8    7   6    5   4   3    2   3    4   5   6
   7        5    4   3                         5

   6             3   2   1    0   1    2       4

   7    6                                      5

   8    7   6    5   4   3    2   3    4   5   6
Robot Navigation

f(N) = h(N), with h(N) = Manhattan distance to the goal


   8    7   6    5   4   3     2   3   4   5   6
   7        5    4   3                         5

   6             3   2   1     0
                              0 1 2         4
                             What happened???
   7
   7    6                                   5

   8    7   6    5   4   3     2   3   4   5   6
Greedy Search


  f(N) = h(N)  greedy best-first
 Is it complete?
 If we eliminate endless loops, yes

 Is it optimal?
More informed search
 We kept looking at nodes closer and closer to
 the goal, but were accumulating costs as we
 got further from the initial state
 Our goal is not to minimize the distance from
 the current head of our path to the goal, we
 want to minimize the overall length of the
 path to the goal!
 Let g(N) be the cost of the best
  path found so far between the initial
  node and N

 f(N) = g(N) + h(N)
Robot Navigation

f(N) = g(N)+h(N), with h(N) = Manhattan distance to goal


   8 7 6+3 5 4 3 2 3
           6
  8+3 7+4 6+5 5+6 4+7 3+8 2+9 3+10 4      5   6
   7
  7+2        5 4 3
            5+6 4+7 3+8                       5

   6
  6+1              2 1 0
                3 2+9 1+10 0+11 1     2       4

   7 6
  7+0 6+1                                     5

   8 7 6 5 4 3 2 3
  8+1 7+2 6+3 5+4 4+5 3+6 2+7 3+8 4       5   6
Can we Prove Anything?
 If the state space is finite and we avoid
 repeated states, the search is complete,
 but in general is not optimal
 If the state space is finite and we do
 not avoid repeated states, the search is
 in general not complete
 If the state space is infinite, the search
 is in general not complete
Admissible heuristic
  Let h*(N) be the true cost of the
 optimal path from N to a goal node
 Heuristic h(N) is admissible if:
         0  h(N)  h*(N)

  An admissible heuristic is always
 optimistic
A* Search
 Evaluation function:
          f(N) = g(N) + h(N)
 where:
    g(N) is the cost of the best path found so far to N
    h(N) is an admissible heuristic

 Then, best-first search with this evaluation function is
 called A* search

 Important AI algorithm developed by Fikes and
 Nilsson in early 70s. Originally used in Shakey robot.
Completeness & Optimality of A*
  Claim 1: If there is a path from the
 initial to a goal node, A* (with no
 removal of repeated states) terminates
 by finding the best path, hence is:
   complete

   optimal

 requirements:
    0 <   c(N,N’)
    Each node has a finite number of successors
Completeness of A*
  Theorem: If there is a finite path from
 the initial state to a goal node, A* will
 find it.
Proof of Completeness
 Let g be the cost of a best path to a
 goal node
 No path in search tree can get longer
 than g/, before the goal node is
 expanded
Optimality of A*
  Theorem: If h(n) is admissable, then
 A* is optimal.
Proof of Optimality

                                    Cost of best path
               f(G1) = g(G1)        to a goal thru N



           N
               f(N) = g(N) + h(N)  g(N) + h*(N)

G1    G2
               f(G1)  g(N) + h(N)  g(N) + h*(N)
Heuristic Function
  Function h(N) that estimates the cost
 of the cheapest path from node N to
 goal node.
  Example: 8-puzzle


 5   8   1 2 3 h(N) = number of misplaced tiles
 4 2 1   4 5 6      =6
 7 3 6   7 8
   N      goal
Heuristic Function
  Function h(N) that estimate the cost of
 the cheapest path from node N to goal
 node.
  Example: 8-puzzle


 5   8   1 2 3 h(N) = sum of the distances of
 4 2 1   4 5 6        every tile to its goal position
 7 3 6   7 8        =2+3+0+1+3+0+3+1
   N      goal      = 13
8-Puzzle
               f(N) = h(N) = number of misplaced tiles


                             3        3        4
           5         3

                             4
                                               2
   4                         2        1
           3         3
                                               0
                             4

           5         4
8-Puzzle     f(N) = g(N) + h(N)
             with h(N) = number of misplaced tiles


                           3+3
           1+5     2+3

                           3+4
                                             5+2
   0+4                     3+2      4+1
           1+3     2+3
                                             5+0
                           3+4

           1+5     2+4
8-Puzzle
               f(N) = h(N) =  distances of tiles to goal




           6         5



                                                 2
   5                          2        1
           4         3
                                                 0
                              4

           6         5
8-Puzzle

         5         8               1    2     3

         4    2    1               4    5     6

         7   3     6               7    8
             N                         goal
• h1(N) = number of misplaced tiles = 6 is admissible
• h2(N) = sum of distances of each tile to goal = 13
        is admissible
• h3(N) = (sum of distances of each tile to goal)
           + 3 x (sum of score functions for each tile) = 49
        is not admissible
Robot navigation

f(N) = g(N) + h(N), with h(N) = straight-line distance from N to goal




      Cost of one horizontal/vertical step = 1
      Cost of one diagonal step = 2
Consistent Heuristic
  The admissible heuristic h is consistent
 (or satisfies the monotone restriction) if
 for every node N and every successor
 N’ of N:
                                       N
 h(N)  c(N,N’) + h(N’)         c(N,N’)

                                  N’       h(N)
 (triangular inequality)           h(N’)
8-Puzzle

     5        8               1    2     3
     4    2   1               4    5     6

     7    3   6               7    8
         N                        goal

 • h1(N) = number of misplaced tiles
 • h2(N) = sum of distances of each tile to goal
 are both consistent
Robot navigation




Cost of one horizontal/vertical step = 1
Cost of one diagonal step = 2

h(N) = straight-line distance to the goal is consistent
Claims
 If h is consistent, then the function f along
 any path is non-decreasing:
                                          N
                                   c(N,N’)
  f(N) = g(N) + h(N)
  f(N’) = g(N) +c(N,N’) + h(N’)      N’       h(N)

                                      h(N’)
Claims
 If h is consistent, then the function f along
 any path is non-decreasing:
                                          N
                                   c(N,N’)
  f(N) = g(N) + h(N)
  f(N’) = g(N) +c(N,N’) + h(N’)      N’       h(N)
  h(N)  c(N,N’) + h(N’)              h(N’)
  f(N)  f(N’)
Claims

 If h is consistent, then the function f along
 any path is non-decreasing:
                                         N
                                  c(N,N’)
  f(N) = g(N) + h(N)
  f(N’) = g(N) +c(N,N’) + h(N’)     N’       h(N)
  h(N)  c(N,N’) + h(N’)
                                     h(N’)
  f(N)  f(N’)

 If h is consistent, then whenever A* expands
 a node it has already found an optimal path
 to the state associated with this node
Avoiding Repeated States in A*
If the heuristic h is consistent, then:
  Let CLOSED be the list of states
  associated with expanded nodes
  When a new node N is generated:
     If its state is in CLOSED, then discard N
     If it has the same state as another node in
      the fringe, then discard the node with the
      largest f
Heuristic Accuracy

  h(N) = 0 for all nodes is admissible and
  consistent. Hence, breadth-first and uniform-
  cost are particular A* !!!
  Let h1 and h2 be two admissible and consistent
  heuristics such that for all nodes N: h1(N) 
  h2(N).
  Then, every node expanded by A* using h2 is
  also expanded by A* using h1.
  h2 is more informed than h1
Iterative Deepening A* (IDA*)
 Use f(N) = g(N) + h(N) with admissible
 and consistent h
 Each iteration is depth-first with cutoff
 on the value of f of expanded nodes
8-Puzzle       f(N) = g(N) + h(N)
               with h(N) = number of misplaced tiles




     4
Cutoff=4


           6
8-Puzzle       f(N) = g(N) + h(N)
               with h(N) = number of misplaced tiles




     4
Cutoff=4   4



           6         6
8-Puzzle       f(N) = g(N) + h(N)
               with h(N) = number of misplaced tiles




     4
Cutoff=4   4         5



           6         6
8-Puzzle       f(N) = g(N) + h(N)
               with h(N) = number of misplaced tiles



                     5




     4
Cutoff=4   4         5



           6         6
8-Puzzle       f(N) = g(N) + h(N)
               with h(N) = number of misplaced tiles



           6         5




     4
Cutoff=4   4         5



           6         6
8-Puzzle       f(N) = g(N) + h(N)
               with h(N) = number of misplaced tiles




     4
Cutoff=5


           6
8-Puzzle       f(N) = g(N) + h(N)
               with h(N) = number of misplaced tiles




     4
Cutoff=5   4



           6         6
8-Puzzle       f(N) = g(N) + h(N)
               with h(N) = number of misplaced tiles




     4
Cutoff=5   4         5



           6         6
8-Puzzle       f(N) = g(N) + h(N)
               with h(N) = number of misplaced tiles




     4
Cutoff=5   4         5

                             7

           6         6
8-Puzzle       f(N) = g(N) + h(N)
               with h(N) = number of misplaced tiles




     4                       5
Cutoff=5   4         5

                             7

           6         6
8-Puzzle       f(N) = g(N) + h(N)
               with h(N) = number of misplaced tiles




     4                       5        5
Cutoff=5   4         5

                             7

           6         6
8-Puzzle       f(N) = g(N) + h(N)
               with h(N) = number of misplaced tiles




     4                       5        5
Cutoff=5   4         5

                             7

           6         6
About Heuristics

   Heuristics are intended to orient the search along
   promising paths
   The time spent computing heuristics must be
   recovered by a better search
   After all, a heuristic function could consist of solving
   the problem; then it would perfectly guide the
   search
   Deciding which node to expand is sometimes called
   meta-reasoning
   Heuristics may not always look like numbers and
   may involve large amount of knowledge
What’s the Issue?
 Search is an iterative local procedure

  Good heuristics should provide some
 global look-ahead (at low computational
 cost)
Another approach…
 for optimization problems
    rather than constructing an optimal
     solution from scratch, start with a
     suboptimal solution and iteratively improve
     it

 Local Search Algorithms
    Hill-climbing or Gradient descent
    Potential Fields
    Simulated Annealing
    Genetic Algorithms, others…
Hill-climbing search
If there exists a successor s for the current state n
such that
    h(s) < h(n)
    h(s) <= h(t) for all the successors t of n,
then move from n to s. Otherwise, halt at n.
Looks one step ahead to determine if any successor
is better than the current state; if there is, move to
the best successor.
Similar to Greedy search in that it uses h, but does
not allow backtracking or jumping to an alternative
path since it doesn’t “remember” where it has been.
Not complete since the search will terminate at "local
minima," "plateaus," and "ridges."
Hill climbing on a surface of
states




                     Height Defined
                       by Evaluation
                       Function
Hill climbing
  Steepest descent (~ greedy best-first
 with no search)  may get stuck into
 local minimum
Robot Navigation


     Local-minimum problem




   f(N) = h(N) = straight distance to the goal
Examples of problems with HC




 applet
Drawbacks of hill climbing
 Problems:
   Local Maxima: peaks that aren’t the
    highest point in the space
   Plateaus: the space has a broad flat region
    that gives the search algorithm no direction
    (random walk)
   Ridges: flat like a plateau, but with
    dropoffs to the sides; steps to the North,
    East, South and West may go down, but a
    step to the NW may go up.
 Remedy:
     Introduce randomness
        Random restart.
 Some problem spaces are great for hill climbing
 and others are terrible.
What’s the Issue?
 Search is an iterative local procedure

  Good heuristics should provide some
 global look-ahead (at low computational
 cost)
 Hill climbing example
         2 8 3                               1 2 3
start    1 6 4   h = -4               goal   8   4 h=0
         7   5                               7 6 5

  -5              -5                 -2
         2 8 3                               1 2 3
         1   4 h = -3                          8 4 h = -1
         7 6 5                               7 6 5

 -3               -4
         2   3                                  2 3
         1 8 4                                1 8 4 h = -2
         7 6 5                                7 6 5
h = -3                 -4
             f(n) = -(number of tiles out of place)
Example of a local maximum

         1 2 5
                 -4
           7 4
start    8 6 3           goal
1 2 5     1 2 5         1 2 5
  7 4       7 4 -4        7 4 0
8 6 3     8 6 3         8 6 3
 -3
          1 2 5
            7 4 -4
          8 6 3
Potential Fields
 Idea: modify the heuristic function
 Goal is gravity well, drawing the robot toward
 it
 Obstacles have repelling fields, pushing the
 robot away from them
 This causes robot to “slide” around obstacles
 Potential field defined as sum of attractor
 field which get higher as you get closer to the
 goal and the indivual obstacle repelling field
 (often fixed radius that increases
 exponentially closer to the obstacle)
Does it always work?
 No.
 But, it often works very well in practice
 Advantage #1: can search a very large
 search space without maintaining fringe
 of possiblities
 Scales well to high dimensions, where
 no other methods work
 Example: motion planning
 Advantage #2: local method. Can be
 done online
Example: RoboSoccer
All robots have same field: attracted to the ball
   Repulsive potential to other players
   Kicking field: attractive potential to the ball
   and local repulsive potential if clase to the
   ball, but not facing the direction of the
   opponent’s goal. Result is tangent, player
   goes around the ball.
   Single team: kicking field + repulsive field to
   avoid hitting other players + player position
   fields (paraboilic if outside your area of the
   field, 0 inside). Player nearest to the ball has
   the largest attractive coefficient, avoids all
   players crowding the ball.
Simulated annealing
Simulated annealing (SA) exploits an analogy between
the way in which a metal cools and freezes into a
minimum-energy crystalline structure (the annealing
process) and the search for a minimum [or maximum] in
a more general system.
SA can avoid becoming trapped at local minima.
SA uses a random search that accepts changes that
increase objective function f, as well as some that
decrease it.
SA uses a control parameter T, which by analogy with the
original application is known as the system
“temperature.”
T starts out high and gradually decreases toward 0.
Simulated annealing (cont.)

 A “bad” move from A to B is accepted with
 a probability
              (f(B)-f(A)/T)
          e
 The higher the temperature, the more
 likely it is that a bad move can be made.
 As T tends to zero, this probability tends to
 zero, and SA becomes more like hill
 climbing
 If T is lowered slowly enough, SA is
 complete and admissible.
The simulated annealing
algorithm
Simulated Annealing
 applet

 Successful application: circuit routing,
 traveling sales person (TSP)
Summary: Local Search Algorithms
   Steepest descent (~ greedy best-first
  with no search)  may get stuck into
  local minimum
   Better Heuristics: Potential Fields
   Simulated annealing
   Genetic algorithms
When to Use Search Techniques?
 The search space is small, and
    There is no other available techniques, or
    It is not worth the effort to develop a more
     efficient technique
 The search space is large, and
    There is no other available techniques, and
    There exist “good” heuristics
Summary
 Heuristic function
 Best-first search
 Admissible heuristic and A*
 A* is complete and optimal
 Consistent heuristic and repeated
states
 Heuristic accuracy
 IDA*

				
DOCUMENT INFO