Heuristic search

					                                                                                                                                                                                                                                                1/4




                          Heuristic search                                                                                            Definitions                                                                   Definitions (2)
                                                                                      Heuristics (Greek heuriskein = find, discover): "the study                            For heuristic search to work, we must be able to rank
                        Points                                                        of the methods and rules of discovery and invention".                                 the children of a node. A heuristic function takes a
                        Definitions                                                                                                                                         state and returns a numeric value – a composite
                                                                                      We use our knowledge of the problem to consider some
                        Best-first search                                             (not all) successors of the current state (preferably just                            assessment of this state. We then choose a child with
                        Hill climbing                                                 one, as with an oracle). This means pruning the state                                 the best score (this could be a maximum or minimum).
                        Problems with hill climbing                                   space, gaining speed, but perhaps missing the solution!                               A heuristic function can help gain or lose a lot, but
                         An example: local and global                                 In chess: consider one (apparently best) move, maybe a few – but                      finding the right function is not always easy.
                         heuristic functions                                          not all possible legal moves.
                                                                                                                                                                            The 8-puzzle: how many misplaced tiles? how many
                        Simulated annealing                                           In the travelling-salesman problem: select one nearest city, give up                  slots away from the correct place? and so on.
                        The A* procedure                                              complete search (the greedy technique). This gives us, in polynomial
                                                                                      time, an approximate solution of the inherently exponential problem; it               Water jugs: ???
                        Means-ends analysis                                           can be proven that the approximation error is bounded.
                                                                                                                                                                            Chess: no simple counting of pieces is adequate.



CSI 4106, Winter 2012                             Heuristic search, page 1   CSI 4106, Winter 2012                                     Heuristic search, page 2   CSI 4106, Winter 2012                              Heuristic search, page 3




                                                  Definitions (3)                                           Best-first search                                                                   Hill climbing
          The gain (often spectacular) gain is the                                                             The algorithm                                                This is a greedy algorithm: go as high up as possible as
          reduction of the state space. For example, the                              select a heuristic function (e.g., distance to the goal);                             fast as possible, without looking around or keeping track.
          full tree for Tic-Tac-Toe has 9! = 362880 leaves.                           put the initial node(s) on the open list;                                                                    The algorithm
          If we consider symmetries, the tree becomes six                             repeat                                                                                select a heuristic function;
          times smaller, but it is still quite large.                                   select N, the best node on the open list;                                           let the current node C be the highest-valued initial node;
          With a fairly simple heuristic function we can get                            succeed if N is a goal node; otherwise put N on the
                                                                                        closed list and add N's children to the open list;                                  repeat
          the tree down to 40 states. (More on this when                                                                                                                     select N, the highest-valued child of C;
                                                                                      until
          we discuss games.)
                                                                                        we succeed or the open list becomes empty (we fail);                                 until
          Heuristics can also help speed up exhaustive,                               A closed node reached on a different path is reopened.                                 the goal has been reached or
          blind search, such as depth-first and breadth-                                                                                                                     no further improvement is possible;
                                                                                      NOTE: "the best" only means "currently appearing the best"...
          first search.


CSI 4106, Winter 2012                             Heuristic search, page 4   CSI 4106, Winter 2012                                     Heuristic search, page 5   CSI 4106, Winter 2012                              Heuristic search, page 6
                                                                                                                                                                                                                                                       1/4




                        Problems with hill climbing                                                        A hill climbing example                                                                       A hill climbing example (2)
           1.     Local maximum, or the foothill problem: there is a                                                                                                         A local heuristic function
                  peak, but it is lower than the highest peak in the                                                                                                         Count +1 for every block that sits on the correct
                  whole space.                                                                                                                                               thing. The goal state has the value +8.
           2.     The plateau problem: all local moves are equally                                                                                                           Count -1 for every block that sits on an incorrect
                  unpromising, and all peaks seem far away.                                                                                                                  thing. In the initial state blocks C, D, E, F, G, H
           3.     The ridge problem: almost every move takes us                                                                                                              count +1 each. Blocks A, B count -1 each , for the
                  down.                                                                                                                                                      total of +4.
                  Random-restart hill climbing is a series of hill-                                                                                                          Move 1 gives the value +6 (A is now on the correct
                  climbing searches with a randomly selected start                                                                                                           support). Moves 2a and 2b both give +4 (B and H
                  node whenever the current search gets stuck.                                                                                                               are wrongly situated). This means we have a local
                                                                                                                                                                             maximum of +6.
           See also simulated annealing – in a moment.



CSI 4106, Winter 2012                                  Heuristic search, page 7    CSI 4106, Winter 2012                             Heuristic search, page 8    CSI 4106, Winter 2012                                     Heuristic search, page 9




                                        A hill climbing example (3)                                                    A hill climbing example (4)                                         Simulated annealing
                                                                                                                                                                          anneal heat and then cool (as steel or glass)
                                                                                                                                                                          usually for softening and making less brittle;
                 A global heuristic function                                                      Move 1 gives the value -21 (A is now on the                             also cool slowly usually in a furnace
                 Count +N for every block that sits on a correct                                  correct support).
                 stack of N things. The goal state has the value                                                                                                                The intuition for this search method, an
                                                                                                  Move 2a gives -16, because C, D, E, F, G, H
                 +28.                                                                             count -1, -2, -3, -4, -5, -1.                                                 improvement on hill-climbing, is the process
                 Count -N for every block that sits on an                                         Move 2b gives -15, because C, D, E, F, G                                      of gradually cooling a liquid. A schedule of
                 incorrect stack of N things. That is, there is a                                 count -1, -2, -3, -4, -5.                                                     "temperatures" changes in time. Reaching
                 large penalty for blocks tied up in a wrong
                 structure.                                                                       There is no local maximum!                                                    the "freezing point" (temperature 0) stops
                                                                                                  Moral: sometimes another heuristic function is all
                                                                                                                                                                                the process.
                 In the initial state C, D, E, F, G, H count -1, -2,
                 -3, -4, -5, -6. A counts -7 , for the total of -28.                              we need.                                                                      We replace random restart with a more
                                                                                                                                                                                systematic method.
                                                                continued


CSI 4106, Winter 2012                                  Heuristic search, page 10   CSI 4106, Winter 2012                             Heuristic search, page 11   CSI 4106, Winter 2012                                     Heuristic search, page 12
                                                                                                                                                                                                                                                                                           1/4




                                            Simulated annealing (2)                                                                                                                                    The A* procedure
                                                                                                              A few other search ideas
                        The algorithm, one of the versions
                                                                                                                                                                                     Hill-climbing (and its clever versions) may miss an optimal solution.
           select a heuristic function f;                                                                                                                                            Here is a search method which ensures optimality of the solution.
                                                                                                                                                                                                                    The algorithm
           let the current node C be any initial node;                                               Stochastic beam search:
                                                                                                                                                                                     keep a list of partial paths (initially root to root, length 0);
           let the current time t be 0;                                                              select w nodes at random; nodes with higher                                     repeat
           let schedule(x) be a table of "temperatures”;                                             values have a higher probability of selection.                                    if the first path P reaches the goal node, return success;
           loop                                                                                      Genetic algorithms:                                                               otherwise remove path P from the list;
                                                                                                                                                                                       extend P in all possible ways, add new paths to the list;
             t = t + 1;                                                                              generate nodes like in stochastic beam search,
                                                                                                                                                                                       sort the list by the sum of two values: the real cost of P till now,
                                                                                                     but from two parents rather than from one.                                        and an estimate of the remaining distance;
             if schedule(t) = 0, return C;
             select at random N, any child of C;                                                     (This topic is worthy of a separate lecture...)                                   prune the list by leaving only the shortest path for each node
                                                                                                                                                                                       reached so far;
             if f(N) > f(C), set C to N,                                                                                                                                             until
             otherwise                                                                                                                                                                 success or the list of paths becomes empty;
                   set C to N with probability e(f(N)-f(C))/schedule(t);


CSI 4106, Winter 2012                                     Heuristic search, page 13   CSI 4106, Winter 2012                                    Heuristic search, page 14   CSI 4106, Winter 2012                                                               Heuristic search, page 15




                                                The A* procedure (2)                                                               The A* procedure (3)                                                                                          The A* procedure (4)
                                                                                                                                                                                       A simple search problem: S is the start node, G is
                                                                                                Here is a sketchy proof of the optimality of the goal node                             the goal node, the real distances are shown.
           A* requires a lower-bound estimate of the                                            found by A*. Assume that m is a non-optimal goal:                                                                           4                4
                                                                                                                                                                                                                   A                 B               C
           distance of any node to the goal node. A                                                                                                                                                        3
           heuristic which never overestimates is also                                                                   g(m) > g(nk)
                                                                                                                                                                                                       S               5             5
                                                                                                                                                                                                                                                         G
           called optimistic or admissible.                                                     Let n0, n1, ..., nk be a path leading to nk.                                                                   4                                 3.5
                                                                                                                                                                                                                           2             4
                                                                                                                                                                                                                   D             E               F
           We consider three functions with values ≥ 0:                                         Since m is a goal, h(m) = 0 and thus h'(m) = 0.
                                                                                                Every step of A* selects the next ni in such a way that                                A lower-bound estimate of the distance to G could
           • g(n) is the actual cost of reaching node n,                                                                                                                               be as follows (note that we do not need it for F):
                                                                                                g(ni) + h'(ni) ≤ g(ni) + h(ni) = g(nk) < g(m) = g(m) + h'(m)
           • h(n) is the actual unknown remaining cost,                                                                                                                                                            A                 B 5.8           C
                                                                                                In other words, g(ni) + h'(ni) < g(m) + h'(m), so that A*                                                          10.1                                  3.4
                                                                                                                                                                                                       S
           • h'(n) is the optimistic estimate of h(n).                                          cannot have selected m: the value of g + h' is too high.                                                   11.5                                          G
                                                                                                                                                                                                                           9.2
                                                                                                                                                                                                                   D                                   3.5
                                                                                                                                                                                                                                 E   7.1         F




CSI 4106, Winter 2012                                     Heuristic search, page 16   CSI 4106, Winter 2012                                    Heuristic search, page 17   CSI 4106, Winter 2012                                                               Heuristic search, page 18
                                                                                                                                                                                                                                                                                                 1/4




          Hill-climbing                                 The A* procedure (5)                                                                          The A* procedure (6)                                                                             The A* procedure (7)
                                                                                                                                                S                                                                                              S
          happens to
         succeed here:                       S
                                                                                                                                    3+10.1          4+9.2                                                                            13.1            13.2
                                                                                                        A*, step 1                                                                                     A*, step 2
                                    10.1            9.2                                                                                 A            D                                                                                 A              D
                                                                                                                                                                                                                         3+4+5.8           3+5+9.2
                                     A              D

                                            10.1      11.5   7.1                                                                                                                                                               B       D

                                                A   S        E

                                                     5.8      9.2     3.5

                                                      B      D       F
                                                             7.1            0.0

                                                             E               G




CSI 4106, Winter 2012                                               Heuristic search, page 19   CSI 4106, Winter 2012                                              Heuristic search, page 20   CSI 4106, Winter 2012                                                 Heuristic search, page 21




                                                S
                                                        The A* procedure (8)                                                                    S
                                                                                                                                                      The A* procedure (9)                                                                     S
                                                                                                                                                                                                                                                       The A* procedure (10)
                                     13.1           13.2                                                                              13.1          13.2                                                                             13.1            13.2
        A*, step 3                                                                                      A*, step 4                                                                                     A*, step 5
                                      A               D                                                                                 A            D                                                                                 A              D
                             12.8        17.2                                                                                  12.8      17.2                4+5+10.1                                                         12.8      17.2                  19.1

                              B       D                                                                                         B       D                     A                                                                B       D                       A
                 3+4+4+3.4        3+4+5+7.1                                                                             14.4     19.1                    4+2+7.1                                                       14.4     19.1                   13.1

                        C     E                                                                                         C       E                    E                                                                 C       E                      E
                                                                                                                                                                                                                                       4+2+5+5.8              4+2+4+3.5

                                                                                                                                                                                                                                               B               F




CSI 4106, Winter 2012                                               Heuristic search, page 22   CSI 4106, Winter 2012                                              Heuristic search, page 23   CSI 4106, Winter 2012                                                 Heuristic search, page 24
                                                                                                                                                                                                                                                 1/4




                                                  S
                                                            The A* procedure (11)                                                 The A* procedure (12)                                     Means-ends analysis
                                      13.1                13.2
        A*, step 6                                                                                                                                                            The idea is due to Newell and Simon (1957): reduce the
                                          A                D                                                                                                                  difference between states, and so approach the goal
                               12.8       17.2                        19.1                                                             The 8-puzzle with                      state. There are procedures, indexed by differences,
                                                                                                                                       optimistic heuristics.                 which change states.
                                B         D                               A                                                                                                              The General Problem Solver algorithm
                                                                                                                                       A, ..., N: states;
                        14.4     19.1                          13.1                                                                    m = misplaced;                         let the current node C be any initial node;
                                                                                                                                       d = depth;
                        C       E                          E                                                                           f(x) = m(x) + d(x).
                                                                                                                                                                              loop
                                                  16.8            13.5                                                                                                          succeed if the goal state has been reached, otherwise
                                                                                                                                                                                find the difference between C and the goal state;
                                                   B                   F                                                                                                        choose a procedure that reduces this difference, apply
                                                                                                                                                                                it to C to produce the new current state;

                                                                       G



CSI 4106, Winter 2012                                                         Heuristic search, page 25   CSI 4106, Winter 2012         Heuristic search, page 26   CSI 4106, Winter 2012                            Heuristic search, page 27




                                                  Means-ends analysis (2)
                     An incomplete toy example: planning a trip.
                                      P       r    o       c      e           d    u     r   e

          D                      Plane            Train          Car              Taxi       Walk
                 miles
          i
          s     > 1000                X
          t
               100-1000                               X               X
          a
          n     10-100                                                X             X
          c
          e        <1                                                                         X

               Prerequisite     At plane At train                At car           At taxi




CSI 4106, Winter 2012                                                         Heuristic search, page 28

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:2
posted:2/29/2012
language:German
pages:5