pptx by liuqingyan


									CISC453 Winter 2010

Classical Planning
AIMA3e Ch 10
What is Planning?
 generate sequences of actions to perform tasks and
  achieve objectives
   involving states, actions & goals
 a search for a solution over an abstract space of plans
 the assumptions for classical planning problems
   fully observable
      we see everything that matters
   deterministic
      the effects of actions are known exactly
   static
      no changes to environment other than those caused by agent
   discrete
      changes in time and space occur in quantum amounts
   single agent
      no competition or cooperation to account for

                                                            Classical Planning
Background: Planning
 real systems help humans in practical applications
   design and manufacturing environments
   military operations, games, space exploration

 notes:
   the methods of Ch 10 (Classical Planning)
      assume the classical environment assumptions
   in Ch 11 (Planning & Acting in the Real World)
      we'll introduce methods to handle real-world situations where the
       classical assumptions may not hold

                                                            Classical Planning
Planning: Language
 What is a good language?
   expressive enough to describe a wide variety of problems
   restrictive enough for efficient algorithms to operate on it
   planning algorithm should be able to take advantage
      of the logical structure of the problem

 historical AI planning languages
   STRIPS was used in classical planners
      Stanford Research Institute Problem Solver
   ADL addresses expressive limitations of STRIPS
      Action Description Language
      adds features not in STRIPS
          negative literals, quantified variables, conditional effects, equality
   we'll use one version of the now de facto standard PDDL

                                                                       Classical Planning
Planning language
 there's now wide agreement on some planning
  language basics
 one key is adopting of a factored representation for
   each state is represented as a collection of variables
   this contrasts with
      the atomic representation typical of our previous search
       algorithms (recall the Romanian driving problem: state = city)
 the language we'll use is a variant of PDDL
   Planning Domain Definition Language
      to see its expressive power, recall propositional agent in the
       Wumpus World, which requires 4Tn2 actions to describe a
       movement of 1 square
   PDDL captures this with a single Action Schema

                                                              Classical Planning
State & Action Representations
 each state is represented as a conjunction of fluents
   these are ground, functionless atoms
   in addition, we'll use Database semantics
   1. the Close World Assumption
      fluents not explicitly mentioned are false
   2. the Unique Names Assumption
      different ground terms are different objects: Plane1, Plane2
   this state representation allows alternative algorithms
      it can be manipulated either by logical inference techniques or by
       set operations
 actions are defined by a set of action schemas
   these implicitly define the ACTIONS(s) & RESULT(s, a)
    functions required to apply search techniques

                                                             Classical Planning
Action Schemas
 PDDL & the Frame Problem
   recall the representational issue of capturing what stays the
    same given some action
   in PDDL we specify what changes, and if something is not
    mentioned, it stays the same
 Action Schemas
   are a lifted representation (recall Generalized Modus Ponens)
   lifts from propositional logic to a restricted subset of FOL
   each one stands for a set of variable-free actions
   each includes the schema name, list of variables used,
    preconditions & effects
   we consider variables as universally quantified, choose values
    to instantiate them
   PRECOND: defines states in which an action can be executed
   EFFECT: defines the result of executing the action

                                                        Classical Planning
Action Schemas
 each represents a set of variable-free actions
    form: Action Schema = predicate + preconditions + effects
    example:
      Action(Fly(p, from, to),
        PRECOND: At(p, from)  Plane(p)  Airport(from)  Airport(to)
        EFFECT: ¬AT(p, from)  At(p, to))

       an action schema in which (p, from, to) need to be instantiated
       the action name and its parameter list
       its preconditions
           a conjunction of function-free literals
       its effects
           a conjunction of function-free literals

                                                             Classical Planning
Applying Action Schemas
 action a is applicable in state s
    if its preconditions are satisfied in s
         its preconditions are entailed by s
    given variables, there can be multiple applicable instantiations
    for v variables in a domain with k unique object names, worst
     case time to find applicable ground actions is O(vk)
    leads to 1 approach for solving PDDL planning problems
         propositionalize by replacing action schemas with sets of ground
          actions & applying a propositional solver like SATPlan
         impractical for large v & k
 result of executing a in s is s'
      s' is derived from s & action a's EFFECTs
      remove -ve literal fluents (the delete list, DEL(a))
      add +ve literals in EFFECTs (the add list, ADD(a))
      RESULT(s, a) = (s - DEL(a))  ADD(a)
                                                               Classical Planning
Action Schemas
 PDDL schemas
   1. variables & ground terms
      variables in effects must also be in precondition
      so matching to state s yields results with all variables bound
      i.e. that contain only ground terms
   2. handling of time
      no explicit time terms, unlike axioms we saw for logical agents
      instead time is implicitly represented in PDDL schemas
      preconditions always refer to time: t
      effects always refer to time: t + 1
   3. a set of schemas defines a planning domain
      a specific problem adds initial & goal states

                                                              Classical Planning
Initial States, Goals, Solutions
 initial state
    conjunction of ground terms
 goal
    conjunction of +ve & -ve literals
    both ground terms & those containing variables
    variables are treated as existentially quantified
 solution
    a sequence of actions ending in s that entails the goal
    example:
    Plane(P1)  At (P1, SFO) entails At(p, SFO)  Plane (p)
 defines planning as a search problem

                                                         Classical Planning
Some Language Semantics
 How do actions affect states?
   an action is applicable in any state that satisfies its
   applicability involves a substitution  for the variables in the
      example

     At(P1,JFK)  At(P2,SFO)  Plane(P1)  Plane(P2)  Airport(JFK) 
     Satisfies PRECOND of Fly action:
     At(p, from)  Plane(p)  Airport(from)  Airport(to)
     With substitution  ={p/P1, from/JFK, to/SFO}
     Thus the action is applicable.

                                                            Classical Planning
Language Semantics
 result of executing action a in state s is the state s’
    s’ is same as s except
       any positive literal P in the effect of a is added to s’
       any negative literal ¬P is removed from s’
       example

      Fly(p, from, to)
      EFFECT: ¬AT(p, from)  At(p, to),  ={p/P1, from/JFK, to/SFO}
      s: At(P1,JFK)  At(P2,SFO)  Plane(P1)  Plane(P2)  Airport(JFK) 
      s': At(P1,SFO)  At(P2,SFO)  Plane(P1)  Plane(P2)  Airport(JFK) 

                                                                    Classical Planning
Example: air cargo transport
 simple actions to illustrate
     moving cargo between New York(JFK) & San Francisco(SFO)

Init(At(C1, SFO)  At(C2,JFK)  At(P1,SFO)  At(P2,JFK)  Cargo(C1)  Cargo(C2) 
    Plane(P1)  Plane(P2)  Airport(JFK)  Airport(SFO))
Goal(At(C1,JFK)  At(C2,SFO))

Action(Load(c, p, a)
   PRECOND: At(c, a) At(p, a) Cargo(c) Plane(p) Airport(a)
   EFFECT: ¬At(c, a) In(c, p))
Action(Unload(c, p, a)
   PRECOND: In(c, p) At(p, a) Cargo(c) Plane(p) Airport(a)
   EFFECT: At(c, a)  ¬In(c, p))
Action(Fly(p, from, to)
   PRECOND: At(p, from) Plane(p) Airport(from) Airport(to)
   EFFECT: ¬ At(p, from)  At(p, to))

                                                                     Classical Planning
Air Cargo Example
 this planning domain involves 3 actions
   and 2 affects predicates, In(c, p) & At(x, a)
   properly maintaining At predicates could be an issue
      where is cargo that is in a plane?
      without quantifiers, problem of expressing what happens to cargo
       that is in a plane
      solve by treating At predicate as applying only to cargo after it is
       unloaded, in effect not At anywhere when it is in a plane
   another issue is that this representation allows "empty"
    actions that produce contradictory effects
      Fly(P1, JFK, JFK) yields At(P1, JFK)  ¬At(P1, JFK)
      can ignore issues like this, rarely produce incorrect plans, or
      add inequality precondition: from  to

                                                               Classical Planning
Air Cargo Example
 the solution is pretty obvious
Init(At(C1, SFO)  At(C2,JFK)  At(P1,SFO)  At(P2,JFK)  Cargo(C1)  Cargo(C2) 
    Plane(P1)  Plane(P2)  Airport(JFK)  Airport(SFO))
Goal(At(C1,JFK)  At(C2,SFO))

Action(Load(c, p, a)
   PRECOND: At(c, a) At(p, a) Cargo(c) Plane(p) Airport(a)
   EFFECT: ¬At(c, a) In(c, p))
Action(Unload(c, p, a)
   PRECOND: In(c, p) At(p, a) Cargo(c) Plane(p) Airport(a)
   EFFECT: At(c, a)  ¬In(c, p))
Action(Fly(p, from, to)
   PRECOND: At(p, from) Plane(p) Airport(from) Airport(to)
   EFFECT: ¬ At(p, from)  At(p, to))

   [Load(C1, P1, SFO), Fly(P1, SFO, JFK), Unload (C1, P1, JFK),
   Load(C2, P2, JFK), Fly(P2, JFK, SFO), Unload(C2, P2, SFO)]

                                                                     Classical Planning
Example: flat tire problem
 simple actions to illustrate
     changing a flat tire (in a bad neighbourhood)
Init(At(Flat, Axle)  At(Spare, Trunk))
Goal(At(Spare, Axle))
Action(Remove(Spare, Trunk)
   PRECOND: At(Spare, Trunk)
   EFFECT: ¬At(Spare, Trunk)  At(Spare, Ground))
Action(Remove(Flat, Axle)
   PRECOND: At(Flat, Axle)
   EFFECT: ¬At(Flat, Axle)  At(Flat, Ground))
Action(PutOn(Spare, Axle)
   PRECOND: At(Spare, Ground) ¬At(Flat, Axle)
   EFFECT: At(Spare, Axle)  ¬At(Spare, Ground))
   EFFECT: ¬ At(Spare, Ground)  ¬ At(Spare, Axle)  ¬ At(Spare, Trunk)  ¬
   At(Flat, Ground)  ¬ At(Flat, Axle) )

                                                                    Classical Planning
Flat Tire example
 highly simplified, abstracted version of the problem of
  constructing a plan to fix a flat tire
    4 actions
    note capturing of "bad neighbourhood" as tires "disappearing"
     if the car is left overnight
    the simple solution

  [Remove (Flat, Axle), Remove(Spare, Trunk), PutOn(Spare,

                                                        Classical Planning
Example: Blocks World
 simple actions to illustrate
     building a tower in Blocks World
Init(On(A, Table)  On(B, Table)  On(C,A)  Block(A)  Block(B)  Block(C) 
    Clear(B)  Clear(C))
Goal(On(A,B)  On(B,C))
Action(Move(b, x, y)
   PRECOND: On(b, x)  Clear(b)  Clear(y)  Block(b)  (b  x)  (b  y)  (x  y)
   EFFECT: On(b, y)  Clear(x)  ¬ On(b, x)  ¬ Clear(y))
Action(MoveToTable(b, x)
   PRECOND: On(b, x)  Clear(b)  Block(b)  (b  x)
   EFFECT: On(b, Table)  Clear(x)  ¬ On(b, x))

     simple plan given the initial state
         note PRECOND of Move uses inequality to prevent empty moves
         Move(B,C,C)

                                                                       Classical Planning
Blocks World example
 illustration of the problem

                                Classical Planning
Blocks World example
 problem domain:
   stack cube-shaped blocks that are on a table
 Notes:
   only 1 block can fit on top of another
   moves can be onto another block or onto the table
   FOL would use quantifiers to express
      there's no block on top of some other block
      without quantifiers, PDDL requires Clear(x) predicate
   Move action schema must be complemented by MoveToTable
    to avoid errors with the Clear predicate
      since Clear(Table) is always true
   residual problem if bind y to Table in Move(b, x, y)
      search space grows, though answers still correct
      fix with a Block(m) predicate & adding Block(b)  Block(y) to the
       preconditions for Move

                                                               Classical Planning
Complexity & Classical Planning
 first, we distinguish 2 versions of the planning
    PlanSAT:
       is there a plan that solves the problem?
    Bounded PlanSAT:
       is there a solution of k or fewer steps?
       this can be used to find optimal plans
    so long as we don't allow functions the number of states is
     finite & both categories are decidable
    if we allow functions, PlanSAT becomes semi-decidable (may
     not terminate on unsolvable problems) though Bounded
     PlanSAT remains decidable even with functions

                                                       Classical Planning
Complexity & Classical Planning
 sadly, in general, both problems are NP-hard
   putting restrictions on the expressiveness may help
   for example, disallowing -ve preconditions lets PlanSAT reduce
    to P
   fortunately for planners, many useful problems are easier than
    the worst case
      for blocks world & air cargo domains, Bounded PlanSAT is NP-
       Complete, while PlanSAT is in P!
   so, optimal planning is hard, sometimes sub-optimal planning is
   reminder of complexity category relationships
Planning with state-space search
 given the problem formulation
   we can use search algorithms to generate plans
      systematic heuristic search
      local search (while recording trail of states)
 search can be
   in either the forward or the backward direction
      recall the bi-directional example from search topic
          problems related to non-invertible successor functions
 Progression planners
   do a forward state-space search
   consider the effects of all possible actions in a given state
 Regression planners
   do a backward state-space search
   to achieve a goal
      consider what must have been true in the previous state
   from effects through actions to preconditions
Progression & Regression
 partial samples: plane moving example

                                          Classical Planning
Progression algorithm
 formulation as a state-space search problem
   Initial state = initial state of the planning problem
      literals not appearing are false
   Actions = apply those whose preconditions are satisfied
      add positive effects, delete negative
   Goal test = does the state satisfy the goal
   Step cost = for simplicity, each action costs 1
   recall, functions are not permitted in the representation
      any graph search that is complete is a complete planning
      for example, A*

                                                            Classical Planning
Progression algorithm
 Progression approach
   while complete is inefficient
      any applicable action is a possible next step
   typically its problems are related to branching factor
      (1) irrelevant actions
          just because the preconditions of an action are satisfied
          does not mean that it should be done
      (2) requirement for a good heuristic
          in order for improved efficiency of search

                                                                       Classical Planning
Progression: issues
 illustrate with the air cargo example
    assume 10 airports, 5 planes & 20 pieces of cargo at each
    problem: move all cargo at airport A to airport B
    solution is pretty simple, but uninformed search has serious
     complexity difficulties
       just considering planes: 50 x 9 = 450 possible flights
       suppose all planes & packages were at 1 airport: 10450 possible
        actions (200 x 50 + 50 x 9)
       if 2000 actions available on average, 200041 nodes in search
    fortunately, good (domain-independent) heuristics are
     possible, & can be derived automatically

                                                            Classical Planning
Regression algorithm
 what about Regression planners
   issues:
      How do we determine predecessors?
      as we saw for bi-directional search, the problem may not allow
       predecessor determination (n-Queens)
      fortunately PDDL formulation facilitates it
      terminology: predecessor is g', action a, goal g
      g' = (g - Add(a))  Precond(a), since any effects added by a
       might not have been true previously & preconditions must have
       been met or could not execute a
   this process requires partially uninstantiated actions & states
      use standardized variable names (avoid clashes, retain generality
       of variables)
      implicitly quantify over these so 1 description summarizes
       possibility of using any appropriate ground term

                                                            Classical Planning
Regression: relevant actions
 which action?
   forward uses applicable actions (their preconditions are met)
   backward uses relevant actions (they could be the last step
    leading to current state)

   relevant: at least 1 effect (+ve or -ve) must unify with a term
    in the (current) goal & there can't be any effect that negates
    a term in the goal
   process: use the action formed by substituting the most
    general unifier into the standardized action schema
   more formal definition of relevant actions
   if goal g contains literal gi & action schema A (standardized to
    produce A') has effect literal ej' where Unify (gi, ej') =  & a' =
    SUBST(, A') & there's no effect in a' that is the negation of a
    literal in g, then a' is a relevant action toward g
   terminates when a predecessor is satisfied by the initial state

                                                            Classical Planning
Progression & Regression
 status of the regression approach
   1. there's a lower branching factor in the backward direction,
    the result of only considering relevant actions
   2. but each state is really a set of states
      the state defined by true or false for the ground fluents, and the
       states not mentioned (see mid p 374)
      this representation of stages in the backward search as sets of
       states complicates the process of developing heuristics

 progression versus regression
   the greater availability of accurate heuristics for forward
    search has resulted in it being used in many more planning

                                                              Classical Planning
Heuristics for planning
 the complexity of planning problems
   means that we need good heuristics
   below: 2 general approaches to finding admissible heuristics
      recall that an admissible h(n) (i.e. one that never overestimates
       cost) allows A* search for optimal solutions
   recall from our (informed search) discussion of finding
      as the optimal solution to a relaxed version of the problem
      from problem decomposition
          if we can make the subgoal independence assumption we can
           approximate the cost of solving a conjunction of subgoals as the sum
           of the costs of solving the corresponding subproblems
   (1) relaxation: search problem viewed as a graph allows
      (a) adding edges to make a path to a solution easier to find
      (b) merging states in an abstraction of the original problem so
       searching is in a space with fewer nodes

                                                                  Classical Planning
Heuristics by problem relaxation
 (a) adding edges to the search space
   (1) ignore all preconditions
      assume all actions are applicable in all states, so any goal fluent
       takes just a single step
      apparently #steps ≈ #unsatisfied goals, but
          note: some actions may achieve > 1 goal, & others may undo goals
          ignoring the latter, h(n) = min # actions for which the union of their
           effects satisfies the goal
          this is a version of the "set-cover problem", known to be NP-hard!
          a greedy algorithm yields set covering within log n of the true min (n
           is # literals in goal), but is not guaranteed admissible
   (2) ignore selected PRECONDs of actions
      as in the N-puzzle heuristics: misplaced tiles, manhattan distance
      the factored representation in action schemata for planning
       problems allows automated derivation of such heuristics
   (3) ignore delete lists
      if all goals & preconditions have only +ve literals (true for many
       problems anyway & others can be converted to this form)

                                                                    Classical Planning
Heuristics by problem relaxation
 (a) adding edges to search space
   (3) ignore delete lists
      with only +ve literals in all goals & preconditions
      then remove -ve literals from effects
      now steps are monotonic progress towards the goal w/o undoing
      but, it's still NP-hard to find an optimal solution, though it can
       be approximated in polynomial time with hill climbing
   summary
      adding edges relaxes a planning problem to yield heuristics but
       they're still expensive to calculate
 (b) reduce nodes by abstraction to merge states
   return to the air cargo example
      10 airports, 50 planes, 200 packages means the general case has
       5010 x 20050+10 or 10155 states
      a specific problem: say all pkgs are at 5 airports & all pkgs at any
       airport have 1 destination
                                                               Classical Planning
Heuristics by problem relaxation
 (b) reduce nodes by abstraction to merge states
   in the air cargo example, in general, 10155 states
      a specific problem has all pkgs at 5 airports & pkgs at any airport
       have 1 destination
      an abstraction of this version omits At fluents except those re: 1
       plane & 1 pkg at each of 5 airports, reducing # states to 510 x 55+10
       or 1017 states
      so a solution is shorter & an admissible h(n) & is extensible to the
       original problem by adding more Load & Unload actions
 problem decomposition approaches for h(n)'s
   the relevant pattern is:
      subdivide problem  solve subproblems  combine solutions
   the subgoal independence assumption
      is useful in developing a heuristic as a sum of subgoal costs
      is too optimistic if there are -ve interactions between subplans
      is too pessimistic (inadmissible) when there are redundant actions
       in subplans
Heuristics by decomposition
 problem decomposition approaches for h(n)'s
   notation: represent the goal as a set of fluents G, where
    disjoint subsets of G are G1, …Gn, for which we find plans P1,
    … Pn & then use their costs to estimate cost for G
   Cost(Pi) is an estimate, a possible heuristic
   maxi Cost(Pi) is admissible, overly optimistic
   sumi Cost(Pi) is not generally admissible unless all Gi & Gj are
    independent (Pi does not affect preconds or goals of Pj) in
    which case the sum is admissible & more accurate

                                                          Classical Planning
Heuristics by decomposition
 the challenge of abstraction is
    to find one that has a significantly lower total cost (defining the
     abstraction + doing the abstracted search + mapping back to
     original problem) than the original problem
    the pattern database techniques may be useful
       recall them from informed search discussion of heuristics
       amortize the cost of building the database over multiple problem
        solution instances

 the FF (FASTFORWARD) planning system is a hybrid
    it uses heuristics from a planning graph (our next discussion)
    it uses local search (storing plan steps)
    at a plateau or local maximum it does iterative deepening
     systematic search for a better state or gives up & restarts
Planning Graphs
 Planning Graphs
   are an alternative intermediate representation/data structure
      they are polynomial complexity approximations of full
       (exponential) trees of all states & actions
      they can't answer the question: is G reachable?
      though they are correct when they say it's not reachable
      they provide an estimate of the number of steps to G
      any such estimate is optimistic, so is an admissible heuristic

   planning graphs
      provide a possible basis for better search heuristics
      and they can also be used directly, for extracting a solution to a
       planning problem, by applying the GRAPHPLAN algorithm

                                                              Classical Planning
Planning Graphs
 a planning graph
   is a directed graph organized in levels
   the levels of nodes correspond to time steps in a plan
      they consist of alternating S levels (in which the nodes are
       fluents that hold) & A levels (in which the nodes are actions that
       might be applicable)
   level S0 is the initial state
   each level is a set of literals Si or a set of actions Ai
      Si: all the literals that could be true at that time step
          depending on the actions executed at the previous steps
      Ai: all actions that could have PRECONDs satisfied at that step
          depending on which of the literals actually hold

                                                                     Classical Planning
Planning Graphs
 explanation regarding could on the previous slide
   a planning graph only captures a restricted subset of the
    possible -ve interactions
      so a literal might appear at a level earlier than it actually would
       (if it would at all), though it never appears too late
      despite this error, the level j of the first appearance is a good
       estimate of how difficult it is to achieve from the initial state
      refers to the approximate nature of the state/action lists
      in part, this allows efficient construction by recording a restricted
       subset of possible -ve interactions among actions
 note:
   planning graphs apply only for propositional problems
   action schemas for problems with variables can be
   their advantages may make this worthwhile, despite the
    increased size of the problem description

                                                                Classical Planning
Planning Graphs
 our simple example: "have your cake & eat it too"
   the problem description
     Goal(Have(Cake)  Eaten(Cake))
       PRECOND: Have(Cake)
       EFFECT: ¬Have(Cake)  Eaten(Cake))
       PRECOND: ¬ Have(Cake)
       EFFECT: Have(Cake))

   the corresponding planning graph
 Planning Graph Cake Example                                              42

 start at level S0, determine action level A0 & next level S1
    A0: all actions whose preconditions are satisfied in the previous
     level (initial state)
       actions are shown in rectangular boxes
       lines connect PRECONDs at S0 to EFFECTs at S1
    also, for each literal in Si, there's a persistence action (square box)
     & line to it in the next level Si+1
 level A0 contains the actions that could occur
    conflicts between actions are represented by arcs: mutual
     exclusion or mutex links
 Planning Graph Cake Example                                               43

 level S1 contains all the literals that could result
   from picking any subset of actions in A0
   so S1 is a belief state consisting of the set of all possible states
       each is a subset of literals with no mutex links between members
    conflicts between literals that cannot occur together are
     represented by the mutex links.
 the level generation process is repeated
    eventually consecutive levels are identical: leveling off
Planning Graph Cake Example                                                   44

 mutex relation holds between 2 actions at a level when
   1. inconsistent effects
      one action negates the effect of another
   2. interference
      an effect of one action negates a precondition of the other
   3. competing needs
      a precondition of one action is mutex with a precondition of the

                                                              Classical Planning
Planning Graph Cake Example                                                 45

 mutex relation holds between 2 literals at a level when
   1. one is the negation of the other
   2. if each possible action pair that could achieve the literals is
    mutex (Have(Cake) & Eaten(Cake) at S1)
      this is termed inconsistent support

                                                            Classical Planning
Planning Graphs & Heuristics
 Planning Graphs
   construction has complexity polynomial in the size of the
    planning problem
      given l literals, a actions, & a PG of n levels: O(n(a + l)2)
   the completed PG
      provides information about the problem & candidate heuristics
      1. a goal literal g that does not appear in the final level cannot be
       achieved by any plan
      2. the level cost, the level at which a goal literal first appears, is
       useful as a cost estimate of achieving that goal literal
      note that level cost is admissible, though possibly inaccurate
       since it counts levels, not actions
      we could find a better alternative level cost by using a serial
       planning graph variation, restricted to one action per level
          mutex links between every pair of actions except persistence actions

                                                                  Classical Planning
Planning Graphs & Heuristics
 Planning Graph provides
   possible heuristics for the cost of a conjunction of goals
   1. max-level: highest level of any conjunct in the goal
      admissible, possibly not accurate
   2. level sum: the sum of level costs of conjuncts in the goal
      incorporates the subgoal independence assumption
          so may be inadmissible to degree the assumption does not hold
   3. set-level: level where all goal conjuncts are present without
    mutex links
      this one dominates the max level heuristic
      & is good where there are interactions among subplans

                                                                 Classical Planning
Planning Graphs & Heuristics
 a Planning Graph is a relaxed version of the problem
   if a goal literal g does not appear, no plan can achieve it, but
    if it does appear, is not guaranteed to be achievable
   why?
      the PG only captures pairwise conflicts & there could be higher
       order conflicts
      likely not worth the computational expense of checking for them
      similar to Constraint Satisfaction Problems where arc consistency
       was a valuable pruning tool
      3-consistency or even higher order consistency would have made
       finding solutions easier but was not worth the additional work
   an example where PG fails to indicate unsolvable problem
      blocks world problem with goal of A on B, B on C, C on A
      any pair of subgoals are achievable, so no mutexes
      problem only fails at stage of searching the PG

                                                            Classical Planning
The GRAPHPLAN Algorithm
 the GRAPHPLAN algorithm
     generates the Planning Graph & extracts a solution from it

function GRAPHPLAN(problem) return solution or failure
   graph  INITIAL-PLANNING-GRAPH(problem)
   goals  CONJUNCTS(problem. GOAL)
   nogoods  an empty hash table
   for tl = 0 to  do
          if goals all non-mutex in St of graph then
              solution  EXTRACT-SOLUTION(graph, goals,
                                               NUMLEVELS(graph), nogoods)
              if solution  failure then return solution
          if graph and nogoods have both leveled off then return failure
          graph  EXPAND-GRAPH(graph, problem)

                                                                      Classical Planning
Example: Spare Tire Problem
 recall the spare tire problem
Init(At(Flat, Axle)  At(Spare, Trunk))

Goal(At(Spare, Axle))
Action(Remove(Spare, Trunk)
   PRECOND: At(Spare, Trunk)
   EFFECT: ¬At(Spare, Trunk)  At(Spare, Ground))
Action(Remove(Flat, Axle)
   PRECOND: At(Flat, Axle)
   EFFECT: ¬At(Flat, Axle)  At(Flat, Ground))
Action(PutOn(Spare, Axle)
   PRECOND: At(Spare, Ground) ¬At(Flat, Axle)
   EFFECT: At(Spare, Axle)  ¬At(Spare, Ground))
   EFFECT: ¬ At(Spare, Ground)  ¬ At(Spare, Axle)  ¬ At(Spare, Trunk) 
            ¬ At(Flat, Ground)  ¬ At(Flat, Axle) )

                                                                    Classical Planning
GRAPHPLAN Spare Tire Example                                                    51

 Notes:
   this figure shows the complete Planning Graph for the problem
   arcs show mutex relations
      but arcs between literals are omitted to avoid clutter
   it also omits unchanging +ve literals (for example, Tire(Spare))
    & irrelevant -ve literals
   bold boxes & links indicate the solution plan

                                                                Classical Planning
 GRAPHPLAN Spare Tire Example                                              52

 S0 is initialized to 5 literals
    from the problem initial state and the CWA literals
 no goal literal in S0 so EXPAND-GRAPH add actions
    those with preconditions satisfied in S0
    also adds persistence actions for literals in S0
    adds the effects at level S1, analyzes & adds mutex relations
 repeat until the goal is in level Si or failure
                                                           Classical Planning
 GRAPHPLAN Spare Tire Example                                                        53

 EXPAND-GRAPH adds constraints: mutex relations
    inconsistent effects (action x vs action y)
        Remove(Spare, Trunk) & LeaveOvernight: At(Spare, Ground) & ¬At(Spare,
    interference (effect negates a precondition)
        Remove(Flat, Axle) & LeaveOvernight: At(Flat, Axle) as PRECOND &
         ¬At(Flat, Axle) as EFFECT
    competing needs (mutex preconditions)
        PutOn(Spare, Axle) & Remove(Flat, Axle): At(Flat, Axle) & ¬At(Flat, Axle)
    inconsistent support (actions to produce literals are mutex)
        in S2, At(Spare, Axle) & At(Flat, Axle)
Reminder: GRAPHPLAN Algorithm
 the GRAPHPLAN algorithm both generates the
  Planning Graph & extracts a solution from it
function GRAPHPLAN(problem) return solution or failure
   graph  INITIAL-PLANNING-GRAPH(problem)
   goals  CONJUNCTS(problem. GOAL)
   nogoods  an empty hash table
   for tl = 0 to  do
          if goals all non-mutex in St of graph then
              solution  EXTRACT-SOLUTION(graph, goals,
                                               NUMLEVELS(graph), nogoods)
              if solution  failure then return solution
          if graph and nogoods have both leveled off then return failure
          graph  EXPAND-GRAPH(graph, problem)

                                                                      Classical Planning
 GRAPHPLAN Spare Tire Example                                                        55

 in S2, the goal literals exist, & are not mutex
        just 1 goal literal so obviously not mutex with any other goal
    since a solution may exist, EXTRACT-SOLUTION tries to find it
 EXTRACT-SOLUTION may search backwards for a solution
    initial state: last level of the PG + goals of the planning problem
    actions from Si
        select a set of conflict-free actions in Ai-1 with effects covering the goals
        conflict free = no 2 actions are mutex & no pair of preconditions are mutex
    goal: reach a state at level S0 such that all goals are satisfied
    cost: 1 for each action
   at that point it records (level, goals) as a "no-good"
   subsequent calls can fail immediately if they require the same
    goals at that level

   we already know planning problems are computationally hard
      require good heuristics
   greedy search with level cost of literals as a heuristic works
      1. pick literal with highest level cost
      2. to achieve it, pick actions with easier preconds
          action with smallest sum (or max) of level costs for its preconds
 alternative to backward search for a solution
   EXTRACT-SOLUTION could formulate a Boolean CSP
      variables are actions at each level
      values are Boolean: an action is either in or out of the plan
      constraints are mutex relations & the need to satisfy each goal &

                                                             Classical Planning
GRAPHPLAN Termination
 does GRAPHPLAN terminate?
   does it stop & return failure if there's no solution
   the general rationale is presented here
   for some proof details see the textbook & its Ghallab reference
 background
   recall that levelled off means consecutive PG levels are identical
   now note that a graph may level off before a solution can be
    found, on a problem for which there is a solution
   example of 2 airports, 1 plane & n pieces of cargo at one of
    them, plan required to move all n to the other airport
      but only 1 piece at a time fits in the plane
   graph levels off at level 4, from which full solution can't be
    extracted (that would require 4n - 1 steps)
   we need to take account of the no-goods (goals that were not
    achievable) as well
   these may decline even after the graph levels off
GRAPHPLAN Termination
 does GRAPHPLAN terminate?
   given that no-goods might continue to decrease after the graph
    levels off
   we now require a proof that both the graph & no-goods will
    level off
   literals increase monotonically
      once a literal appears, its persistence action causes it to stay
   actions increase monotonically
      once a literal that is a precondition appears, the action stays
   mutexes decrease monotonically
      though the graph simplifying conventions may not show it
          if 2 actions are mutex at Ai, they are also mutex at all previous levels
           where they appear
GRAPHPLAN Termination
 does GRAPHPLAN terminate?
   no-goods decrease monotonically
      if a set of goals is not achievable at level i, they are not achievable
       at any previous level
   so literals & actions increase monotonically
      given a finite number of actions and literals, they will level off
   and mutexes & no-goods decrease monotonically
      since there cannot be fewer than 0, they must level off
   if the graph & no-goods have levelled off & a goal is missing or
    is mutex with another goal, the algorithm can stop & return
Alternative Planning Approaches
 the following slides outline brief discussions of 4
  alternative approaches to solving planning problems
   1. convert the PDDL description into a form solvable by a
    propositional logic satisfiability algorithm: SATPlan
   2. convert into a situational calculus representation solvable
    by First-Order Logic inference techniques
   3. represent & solve as a Constraint Satisfaction Problem
   4. use a "plan refinement" approach to searching in a space of
    partially ordered plans

                                                       Classical Planning
Planning with Propositional Logic
 1. form a propositional logic satisfiability problem for
  submission to SATPlan
    iteratively tries plans of increasing length (increasing number
     time steps) up to the Tmax parameter value, so finds the
     shortest plan, if one exists
 the SATPlan algorithm
    finds models for a (very long) PL sentence that includes initial
     state, goal, successor-state axioms, precondition axioms, &
     action-exclusion axioms
       assigns true to the actions that are part of the correct plan &
        false to the others
           if the planning is unsolvable the sentence will be unsatisfiable
           any model satisfying the sentence will be a valid plan
           an assignment that corresponds to an incorrect plan will not be a
            model because of inconsistency with the assertion that the goal is

                                                                    Classical Planning
Planning with Propositional Logic
 translation from PDDL to PL form for SATPlan
 the process involves 6 steps
     step 1: propositionalize actions, replacing each schema with a
      set of ground actions (constants substituted for variables)
     step 2: define an initial state asserting F0 for all fluents in the
      initial state, ¬F0 for all fluents not in the initial state
     step 3: propositionalize the goal - for each variable replace
      literals containing it with a disjunction over constants
     step 4: include successor-state axioms for each fluent
Ft+1  ActionCausesFt  (Ft  ¬ActionCausesNotFt), where
        ActionCausesF is disjunction of all ground actions with F in their
         add List
        ActionCausesNotF is disjunction of all ground actions with F in
         their delete list
            air cargo example: axioms are required for each plane, airport and
             time step!
                                                                    Classical Planning
Planning with Propositional Logic
 the 6 steps from PDDL to SATPlan format
   step 5: add precondition axioms (for every ground action A,
    add At  PRE(A)t
   step 6: add action exclusion axioms to say every action is
    distinct from every other action
 submit to SATPlan

                                                       Classical Planning
The SATPLAN Algorithm
 the algorithm
     note that in AIMA 3e, this is first presented in the earlier
      section on propositional logic agents

function SATPLAN(init, transition, goal, Tmax) returns solution or failure
   inputs: init, transition, goal form a description of the planning problem
          Tmax, an upper limit to the plan length
   for t = 0 to Tmax do
         cnf  TRANSLATE-TO-SAT(init, transition, goal, t )
         model  SAT-SOLVER(cnf)
         if model is not null then
             return EXTRACT-SOLUTION(model)
   return failure

                                                                         Classical Planning
Planning with Propositional Logic
 some explanatory notes
   distinct propositions for assertions about each time step
      superscripts denote the time step: At(P1,SFO)0  At(P2,JFK)0
   PDDL includes the Closed World Assumption, so conversion
    involves the need to specify which propositions are not true
      ¬At(P1,JFK)0  ¬At(P2,SFO)0
   unknown propositions are left unspecified
   goal is associated with some particular time-step: which one?

                                                            Classical Planning
Planning with Propositional Logic
 some explanatory notes
   we can't know in advance how many steps are required to
    achieve a goal so, to determine the time step T where the
    goal will be reached, the algorithm loops until success or
    some arbitrary time step limit
   repeat
      start at T = 0
          Assert At(P1,JFK)0  At(P2,SFO)0
      on failure: try T = 1
          Assert At(P1,JFK)1  At(P2,SFO)1
          add terms for next time step
   until a solution is found or some max path length Tmax is
    exceeded: setting Tmax assures termination

                                                        Classical Planning
 the pre-condition axioms
   avoid models with illegal actions
      e.g. for T = 1
      Fly(P1, SFO, JFK)0  Fly(P1, JFK, SFO)0  Fly(P2, JFK, SFO)0
      the second action is infeasible, precluded if we include the
       appropriate pre-condition axiom: Fly(P1,JFK,SFO)0  At(P1,JFK)0
      since At(P1, JFK)0 false in initial state, Fly(P1, JFK, SFO)0 false in
       any model

                                                                 Classical Planning
 without action-exclusion axioms
   a model might include actions, for example having 1 plane fly
    to two destinations in a time step
      Fly(P1, SFO, JFK)0  Fly(P2, JFK, SFO)0  Fly(P2, JFK, LAX)0
      though the second P2 action is infeasible, it requires the added
       action-exclusion axioms to avoid such solutions
      ¬(Fly(P2,JFK,SFO)0  Fly(P2,JFK,LAX)0)

                                                              Classical Planning
Planning in First-Order Logic
 alternative 2: convert to the Situation Calculus format
  of FOL & apply FOL inferencing
    recall Situation Calculus as a topic in the 352 Knowledge
     Representation discussion of time & action
    PDDL omits quantifiers which limits expressiveness but helps
     control complexity of algorithms applied to it
    PL representation also has limitations like the requirement
     that a "timestamp" becomes part of each fluent
    Situation Calculus replaces explicit timestamps by using
     situations that correspond to a sequence or history of actions
    1. situations
       initial state is a situation
       if s is a situation & a an action, RESULT(s, a) is also a situation
       situations correspond to sequences or history of actions
       2 situations are the same only if their start & actions are the

                                                                 Classical Planning
 Planning in First-Order Logic
 convert to FOL Situation Calculus
     2. fluents are functions or relations that vary over situations
          syntax: situation is conventionally the last argument
          At(x, l, s) is a relational fluent, true when object x is at location l in
           situation s & Location(x, s) is a functional fluent such that
           Location(x, s) = l in the same situation
     3. a possibility axiom describes preconditions of each action
          (s)  Poss(a, s) indicates when an action can be taken
           is a formula giving preconditions
          Alive(Agent, s)  Have(Agent, Arrow, s)  Poss(Shoot, s)
     4. successor-state axioms for each fluent indicate what
      happens, depending on the action taken
          here are the general form & an example from Wumpus World
Action is possible 
(Fluent is true in result state  Action's effect made it true V
It was true before & the action left it unchanged)
Poss(a, s) 
(At(Agent, y, Result(a, s))  a = Go(x, y) V (At(Agent, y, s) Λ a  Go(y, z)))
Planning in First-Order Logic
 express problem using FOL Situation Calculus
   5. unique action axioms required to allow deductions such as
      a  Go(y, z)
      for each pair of action names Ai & Aj there's an axiom to say they
       are different: Ai(x, …)  Aj(y, …)
      plus, for each action name Aj an axiom indicates 2 uses are only
       equal if all arguments are equal
      Ai (x1, …, xn) = Ai(y1, …, yn)  x1 = y1  …  xn = yn
   6. the solution is a situation (action sequence) that satisfies the
   unfortunately, no major planning systems use situation calculus
      basic efficiency issues of FOL inference
      compounded by lack of good heuristics for planning with SC
Planning: Constraint Satisfaction
 we noticed in discussions in 352
   the similarities between Constraint Satisfaction Problems &
    boolean satisfiability
   conversion to a CSP is similar to that of conversion to a
    propositional logic satisfiability problem
   one difference is that it only requires a single variable Actiont
    at each time step: its domain is the set of all possible actions
   thus we also don't need the action exclusion axioms

                                                           Classical Planning
Partial-order planning
 plans may be developed through refinement of partially
  ordered plans
    approaches describe so far, including both Progression and
     Regression planning, are totally ordered forms of plan search
       they cannot take advantage of problem decomposition
       decisions must be made on how to sequence the actions for all the
        subproblems though some are independent (in the air cargo
        problem, loading cargo at 2 different airports)
    a least commitment strategy
       would, where possible, delay choices during search
    partially ordered planning implements this
       adds actions to plans without committing to an absolute time/step,
        instead dealing mainly with relative constraints
           action A must precede action B
    partially ordered plans are generated by a search in the space of
     plans rather than through the state space
       so the planning operators are different from the real-world actions
                                                                 Classical Planning
A plan to put your shoes on
 an illustration from 2e notes

Goal(RightShoeOn  LeftShoeOn)

   PRECOND: RightSockOn
   EFFECT: RightShoeOn)
   EFFECT: RightSockOn)
   PRECOND: LeftSockOn
   EFFECT: LeftShoeOn)
   EFFECT: LeftSockOn)

                                  Classical Planning
A plan to put your shoes on
 a partially-ordered planner can place two actions into
  a plan without specifying which comes first
   sample POP and corresponding totally ordered plans

                                                     Classical Planning
POP as a search problem
 states are (mostly unfinished) plans
    the empty plan contains only the special start & finish actions
 each state (plan) is a partially ordered structure
    it consists of a set of actions (the steps of the plan) and a set of
     ordering constraints of the form Before(ai, aj) meaning that one
     action occurs before another
       graphically, actions are shown as boxes, ordering constraints as
    the starting point is an empty plan consisting of the initial state
     & goal, with no actions between them
       the search process adds an action to correct a flaw in the plan or if
        it can't, it backtracks
           flaws keep a partial plan from being a solution
    as an example, the next slide shows POPs corresponding to the
     "flat tire" planning problem, both the empty plan and a solution
POP for the flat tire problem
 at the top the empty plan & below it, the solution plan
   Remove(spare, trunk), Remove(flat, axle) can be done in either
    order, so long as both are done before PutOn(spare, axle)
POP for the flat tire problem
 search in plan space seeks to add to the plan to
  correct a flaw
   flaw: not achieving At(spare, axle) in the flat tire empty plan
   adding PutOn(spare, axle) fixes that, but introduces new
    flaws in the form of its (unachieved) preconditions
   process repeats, backtracking as necessary
   each step makes the least commitment (ordering constraints
    & variable bindings) in order to fix a flaw
   Remove(spare, trunk) must be before PutOn(spare, axle) but
    no other ordering requirements

                                                         Classical Planning
Partial-Order Planning
 was previously popular because of explicitly
  representing independent subproblems
   but it lacks an explicit representation of states in the state-
    transition model & was superseded by progression planners
    with heuristics that allowed discovery of independent
   nevertheless it's useful for tasks like operations scheduling,
    enhanced with domain specific heuristics
   also appropriate where humans must understand the resulting
      plan refinement approach is easier to understand & verify
      example: spacecraft/rover plans are generated by this approach

                                                           Classical Planning
Solving POP
 POP in action
   the initial plan contains
      only the Start & Finish steps
          Start step/action has the initial state description as its effect
          Finish step/action has the goal description as its precondition
      the single ordering constraint Before (Start, Finish)
      there's a set of open preconditions: all those in Finish
   the successor function
      picks an open precondition p on an action B and generates a
       successor plan using an action A that achieves p
   then does goal test
      any remaining open preconditions?

                                                                       Classical Planning
Process summary
 operators on partial plans may
   add link from an existing action to an open precondition
   add a step (a new action) to fulfill an open condition
   order one step wrt another to remove possible conflicts
 plan generation involves gradually moving
   from incomplete/vague plans to complete/correct plans
 plan construction may require backtracking
   if an open condition is unachievable or conflict is unresolvable

                                                         Classical Planning
Classical Planning Summary
 approaches rely on techniques of search or logic
   search to assemble a plan or to constructively prove one exists
 handling the inherent complexity of these problems is a
   identifying independent subproblems provides potential for
    exponential speedup, but full decomposability is inconsistent
    with -ve interactions between actions
   heuristically guided search can gain efficiency even if
    subproblems are not completely independent
 some problems allow efficient solution by ruling out -ve
   serializable subgoals = there's a subgoal ordering for a solution
    that avoids having to undo any subgoals
   tower problems in Blocks World are serializable if subgoals are
    ordered bottom to top
Classical Planning Summary
 serializable subgoals
    tower problems in Blocks World are serializable if subgoals are
     ordered bottom to top
    NASA spacecraft control commands are serializable
     (unsurprising since the control systems are designed for easy
    when problem is serializable, a planner can seriously prune
     the planning search
 what's next? where does further progress come from?
    probably not simply factored representations & propositional
    combine efficient heuristics + first-order, hierarchical

                                                         Classical Planning
Classical Planning Summary
 PDDL (Planning Domain Definition Language)
   initial & goal states as conjunctions of literals
   actions as lists of preconditions & effects
 search in state space may be forward (Progression) or
  backward (Regression)
   derive heuristics from subgoal independence assumption &
    relaxed versions of the problems
 Planning Graphs
   build from the initial state to the goal with alternating
    incremental sets of literals or actions representing possibilities
    for each time step, & encoding mutexes between inconsistent
    pairs of literals or actions
      layers are supersets of literals/actions possible at that time step
   the basis of good heuristics for state space searches or can
    yield a plan by application of the GRAPHPLAN algorithm
                                                               Classical Planning
Classical Planning Summary
 possible alternative approaches
     FOL inference on a situation calculus knowledge base
     conversion to boolean satisfiability problem for SATPlan
     encoding as a Constraint Satisfaction Problem
     search backwards over a space of Partially Ordered Plans
 what's best?
   there's no agreement on a single best representation or
   we saw one hybrid that performs well
   a particular domain may provide a best fit to one particular

                                                        Classical Planning

To top