VIEWS: 23 PAGES: 86 POSTED ON: 8/9/2011 Public Domain
CISC453 Winter 2010 Classical Planning AIMA3e Ch 10 2 What is Planning? generate sequences of actions to perform tasks and achieve objectives involving states, actions & goals a search for a solution over an abstract space of plans the assumptions for classical planning problems fully observable we see everything that matters deterministic the effects of actions are known exactly static no changes to environment other than those caused by agent actions discrete changes in time and space occur in quantum amounts single agent no competition or cooperation to account for Classical Planning 3 Background: Planning real systems help humans in practical applications design and manufacturing environments military operations, games, space exploration notes: the methods of Ch 10 (Classical Planning) assume the classical environment assumptions in Ch 11 (Planning & Acting in the Real World) we'll introduce methods to handle real-world situations where the classical assumptions may not hold Classical Planning 4 Planning: Language What is a good language? expressive enough to describe a wide variety of problems restrictive enough for efficient algorithms to operate on it planning algorithm should be able to take advantage of the logical structure of the problem historical AI planning languages STRIPS was used in classical planners Stanford Research Institute Problem Solver ADL addresses expressive limitations of STRIPS Action Description Language adds features not in STRIPS negative literals, quantified variables, conditional effects, equality we'll use one version of the now de facto standard PDDL Classical Planning 5 Planning language there's now wide agreement on some planning language basics one key is adopting of a factored representation for states each state is represented as a collection of variables this contrasts with the atomic representation typical of our previous search algorithms (recall the Romanian driving problem: state = city) the language we'll use is a variant of PDDL Planning Domain Definition Language to see its expressive power, recall propositional agent in the Wumpus World, which requires 4Tn2 actions to describe a movement of 1 square PDDL captures this with a single Action Schema Classical Planning 6 State & Action Representations each state is represented as a conjunction of fluents these are ground, functionless atoms in addition, we'll use Database semantics 1. the Close World Assumption fluents not explicitly mentioned are false 2. the Unique Names Assumption different ground terms are different objects: Plane1, Plane2 this state representation allows alternative algorithms it can be manipulated either by logical inference techniques or by set operations actions are defined by a set of action schemas these implicitly define the ACTIONS(s) & RESULT(s, a) functions required to apply search techniques Classical Planning 7 Action Schemas PDDL & the Frame Problem recall the representational issue of capturing what stays the same given some action in PDDL we specify what changes, and if something is not mentioned, it stays the same Action Schemas are a lifted representation (recall Generalized Modus Ponens) lifts from propositional logic to a restricted subset of FOL each one stands for a set of variable-free actions each includes the schema name, list of variables used, preconditions & effects we consider variables as universally quantified, choose values to instantiate them PRECOND: defines states in which an action can be executed EFFECT: defines the result of executing the action Classical Planning 8 Action Schemas each represents a set of variable-free actions form: Action Schema = predicate + preconditions + effects example: Action(Fly(p, from, to), PRECOND: At(p, from) Plane(p) Airport(from) Airport(to) EFFECT: ¬AT(p, from) At(p, to)) an action schema in which (p, from, to) need to be instantiated the action name and its parameter list its preconditions a conjunction of function-free literals its effects a conjunction of function-free literals Classical Planning 9 Applying Action Schemas action a is applicable in state s if its preconditions are satisfied in s its preconditions are entailed by s given variables, there can be multiple applicable instantiations for v variables in a domain with k unique object names, worst case time to find applicable ground actions is O(vk) leads to 1 approach for solving PDDL planning problems propositionalize by replacing action schemas with sets of ground actions & applying a propositional solver like SATPlan impractical for large v & k result of executing a in s is s' s' is derived from s & action a's EFFECTs remove -ve literal fluents (the delete list, DEL(a)) add +ve literals in EFFECTs (the add list, ADD(a)) RESULT(s, a) = (s - DEL(a)) ADD(a) Classical Planning 10 Action Schemas PDDL schemas 1. variables & ground terms variables in effects must also be in precondition so matching to state s yields results with all variables bound i.e. that contain only ground terms 2. handling of time no explicit time terms, unlike axioms we saw for logical agents instead time is implicitly represented in PDDL schemas preconditions always refer to time: t effects always refer to time: t + 1 3. a set of schemas defines a planning domain a specific problem adds initial & goal states Classical Planning 11 Initial States, Goals, Solutions initial state conjunction of ground terms goal conjunction of +ve & -ve literals both ground terms & those containing variables variables are treated as existentially quantified solution a sequence of actions ending in s that entails the goal example: Plane(P1) At (P1, SFO) entails At(p, SFO) Plane (p) defines planning as a search problem Classical Planning 12 Some Language Semantics How do actions affect states? an action is applicable in any state that satisfies its precondition(s) applicability involves a substitution for the variables in the PRECOND example State: At(P1,JFK) At(P2,SFO) Plane(P1) Plane(P2) Airport(JFK) Airport(SFO) Satisfies PRECOND of Fly action: At(p, from) Plane(p) Airport(from) Airport(to) With substitution ={p/P1, from/JFK, to/SFO} Thus the action is applicable. Classical Planning 13 Language Semantics result of executing action a in state s is the state s’ s’ is same as s except any positive literal P in the effect of a is added to s’ any negative literal ¬P is removed from s’ example Fly(p, from, to) EFFECT: ¬AT(p, from) At(p, to), ={p/P1, from/JFK, to/SFO} s: At(P1,JFK) At(P2,SFO) Plane(P1) Plane(P2) Airport(JFK) Airport(SFO) s': At(P1,SFO) At(P2,SFO) Plane(P1) Plane(P2) Airport(JFK) Airport(SFO) Classical Planning 14 Example: air cargo transport simple actions to illustrate moving cargo between New York(JFK) & San Francisco(SFO) Init(At(C1, SFO) At(C2,JFK) At(P1,SFO) At(P2,JFK) Cargo(C1) Cargo(C2) Plane(P1) Plane(P2) Airport(JFK) Airport(SFO)) Goal(At(C1,JFK) At(C2,SFO)) Action(Load(c, p, a) PRECOND: At(c, a) At(p, a) Cargo(c) Plane(p) Airport(a) EFFECT: ¬At(c, a) In(c, p)) Action(Unload(c, p, a) PRECOND: In(c, p) At(p, a) Cargo(c) Plane(p) Airport(a) EFFECT: At(c, a) ¬In(c, p)) Action(Fly(p, from, to) PRECOND: At(p, from) Plane(p) Airport(from) Airport(to) EFFECT: ¬ At(p, from) At(p, to)) Classical Planning 15 Air Cargo Example this planning domain involves 3 actions and 2 affects predicates, In(c, p) & At(x, a) properly maintaining At predicates could be an issue where is cargo that is in a plane? without quantifiers, problem of expressing what happens to cargo that is in a plane solve by treating At predicate as applying only to cargo after it is unloaded, in effect not At anywhere when it is in a plane another issue is that this representation allows "empty" actions that produce contradictory effects Fly(P1, JFK, JFK) yields At(P1, JFK) ¬At(P1, JFK) can ignore issues like this, rarely produce incorrect plans, or add inequality precondition: from to Classical Planning 16 Air Cargo Example the solution is pretty obvious Init(At(C1, SFO) At(C2,JFK) At(P1,SFO) At(P2,JFK) Cargo(C1) Cargo(C2) Plane(P1) Plane(P2) Airport(JFK) Airport(SFO)) Goal(At(C1,JFK) At(C2,SFO)) Action(Load(c, p, a) PRECOND: At(c, a) At(p, a) Cargo(c) Plane(p) Airport(a) EFFECT: ¬At(c, a) In(c, p)) Action(Unload(c, p, a) PRECOND: In(c, p) At(p, a) Cargo(c) Plane(p) Airport(a) EFFECT: At(c, a) ¬In(c, p)) Action(Fly(p, from, to) PRECOND: At(p, from) Plane(p) Airport(from) Airport(to) EFFECT: ¬ At(p, from) At(p, to)) [Load(C1, P1, SFO), Fly(P1, SFO, JFK), Unload (C1, P1, JFK), Load(C2, P2, JFK), Fly(P2, JFK, SFO), Unload(C2, P2, SFO)] Classical Planning 17 Example: flat tire problem simple actions to illustrate changing a flat tire (in a bad neighbourhood) Init(At(Flat, Axle) At(Spare, Trunk)) Goal(At(Spare, Axle)) Action(Remove(Spare, Trunk) PRECOND: At(Spare, Trunk) EFFECT: ¬At(Spare, Trunk) At(Spare, Ground)) Action(Remove(Flat, Axle) PRECOND: At(Flat, Axle) EFFECT: ¬At(Flat, Axle) At(Flat, Ground)) Action(PutOn(Spare, Axle) PRECOND: At(Spare, Ground) ¬At(Flat, Axle) EFFECT: At(Spare, Axle) ¬At(Spare, Ground)) Action(LeaveOvernight PRECOND: EFFECT: ¬ At(Spare, Ground) ¬ At(Spare, Axle) ¬ At(Spare, Trunk) ¬ At(Flat, Ground) ¬ At(Flat, Axle) ) Classical Planning 18 Flat Tire example highly simplified, abstracted version of the problem of constructing a plan to fix a flat tire 4 actions note capturing of "bad neighbourhood" as tires "disappearing" if the car is left overnight the simple solution [Remove (Flat, Axle), Remove(Spare, Trunk), PutOn(Spare, Axle)] Classical Planning 19 Example: Blocks World simple actions to illustrate building a tower in Blocks World Init(On(A, Table) On(B, Table) On(C,A) Block(A) Block(B) Block(C) Clear(B) Clear(C)) Goal(On(A,B) On(B,C)) Action(Move(b, x, y) PRECOND: On(b, x) Clear(b) Clear(y) Block(b) (b x) (b y) (x y) EFFECT: On(b, y) Clear(x) ¬ On(b, x) ¬ Clear(y)) Action(MoveToTable(b, x) PRECOND: On(b, x) Clear(b) Block(b) (b x) EFFECT: On(b, Table) Clear(x) ¬ On(b, x)) simple plan given the initial state note PRECOND of Move uses inequality to prevent empty moves Move(B,C,C) Classical Planning 20 Blocks World example illustration of the problem Classical Planning 21 Blocks World example problem domain: stack cube-shaped blocks that are on a table Notes: only 1 block can fit on top of another moves can be onto another block or onto the table FOL would use quantifiers to express there's no block on top of some other block without quantifiers, PDDL requires Clear(x) predicate Move action schema must be complemented by MoveToTable to avoid errors with the Clear predicate since Clear(Table) is always true residual problem if bind y to Table in Move(b, x, y) search space grows, though answers still correct fix with a Block(m) predicate & adding Block(b) Block(y) to the preconditions for Move Classical Planning 22 Complexity & Classical Planning first, we distinguish 2 versions of the planning problems PlanSAT: is there a plan that solves the problem? Bounded PlanSAT: is there a solution of k or fewer steps? this can be used to find optimal plans so long as we don't allow functions the number of states is finite & both categories are decidable if we allow functions, PlanSAT becomes semi-decidable (may not terminate on unsolvable problems) though Bounded PlanSAT remains decidable even with functions Classical Planning 23 Complexity & Classical Planning sadly, in general, both problems are NP-hard putting restrictions on the expressiveness may help for example, disallowing -ve preconditions lets PlanSAT reduce to P fortunately for planners, many useful problems are easier than the worst case for blocks world & air cargo domains, Bounded PlanSAT is NP- Complete, while PlanSAT is in P! so, optimal planning is hard, sometimes sub-optimal planning is easy reminder of complexity category relationships 24 Planning with state-space search given the problem formulation we can use search algorithms to generate plans systematic heuristic search local search (while recording trail of states) search can be in either the forward or the backward direction recall the bi-directional example from search topic problems related to non-invertible successor functions Progression planners do a forward state-space search consider the effects of all possible actions in a given state Regression planners do a backward state-space search to achieve a goal consider what must have been true in the previous state from effects through actions to preconditions 25 Progression & Regression partial samples: plane moving example Classical Planning 26 Progression algorithm formulation as a state-space search problem Initial state = initial state of the planning problem literals not appearing are false Actions = apply those whose preconditions are satisfied add positive effects, delete negative Goal test = does the state satisfy the goal Step cost = for simplicity, each action costs 1 recall, functions are not permitted in the representation any graph search that is complete is a complete planning algorithm for example, A* Classical Planning 27 Progression algorithm Progression approach while complete is inefficient any applicable action is a possible next step typically its problems are related to branching factor (1) irrelevant actions just because the preconditions of an action are satisfied does not mean that it should be done (2) requirement for a good heuristic in order for improved efficiency of search Classical Planning 28 Progression: issues illustrate with the air cargo example assume 10 airports, 5 planes & 20 pieces of cargo at each problem: move all cargo at airport A to airport B solution is pretty simple, but uninformed search has serious complexity difficulties just considering planes: 50 x 9 = 450 possible flights suppose all planes & packages were at 1 airport: 10450 possible actions (200 x 50 + 50 x 9) if 2000 actions available on average, 200041 nodes in search graph fortunately, good (domain-independent) heuristics are possible, & can be derived automatically Classical Planning 29 Regression algorithm what about Regression planners issues: How do we determine predecessors? as we saw for bi-directional search, the problem may not allow predecessor determination (n-Queens) fortunately PDDL formulation facilitates it terminology: predecessor is g', action a, goal g g' = (g - Add(a)) Precond(a), since any effects added by a might not have been true previously & preconditions must have been met or could not execute a this process requires partially uninstantiated actions & states use standardized variable names (avoid clashes, retain generality of variables) implicitly quantify over these so 1 description summarizes possibility of using any appropriate ground term Classical Planning 30 Regression: relevant actions which action? forward uses applicable actions (their preconditions are met) backward uses relevant actions (they could be the last step leading to current state) relevant: at least 1 effect (+ve or -ve) must unify with a term in the (current) goal & there can't be any effect that negates a term in the goal process: use the action formed by substituting the most general unifier into the standardized action schema more formal definition of relevant actions if goal g contains literal gi & action schema A (standardized to produce A') has effect literal ej' where Unify (gi, ej') = & a' = SUBST(, A') & there's no effect in a' that is the negation of a literal in g, then a' is a relevant action toward g terminates when a predecessor is satisfied by the initial state Classical Planning 31 Progression & Regression status of the regression approach 1. there's a lower branching factor in the backward direction, the result of only considering relevant actions 2. but each state is really a set of states the state defined by true or false for the ground fluents, and the states not mentioned (see mid p 374) this representation of stages in the backward search as sets of states complicates the process of developing heuristics progression versus regression the greater availability of accurate heuristics for forward search has resulted in it being used in many more planning systems Classical Planning 32 Heuristics for planning the complexity of planning problems means that we need good heuristics below: 2 general approaches to finding admissible heuristics recall that an admissible h(n) (i.e. one that never overestimates cost) allows A* search for optimal solutions recall from our (informed search) discussion of finding heuristics as the optimal solution to a relaxed version of the problem from problem decomposition if we can make the subgoal independence assumption we can approximate the cost of solving a conjunction of subgoals as the sum of the costs of solving the corresponding subproblems (1) relaxation: search problem viewed as a graph allows (a) adding edges to make a path to a solution easier to find (b) merging states in an abstraction of the original problem so searching is in a space with fewer nodes Classical Planning 33 Heuristics by problem relaxation (a) adding edges to the search space (1) ignore all preconditions assume all actions are applicable in all states, so any goal fluent takes just a single step apparently #steps ≈ #unsatisfied goals, but note: some actions may achieve > 1 goal, & others may undo goals ignoring the latter, h(n) = min # actions for which the union of their effects satisfies the goal this is a version of the "set-cover problem", known to be NP-hard! a greedy algorithm yields set covering within log n of the true min (n is # literals in goal), but is not guaranteed admissible (2) ignore selected PRECONDs of actions as in the N-puzzle heuristics: misplaced tiles, manhattan distance the factored representation in action schemata for planning problems allows automated derivation of such heuristics (3) ignore delete lists if all goals & preconditions have only +ve literals (true for many problems anyway & others can be converted to this form) Classical Planning 34 Heuristics by problem relaxation (a) adding edges to search space (3) ignore delete lists with only +ve literals in all goals & preconditions then remove -ve literals from effects now steps are monotonic progress towards the goal w/o undoing but, it's still NP-hard to find an optimal solution, though it can be approximated in polynomial time with hill climbing summary adding edges relaxes a planning problem to yield heuristics but they're still expensive to calculate (b) reduce nodes by abstraction to merge states return to the air cargo example 10 airports, 50 planes, 200 packages means the general case has 5010 x 20050+10 or 10155 states a specific problem: say all pkgs are at 5 airports & all pkgs at any airport have 1 destination Classical Planning 35 Heuristics by problem relaxation (b) reduce nodes by abstraction to merge states in the air cargo example, in general, 10155 states a specific problem has all pkgs at 5 airports & pkgs at any airport have 1 destination an abstraction of this version omits At fluents except those re: 1 plane & 1 pkg at each of 5 airports, reducing # states to 510 x 55+10 or 1017 states so a solution is shorter & an admissible h(n) & is extensible to the original problem by adding more Load & Unload actions problem decomposition approaches for h(n)'s the relevant pattern is: subdivide problem solve subproblems combine solutions the subgoal independence assumption is useful in developing a heuristic as a sum of subgoal costs is too optimistic if there are -ve interactions between subplans is too pessimistic (inadmissible) when there are redundant actions in subplans 36 Heuristics by decomposition problem decomposition approaches for h(n)'s notation: represent the goal as a set of fluents G, where disjoint subsets of G are G1, …Gn, for which we find plans P1, … Pn & then use their costs to estimate cost for G Cost(Pi) is an estimate, a possible heuristic maxi Cost(Pi) is admissible, overly optimistic sumi Cost(Pi) is not generally admissible unless all Gi & Gj are independent (Pi does not affect preconds or goals of Pj) in which case the sum is admissible & more accurate Classical Planning 37 Heuristics by decomposition the challenge of abstraction is to find one that has a significantly lower total cost (defining the abstraction + doing the abstracted search + mapping back to original problem) than the original problem the pattern database techniques may be useful recall them from informed search discussion of heuristics amortize the cost of building the database over multiple problem solution instances the FF (FASTFORWARD) planning system is a hybrid it uses heuristics from a planning graph (our next discussion) it uses local search (storing plan steps) at a plateau or local maximum it does iterative deepening systematic search for a better state or gives up & restarts 38 Planning Graphs Planning Graphs are an alternative intermediate representation/data structure they are polynomial complexity approximations of full (exponential) trees of all states & actions they can't answer the question: is G reachable? though they are correct when they say it's not reachable they provide an estimate of the number of steps to G any such estimate is optimistic, so is an admissible heuristic planning graphs provide a possible basis for better search heuristics and they can also be used directly, for extracting a solution to a planning problem, by applying the GRAPHPLAN algorithm Classical Planning 39 Planning Graphs a planning graph is a directed graph organized in levels the levels of nodes correspond to time steps in a plan they consist of alternating S levels (in which the nodes are fluents that hold) & A levels (in which the nodes are actions that might be applicable) level S0 is the initial state each level is a set of literals Si or a set of actions Ai Si: all the literals that could be true at that time step depending on the actions executed at the previous steps Ai: all actions that could have PRECONDs satisfied at that step depending on which of the literals actually hold Classical Planning 40 Planning Graphs explanation regarding could on the previous slide a planning graph only captures a restricted subset of the possible -ve interactions so a literal might appear at a level earlier than it actually would (if it would at all), though it never appears too late despite this error, the level j of the first appearance is a good estimate of how difficult it is to achieve from the initial state refers to the approximate nature of the state/action lists in part, this allows efficient construction by recording a restricted subset of possible -ve interactions among actions note: planning graphs apply only for propositional problems action schemas for problems with variables can be propositionalized their advantages may make this worthwhile, despite the increased size of the problem description Classical Planning 41 Planning Graphs our simple example: "have your cake & eat it too" the problem description Init(Have(Cake)) Goal(Have(Cake) Eaten(Cake)) Action(Eat(Cake) PRECOND: Have(Cake) EFFECT: ¬Have(Cake) Eaten(Cake)) Action(Bake(Cake) PRECOND: ¬ Have(Cake) EFFECT: Have(Cake)) the corresponding planning graph Planning Graph Cake Example 42 start at level S0, determine action level A0 & next level S1 A0: all actions whose preconditions are satisfied in the previous level (initial state) actions are shown in rectangular boxes lines connect PRECONDs at S0 to EFFECTs at S1 also, for each literal in Si, there's a persistence action (square box) & line to it in the next level Si+1 level A0 contains the actions that could occur conflicts between actions are represented by arcs: mutual exclusion or mutex links Planning Graph Cake Example 43 level S1 contains all the literals that could result from picking any subset of actions in A0 so S1 is a belief state consisting of the set of all possible states each is a subset of literals with no mutex links between members conflicts between literals that cannot occur together are represented by the mutex links. the level generation process is repeated eventually consecutive levels are identical: leveling off Planning Graph Cake Example 44 mutex relation holds between 2 actions at a level when 1. inconsistent effects one action negates the effect of another 2. interference an effect of one action negates a precondition of the other 3. competing needs a precondition of one action is mutex with a precondition of the other Classical Planning Planning Graph Cake Example 45 mutex relation holds between 2 literals at a level when 1. one is the negation of the other 2. if each possible action pair that could achieve the literals is mutex (Have(Cake) & Eaten(Cake) at S1) this is termed inconsistent support Classical Planning 46 Planning Graphs & Heuristics Planning Graphs construction has complexity polynomial in the size of the planning problem given l literals, a actions, & a PG of n levels: O(n(a + l)2) the completed PG provides information about the problem & candidate heuristics 1. a goal literal g that does not appear in the final level cannot be achieved by any plan 2. the level cost, the level at which a goal literal first appears, is useful as a cost estimate of achieving that goal literal note that level cost is admissible, though possibly inaccurate since it counts levels, not actions we could find a better alternative level cost by using a serial planning graph variation, restricted to one action per level mutex links between every pair of actions except persistence actions Classical Planning 47 Planning Graphs & Heuristics Planning Graph provides possible heuristics for the cost of a conjunction of goals 1. max-level: highest level of any conjunct in the goal admissible, possibly not accurate 2. level sum: the sum of level costs of conjuncts in the goal incorporates the subgoal independence assumption so may be inadmissible to degree the assumption does not hold 3. set-level: level where all goal conjuncts are present without mutex links this one dominates the max level heuristic & is good where there are interactions among subplans Classical Planning 48 Planning Graphs & Heuristics a Planning Graph is a relaxed version of the problem if a goal literal g does not appear, no plan can achieve it, but if it does appear, is not guaranteed to be achievable why? the PG only captures pairwise conflicts & there could be higher order conflicts likely not worth the computational expense of checking for them similar to Constraint Satisfaction Problems where arc consistency was a valuable pruning tool 3-consistency or even higher order consistency would have made finding solutions easier but was not worth the additional work an example where PG fails to indicate unsolvable problem blocks world problem with goal of A on B, B on C, C on A any pair of subgoals are achievable, so no mutexes problem only fails at stage of searching the PG Classical Planning 49 The GRAPHPLAN Algorithm the GRAPHPLAN algorithm generates the Planning Graph & extracts a solution from it function GRAPHPLAN(problem) return solution or failure graph INITIAL-PLANNING-GRAPH(problem) goals CONJUNCTS(problem. GOAL) nogoods an empty hash table for tl = 0 to do if goals all non-mutex in St of graph then solution EXTRACT-SOLUTION(graph, goals, NUMLEVELS(graph), nogoods) if solution failure then return solution if graph and nogoods have both leveled off then return failure graph EXPAND-GRAPH(graph, problem) Classical Planning 50 Example: Spare Tire Problem recall the spare tire problem Init(At(Flat, Axle) At(Spare, Trunk)) Goal(At(Spare, Axle)) Action(Remove(Spare, Trunk) PRECOND: At(Spare, Trunk) EFFECT: ¬At(Spare, Trunk) At(Spare, Ground)) Action(Remove(Flat, Axle) PRECOND: At(Flat, Axle) EFFECT: ¬At(Flat, Axle) At(Flat, Ground)) Action(PutOn(Spare, Axle) PRECOND: At(Spare, Ground) ¬At(Flat, Axle) EFFECT: At(Spare, Axle) ¬At(Spare, Ground)) Action(LeaveOvernight PRECOND: EFFECT: ¬ At(Spare, Ground) ¬ At(Spare, Axle) ¬ At(Spare, Trunk) ¬ At(Flat, Ground) ¬ At(Flat, Axle) ) Classical Planning GRAPHPLAN Spare Tire Example 51 Notes: this figure shows the complete Planning Graph for the problem arcs show mutex relations but arcs between literals are omitted to avoid clutter it also omits unchanging +ve literals (for example, Tire(Spare)) & irrelevant -ve literals bold boxes & links indicate the solution plan Classical Planning GRAPHPLAN Spare Tire Example 52 S0 is initialized to 5 literals from the problem initial state and the CWA literals no goal literal in S0 so EXPAND-GRAPH add actions those with preconditions satisfied in S0 also adds persistence actions for literals in S0 adds the effects at level S1, analyzes & adds mutex relations repeat until the goal is in level Si or failure Classical Planning GRAPHPLAN Spare Tire Example 53 EXPAND-GRAPH adds constraints: mutex relations inconsistent effects (action x vs action y) Remove(Spare, Trunk) & LeaveOvernight: At(Spare, Ground) & ¬At(Spare, Ground) interference (effect negates a precondition) Remove(Flat, Axle) & LeaveOvernight: At(Flat, Axle) as PRECOND & ¬At(Flat, Axle) as EFFECT competing needs (mutex preconditions) PutOn(Spare, Axle) & Remove(Flat, Axle): At(Flat, Axle) & ¬At(Flat, Axle) inconsistent support (actions to produce literals are mutex) in S2, At(Spare, Axle) & At(Flat, Axle) 54 Reminder: GRAPHPLAN Algorithm the GRAPHPLAN algorithm both generates the Planning Graph & extracts a solution from it function GRAPHPLAN(problem) return solution or failure graph INITIAL-PLANNING-GRAPH(problem) goals CONJUNCTS(problem. GOAL) nogoods an empty hash table for tl = 0 to do if goals all non-mutex in St of graph then solution EXTRACT-SOLUTION(graph, goals, NUMLEVELS(graph), nogoods) if solution failure then return solution if graph and nogoods have both leveled off then return failure graph EXPAND-GRAPH(graph, problem) Classical Planning GRAPHPLAN Spare Tire Example 55 in S2, the goal literals exist, & are not mutex just 1 goal literal so obviously not mutex with any other goal since a solution may exist, EXTRACT-SOLUTION tries to find it EXTRACT-SOLUTION may search backwards for a solution initial state: last level of the PG + goals of the planning problem actions from Si select a set of conflict-free actions in Ai-1 with effects covering the goals conflict free = no 2 actions are mutex & no pair of preconditions are mutex goal: reach a state at level S0 such that all goals are satisfied cost: 1 for each action 56 GRAPHPLAN Solutions if EXTRACT-SOLUTION fails at that point it records (level, goals) as a "no-good" subsequent calls can fail immediately if they require the same goals at that level we already know planning problems are computationally hard require good heuristics greedy search with level cost of literals as a heuristic works well 1. pick literal with highest level cost 2. to achieve it, pick actions with easier preconds action with smallest sum (or max) of level costs for its preconds 57 GRAPHPLAN Solutions alternative to backward search for a solution EXTRACT-SOLUTION could formulate a Boolean CSP variables are actions at each level values are Boolean: an action is either in or out of the plan constraints are mutex relations & the need to satisfy each goal & precondition Classical Planning 58 GRAPHPLAN Termination does GRAPHPLAN terminate? does it stop & return failure if there's no solution the general rationale is presented here for some proof details see the textbook & its Ghallab reference background recall that levelled off means consecutive PG levels are identical now note that a graph may level off before a solution can be found, on a problem for which there is a solution example of 2 airports, 1 plane & n pieces of cargo at one of them, plan required to move all n to the other airport but only 1 piece at a time fits in the plane graph levels off at level 4, from which full solution can't be extracted (that would require 4n - 1 steps) we need to take account of the no-goods (goals that were not achievable) as well these may decline even after the graph levels off 59 GRAPHPLAN Termination does GRAPHPLAN terminate? given that no-goods might continue to decrease after the graph levels off we now require a proof that both the graph & no-goods will level off literals increase monotonically once a literal appears, its persistence action causes it to stay actions increase monotonically once a literal that is a precondition appears, the action stays mutexes decrease monotonically though the graph simplifying conventions may not show it if 2 actions are mutex at Ai, they are also mutex at all previous levels where they appear 60 GRAPHPLAN Termination does GRAPHPLAN terminate? no-goods decrease monotonically if a set of goals is not achievable at level i, they are not achievable at any previous level so literals & actions increase monotonically given a finite number of actions and literals, they will level off and mutexes & no-goods decrease monotonically since there cannot be fewer than 0, they must level off if the graph & no-goods have levelled off & a goal is missing or is mutex with another goal, the algorithm can stop & return failure 61 Alternative Planning Approaches the following slides outline brief discussions of 4 alternative approaches to solving planning problems 1. convert the PDDL description into a form solvable by a propositional logic satisfiability algorithm: SATPlan 2. convert into a situational calculus representation solvable by First-Order Logic inference techniques 3. represent & solve as a Constraint Satisfaction Problem 4. use a "plan refinement" approach to searching in a space of partially ordered plans Classical Planning 62 Planning with Propositional Logic 1. form a propositional logic satisfiability problem for submission to SATPlan iteratively tries plans of increasing length (increasing number time steps) up to the Tmax parameter value, so finds the shortest plan, if one exists the SATPlan algorithm finds models for a (very long) PL sentence that includes initial state, goal, successor-state axioms, precondition axioms, & action-exclusion axioms assigns true to the actions that are part of the correct plan & false to the others if the planning is unsolvable the sentence will be unsatisfiable any model satisfying the sentence will be a valid plan an assignment that corresponds to an incorrect plan will not be a model because of inconsistency with the assertion that the goal is true Classical Planning 63 Planning with Propositional Logic translation from PDDL to PL form for SATPlan the process involves 6 steps step 1: propositionalize actions, replacing each schema with a set of ground actions (constants substituted for variables) step 2: define an initial state asserting F0 for all fluents in the initial state, ¬F0 for all fluents not in the initial state step 3: propositionalize the goal - for each variable replace literals containing it with a disjunction over constants step 4: include successor-state axioms for each fluent Ft+1 ActionCausesFt (Ft ¬ActionCausesNotFt), where ActionCausesF is disjunction of all ground actions with F in their add List ActionCausesNotF is disjunction of all ground actions with F in their delete list air cargo example: axioms are required for each plane, airport and time step! Classical Planning 64 Planning with Propositional Logic the 6 steps from PDDL to SATPlan format step 5: add precondition axioms (for every ground action A, add At PRE(A)t step 6: add action exclusion axioms to say every action is distinct from every other action submit to SATPlan Classical Planning 65 The SATPLAN Algorithm the algorithm note that in AIMA 3e, this is first presented in the earlier section on propositional logic agents function SATPLAN(init, transition, goal, Tmax) returns solution or failure inputs: init, transition, goal form a description of the planning problem Tmax, an upper limit to the plan length for t = 0 to Tmax do cnf TRANSLATE-TO-SAT(init, transition, goal, t ) model SAT-SOLVER(cnf) if model is not null then return EXTRACT-SOLUTION(model) return failure Classical Planning 66 Planning with Propositional Logic some explanatory notes distinct propositions for assertions about each time step superscripts denote the time step: At(P1,SFO)0 At(P2,JFK)0 PDDL includes the Closed World Assumption, so conversion involves the need to specify which propositions are not true ¬At(P1,JFK)0 ¬At(P2,SFO)0 unknown propositions are left unspecified goal is associated with some particular time-step: which one? Classical Planning 67 Planning with Propositional Logic some explanatory notes we can't know in advance how many steps are required to achieve a goal so, to determine the time step T where the goal will be reached, the algorithm loops until success or some arbitrary time step limit repeat start at T = 0 Assert At(P1,JFK)0 At(P2,SFO)0 on failure: try T = 1 Assert At(P1,JFK)1 At(P2,SFO)1 add terms for next time step until a solution is found or some max path length Tmax is exceeded: setting Tmax assures termination Classical Planning 68 Models the pre-condition axioms avoid models with illegal actions e.g. for T = 1 Fly(P1, SFO, JFK)0 Fly(P1, JFK, SFO)0 Fly(P2, JFK, SFO)0 the second action is infeasible, precluded if we include the appropriate pre-condition axiom: Fly(P1,JFK,SFO)0 At(P1,JFK)0 since At(P1, JFK)0 false in initial state, Fly(P1, JFK, SFO)0 false in any model Classical Planning 69 Models without action-exclusion axioms a model might include actions, for example having 1 plane fly to two destinations in a time step Fly(P1, SFO, JFK)0 Fly(P2, JFK, SFO)0 Fly(P2, JFK, LAX)0 though the second P2 action is infeasible, it requires the added action-exclusion axioms to avoid such solutions ¬(Fly(P2,JFK,SFO)0 Fly(P2,JFK,LAX)0) Classical Planning 70 Planning in First-Order Logic alternative 2: convert to the Situation Calculus format of FOL & apply FOL inferencing recall Situation Calculus as a topic in the 352 Knowledge Representation discussion of time & action PDDL omits quantifiers which limits expressiveness but helps control complexity of algorithms applied to it PL representation also has limitations like the requirement that a "timestamp" becomes part of each fluent Situation Calculus replaces explicit timestamps by using situations that correspond to a sequence or history of actions 1. situations initial state is a situation if s is a situation & a an action, RESULT(s, a) is also a situation situations correspond to sequences or history of actions 2 situations are the same only if their start & actions are the same Classical Planning 71 Planning in First-Order Logic convert to FOL Situation Calculus 2. fluents are functions or relations that vary over situations syntax: situation is conventionally the last argument At(x, l, s) is a relational fluent, true when object x is at location l in situation s & Location(x, s) is a functional fluent such that Location(x, s) = l in the same situation 3. a possibility axiom describes preconditions of each action (s) Poss(a, s) indicates when an action can be taken is a formula giving preconditions Alive(Agent, s) Have(Agent, Arrow, s) Poss(Shoot, s) 4. successor-state axioms for each fluent indicate what happens, depending on the action taken here are the general form & an example from Wumpus World Action is possible (Fluent is true in result state Action's effect made it true V It was true before & the action left it unchanged) Poss(a, s) (At(Agent, y, Result(a, s)) a = Go(x, y) V (At(Agent, y, s) Λ a Go(y, z))) 72 Planning in First-Order Logic express problem using FOL Situation Calculus 5. unique action axioms required to allow deductions such as a Go(y, z) for each pair of action names Ai & Aj there's an axiom to say they are different: Ai(x, …) Aj(y, …) plus, for each action name Aj an axiom indicates 2 uses are only equal if all arguments are equal Ai (x1, …, xn) = Ai(y1, …, yn) x1 = y1 … xn = yn 6. the solution is a situation (action sequence) that satisfies the goal unfortunately, no major planning systems use situation calculus basic efficiency issues of FOL inference compounded by lack of good heuristics for planning with SC 73 Planning: Constraint Satisfaction we noticed in discussions in 352 the similarities between Constraint Satisfaction Problems & boolean satisfiability conversion to a CSP is similar to that of conversion to a propositional logic satisfiability problem one difference is that it only requires a single variable Actiont at each time step: its domain is the set of all possible actions thus we also don't need the action exclusion axioms Classical Planning 74 Partial-order planning plans may be developed through refinement of partially ordered plans approaches describe so far, including both Progression and Regression planning, are totally ordered forms of plan search they cannot take advantage of problem decomposition decisions must be made on how to sequence the actions for all the subproblems though some are independent (in the air cargo problem, loading cargo at 2 different airports) a least commitment strategy would, where possible, delay choices during search partially ordered planning implements this adds actions to plans without committing to an absolute time/step, instead dealing mainly with relative constraints action A must precede action B partially ordered plans are generated by a search in the space of plans rather than through the state space so the planning operators are different from the real-world actions Classical Planning 75 A plan to put your shoes on an illustration from 2e notes Goal(RightShoeOn LeftShoeOn) Init() Action(RightShoe, PRECOND: RightSockOn EFFECT: RightShoeOn) Action(RightSock, PRECOND: EFFECT: RightSockOn) Action(LeftShoe, PRECOND: LeftSockOn EFFECT: LeftShoeOn) Action(LeftSock, PRECOND: EFFECT: LeftSockOn) Classical Planning 76 A plan to put your shoes on a partially-ordered planner can place two actions into a plan without specifying which comes first sample POP and corresponding totally ordered plans Classical Planning 77 POP as a search problem states are (mostly unfinished) plans the empty plan contains only the special start & finish actions each state (plan) is a partially ordered structure it consists of a set of actions (the steps of the plan) and a set of ordering constraints of the form Before(ai, aj) meaning that one action occurs before another graphically, actions are shown as boxes, ordering constraints as arrows the starting point is an empty plan consisting of the initial state & goal, with no actions between them the search process adds an action to correct a flaw in the plan or if it can't, it backtracks flaws keep a partial plan from being a solution as an example, the next slide shows POPs corresponding to the "flat tire" planning problem, both the empty plan and a solution 78 POP for the flat tire problem at the top the empty plan & below it, the solution plan Remove(spare, trunk), Remove(flat, axle) can be done in either order, so long as both are done before PutOn(spare, axle) 79 POP for the flat tire problem search in plan space seeks to add to the plan to correct a flaw flaw: not achieving At(spare, axle) in the flat tire empty plan adding PutOn(spare, axle) fixes that, but introduces new flaws in the form of its (unachieved) preconditions process repeats, backtracking as necessary each step makes the least commitment (ordering constraints & variable bindings) in order to fix a flaw Remove(spare, trunk) must be before PutOn(spare, axle) but no other ordering requirements Classical Planning 80 Partial-Order Planning was previously popular because of explicitly representing independent subproblems but it lacks an explicit representation of states in the state- transition model & was superseded by progression planners with heuristics that allowed discovery of independent subproblems nevertheless it's useful for tasks like operations scheduling, enhanced with domain specific heuristics also appropriate where humans must understand the resulting plans plan refinement approach is easier to understand & verify example: spacecraft/rover plans are generated by this approach Classical Planning 81 Solving POP POP in action the initial plan contains only the Start & Finish steps Start step/action has the initial state description as its effect Finish step/action has the goal description as its precondition the single ordering constraint Before (Start, Finish) there's a set of open preconditions: all those in Finish the successor function picks an open precondition p on an action B and generates a successor plan using an action A that achieves p then does goal test any remaining open preconditions? Classical Planning 82 Process summary operators on partial plans may add link from an existing action to an open precondition add a step (a new action) to fulfill an open condition order one step wrt another to remove possible conflicts plan generation involves gradually moving from incomplete/vague plans to complete/correct plans plan construction may require backtracking if an open condition is unachievable or conflict is unresolvable Classical Planning 83 Classical Planning Summary approaches rely on techniques of search or logic search to assemble a plan or to constructively prove one exists handling the inherent complexity of these problems is a key identifying independent subproblems provides potential for exponential speedup, but full decomposability is inconsistent with -ve interactions between actions heuristically guided search can gain efficiency even if subproblems are not completely independent some problems allow efficient solution by ruling out -ve interactions serializable subgoals = there's a subgoal ordering for a solution that avoids having to undo any subgoals tower problems in Blocks World are serializable if subgoals are ordered bottom to top 84 Classical Planning Summary serializable subgoals tower problems in Blocks World are serializable if subgoals are ordered bottom to top NASA spacecraft control commands are serializable (unsurprising since the control systems are designed for easy control) when problem is serializable, a planner can seriously prune the planning search what's next? where does further progress come from? probably not simply factored representations & propositional approaches combine efficient heuristics + first-order, hierarchical representations Classical Planning 85 Classical Planning Summary PDDL (Planning Domain Definition Language) initial & goal states as conjunctions of literals actions as lists of preconditions & effects search in state space may be forward (Progression) or backward (Regression) derive heuristics from subgoal independence assumption & relaxed versions of the problems Planning Graphs build from the initial state to the goal with alternating incremental sets of literals or actions representing possibilities for each time step, & encoding mutexes between inconsistent pairs of literals or actions layers are supersets of literals/actions possible at that time step the basis of good heuristics for state space searches or can yield a plan by application of the GRAPHPLAN algorithm Classical Planning 86 Classical Planning Summary possible alternative approaches FOL inference on a situation calculus knowledge base conversion to boolean satisfiability problem for SATPlan encoding as a Constraint Satisfaction Problem search backwards over a space of Partially Ordered Plans what's best? there's no agreement on a single best representation or algorithm we saw one hybrid that performs well a particular domain may provide a best fit to one particular technique Classical Planning