Model-based Programming of
Cooperating Explorers
Brian C. Williams
CSAIL
Dept. Aeronautics and Astronautics
Massachusetts Institute of Technology
Programming Long-lived Embedded Systems
Large collections of devices must work in concert to achieve goals
• Devices indirectly observed and controlled
• Need quick, robust response to anomalies throughout life
• Must manage large levels of redundancy
Coordination Recapitulated At The
Level of Cooperating Explorers
(Courtesy of Jonathan How. Used with permission.)
Coordination Issues Increase For
Dexterous Explorers
(Courtesy of Frank Kirchner. Used with permission.)
Outline
• Model-based Programming
• Autonomous Engineering Operations
– An Example
– Model based Execution
– Fast Reasoning using Conflicts
• Cooperating Mobile Vehicles
– Predictive Strategy Selection
– Planning Out The Strategy
Approach
Elevate programming and operation to
system-level coaching.
Model-based Programming
– State Aware: Coordinates behavior at the level
of intended state.
Model-based Execution
– Fault Aware: Uses models to achieve intended
behavior under normal and faulty conditions.
Why Model-based Programming?
Polar Lander Leading Diagnosis:
• Legs deployed during descent.
• Noise spike on leg sensors
latched by software monitors.
• Laser altimeter registers 40m.
• Begins polling leg monitors to
determine touch down.
• Read latched noise spike as touchdown.
Objective: Support programmers
• Engine shutdown at ~40m. with embedded languages that
avoid these mistakes, by
Programmers often make reasoning about hidden state
commonsense mistakes when automatically.
reasoning about hidden state.
Reactive Model-based
Programming Language (RMPL)
Model-based Programs
Interact Directly with State
Embedded programs interact with Model-based programs
plant sensors and actuators: interact with plant state:
• Read sensors • Read state
• Set actuators • Write state
Model-based
Embedded Program Embedded Program
Obs Cntrl S’
Model-based Executive
S Obs Cntrl
Plant S
Plant
Programmer must map between Model-based executive maps
state and sensors/actuators. between state and
RMPL Model-based Program Titan Model-based Executive
Control Program
• Executes concurrently
• Preempts Generates target goal states
• Queries (hidden) states conditioned on state estimates
• Asserts (hidden) state
System Model State estimates State
goals
Tracks
likely Tracks least
Valve plant states cost goal states
0.01 Stuck
Open open
0. 01
Open Close
0. Stuck
01
Closed 0. 01 closed Observations Command
inflow = outflow = 0 s
Plant
Outline
• Model-based Programming
• Autonomous Engineering Operations
– An Example
– Model based Execution
– Fast Reasoning using Conflicts
• Cooperating Mobile Vehicles
– Predictive Strategy Selection
– Planning Out The Strategy
Motivation
Mission-critical sequences:
• Launch & deployment
• Planetary fly-by
• Orbital insertion
• Entry, descent & landing
images courtesy
of NASA
(Courtesy of Mitch Ingham. Used with permission.)
Mars Entry Example
engine to standby
planetary approach
switch to
inertial nav rotate to entry-orient
& hold attitude
separate
lander
(Courtesy of Mitch Ingham. Used with permission.)
Mars Entry Example
engine to standby
planetary approach
switch to
inertial nav rotate to entry-orient
& hold attitude
Descent engine to “standby”: separate
lander
(Courtesy of Mitch Ingham. Used with permission.)
Mars Entry Example
engine to standby
planetary approach
switch to
inertial nav rotate to entry-orient
& hold attitude
separate
Spacecraft approach: lander
• 270 mins delay
• relative position wrt Mars not
observable
• based on ground computations
of cruise trajectory
(Courtesy of Mitch Ingham. Used with permission.)
Mars Entry Example
engine to standby
planetary approach
switch to
inertial nav rotate to entry-orient
& hold attitude
separate
Switch navigation mode: lander
“Inertial” = IMU only
(Courtesy of Mitch Ingham. Used with permission.)
Mars Entry Example
engine to standby
planetary approach
switch to
inertial nav rotate to entry-orient
& hold attitude
separate
Rotate spacecraft: lander
• command ACS to entry orientation
(Courtesy of Mitch Ingham. Used with permission.)
Mars Entry Example
engine to standby
planetary approach
switch to
inertial nav rotate to entry-orient
& hold attitude
separate
Rotate spacecraft: lander
• once entry orientation achieved,
ACS holds attitude
(Courtesy of Mitch Ingham. Used with permission.)
Mars Entry Example
engine to standby
planetary approach
switch to
inertial nav rotate to entry-orient
& hold attitude
Separate lander from cruise stage: separate
lander
cruise
stage
pyro lander
latches stage
(Courtesy of Mitch Ingham. Used with permission.)
Mars Entry Example
engine to standby
planetary approach
switch to
inertial nav rotate to entry-orient
& hold attitude
Separate lander from cruise stage: separate
lander
• when entry orientation achieved,
fire primary pyro latch
cruise
stage
pyro lander
latches stage
(Courtesy of Mitch Ingham. Used with permission.)
Mars Entry Example
engine to standby
planetary approach
switch to
inertial nav rotate to entry-orient
& hold attitude
Separate lander from cruise stage: separate
lander
• when entry orientation achieved,
fire primary pyro latch
cruise
stage
lander
stage
(Courtesy of Mitch Ingham. Used with permission.)
Mars Entry Example
engine to standby
planetary approach
switch to
inertial nav rotate to entry-orient
& hold attitude
Separate lander from cruise stage: separate
lander
• in case of failure of primary latch,
fire backup pyro latch
cruise
stage
lander
stage
(Courtesy of Mitch Ingham. Used with permission.)
Mars Entry Example
engine to standby
planetary approach
switch to
inertial nav rotate to entry-orient
& hold attitude
Separate lander from cruise stage: separate
lander
• in case of failure of primary latch,
fire backup pyro latch
cruise
stage
lander
stage
(Courtesy of Mitch Ingham. Used with permission.)
Mars Entry Example
engine to standby
planetary approach
switch to
inertial nav rotate to entry-orient
& hold attitude
• simple state-based control
specifications separate
lander
• models are writable/inspectable by
systems engineers
• handle timed plant & control
behavior
• automated reasoning through low-
level plant interactions
• fault-aware (in-the-loop recoveries)
(Courtesy of Mitch Ingham. Used with permission.)
Descent Example
Turn camera off and engine on
EngineA EngineB EngineA EngineB
Science Camer Science Camera
Model-based Program
Control program specifies OrbitInsert()::
state trajectories: (do-watching ((EngineA = Thrusting) OR
(EngineB = Thrusting))
• fires one of two engines (parallel
(EngineA = Standby)
• sets both engines to „standby‟
(EngineB = Standby)
• prior to firing engine, camera must be (Camera = Off)
turned off to avoid plume contamination (do-watching (EngineA = Failed)
(when-donext ( (EngineA = Standby) AND
• in case of primary engine failure, fire (Camera = Off) )
backup engine instead (EngineA = Thrusting)))
(when-donext ( (EngineA = Failed) AND
Plant Model describes behavior (EngineB = Standby) AND
of each component: (Camera = Off) )
– Nominal and Off nominal (EngineB = Thrusting))))
– qualitative constraints
– likelihoods and costs
Plant Model
component modes…
described by finite domain constraints on variables…
deterministic and probabilistic transitions
cost/reward
Engine Model Camera Model
(thrust = zero) AND
(power_in = zero) Off (power_in = zero) AND
(shutter = closed) Off
off- standby-
cmd
(thrust = zero) AND cmd Failed
(power_in = nominal)
turnoff – turnoff –
cmd
Standby cmd
standby- fire-
(thrust = full) AND cmd cmd
(power_in = nominal) (power_in = nominal) AND
(shutter = open) On
Firing
one per component … operating concurrently
Example: The model-based program sets engine = thrusting, and the
deductive controller . . . .
Mode Estimation Mode Reconfiguration
Oxidizer tank Fuel tank
Selects valve
Deduces that configuration;
thrust is off, and plans actions
the engine is healthy to open Deduces that a valve
six valves failed - stuck closed
Determines valves
on backup engine that
will achieve thrust, and
plans needed actions
Mode Reconfiguration Mode Estimation
Outline
• Model-based Programming
• Autonomous Engineering Operations
– An Example
– Model based Execution
– Fast Reasoning using Conflicts
• Cooperating Mobile Vehicles
– Predictive Strategy Selection
– Planning Out The Strategy
Modeling Plant Dynamics using Probabilistic
Concurrent, Constraint Automata (PCCA)
Compact Encoding:
– Concurrent probabilistic transitions
– State constraints between variables
Engine Model Camera Model
(thrust = zero) AND
(power_in = zero) AND
(power_in = zero) (shutter = closed) Off
off- standby-
cmd cmd Failed
turnoff – turnoff –
Off
Standby cmd cmd
standby- fire-
(thrust = full) AND cmd cmd
(thrust = zero) AND
(power_in = nominal) (power_in = nominal) AND
(power_in = nominal)
(shutter = open) On
Firing
Typical Example (DS1 spacecraft):
– 80 Automata, 5 modes on average
– 3000 propositional variables, 12,000 propositional clauses
•Assigns a value to each • A set of concurrent transitions,
variable (e.g.,3,000 vars). one per automata (e.g., 80).
•Consistent with all state •Previous & Next states
constraints (e.g., 12,000). consistent with source & target
of transitions
RMPL Model-based Program Titan Model-based Executive
Control Program Control Sequencer:
Executes concurrently
Generates goal states
Preempts
Asserts and queries states conditioned on state estimates
Chooses based on reward
State estimates State goals
Mode
Estimation: MAINTAIN(EAR OR EBR)
Orbitlnsert(): Tracks likely LEGENO:
EAS (EngineA=Standby)
(do-watching((EngineA=Firing)OR EAS EAF (EngineA=Failed)
(EngineB=Firing))
States EAR (EngineA=Firing)
EBS (EngineB=Standby)
EBS
EBF (EngineB=Failed)
(parallel EBR (EngineB=Firing)
(EngineA=Standby) CO CO (Camera=Off)
(EngineB=Standby)
MAINTAIN(EAF)
(Camera=Off) Observations (EAS AND CO)
(do-watching(EngineA=Failed)
(when-donext((EngineA=Standby)AND Plant EAS AND CO
EAR
(Camera=Off))
(EngineA=Firing))) (EAF AND EBS AND CO)
(when-donext((EngineA=Failed)AND
(EngineB=Standby)AND EAF AND EBS
EBR
AND CO
(Camera=Off))
(EngineB=Firing)))) hierarchical constraint
automata on state s
RMPL Model-based Program Titan Model-based Executive
Control Program Control Sequencer:
Executes concurrently
Generates goal states
Preempts
Asserts and queries states conditioned on state estimates
Chooses based on reward
State estimates State goals
Mode Mode
Estimation: Reconfiguration:
Tracks likely Tracks least-cost
States state goals
Commands
Observations
Valve fails
stuck closed
Plant Fire backup
engine
X0
X0
X1 X1
XN-1 XN-1 XN
X X0 X1
XN-1 XN
X0 X1 XN-1 XN
N
S T
least cost reachable
Current Belief State First Action goal state
arg max PT(m') arg min RT*(m')
s.t. M(m')^O(m') is satisfiable s.t. M(m') entails G(m')
s.t. M(m') is satisfiable
State estimates State goals
OpSat:
Mode Mode
arg min f(x) Mode
Estimation: Reconfiguration:
s.t. C(x) is satisfiable Tracks likely Tracks least-cost
D(x) is unsatisfiable States state goals
Commands
Observations
Valve fails
stuck closed
Plant Fire backup
engine
X0 X1 XN-1 XN
X XN
N
X0 X1 XN-1 XN
S T
least cost reachable
Current Belief State First Action goal state
Outline
• Model-based Programming
• Autonomous Engineering Operations
– An Example
– Model based Execution
– Fast Reasoning using Conflicts
• Cooperating Mobile Vehicles
– Predictive Strategy Selection
– Planning Out The Strategy
Diagnosis Formulation
Consistency-based Diagnosis: Given symptoms, find
diagnoses that are consistent with symptoms.
Handle Novel Failures by Suspending Constraints:
Make no presumptions about faulty component
behavior.
Symptom
1A 1 1
Or1 X 0
B F
1 AND1
1
1C Or2 Y 1
D G
1 AND2
Z
0E Or3
Diagnosis Formulation
Consistency-based Diagnosis: Given symptoms,
find diagnoses that are consistent with symptoms.
Handle Novel Failures by Suspending Constraints:
Make no presumptions about faulty component
behavior.
Symptom
1A 1 1
X F 0
B
1 AND1
1
1C Or2 Y 1
D G
1 AND2
Z
0E Or3
Fast Reasoning Through Conflict
• When you have eliminated the impossible,
whatever remains, however improbable, must
be the truth.
-Sherlock Holmes. The Sign of the Four.
1. Test Hypothesis
2. If inconsistent, learn reason for inconsistency (a
Conflict).
3. Use conflicts to leap over similarly infeasible options to
next best hypothesis.
Compare Most Likely Hypothesis
to Observations
Helium tank
Oxidizer tank Fuel tank
Flow1 = zero Pressure2 = nominal
Pressure1 = nominal
Main
Engines
Acceleration = zero
It is most likely that all components are okay.
Isolate Conflicting Information
Helium
tank
Oxidizer tank Fuel tank
Flow1= zero
Main
Engines
The red component modes conflict with the model
and observations.
Leap to the Next Most Likely Hypothesis
that Resolves the Conflict
Helium
tank
Oxidizer tank Fuel tank
Flow1= zero
Main
Engine
s
The next hypothesis must remove the conflict
New Hypothesis Exposes Additional Conflicts
Helium
tank
Oxidizer tank Fuel tank
Pressure1 = nominal Pressure2 = nominal
Main
Acceleration = zero Engine
s
Another conflict, try removing both
Final Hypothesis Resolves all Conflicts
Helium
tank
Oxidizer tank Fuel tank
Pressure1 = nominal Pressure2= nominal
Flow1 = zero Flow2 = positive
Main
Acceleration = zero Engine
s
Implementation: Conflict-directed A* search.
A*
Increasin
g
Cost
Infeasible
Feasibl
e
Conflict-directed A*
Increasin
g
Cost
Infeasible
Feasibl
e
Conflict-directed A*
Increasing
Cost
Conflict 1
Infeasible
Feasible
Conflict-directed A*
Increasing
Cost Conflict 1
Infeasible
Conflict 2
Feasible
• Conflicts are mapped to
feasible regions as
implicants
(Kernel Assignments)
•Want kernel assignment
containing the best cost state.
Outline
• Model-based Programming
• Autonomous Engineering Operations
– An Example
– Model based Execution
– Fast Reasoning using Conflicts
• Cooperating Mobile Vehicles
– Predictive Strategy Selection
– Planning Out The Strategy
Coordination is Recapitulated at the
Level of Cooperating Explorers
(Courtesy of Jonathan How. Used with permission.)
Traditional Robot Architectures
Goals Action Planning and Scheduling
descriptions
Goal-directed Goal-directed Execution
Programs
Reactive Behaviors
• Explicit human guidance is at the lowest levels
RMPL for Robotics
Reactive Model-based
Programming Model-based Executive
Language(PMPL)
Control Goal-directed
Programs Execution
Plant Deductive
Models Controllers
What types of reasoning should the programmer/operator guide?
•State/mode inference •Method selection
•Machine control •Roadmap path planning
•Scheduling •Optimal trajectory planning
•Generative temporal planning
RMPL Model-based Program Kirk Model-based Executive
Control Program
z Executes concurrently Control Sequencer
z Preempts Predictive Strategy Selection
z non-deterministic choice
z A[l,u] timing Dynamic Scheduling
z A at l location
Ensures Safe Execution
Environment location location goals
Model TT OO estimates
Deductive
HOME WV N
OO EN
HOME E
COLLECTION POINT RENDEZVOUS
COLLECTION POINT
Landing Site: Enroute
Landing Site: ABC RENDEZVOUS
ABC COLLECTION POINT
State via Path
Achieves Controller Planning
Enroute Diverge
Landing Site: XYZ
Landing Site: XYZ Diverge
SCIENCE AREA 1 SCIENC
SCIENCE AREA 1‟
SCIENCE AREA E AREA
1SCIENCE AREA 1‟
1
SCIENCE AREA 3
SCIENCE
AREA 3
Estimates using Localization
Observations Commands
Plant
Example Scenario
TW ON
O E
HOME
Landing Site: ABC COLLECTION POINT Enroute RENDEZVOU
S
Landing Site: Diverge
XYZ SCIENCE AREA 1‟
SCIENCE AREA 1
Properties: SCIENCE AREA 3
• Mars rover operators have been leery of generative planners.
• z Are more comfortable with specifying contingencies.
• z Want strong guarantees of safety and robust to uncertainty.
• z Global path planning is on the edge
Extend RMPL with planner-like capabilities ..except planning
Reactive Model-based Programming
Idea: To describe group behaviors, start with concurrent language:
• p • Primitive activities
• If c next A • Conditional execution
• Unless c next A • Preemption
• A, B • Full concurrency
• Always A • Iteration
•Add temporal constraints:
• A [l,u] • Timing
•Add choice (non-deterministic or decision-theoretic):
• Choose {A, B} • Contingency
•Parameterize by location:
• A at [l]
Example Enroute Activity:
Enroute
Corridor 2
Rescue Area
Rendezvou
s
Corridor 1
RMPL for Group-Enroute
Temporal Constraints:
Group-Enroute()[l,u] = {
choose {
do {
Group-Fly-
Path(PATH1_1,PATH1_2,PATH1_3,RE_POS)[l*90%,u*90%];
} maintaining PATH1_OK,
do {
Group-Fly-
Path(PATH2_1,PATH2_2,PATH2_3,RE_POS)[l*90%,u*90%];
} maintaining PATH2_OK
};
{
Group-Transmit(OPS,ARRIVED)[0,2],
do {
Group-Wait(HOLD1,HOLD2)[0,u*10%]
} watching PROCEED
} at RE_POS
}
RMPL for Group-Enroute
Location Constraints:
Group-Enroute()[l,u] = {
choose {
do {
Group-Fly-
Path(PATH1_1,PATH1_2,PATH1_3,RE_POS)[l*90%,u*90%];
} maintaining PATH1_OK,
do {
Group-Fly-
Path(PATH2_1,PATH2_2,PATH2_3,RE_POS)[l*90%,u*90%];
} maintaining PATH2_OK
};
{
Group-Transmit(OPS,ARRIVED)[0,2],
do {
Group-Wait(HOLD1,HOLD2)[0,u*10%]
} watching PROCEED
} at RE_POS
}
RMPL for Group-Enroute
Non-deterministic
Group-Enroute()[l,u] = {
choose { choice:
do {
Group-Fly-
Path(PATH1_1,PATH1_2,PATH1_3,RE_POS)[l*90%,u*90%];
} maintaining PATH1_OK,
do {
Group-Fly-
Path(PATH2_1,PATH2_2,PATH2_3,RE_POS)[l*90%,u*90%];
} maintaining PATH2_OK
};
{
Group-Transmit(OPS,ARRIVED)[0,2],
do {
Group-Wait(HOLD1,HOLD2)[0,u*10%]
} watching PROCEED
} at RE_POS
}
Outline
• Model-based Programming
• Autonomous Engineering Operations
– An Example
– Model based Execution
– Fast Reasoning using Conflicts
• Cooperating Mobile Vehicles
– Predictive Strategy Selection
– Planning Out The Strategy
RMPL Model-based Program Kirk Model-based Executive
Control Program
z Executes concurrently Selects consistent threads of
z Preempts activity from redundant methods
z non-deterministic choice
z A[l,u] timing
z A at l location
Environment location estimates location goals
HOME
Model TT
WV
O
O
N
O
EN
O
HOME E
COLLECTION POINT RENDEZVOUS
COLLECTION POINT
Tracks Finds least
Landing Site: Enroute
Landing Site: ABC RENDEZVOUS
ABC COLLECTION POINT
Enroute Diverge
Landing Site: XYZ
Landing Site: XYZ Diverge
SCIENCE AREA 1 SCIENC
SCIENCE AREA 1‟
SCIENCE AREA
1SCIENCE AREA
1
E AREA
1‟ location cost paths
SCIENCE AREA 3
SCIENCE
AREA 3
Executive
• pre-plans activities Observations Commands
• pre-plans paths
• dynamically schedules [Tsmardinos et al.] Plant
Enroute Activity Encoded as a Temporal Plan Network
• Start with flexible plan representation
Enroute [450,540]
[0, 0]
Group Traverse Group Wait
[0, 0]
[405, 486] [0, 54]
[0, 0] [0, 0]
[0, 0]
Science Target
[0, 0]
Group Transmit [0, ]
[0, 2]
Activity (or sub-activity)
Duration (temporal constraint)
Enroute Activity Encoded as a Temporal Plan Network
• Add conditional nodes
Enroute [450,540]
[0, 0]
Group Traverse Group Wait
[0, 0]
[405, 486] [0, 54]
[0, 0] [0, 0]
[0, 0]
Science Target
3 8
[0, 0]
[0, 0] Group Transmit [0, ]
[0, 0]
Group Traverse [0, 2]
[405, 486]
Activity (or sub-activity)
Duration (temporal constraint)
Conditional node
Enroute Activity Encoded as a Temporal Plan Network
•Add temporally extended, symbolic constraints
Enroute [450,540]
[0, 0]
Group Traverse Group Wait
[0, 0]
[405, 486] [0, 54]
[0, 0] [0, 0]
Ask( PATH1 = OK) Ask( EXPLORE = OK) [0, 0]
Science Target
3 8 13
[0, 0]
[0, 0] Group Traverse
[0, 0] Group Transmit [0, ]
[0, 2]
6 [405, 486] 7 11 12
Ask( PATH2 = OK)
Activity (or sub-activity)
Duration (temporal constraint)
Conditional node
Symbolic constraint (Ask,Tell)
Instantiated Enroute Activity
•Add environmental constraints Group-Enroute
[500,800]
[0,∞]
[0,∞]
[450,540]
Group Traverse Group Wait
Ask( EXPLORE = OK)
Ask( PATH1 = OK)
[405, 486] 5 [0, 54]
Science Target
3 1
8
Group Traverse Group Transmit
Ask( PATH2 = OK)
6 [405, 486] 7 11 [0, 2] 12
Tell( PROCEED)
Tell(PATH1=OK)
[0,∞]
[10,10]
14 15 16 17
[200,200]
[450,450]
Activity (or sub-activity)
Duration (temporal constraint)
Conditional node
Symbolic constraint (Ask,Tell) External constraints
Generates Schedulable Plan
Group-Enroute
[500,800]
[0,∞] [0,∞]
[450,540]
Group Traverse Group Wait
Ask(PATH1=OK) Ask(PROCEED)
5
[405,486] [0,54]
Science Target
3 8 1
Group Traverse Group Transmit
Ask(PATH2=OK)
[0,∞]
6 7 11 12
[405,486] [0,2]
[10,10] Tell(PATH1=OK) Tell(PROCEED) [0,∞]
14 15 16 17
[450,450] [200,200]
To Plan, . . . perform the following hierarchically:
• Trace trajectories
• Check schedulability
• Supporting and protecting goals (Asks)
Supporting and Protecting Goals
Unsupported Subgoal Threatened Activities
Goal: any UCAV at Target
Activity: UAV1 at Base
1 2 1 2
Activity: UCAV1 at Target
Activity:UAVI at Targer
3 4 3 4
Close open goals Activities can’t co-occur
Resolving Unsupported Subgoals:
• Scan plan graph,identifying activities that support open sub-goals; force to co-occur.
Resolving Threatened Subgoals:
• Search for inconsistent activities that co-occur, and impose ordering.
Key computation is bound time of occurrence:
• Used Floyd-Warshall APSP algorithm O(V3).
Randomized Experiments for Assessing
Scaling and Robustness
Randomized Experiments:
• Randomly generated range of scenarios with 1-50 vehicles.
• Each vehicle has two scenario options, each with five actions and 2
waypoints:
1. Go to waypoint 1
2. Observe science
3. Go to waypoint 2
4. Observe science
5. Return to collection point
• Waypoints generated randomly from environment with uniform
distribution.
Strategy Selection:
• TPN planner chooses one option per vehicle.
• Combined choices must be consistent with timing constraints and vehicle
paths.
Kirk Strategy Selection:
Scaling and Robustness
Planning Time vs. Number of UAVs
35
Each vehicle visits 2
30
science sites and returns
25
to collection point
Planning Time (min)
20 Kirk Oct. ‟02
Kirk April ‟03
15
10
5
0
0 5 10 15 20 25 30 35 40 45 50
Number of UAVs
Performance Improvement Through
• Incremental temporal consistency
• Conflict-directed Search (in progress)
RMPL Model-based Program Kirk Model-based Executive
Control Program
z Executes concurrently Selects consistent threads of
z Preempts activity from redundant methods
z non-deterministic choice
z A[l,u] timing
z A at l location
Environment location estimates location goals
HOME
Model TT
WV
O
O
N
O
EN
O
HOME E
COLLECTION POINT RENDEZVOUS
COLLECTION POINT
Tracks Finds least
Landing Site: Enroute
Landing Site: ABC RENDEZVOUS
ABC COLLECTION POINT
Enroute Diverge
Landing Site: XYZ
Landing Site: XYZ Diverge
SCIENCE AREA 1 SCIENC
SCIENCE AREA 1‟
SCIENCE AREA
1SCIENCE AREA
1
E AREA
1‟ location cost paths
SCIENCE AREA 3
SCIENCE
AREA 3
Executive
• pre-plans activities Observations Commands
• pre-plans paths
• dynamically schedules [Tsmardinos et al.] Plant
Achieving Program States Combines
Logical Decisions and Trajectory Planning
Vehicle
Obstacle
Waypoint
Explorers Will Need to Be Dexterous
(Courtesy of Frank Kirchner. Used with permission.)
Outline
• Model-based Programming
• Autonomous Engineering Operations
– An Example
– Model based Execution
– Fast Reasoning using Conflicts
• Cooperating Mobile Vehicles
– Predictive Strategy Selection
– Planning Out The Strategy
Example:
Coaching Heterogeneous Teams
• Search and Rescue
• Ocean Exploration
(Courtesy of Jonathan How. Used with permission.)
• A dozen vehicles is too many to micro manage
→ Act as a coach:
• Specify evolution of state and location.
Forest Fire Rescue
• Goal: retrieve
Fire Line
family from fire.
Forest
• Rescue cannot take
place until the local
fire is suppressed. Rescue Point
Ambulance
Fire
• Retrofit one rescue
vehicle for fire
suppression
Kirk Model-based Execution System Overview
PMPL Mission Developer
control Strategy Selection Strategy macro
program decomposition
Mission Controller TPN Planner Strategy Macro
Library
state configuration goals
Activity Planning
environment
Visibility Graph and action data
• Strategy Selection Operators,
determines the optimal rules / Generative Tactics,
strategies to accomplish mission
Activity Planner Scenario Model
goals.
• Activity Planning figures out
within how to achieve mission goals
within strategic framework using Human / Computer
available low-level actions. Interface
schedulable plan
with rationale
MILP Path-Planning
RMPL Control Program
• (defclass rescue-team
(execute ()
(sequence
(parallel [l1,u1]
(tell-start(at family RescuePoint))
Initial State (tell-start(at uav1 Ambulance))
(tell-start(at uav2 Ambulance)) Phase 1
)
Intermediat (parallel [l2,u2]
e State (tell-atart(at family RescuePoint))
(ask-end(rescued family))
(ask-end(at uav1 Ambulance))
Goal State (ask-end(at uav2 Ambulance)) Phase 2
)
)
)
)
Environment Model
• Terrain Map
• Object instantiations:
– UAV uav1
– UAV uav2
– RESCU-READY uav1
– RESCUE-READY uav2
– IN-DISTRESS family
– LOCATION Ambulance
– LOCATION Fire
– LOCATION RescuePoint
Vehicle Specifications
• Vehicle linearized dynamics
• Vehicle primitive operators:
– Fly(V,A,B)
• move UAV “V” from location “A” to location “B”
– Refit(V)
• Prepare UAV “V” to drop fire retardant
– Drop(V,A)
• Drop fire retardant at location “A” with UAV “V”
– Rescue(V,P,A)
• Rescue people “P” in distress with UAV “V” at location “A”
Kirk Model-based Execution System Overview
PMPL Mission Developer
control Strategy Selection Strategy macro
program decomposition
Mission Controller TPN Planner Strategy Macro
Library
state configuration goals
Activity Planning
environment
Visibility Graph and action data
• Strategy Selection Operators,
determines the optimal rules / Generative Tactics,
strategies to accomplish mission
Activity Planner Scenario Model
goals.
• Activity Planning figures out
within how to achieve mission goals
within strategic framework using Human / Computer
available low-level actions. Interface
schedulable plan
with rationale
MILP Path-Planning
Kirk Constructs Vehicle Activity Plan
Using a Generative Temporal Planner
Approach:
• Encode Goal Plan using an LPGP-style encoding
• Prototype using LPGP [Fox/Long, CP03]
Mission Goal State Plan
Translate to Planning Problem
with Atomic Operators
Vehicle
Operator Use Atomic Generative Planner
Definitions (GraphPlan-Blum & Furst)
To Generate Operators and Precedence
Extract Temporal Plan and
Check Schedulability
Vehicle Activity Plan
Generated Activity Plan
Refit-Inv Fly-Inv Fly-Inv
Refit-Start Refit-End Fly-Start
[10,+lNF] [20,+lNF] [20,+lNF]
[20,+lNF] [10,20]
Fly-Start Fly-lnv Suppress-Start Suppress-lnv Suppress-End
Fly-End
Fire
Fire [20,+lNF] [10,20]
CP-Start CP-lnv-1 CP-lnv-1 CP-lnv-1 CP-lnv-1 CP-lnv-1 CP-lnv-1 CP-lnv-1 CP-Midpoint
[0,100] [0,100] [0,100] [0,100]
[0,100] [0,100] [0,100] [0,100]
Kirk extracts a least commitment plan and generates a rationale
Kirk Model-based Execution System Overview
PMPL Mission Developer
control Strategy Selection Strategy macro
program decomposition
Mission Controller TPN Planner Strategy Macro
Library
state configuration goals
Activity Planning
environment
Visibility Graph and action data
• Strategy Selection Operators,
determines the optimal rules / Generative Tactics,
strategies to accomplish mission
Activity Planner Scenario Model
goals.
• Activity Planning figures out
within how to achieve mission goals
within strategic framework using Human / Computer
available low-level actions. Interface
schedulable plan
with rationale
MILP Path-Planning
Output: Least Commitment Plan
with Rationale
Plan layered with rationale Rescue(UAV1,Troops,RSQ)
Refit(UAV1) Fly(UAV1,RSQ,Base)
Fly(UAV1,Base,RSQ)
[10,+INF] [20,+INF] [30,60] [20,+INF]
[10,20] [20,+INF]
[20,+INF]
[0,100]
[0,100]
Control Program Phase II
Control Program Phase I
Attack(UAV2,Radar) Fly(UAV2,Radar,Base)
Fly(UAV2,Base,Radar)
Kirk Ensures Plan Completeness, Consistency
and Minimality
Activity-A fact-L Activity-C
fact-J fact-O
[l1,u1] [l3,u3]
Start fact-M
End
Activity-B Activity-D
fact-K [l2,u2] fact-P
fact-N [l4,u4]
• Complete Plan
• A plan is complete IFF every precondition of every activity is achieved.
• An activity’s precondition is achieved IFF:
• The precondition is the effect of a preceding activity (support), and
• No intervening step conflicts with the precondition (mutex).
• Consistent Plan
• The plan is consistent IFF the temporal constraints of its activities are
consistent (the associated distance graph has no negative cycles), and
• no conflicting (mutex) activities can co-occur.
• Minimal Plan
• The plan is minimal IFF every constraint serves a purpose, i.e.,
• If we remove any temporal or symbolic constraint from a minimal plan, the
new plan is not equivalent to the original plan
Plan-based HCI Proof of Concept:
Coaching through Coordinated Views
Action: FLY-TO
Actor: UCAV2
Objects: BASE1
[5,NA]
WAYPOINT1 Action: FLY-TO
Actor: UCAV1
Objects: BASE1 Plan
Top Level
UCAV2 Action: ATTACK
Action: FLY-TO Actor: UCAV1
Actor: UCAV1 Objects: SAM1
Objects: SAM1 [10, 10]
HVT1 [5,NA]
SAM1 Action: ATTACK
UCAV1 Action: FLY-TO Actor: UCAV2
Action: FLY-TO
Actor: UCAV2 Objects: HVT1
Actor: UCAV2
Objects: HVT1 [10, 10]
Objects: WAYPOINT1
RADAR1
BASE1
Activity UCAV2-FLY-TO-WAYPOINT beginning
UCAV2-FLY-TO-WAYPOINT Establishes the following
prerequisites:
UCAV2-FLY-TO-WAYPOINT helps establish (AT UCAV2
WAYPOINT) a prerequisite of UCAV2-FLY-TO-TARGET
UCAV2-FLY-TO-WAYPOINT establishes (AT UCAV2 WAYPOINT)
Action: FLY-TO Action: FLY-TO Action: ATTACK
UCAV2 executes a FLY-TO operation on WAYPOINT1
Actor: UCAV2 Actor: UCAV2 Actor: UCAV2
Objects: WAYPOINT1 Objects: HVT1 Objects: HVT1
Activity UCAV1-FLY-TO-SAM beginning
Action: FLY-TO UCAV1-FLY-TO-SAM Establishes the following
[10, 10] Actor: UCAV2
prerequisites:
Objects: BASE1
Action: FLY-TO Action: ATTACK [5, NA]
UCAV1-FLY-TO-SAM helps establish (AT UCAV1 SAM1) a
Actor UCAV1 Actor UCAV1 Action: FLY-TO prerequisite of UCAV1-FLY-TO-SAM
Objects: SAM1 Objects: SAM1 Actor UCAV1 UCAV1-FLY-TO-SAM establishes (AT UCAV1 SAM1)
Set Current Plan Revert Clear Screen UCAV1 executes a FLY-TO operation on SAM1
UCAV1-FLY-TO-SAM finished, go on? [default Yes]: ▌
(Courtesy of Howard Shrobe, Principal Research Scientist, MIT CSAIL. Used with permission.)
Plan & Geography View
WAYPOINT1
HVT1
SAM1
UCAV1
RADAR1
BASE1
UCAV2
Sequencing: Action: FLY-TO Action: FLY-TO Action: ATTACK
Actor: UCAV2 Actor: UCAV2 Actor: UCAV2
Objects: Objects: HVT1 Objects: HVT1 Action: FLY-TO
WAYPOINT1 [10, 10] Actor: UCAV2
Objects: BASE1
Action: FLY-TO Action: ATTACK [5, NA]
Action: FLY-TO
Actor UCAV1 Actor UCAV1
Actor UCAV1
Objects: SAM1 Objects: SAM1
Objects: BASE1
[5, NA] [10, 10]
(Courtesy of Howard Shrobe, Principal Research Scientist, MIT CSAIL. Used with permission.)
Causal View
Causality
Action: FLY-TO
Actor: UCAV2
Objects: BASE1
[5,NA]
Action: FLY-TO
Actor: UCAV1
Objects: BASE1
Action: FLY-TO Action: ATTACK Plan
Actor: UCAV1 Actor: UCAV1 Top Level
Objects: SAM1 Objects: SAM1
[5,NA] [10, 10]
Action: ATTACK
Action: FLY-TO Action: FLY-TO Actor: UCAV2
Actor: UCAV2 Actor: UCAV2 Objects: HVT1
Objects: WAYPOINT1 Objects: HVT1 [10, 10]
Explanation
UCAV2-ATTACK-TO-TARGET Has the following prerequisites:
UCAV2-ATTACK-TO-TARGET requires (AT UCAV2 HVT1)
This was established by
Activity UCAV2-FLY-TO=TARGET achieving (AT UCAV2 HVT1)
UCAV2-ATTACK-TO-TARGET Establishes the following
prerequisites:
UCAV2-ATTACK-TO-TARGET helps establish (DESTROYED HVT1)
prerequisite of NEW
UCAV2-ATTACK-TO-TARGET establishes (DESTROYED HVT1)
(Courtesy of Howard Shrobe, Principal Research Scientist, MIT CSAIL. Used with permission.)
Model-based Programming
of Robust Robotic Networks
• Long-lived systems achieve robustness by coordinating a
complex network of internal devices.
• Programmers make a myriad of mistakes when programming
these autonomic processes.
• Model-based programming simplifies this task by elevating the
programmer to the level of a coach:
– Makes hidden states directly accessible to the programmer.
– Automatically mapping between states, observables and control
variables.
• Model-based executives reasoning quickly and extensively by
exploiting conflicts.
• Mission-level executives combine activity planning, logical
decision making and control into a single hybrid decision
problem.