A Software Component for Plan Management in Robotics Sylvain by Nowandforever

VIEWS: 0 PAGES: 115

									A Software Component for Plan
   Management in Robotics

           Sylvain Joyeux

      Thèse préparée au LAAS/CNRS


         6 December, 2007
1. Introduction
                      Autonomous robotic systems

   A Software
Component for Plan
 Management in
    Robotics
  Sylvain Joyeux


Introduction
Context
Thesis focus
Existing approaches     I can’t define a robot, but I know one when I see one
Our approach
Scenario
                                                               J. Engelberger
Model
Execution
Adaptation
Results
Conclusion




     3 / 75
                      Autonomous robotic systems

   A Software
Component for Plan
 Management in
    Robotics
  Sylvain Joyeux
                      robotic systems electro-mechanical systems able to
Introduction
Context
                                   sense their environment and act in it
Thesis focus          autonomous able to decide its course of action in order
Existing approaches
Our approach                    to perform a mission
Scenario
Model
Execution
Adaptation
Results               i.e. embodied sense-plan-act loops
Conclusion




     3 / 75
                      Robotic architectures

   A Software
Component for Plan
 Management in
    Robotics          In software development, an architecture
  Sylvain Joyeux
                          separates functions into components
Introduction
Context
                          defines the interactions between those components
Thesis focus
Existing approaches
Our approach
Scenario                  solves the common problems
Model
                          must allow integration to solve the not-so-common
Execution
                          ones
Adaptation
Results                   in robotics, two main concerns:
Conclusion                    modular integration of diverse data processing
                              services
                              integration of decision-making

     4 / 75
                      Robotic architectures

   A Software
Component for Plan
 Management in
    Robotics          In software development, an architecture
  Sylvain Joyeux
                          separates functions into components
Introduction
Context
                          defines the interactions between those components
Thesis focus
Existing approaches
Our approach
Scenario                  solves the common problems
Model
                          must allow integration to solve the not-so-common
Execution
                          ones
Adaptation
Results                   in robotics, two main concerns:
Conclusion                    modular integration of diverse data processing
                              services
                              integration of decision-making

     4 / 75
                      Decision-making

   A Software
Component for Plan
 Management in
    Robotics
  Sylvain Joyeux      Two kinds of decision-making:
Introduction             reactive given the situation and the goals, what
Context                           should be the next step.
Thesis focus
Existing approaches
Our approach
Scenario
Model
Execution
Adaptation
Results
Conclusion




     5 / 75
                      Decision-making

   A Software
Component for Plan
 Management in
    Robotics
  Sylvain Joyeux      Two kinds of decision-making:
Introduction              reactive given the situation and the goals, what
Context                            should be the next step.
Thesis focus
Existing approaches   by anticipation given the situation and the goals,
Our approach
Scenario                  explore the possible futures
Model
                          determine the “desired” ones
Execution
Adaptation                determine the set(s) of actions required to reach
Results                   them
Conclusion                = the plan(s)



     5 / 75
                      The two-layer model

   A Software
Component for Plan
 Management in
    Robotics
  Sylvain Joyeux                     Decision layer

Introduction
Context                       Commands                 Information
Thesis focus
Existing approaches
Our approach                         Functional layer
Scenario
Model
                              Commands                 Information
Execution
Adaptation
Results                              Robot Hardware
Conclusion
                                         Environment



     6 / 75
                      The two-layer model

   A Software
Component for Plan
 Management in
    Robotics
                                                     Multiple decision
  Sylvain Joyeux        Decision layer                making tools
Introduction
Context                                Information
Thesis focus
Existing approaches
                                                     There’s decision−making
Our approach            Functional layer
Scenario                                                  in here too !
Model
                                       Information
Execution
Adaptation
                       Robot Hardware
Results
Conclusion
                         Environment



     6 / 75
                      Focus of this thesis
                      Integration of decision-making

   A Software
Component for Plan
 Management in         Our focus
    Robotics
  Sylvain Joyeux                     An architecture allowing . . .
                              the cooperative integration of heterogeneous
Introduction
Context
                                         decision-making tools
Thesis focus
Existing approaches
Our approach
Scenario
Model
Execution
Adaptation
Results
Conclusion




     7 / 75
                      Focus of this thesis
                      Integration of decision-making

   A Software
Component for Plan
 Management in         Our focus
    Robotics
  Sylvain Joyeux                     An architecture allowing . . .
                              the cooperative integration of heterogeneous
Introduction
Context
                                         decision-making tools
Thesis focus
Existing approaches
Our approach           heterogeneous            choose the right planner for the task
Scenario
Model
                            because there is no one-fits-all planner
Execution                   because planning is complex
Adaptation                    ⇒ split big problems into smaller ones
Results
Conclusion




     7 / 75
                      Focus of this thesis
                      Integration of decision-making

   A Software
Component for Plan
 Management in         Our focus
    Robotics
  Sylvain Joyeux                     An architecture allowing . . .
                              the cooperative integration of heterogeneous
Introduction
Context
                                         decision-making tools
Thesis focus
Existing approaches
Our approach           heterogeneous            choose the right planner for the task
Scenario
Model
                            because there is no one-fits-all planner
Execution                   because planning is complex
Adaptation                    ⇒ split big problems into smaller ones
Results
Conclusion             cooperative
                                          through planner-planner negotiation
                       integration
                         ⇒ one planner’s decision must not overconstrain the
                            others’
     7 / 75
                      Integration through partial plans

   A Software
Component for Plan
 Management in
    Robotics             Alami et al. – 1998
  Sylvain Joyeux         An Architecture for
Introduction
                         Autonomy
                                                      multiple planners
Context
Thesis focus
Existing approaches      Muscettola et al. – 2002     integration is done
Our approach
                         IDEA: Planning at the Core   by exchanging
Scenario
Model
                         of Autonomous Agents         partial plans
Execution
                                                      no representation of
Adaptation               Chouinard et al. – 2005      the whole system
Results                  Intelligent rover            plan
Conclusion               decision-making in
                         response to exogenous
                         events
     8 / 75
                      Concurrent Reactive Plans
                      Execution of plan libraries

   A Software
Component for Plan
 Management in
    Robotics
  Sylvain Joyeux             a plan library
                             tools to adapt plans continuously
Introduction
Context                       ⇒ transformation planning
Thesis focus
Existing approaches
                             state projection
Our approach
Scenario
Model
Execution              Demonstrated the validity of a “plan management”
Adaptation
                       scheme
Results
Conclusion                  Beetz – 2002
                            Concurrent Reactive Plans

     9 / 75
                      GPGP/TAEMS
                      Plan-based control of multi-robot systems

   A Software
Component for Plan
 Management in
    Robotics
  Sylvain Joyeux
                            plan model based only on tasks and task graphs
Introduction
Context
                              ⇒ rich set of task relations
Thesis focus                  ⇒ designed for scheduling
Existing approaches
Our approach
                            designed for multi-robot systems
Scenario
                            whiteboard-based approach of plan building
Model
Execution
                            Lesser, Decker et al. – 2004
Adaptation
                            Evolution of the GPGP/TAEMS
Results
                            Domain-Independent Coordination Framework
Conclusion




     10 / 75
                      Our approach

   A Software
Component for Plan
 Management in
    Robotics
  Sylvain Joyeux
                      General principle
Introduction
Context
                      One central place where the whole system’s plan is
Thesis focus          represented managed
Existing approaches
Our approach
Scenario              Fit to be the integration place for
Model                     planners
Execution
                          reactive decision-making
Adaptation
Results                   plan analysis tools: situation assessment,
Conclusion                diagnostics, . . .



     11 / 75
                      Our approach

   A Software
Component for Plan
 Management in
                        1   A multi-robot plan model
    Robotics                    as planner-agnostic as possible
  Sylvain Joyeux
                                 ⇒ representation based on actions, not state
Introduction                    aimed at supervision, not planning
Context
                                 ⇒ represents from high-level actions to low-level ones
Thesis focus
Existing approaches              ⇒ rich situation representation
Our approach
Scenario
                                designed with multi-robot in mind
Model
Execution                                Plan
                                         object           Models
Adaptation
                                       instances         application
Results
                                         object            code
Conclusion                               graphs
                                                           object
                                       application         models
                                         code

     12 / 75
                      Our approach

   A Software
Component for Plan
                        1   A multi-robot plan model
 Management in
    Robotics
                        2   A plan management scheme
  Sylvain Joyeux
                                execution of the said plan
                                  ⇒ in particular, representation of errors and of their
Introduction                        recovery process
Context                         adaptation as it is being executed
Thesis focus
Existing approaches
                                  ⇒ dynamic plans
Our approach
Scenario
Model                                     Controller
Execution                                                              Plan
Adaptation             Other robots                                 application
Results               Functional layer                                code
                           User           Executive                                Modifies
Conclusion
                                                                    application
                                                                      code
                                                                      Models

     12 / 75
                      Our approach

   A Software
Component for Plan      1   A multi-robot plan model
 Management in
    Robotics            2   A plan management scheme
  Sylvain Joyeux        3   Decision-making for plan execution
Introduction                    online plan fault resolution
Context                         planning-execution conflict resolution
Thesis focus
Existing approaches
Our approach
                                                            Plan
Scenario                               Decision          Planning
                                                         Generation
Model                                  Control             Tools
                                                           Tools
Execution
Adaptation
Results
Conclusion                                               Plan

                                        Executive
                                                       Models



     12 / 75
                      Outline of the presentation

   A Software
Component for Plan
 Management in
    Robotics
                       1   Introduction
  Sylvain Joyeux


Introduction
Context
                       2   Plan Model
Thesis focus
Existing approaches
Our approach
                       3   Plan Execution
Scenario
Model
                       4   Plan Adaptation
Execution
Adaptation
Results                5   Results
Conclusion
                       6   Conclusion & Future Work


     13 / 75
                      Illustration and validation of our approach

   A Software
Component for Plan
 Management in
    Robotics
  Sylvain Joyeux


Introduction              through a complete robot
Context                    ⇒ must manage a complex set of functions
Thesis focus
                           ⇒ “big enough” plan sizes to test the execution
Existing approaches
Our approach                 scalability
Scenario
                          through the management of a bi-robot plan
Model
                           ⇒ model fit to control multiple robots
Execution
                           ⇒ building joint plans
Adaptation
                           ⇒ joint plan execution
Results
Conclusion




     14 / 75
                      Mono-robot scenario: the Dala rover

   A Software
Component for Plan
 Management in
    Robotics
                         Mission: navigation in
  Sylvain Joyeux
                         unknown environment
Introduction
Context
Thesis focus
Existing approaches      Bitmap builds a global
Our approach
Scenario
                         traversability map
Model                    Nav long-range path
Execution                planning in traversability map
Adaptation
                         Dtm builds a local elevation
Results
                         map
Conclusion
                         P3d short-range path
                         planning in elevation map
                         Pom position fusion
     15 / 75
                      Bi-robot scenario: rover/UAV cooperation

   A Software
Component for Plan
 Management in
    Robotics
                                                       UAV
  Sylvain Joyeux              Rover
Introduction                                  ⇒ long-range perception
Context                  ⇒ local perception
Thesis focus
Existing approaches
Our approach
Scenario
Model
Execution
Adaptation
Results
Conclusion




     16 / 75
                      Simulated bi-robot setup

   A Software
Component for Plan
 Management in
    Robotics
  Sylvain Joyeux


Introduction                 Terrain
Context
Thesis focus                                     + rich simulation: the
Existing approaches
Our approach
                                                   robot software is used
Scenario                                           almost as-is
Model
                                                 + fault injection to test
Execution
                                                   error handling
Adaptation
Results
Conclusion




     17 / 75
                      Experimental mono-robot setup

   A Software
Component for Plan
 Management in
    Robotics
                                                 Fault injection devices
  Sylvain Joyeux
                        The Dala rover
Introduction
Context
Thesis focus
Existing approaches
Our approach
Scenario
                                                                     Caylus, military
Model                                       "Algeco", here at LAAS   training ground
Execution
Adaptation
Results
Conclusion

                                             RF emergency
                                                                 Esperce airfield
                                                 brakes
                      Thanks to Thierry, Matthieu and Arnaud
     18 / 75
                      Experimental bi-robot setup

   A Software
Component for Plan
 Management in
    Robotics
                                       The Ressac UAV
  Sylvain Joyeux


Introduction
Context
Thesis focus
Existing approaches
Our approach
Scenario
Model
Execution
Adaptation
Results
Conclusion



                      Thanks to Roger, Vincent and Alain at ONERA
     19 / 75
2. Plan Model
                     Overview

   A Software
Component for Plan
 Management in
    Robotics
                                      What do we need ?
  Sylvain Joyeux


Introduction
                        a representation of activities
                          ⇒ tasks
Model
Overview
                        The robot has already reached a given point, and is now
Activities              moving towards another. It will update its map in the
Execution flow           future.
Execution
Adaptation
Results
Conclusion




      21 / 75
                     Overview

   A Software
Component for Plan
 Management in
    Robotics
                                      What do we need ?
  Sylvain Joyeux


Introduction            a representation of activities
Model                    ⇒ tasks
Overview                a representation of the activity interactions
Activities
Execution flow            ⇒ task graphs
Execution                   how an activity influences another
Adaptation                  What impact the terrain mapping has on the
Results                     movement ?
Conclusion




      21 / 75
                     Overview

   A Software
Component for Plan
 Management in
    Robotics                          What do we need ?
  Sylvain Joyeux


Introduction            a representation of activities
Model                    ⇒ tasks
Overview
Activities              a representation of the activity interactions
Execution flow
                         ⇒ task graphs
Execution
Adaptation                  how an activity influences another
Results                     constraints on the activities
Conclusion                  Is it allowed for the localization to stop ?




      21 / 75
                     Overview

   A Software
Component for Plan
 Management in
    Robotics
                                      What do we need ?
  Sylvain Joyeux


Introduction            a representation of activities
Model                    ⇒ tasks
Overview                a representation of the activity interactions
Activities
Execution flow            ⇒ task graphs
Execution                   how an activity influences another
Adaptation                  constraints on the activities
Results
                        a representation of the activities evolution
Conclusion
                          ⇒ events
                        The localization has already started. The movement has
                        just blocked.


      21 / 75
                     Overview

   A Software
Component for Plan
 Management in
    Robotics
                                      What do we need ?
  Sylvain Joyeux


Introduction            a representation of activities
Model
                         ⇒ tasks
Overview                a representation of the activity interactions
Activities
                         ⇒ task graphs
Execution flow
Execution                   how an activity influences another
Adaptation                  constraints on the activities
Results                 a representation of the activities evolution
Conclusion               ⇒ events
                        a representation of the whole system evolution
                         ⇒ event graphs
                        When the mapping is started and the path is computed,
      21 / 75
                        the robot can start moving
                     Tasks: representation of activities

   A Software
Component for Plan
 Management in
    Robotics
  Sylvain Joyeux         parametrized activity model
Introduction                 MoveTo(x, y)
Model
                             Pom::Localization()
Overview                     MapZone(x_start, y_start, x_final, y_final)
Activities                   TrackPath(path)
Execution flow
Execution
Adaptation
Results
Conclusion




      22 / 75
                     Tasks: representation of activities

   A Software
Component for Plan
 Management in
    Robotics              parametrized activity model
  Sylvain Joyeux
                          hierarchical organization of task models
Introduction                ⇒ allows to represent equivalence between activities
Model
Overview
Activities
Execution flow
Execution                                          Roby::Task()

Adaptation
Results                       MoveTo(x, y, yaw)                   TraversabilityMapping
Conclusion

                                                             Bitmap::Mapping RemoteMapping
                     P3D::MoveTo NDD::MoveTo Nav::MoveTo
                      (x, y, yaw) (x, y, yaw) (x, y, yaw)




      22 / 75
                     Tasks: representation of activities

   A Software
Component for Plan
 Management in
    Robotics
  Sylvain Joyeux
                         parametrized activity model
Introduction
Model                    hierarchical organization of task models
Overview                 notion of non-executable tasks
Activities
Execution flow
                          ⇒ partially instantiated activities
Execution                 ⇒ activities from abstract models
Adaptation
Results
Conclusion




      22 / 75
                     Task relations: activity interactions

   A Software
Component for Plan
 Management in
    Robotics
                                                      Nav::MoveTo(x, y, yaw)
  Sylvain Joyeux
                                                                           planned_by
Introduction
                      Different kinds of     depends_on      Nav::Path     PlanningTask
Model
Overview                   relations
Activities
Execution flow                                  P3d::TrackPath      Nav::PathPlanning
Execution                hard dependency
Adaptation               planning
Results                                      Pom::Localization       Bmap::Mapping
                         execution support
Conclusion
                                                      executed_by
                      ⇒ task graphs !          Pom::Runner    Nav::Runner




      23 / 75
                     Multi-robot plans: ownership and roles

   A Software
Component for Plan
 Management in
    Robotics
                                   Notion of ownership and roles
  Sylvain Joyeux
                       ownership who is doing what ?
Introduction
                            role who will be doing what in team interest ?
Model
Overview
Activities
                                    Nav::MoveTo(to: C)           RemoteMapping
Execution flow
Execution                                                                    DataTransfer
Adaptation                              Nav::Path
                                                       RegionsOfInterest
Results
Conclusion                 P3d::TrackPath
                                             Nav::PathPlanning


                         POM::Localization
                                                            Bmap::TravMapping


      24 / 75
                     Execution flow: what for ?
                     Example: perception on the Dala rover

   A Software
Component for Plan    The needs
 Management in
    Robotics              trigger the updates at the appropriate times
  Sylvain Joyeux
                          need to sequence the perception activities
Introduction                         when does a perception should start/stop ?
Model                                when are the maps updated ?
Overview                             ...
Activities
Execution flow
Execution
Adaptation                                            Bitmap::Mapping
Results
Conclusion
                      PlanningLoop           Dtm::Mapping      Bitmap::Perception   Bitmap::Perception




                         PlanningLoop       Dtm::Perception   Dtm::Perception
      25 / 75
                     Events
                     A representation of milestones during execution


   A Software
Component for Plan
 Management in
    Robotics
  Sylvain Joyeux

                           represents a noticeable situation
Introduction
Model
                             ⇒ the robot moved more than x meters
Overview
                             ⇒ timeout of 5 seconds
Activities                   ⇒ ...
Execution flow
Execution
Adaptation
Results
Conclusion




      26 / 75
                     Events
                     A representation of milestones during execution


   A Software
Component for Plan         represents a noticeable situation
 Management in
    Robotics                 ⇒ the robot moved more than x meters
  Sylvain Joyeux             ⇒ timeout of 5 seconds
                             ⇒ tasks are composed of events
Introduction
Model
                                                    MoveTo(x, y, yaw)
Overview
Activities
Execution flow                                               start
Execution
                                                            success
Adaptation
                                                  blocked              stop
Results                                                      failed
Conclusion                                        aborted




      26 / 75
                     Events
                     A representation of milestones during execution


   A Software
Component for Plan
 Management in
    Robotics
  Sylvain Joyeux


Introduction               represents a noticeable situation
Model                        ⇒ the robot moved more than x meters
Overview                     ⇒ timeout of 5 seconds
Activities
Execution flow
                             ⇒ tasks are composed of events
Execution                  interface to control the system
Adaptation                   ⇒ start an activity/stop it
Results                      ⇒ update the map
Conclusion                   ⇒ ...




      26 / 75
                     Reacting to events: signalling

   A Software
Component for Plan
 Management in
    Robotics         Describes the reaction to events
  Sylvain Joyeux     When there is a signal e1 → e2 the command of e2 is
Introduction         called when e1 is emitted
Model
Overview
Activities
Execution flow
                                                          start   success   stop
Execution                                                    Dtm::Perception
Adaptation
Results
Conclusion


                                   start   success
                                Stereopixel::Perception



      27 / 75
                     Reacting to events: signalling

   A Software
Component for Plan
 Management in
    Robotics         Describes the reaction to events
  Sylvain Joyeux     When there is a signal e1 → e2 the command of e2 is
Introduction         called when e1 is emitted
Model
Overview
Activities
Execution flow
                                                   start   success     stop
Execution                                             Dtm::Perception
Adaptation
Results
Conclusion


                                start   success   stop     start    success   stop
                                Stereopixel::Perception            Dtm::FuseDtm



      27 / 75
                     Signal is not enough !

   A Software
Component for Plan
 Management in
    Robotics
  Sylvain Joyeux


Introduction                                                    ?
                                              start   success       stop
Model
                                                 Dtm::Perception
Overview
Activities
Execution flow                                                                    ?
Execution
Adaptation
Results                                   ?                                ?                            ?
                        start   success       stop    start    success         stop   start   success       stop
Conclusion              Stereopixel::Perception               Dtm::FuseDtm              Dtm::FillP3dPoster




      28 / 75
                     Event generalization: forwarding

   A Software
Component for Plan
 Management in       Generalization semantic between events
    Robotics
  Sylvain Joyeux     When e1 is forwarded on e2 , e2 is emitted when e1 is.
                     Therefore the situations represented by e2 are a
Introduction
                     superset of those represented by e1 .
Model
Overview
Activities
Execution flow
Execution                                  start   success     stop
                                              Dtm::Perception
Adaptation
Results
Conclusion



                        start   success   stop     start    success   stop   start   success   stop
                        Stereopixel::Perception            Dtm::FuseDtm        Dtm::FillP3dPoster


      29 / 75
                     Event generalization: forwarding

   A Software
Component for Plan   Generalization semantic between events
 Management in
    Robotics
                     When e1 is forwarded on e2 , e2 is emitted when e1 is.
  Sylvain Joyeux
                     Therefore the situations represented by e2 are a
Introduction         superset of those represented by e1 .
Model
Overview
Activities
Execution flow
                                          loop_success              updated_data
Execution                                                          Dtm::Mapping
                                          PlanningLoop
Adaptation
Results
Conclusion
                                  start    success   stop
                                     Dtm::Perception



                                                     start   success    stop
                                                         Dtm::FillP3dPoster
      29 / 75
                     Back to Dala perception loops

   A Software
Component for Plan
 Management in
    Robotics
  Sylvain Joyeux
                           implemented using two loop structures
Introduction
                              ⇒ generic structures built with the standard model
Model
Overview                   triggers are state events
Activities
                              ⇒ in this case, represent a delta in position, heading
Execution flow
                                or time
Execution
Adaptation
Results
Conclusion
                       The loop construct




      30 / 75
                     Summary

   A Software
Component for Plan
 Management in
    Robotics
  Sylvain Joyeux


Introduction
Model                   with separated activity and temporal structures
Overview
Activities              allows to represent progressive tasks and their
Execution flow
                        interaction
Execution
Adaptation              able to represent joint plans
Results
Conclusion




      31 / 75
3. Plan Execution
                     Overview

   A Software
Component for Plan
 Management in
    Robotics
  Sylvain Joyeux


Introduction
Model
Execution               event propagation
Multi-robot              ⇒ synchronous model
Errors
GC
                         ⇒ global propagation algorithm
Adaptation
Results
Conclusion




     33 / 75
                     Overview

   A Software
Component for Plan
 Management in
    Robotics
  Sylvain Joyeux


Introduction            event propagation
Model
                         ⇒ synchronous model
Execution
                         ⇒ global propagation algorithm
Multi-robot
Errors                  multi-robot execution
GC
Adaptation              rich error definition and handling
Results                 automatic cleanup of the plan
Conclusion




     33 / 75
                     Composition of plans in multi-robot context

   A Software
Component for Plan
 Management in
    Robotics
  Sylvain Joyeux


Introduction             cannot have all tasks of all robots in all plans
Model                    selection of relevant tasks is based on:
Execution                    direct relation to one’s own plan
Multi-robot
Errors
                               ⇒ there is a relation between a local task and the
GC                               remote task
Adaptation                   an explicit subscription mechanism
Results                        ⇒ a plan manager can explicitly ask another to send
Conclusion                       updates about its tasks




     34 / 75
                      Example: partial views of Dala and Ressac
                      Live data from Caylus 25/10

   A Software            Joint plan as seen by Dala: ∼ 65 tasks, 2 remote tasks
Component for Plan
 Management in
    Robotics
                                                    Nav::MoveTo

  Sylvain Joyeux
                                                                                         Genom::Bitmap::Mapping
                                    P3d::TrackPath                Nav::Path
Introduction
                     Rflex::TrackSpeedStart

Model                                                                    Pom::Localization        PlanningLoop     Bitmap::Perception
                                       P3d::Track                                                                                          UAVMapping::MapAlongPath
                                                       Dtm::Mapping
Execution                                                                                Nav::RegionsOfInterest
                                                                                                                         TransferSink
                                                                        Nav::PathPlanning
Multi-robot                                                                                                                                                         TransferSource

Errors
GC
Adaptation
                         Joint plan as seen by Ressac: ∼ 17 tasks, 8 remote tasks
Results                                                                                    Genom::Bitmap::Mapping



Conclusion
                                                                              Pom::Localization     PlanningLoop     Bitmap::Perception
                                                                                                                                            UAVMapping::MapAlongPath
                                                         Dtm::Mapping

                                                                                           Nav::RegionsOfInterest
                                                                                                                            TransferSink
                                                                              Nav::PathPlanning
                                                                                                                                           PlanningLoop   MapPath     TransferSource



                                                                                                                             ZoneMapping::Export




     35 / 75
                     Synchronization problems in multi-robot

   A Software
Component for Plan
                     The protocol has synchronization guaranties
 Management in           either it guarantees synchronization
    Robotics
                            ⇒ example: plan building tools
  Sylvain Joyeux
                           or it reports plan-related errors
Introduction                ⇒ lack of synchronization in the plan
Model
                     Example
Execution
Multi-robot
Errors
GC                                       push           pull   failed
Adaptation                           TransferSource    TransferSink

Results                1   Ressac sends a notification to Dala
Conclusion
                       2   communication delays
                            ⇒ TransferSink fails in the meantime
                       3   Dala receives the notification but the target does
                           not exist anymore
     36 / 75
                            ⇒ SynchronizationError on common parent
                     Error handling

   A Software
Component for Plan
 Management in
    Robotics
  Sylvain Joyeux


Introduction             a generic description of errors as
Model                        a type
Execution                    a fault point
Multi-robot
Errors
                         means to constrain the system
GC                        ⇒ to define what’s allowed and what is not
Adaptation
                         means to react to errors
Results
Conclusion               and a default mechanism for unhandled errors




     37 / 75
                     Managing fault modes

   A Software
Component for Plan
 Management in                                    MoveTo(x, y, yaw)
    Robotics
  Sylvain Joyeux                                           start
Introduction           ⇒ forwarding is used to             success
Model                    represent fault modes   blocked             stop
Execution                                                   failed
Multi-robot                                      aborted
Errors
GC
Adaptation
Results
Conclusion




     38 / 75
                     Managing fault modes

   A Software
Component for Plan
 Management in                                               MoveTo(x, y, yaw)
    Robotics
  Sylvain Joyeux                                                     start
Introduction           ⇒ forwarding is used to                       success
Model                    represent fault modes             blocked             stop
Execution                                                             failed
Multi-robot                                                aborted
Errors
GC
Adaptation
Results              Forwarding and failure points
Conclusion               a handler which applies to a given erroneous
                         situation . . .
                         . . . also applies to the subsets of this situation


     38 / 75
                     Task relations as constraint definition
                     The link between events and task relations

   A Software
Component for Plan
 Management in
    Robotics
  Sylvain Joyeux
                      Task relations describe which situations are desired and
Introduction          which are forbidden
Model
Execution
Multi-robot
Errors
GC                                                       start   success   stop

Adaptation                                                  Dtm::Perception

Results                                                     depends_on
Conclusion
                                          start   success
                                       Stereopixel::Perception




     39 / 75
                     Task relations as constraint definition
                     The link between events and task relations

   A Software
Component for Plan
 Management in
    Robotics
  Sylvain Joyeux      Task relations describe which situations are desired and
                      which are forbidden
Introduction
Model
Execution
Multi-robot
                                              start     success   stop
Errors
                                                     Dtm::Perception
GC
                                                                   depends_on
Adaptation
Results
Conclusion                         start    failed      stop      start    success   stop
                                   Stereopixel::Perception                Dtm::FuseDtm
                       Type                  failure point failed event of
                       DependencyFailedError Stereopixel::Perception

     39 / 75
                     Error reparation

   A Software
Component for Plan
 Management in
    Robotics
  Sylvain Joyeux


Introduction
Model                             Three error handling methods
Execution
Multi-robot
Errors
GC
                       1   repair errors during event propagation
Adaptation             2   repair using subplans
Results
                       3   exception propagation
Conclusion




     40 / 75
                     Repairing errors during event propagation

   A Software
Component for Plan
 Management in
    Robotics
  Sylvain Joyeux
                     An event handler of a specific erroneous event repairs
Introduction         the plan
Model
Execution
Multi-robot
Errors
GC
                                              poster_not_updated   failed stop
Adaptation
                               P3d::PathTracking    Rflex::TrackSpeedStart
Results
Conclusion
                                            start
                                     Rflex::TrackSpeedStart




     41 / 75
                     Asynchronous repairs

   A Software
Component for Plan
 Management in
    Robotics             mark a task as repairing a failure point
  Sylvain Joyeux
                          ⇒ plan repair
Introduction             during the lifetime of the repair task
Model                    and while a timeout is not reached
Execution                 ⇒ the error is inhibited
Multi-robot
Errors
GC
Adaptation           The repair_with relation
Results
Conclusion
                     This relation associates a task and an event which may
                     be a failure point

                      ⇒ activated when the failure point is the event


     42 / 75
                     Example: P3D’s blocked event

   A Software
Component for Plan
 Management in
    Robotics
                     Cascade of error handlers
  Sylvain Joyeux


Introduction               When the motion control reports “blocked”
Model
Execution
Multi-robot            1   update the elevation map locally
Errors
GC                     2   if it still fails, reinitialize the elevation map
Adaptation             3   if it still fails, the robot is really blocked
Results
                               ⇒ real error
Conclusion

                       +   if the robot moves after a repair, reinitialize to   1




                       Video           brakes_on on Rflex

     43 / 75
                     Exception handling

   A Software
Component for Plan
 Management in
    Robotics               a failure
  Sylvain Joyeux                  is not handled by a repair_with relation
Introduction
                                  has no associated plan repair
Model                      a repair task has failed
Execution
                                                     Nav::MoveTo
Multi-robot
Errors
GC                                                           Nav::Path
                                     P3d::TrackPath
Adaptation
                                                                   Nav::PathPlanning
Results               Rflex::TrackSpeedStart

Conclusion                                                               Genom::Bitmap::Mapping
                                        P3d::Track
                                                       Dtm::Mapping




                                                            failed stop       PlanningLoop   Bitmap::Perception
                                                          Pom::Localization


     44 / 75
                     Exception handling call order

   A Software
Component for Plan
 Management in
    Robotics
  Sylvain Joyeux

                                                     Nav::MoveTo
Introduction
Model                                                        Nav::Path

Execution                            P3d::TrackPath
Multi-robot                                                        Nav::PathPlanning
Errors                Rflex::TrackSpeedStart                                           1
GC                                                                       Genom::Bitmap::Mapping
                                        P3d::Track
Adaptation                                             Dtm::Mapping

Results
Conclusion                                                  Pom::Localization PlanningLoop   Bitmap::Perception




     45 / 75
                     Exception handling call order

   A Software
Component for Plan
 Management in
    Robotics
  Sylvain Joyeux

                                                     Nav::MoveTo
Introduction
Model                                                        Nav::Path

Execution                            P3d::TrackPath
Multi-robot                                                        Nav::PathPlanning
Errors                Rflex::TrackSpeedStart                                           1
GC                                                                       Genom::Bitmap::Mapping
                                        P3d::Track
Adaptation                                             Dtm::Mapping

Results                                                                                2
Conclusion                                                  Pom::Localization PlanningLoop   Bitmap::Perception




     45 / 75
                     Exception handling call order

   A Software
Component for Plan
 Management in
    Robotics
  Sylvain Joyeux
                                                           5
Introduction                                         Nav::MoveTo
                                                                     4
Model
                                                                Nav::Path
Execution                                                                      3
                                     P3d::TrackPath
Multi-robot
                                                                     Nav::PathPlanning
Errors                Rflex::TrackSpeedStart                                             1
GC
                                                                            Genom::Bitmap::Mapping
Adaptation                              P3d::Track
                                                       Dtm::Mapping
Results
                                                                                         2
Conclusion
                                                               Pom::Localization PlanningLoop   Bitmap::Perception




     45 / 75
                     Garbage collection

   A Software
Component for Plan
 Management in
    Robotics
  Sylvain Joyeux
                         must remove tasks which are harmful
Introduction             must remove tasks that are not useful anymore
Model                        not used by any missions
Execution                    not meaningful in the robot-robot interaction
Multi-robot
Errors
GC
Adaptation
Results
Conclusion           Our plans are made of task graphs
                      ⇒ global algorithm



     46 / 75
                     Summary: the execution cycle

   A Software
Component for Plan
 Management in
    Robotics
  Sylvain Joyeux
                          1. Event
Introduction            Propagation
Model
Execution
Multi-robot
Errors
GC
                                                2. Error
Adaptation                                      Handling
Results
Conclusion




                            3. Garbage
     47 / 75
4. Plan Adaptation
                     Transactions: motivation

   A Software
Component for Plan
 Management in
    Robotics             our plan manager relies on task and event structure
  Sylvain Joyeux         to manage the system
Introduction                 execution flow
Model                        determination of errors
Execution                    garbage collection
Adaptation                   ...
Transactions
Conflicts
Results
Conclusion




     49 / 75
                     Transactions: motivation

   A Software
Component for Plan
 Management in
    Robotics
  Sylvain Joyeux
                         our plan manager relies on task and event structure
Introduction             to manage the system
Model                 ⇒ we need a tool to change the old structure into the
Execution               new one in a single step
Adaptation
Transactions
Conflicts
Results
Conclusion




     49 / 75
                     Transactions: motivation

   A Software
Component for Plan
 Management in
    Robotics
  Sylvain Joyeux         our plan manager relies on task and event structure
                         to manage the system
Introduction
Model
                      ⇒ we need a tool to change the old structure into the
Execution               new one in a single step
Adaptation
Transactions
Conflicts                 plan generation is not a fast process
Results
                             planning is a complex task
Conclusion                   communication latency in multi-robot




     49 / 75
                     Transactions: motivation

   A Software
Component for Plan
 Management in
    Robotics
  Sylvain Joyeux
                         our plan manager relies on task and event structure
Introduction             to manage the system
Model                 ⇒ we need a tool to change the old structure into the
Execution               new one in a single step
Adaptation
Transactions
Conflicts
Results
                         plan generation is not a fast process
Conclusion
                      ⇒ we need a tool to allow simultaneous plan
                        generation and execution



     49 / 75
                     Transaction

   A Software
Component for Plan
 Management in
    Robotics
  Sylvain Joyeux         originally a concept from databases
Introduction             represents the transformation
Model                        from the plan as it is being executed
Execution                    into the new plan
Adaptation               negotiation: allows to sandbox every kind of plan
Transactions
                         change
Conflicts
Results                      structure
Conclusion
                             ownership
                             roles
                             subscription
                             ...



     50 / 75
                     Example: Dala/Ressac negotiation
                     Live data from Caylus 25/10

   A Software
Component for Plan
 Management in
    Robotics
  Sylvain Joyeux


Introduction                                    Ressac’s plan
Model
Execution
Adaptation
Transactions                                   Services::RemoteTraversabilityMapping
Conflicts
Results
Conclusion                         Bitmap::Mapping

                                                                  start

                                                      RemoteTrvMapping::Planning




     51 / 75
                     Example: Dala/Ressac negotiation
                     Live data from Caylus 25/10

   A Software
Component for Plan
 Management in
    Robotics
  Sylvain Joyeux


Introduction                                     Ressac’s plan
Model
Execution
Adaptation                                              Services::RemoteTraversabilityMapping
Transactions                     Nav::PathPlanning
Conflicts
Results                                                     RemoteTrvMapping::Planning

Conclusion                         Bitmap::Mapping



                           PlanningLoop   Dtm::Mapping Pom::Localization




     51 / 75
                     Example: Dala/Ressac negotiation
                     Live data from Caylus 25/10

   A Software
Component for Plan
 Management in
    Robotics
  Sylvain Joyeux
                                      Abstract proposal for Dala
Introduction
Model
Execution
                                       Transaction built by Ressac
Adaptation
Transactions
Conflicts
Results                                            Bitmap::Mapping

Conclusion




                                        Services::RemoteTraversabilityMapping




     52 / 75
                     Example: Dala/Ressac negotiation
                     Live data from Caylus 25/10

   A Software
Component for Plan
 Management in
    Robotics
  Sylvain Joyeux
                         Dala’s accepts and adds the RegionsOfInterest
Introduction                                  task
Model
Execution
Adaptation                            Transaction as modified by Dala
Transactions
Conflicts
Results
Conclusion                                                             Nav::PathPlanning
                                            Bitmap::Mapping


                                                                     Nav::RegionsOfInterest

                             Services::RemoteTraversabilityMapping



     53 / 75
                        Example: Dala/Ressac negotiation
                        Live data from Caylus 25/10

   A Software
Component for Plan
 Management in
    Robotics
  Sylvain Joyeux                              Final state of the transaction
Introduction
Model                                                                                     Nav::PathPlanning
Execution                                                            Bitmap::Mapping
Adaptation
Transactions                                                                            Nav::RegionsOfInter
Conflicts
                   Services::RemoteTraversabilityMapping     UAVMapping::MapAlongPath
Results
Conclusion
                                        TransferSource     MapPath     PlanningLoop
                                                                                           TransferSink



                                                      PlanningLoop




     54 / 75
                     Planning/execution conflict

   A Software
Component for Plan
 Management in
    Robotics
  Sylvain Joyeux


Introduction         The plan is executed while the transactions are being
Model                built
Execution
Adaptation
Transactions
Conflicts
Results
Conclusion           Every time the executed plan changes . . .
                     . . . compare the transaction with the new executed plan




     55 / 75
                     Example: replanning P3d::TrackPath

   A Software
Component for Plan
 Management in
    Robotics
  Sylvain Joyeux
                       Main Plan           Nav::MoveTo
Introduction
Model                              P3d::TrackPath
                                                    PlanningTask

Execution                                                             Nav::MoveTo
                      Pom::Localization      Rflex::TrackSpeedStart
Adaptation
                                    PlanningTask
Transactions
Conflicts
Results
Conclusion                                    P3d::TrackPath          P3d::TrackPath

                       Rflex::TrackSpeedStart                                 Rflex::TrackSpee


                                               PlanningTask Pom::Localization
     56 / 75
                     Planning/execution conflict

   A Software
Component for Plan
 Management in
    Robotics
  Sylvain Joyeux            Main Plan             Nav::MoveTo
Introduction
Model                                                       PlanningTask
                                           P3d::TrackPath
Execution                    failed stop                                    Nav::MoveTo
                         Pom::Localization         Rflex::TrackSpeedStart
Adaptation
                                          PlanningTask
Transactions
Conflicts
Results
Conclusion                                            P3d::TrackPath        P3d::TrackPath

                             Rflex::TrackSpeedStart                                 Rflex::Trac


                                                                                    Pom::Loca
                                                       PlanningTask

     57 / 75
                     Edition cycle: handling invalid transactions

   A Software
Component for Plan
 Management in
    Robotics
  Sylvain Joyeux                                                                                                                                Planner
Introduction                  Decision Control
Model
                                                                                                                                                Interaction
Execution              Marks the transaction as invalid                                                                                         w/ decision
Adaptation             Sends the conflict to the planner                                                                                          control
Transactions
Conflicts
Results
Conclusion                                                     Nav::MoveTo
                                                                                                                  Nav::MoveTo



                                                                                               P3d::TrackPath    P3d::TrackPath



                                                                                                                                                   Plan
                                                     P3d::TrackPath        PlanningTask




                                                                                               PlanningTask     Pom::Localization
                                   failed stop       PlanningTask     Rflex::TrackSpeedStart




                                                                                                                                      Change the generation
                                 Pom::Localization




                                                         Conflict                                                                   transaction or
                                                                                                                                       discard it


     58 / 75
                     Planning/execution conflicts

   A Software
Component for Plan
 Management in
    Robotics
  Sylvain Joyeux


Introduction
Model                    simplest solution: discard the transaction
Execution
Adaptation
                         otherwise, adapt the transaction
Transactions             the decision control can also forbid a plan change
Conflicts
                         brought by execution
Results
Conclusion




     59 / 75
                     Summary

   A Software
Component for Plan
 Management in
    Robotics
  Sylvain Joyeux
                                         Because of . . .
Introduction
Model
Execution                the way our system works
Adaptation               need for simultaneous planning & execution
Transactions
Conflicts                 need for a plan-centric negociation tool
Results
Conclusion
                                    we defined transactions


                      ⇒ sandbox to prepare plan transformations


     60 / 75
5. Results
                     Implementation: the Roby software

   A Software
Component for Plan
 Management in
    Robotics
  Sylvain Joyeux
                     A robotics integration framework through plan
Introduction
                     management
Model
Execution
Adaptation               specification and development of tasks/events
Results                  small plan generation tool
Conclusion
                         integrated applications
                         integrated simulation
                         test-driven development




    62 / 75
                     Mono-robot on Dala

   A Software
Component for Plan
 Management in                                    Roby plan
    Robotics
                                                                                                                                                                                                                                                                    Nav::MoveTo




  Sylvain Joyeux                                                                                                                                               P3d::TrackPath                                                                                 Nav::Path   PlanningTask



                                                                                                                      Sequence                           Sequence            Rflex::TrackSpeedStart                 P3d::Track                                      Nav::PathPlanning



                                                                                          P3d::ReadDemP3dPosterP3d::SetGoalParamsControl PlanningTask    VirtualTask       Rflex::BrakesOnFailureHandler                          P3d::Runner   Nav::Runner   PlanningLoop   Bitmap::Mapping Nav::ComputePath Task



                                                                                                                                       Dtm::Mapping                                                                   Bitmap::Perception                                      PlanningLoop   PlanningTask   PlanningTask




Introduction                                                                       Dtm::Perception                                    Dtm::Perception PlanningLoop       Difmap::Difmap PlanningTask


                                                                                                                                                                              successstop
                                                                                                                                                                                                              Bitmap::UpdateRegionMap


                                                                                                                                                                                                           start
                                                                                                                                                                                                                                            start     s
                                                                                                                                                                                                                                                success top
                                                                                                                                                                                                                                                                    Bitmap::Perception
                                                                                                                                                                                                                                        Bitmap::SetPosterNamesControl



                                               Dtm::FillP3dPoster Dtm::FuseDtm                         PlanningTask
                                                                                 Stereopixel::Perception              Dtm::FillP3dPoster Dtm::FuseDtm   Stereopixel::Perception                                                               PlanningTask
                                                                                                                                                                                                                        Bitmap::UpdateRegionMap                                                         Difmap::Difmap
                                                                                                                                                                                                                                                              Bitmap::FuseMap Bitmap::SetPosterNamesControl
                                                                                                                                                                              PlanningTask            Bitmap::FuseMap




Model
                                                                                                                                                                           AndGenerator
                                                                      Camera::OneShot Stereopixel::Compute                             Camera::OneShot Stereopixel::Compute                                                Bitmap::Runner                                                               Difmap::Runner



                                                                                                                                                          Camera::Runner




Execution            Integration and                                                                                                                                                  Pom::Localization




Adaptation           validation on top of an      GenoM functional layer
                                                                                                Path planning

Results              existing functional                                                         Difmap                                        Bitmap                                                     Nav                                                             Locomotion


Conclusion           layer framework                                                                                                                                                                                                                                              Rflex

                                                                                                                  Elevation                                            Rough terrain

                     (GenoM)                                    Camera                                            mapping                                                 navigation
                                                                                                                                                                                                                                              Localization

                                                                                                                                                                                        P3D
                                                                 Stereo                                                    Dtm                                                                                                                       IMU


                                                                                                                                                                                                                                                    Gyro
                                                                           Flat terrain navigation                                                                                                                                                                                                POM

                                                                                                                                                                                                                                                     GPS
                                                                                                                                                                                NDD
                                                                                      Sick                                   Aspect




    63 / 75
                     Mono-robot experiment

   A Software
Component for Plan
 Management in
    Robotics
  Sylvain Joyeux
                        duration 3-4 minutes per run for 15 meters
Introduction
Model
                     nominal cycle size 100 milliseconds
Execution            global execution statistics
Adaptation
Results                  mean execution cycle size: 105
Conclusion               7776 emissions. max: 29/cycle
                         3604 commands called. max: 18/cycle
                         281 transaction commits. max: 3/cycle




    64 / 75
                     Mono-robot experiment

   A Software
Component for Plan   Count of cycles vs. simultaneous event emissions
 Management in
    Robotics                            300

  Sylvain Joyeux
                                        250
Introduction
Model
                                        200
Execution
                      count of cycles




Adaptation
                                        150
Results
Conclusion
                                        100




                                        50




                                         0
                                              0   5   10         15         20   25   30
                                                           events emitted


                              ⇒ numerous cases of simultaneous event emission
    64 / 75
                     Mono-robot experiment

   A Software
Component for Plan   Count of cycles vs. simultaneous event calls
 Management in
    Robotics                            200

  Sylvain Joyeux


Introduction
                                        150
Model
Execution
                      count of cycles




Adaptation
                                        100
Results
Conclusion

                                        50




                                         0
                                              0   5        10         15   20
                                                      events called


                              ⇒ numerous cases of simultaneous event calls
    64 / 75
                     Mono-robot experiment

   A Software
Component for Plan
                     Count of cycles vs. transaction commits
 Management in                          50
    Robotics
  Sylvain Joyeux
                                        40
Introduction
Model
                                        30
Execution
                      count of cycles




Adaptation
Results                                 20

Conclusion

                                        10




                                        0
                                             0   1   2                   3   4   5
                                                         events called


                              ⇒ numerous cases of simultaneous commits of
    64 / 75                     non-conflicting transactions
                     Bi-robot setup

   A Software
Component for Plan
 Management in
                      ⇒ both robots have their own Roby controller
    Robotics
                      ⇒ failed because of functional layer issues on Dala
  Sylvain Joyeux


Introduction
                                                                                  Roby plan                                                                                                                                                                                                                              Roby plan
Model                                                                                                                                                                                                                                         Nav::MoveTo


                                                                                                                                                                                                                                                                                                                                 Genom::Bitmap::Mapping
                                                                                                                                         P3d::TrackPath                                                                                 Nav::Path   PlanningTask



                                                                                                Sequence                           Sequence            Rflex::TrackSpeedStart                 P3d::Track                                      Nav::PathPlanning




Execution                                                           P3d::ReadDemP3dPosterP3d::SetGoalParamsControl PlanningTask




                                                             Dtm::Perception
                                                                                                                 Dtm::Mapping
                                                                                                                                   VirtualTask




                                                                                                                Dtm::Perception PlanningLoop
                                                                                                                                                     Rflex::BrakesOnFailureHandler




                                                                                                                                                   Difmap::Difmap PlanningTask
                                                                                                                                                                                                            P3d::Runner



                                                                                                                                                                                                Bitmap::Perception



                                                                                                                                                                                        Bitmap::UpdateRegionMap
                                                                                                                                                                                                                          Nav::Runner




                                                                                                                                                                                                                          success top
                                                                                                                                                                                                                      start     s
                                                                                                                                                                                                                                        PlanningLoop   Bitmap::Mapping Nav::ComputePath Task



                                                                                                                                                                                                                                                        PlanningLoop



                                                                                                                                                                                                                                              Bitmap::Perception
                                                                                                                                                                                                                                                                       PlanningTask   PlanningTask


                                                                                                                                                                                                                                                                                                     Dtm::Mapping
                                                                                                                                                                                                                                                                                                                    Pom::Localization   PlanningLoop      Bitmap::Perception
                                                                                                                                                                                                                                                                                                                                                                                UAVMapping::MapAlongPath
                                                                                                                                                                                                                  Bitmap::SetPosterNamesControl


                                                                                                                                                        successstop                  start
                                                                                                                                                                                                                                                                                                                                 Nav::RegionsOfInterest
                         Dtm::FillP3dPoster Dtm::FuseDtm                         PlanningTask
                                                           Stereopixel::Perception              Dtm::FillP3dPoster Dtm::FuseDtm   Stereopixel::Perception                                                               PlanningTask
                                                                                                                                                                                                  Bitmap::UpdateRegionMap                                                         Difmap::Difmap
                                                                                                                                                                                                                                        Bitmap::FuseMap Bitmap::SetPosterNamesControl
                                                                                                                                                        PlanningTask            Bitmap::FuseMap                                                                                                                                                                 TransferSink


Adaptation
                                                                                                                                                                                                                                                                                                                    Nav::PathPlanning
                                                Camera::OneShot Stereopixel::Compute                             Camera::OneShot Stereopixel::Compute
                                                                                                                                                     AndGenerator
                                                                                                                                                                                                     Bitmap::Runner                                                               Difmap::Runner                                                                               PlanningLoop   MapPath   TransferSource

                                                                                                                                    Camera::Runner



                                                                                                                                                                Pom::Localization                                                                                                                                                                                ZoneMapping::Export




Results
                                        GenoM
Conclusion                          functional layer                                                                                                                                                                                                                                                       Ressac Server
                                                                      Path planning

                                                                       Difmap                                      Bitmap                                              Nav                                                              Locomotion

                                                                                                                                                                                                                                               Rflex

                                                                                       Elevation                                       Rough terrain
                                        Camera                                         mapping                                            navigation
                                                                                                                                                                                                            Localization

                                                                                                                                                          P3D
                                         Stereo                                                 Dtm                                                                                                                  IMU


                                                                                                                                                                                                                     Gyro
                                                  Flat terrain navigation                                                                                                                                                                                         POM

                                                                                                                                                                                                                     GPS
                                                             Sick                                 Aspect                                         NDD




    65 / 75
                     Bi-robot in simulation

   A Software
Component for Plan
 Management in                                                  Difmap             Bitmap          Nav
    Robotics
  Sylvain Joyeux                               Camera



Introduction                                    Stereo                   Dtm
                                                                                            P3D


Model
                      Simulation engine
                                            A priori
Execution                                   DEM file

Adaptation
                     ⇒ functional layer
                                                                                            NDD
                                                         Sick             Aspect
Results              almost used as-is
Conclusion
                     ⇒ time control: time                                                          Rflex

                                                   Pocosim
                     synchronization           simulation engine
                                                                                            IMU
                     between simulated
                     robots                                                                 Gyro           POM
                                                     Gazebo
                                                  world/sensor
                                                                                            GPS
                                                   simulation



    66 / 75
                     Bi-robot simulation results

   A Software
Component for Plan
 Management in
    Robotics
  Sylvain Joyeux
                         duration 30 minutes per run for 400 meters
Introduction         nominal cycle size 100 milliseconds
Model
Execution
                     global execution statistics
Adaptation
Results
                           mean execution cycle size: 110
Conclusion                 23490 emissions. max: 24/cycle
                           11518 commands called. max: 15/cycle
                           1180 transaction commits. max: 3/cycle
                       Video




    67 / 75
                     Summary

   A Software
Component for Plan
 Management in
    Robotics                   What still requires validation ?
  Sylvain Joyeux


Introduction            extensive use of plan adaptation operators
Model                   transaction conflict resolution
Execution
                        multi-robot validation – only bi-robot
Adaptation
Results
Conclusion                               Conclusion

                        flexible enough to adapt to existing systems
                        fast enough to manage a real-world, complex
                        system
                        built around the aim of being an application
                        integration framework for robotics
    68 / 75
6. Conclusion & Future Work
                     Contributions

   A Software
Component for Plan
 Management in
    Robotics
  Sylvain Joyeux
                       1   A rich plan model
Introduction
                               with separated activity and temporal structures
Model
                               task graphs
Execution                      adapted to multi-robot systems
Adaptation
Results
Conclusion
Contributions
Future Work




     70 / 75
                     Contributions

   A Software
Component for Plan
 Management in
    Robotics
  Sylvain Joyeux
                       1   A rich plan model
Introduction
Model
                       2   A complete execution scheme
Execution
Adaptation
                               able to handle the specificities of the plan model
Results
                               rich error handling procedures
                               multi-robot plan execution
Conclusion
Contributions
Future Work




     70 / 75
                     Contributions

   A Software
Component for Plan
 Management in
    Robotics
  Sylvain Joyeux


Introduction
                       1   A rich plan model
Model
Execution              2   A complete execution scheme
Adaptation
Results                3   A tool for simultaneous plan execution and
Conclusion                 adaptation
Contributions
Future Work                    adaptation of the notion of transaction from DBs
                               usable as whiteboard in multi-robot context




     70 / 75
                     Future work

   A Software
Component for Plan
 Management in
    Robotics
  Sylvain Joyeux


Introduction
                       1   State and time prediction
Model                          explicit representation of time
Execution                      integration of temporal constraint networks
Adaptation
                               integration of a causal model: state projection
Results
Conclusion
Contributions
Future Work




     71 / 75
                     Future work

   A Software
Component for Plan
 Management in
    Robotics
                       1   State and time prediction
  Sylvain Joyeux


Introduction
                       2   Planner/planner interaction through plans
Model                          integration of real planners from diverse plan
                               models using state projection
Execution
                                    plan ahead-of-time
Adaptation
                                    forestall faults related to changes in the projection
Results                             integration of repair capabilities
Conclusion
                               definition of loose planner-planner interaction
Contributions
Future Work
                               protocols
                                ⇒ plan merging
                               integration of multi-robot cooperation protocols




     71 / 75
                     Future work

   A Software
Component for Plan
 Management in
    Robotics
  Sylvain Joyeux
                       1   State and time prediction
Introduction
Model
                       2   Planner/planner interaction through plans
Execution
Adaptation
Results
                       3   Long term goals
Conclusion                     formalization of the approach
Contributions                   ⇒ prove equivalence between our model and others
Future Work                     ⇒ represent execution properties in a formal model
                               getting it used by more than one person




     71 / 75
Thank you for your
   attention !
                     Looping: global structure

   A Software
Component for Plan
 Management in
    Robotics
  Sylvain Joyeux


Appendix                            start    success     stop
                                       PlanningTask(i+1)




                                    start    success     stop
                                            LoopActivity(i)




                       Return




    73 / 75
                     Looping: global structure

   A Software
Component for Plan
 Management in
    Robotics
  Sylvain Joyeux


Appendix                start    success     stop       start    success    stop
                           PlanningTask(i+1)                    PlanningTask(i+2)



                                                    &

                        start    success     stop       start    success    stop
                                LoopActivity(i)             LoopActivity(i+1)




                       Return




    73 / 75
                     Looping: global structure

   A Software
Component for Plan
 Management in
    Robotics
  Sylvain Joyeux


Appendix                start    success     stop       start    success    stop
                            PlanningTask(i+1)                   PlanningTask(i+2)



                                                    &

                        start    success     stop       start    success    stop
                                LoopActivity(i)             LoopActivity(i+1)




                       Return




    73 / 75
                     Looping: global structure

   A Software
Component for Plan
 Management in
    Robotics
  Sylvain Joyeux


Appendix
                      start    success     stop                    start    success    stop
                         PlanningTask(i+1)                                 PlanningTask(i+2)
                                                  Delayed event

                                                  &           ||                               &

                      start    success     stop                    start    success    stop
                              LoopActivity(i)                          LoopActivity(i+1)




                       Return




    73 / 75
                     Looping: global structure

   A Software
Component for Plan
 Management in
    Robotics
  Sylvain Joyeux


Appendix
                      start    success     stop                start    success    stop
                         PlanningTask(i+1)                             PlanningTask(i+2)



                                                  &       ||                               &

                                                      &
                      start    success     stop                start    success    stop
                              LoopActivity(i)                      LoopActivity(i+1)




                       Return




    73 / 75
                     So, what if the algorithms/planning are not
                     fast enough ?
   A Software
Component for Plan
 Management in                  PosDeltaEvent
    Robotics
  Sylvain Joyeux

                                 OrGenerator
Appendix


                                UntilGenerator
                                                  Dtm::Mapping




                                   loop_start
                                                     Task        Dtm::Perception
                                 PlanningLoop




                                                 PlanningTask
                                 AndGenerator



                       Return
    74 / 75
                     So, what if the algorithms/planning are not
                     fast enough ?
   A Software
Component for Plan
 Management in
    Robotics                        success stop
  Sylvain Joyeux
                                    Dtm::Perception

Appendix
                                loop_success   loop_end
                                                          AndGenerator
                                    PlanningLoop



                                         updated_data
                                         Dtm::Mapping



                                               Task



                                         PlanningTask



                       Return
    74 / 75
                     So, what if the algorithms/planning are not
                     fast enough ?
   A Software
Component for Plan
 Management in
    Robotics
                       Dtm::Mapping
  Sylvain Joyeux
                                                                                      AndGenerator
Appendix
                                                                         start
                           Task
                                                                    Dtm::Perception



                                        loop_start   success stop

                                     PlanningLoop    PlanningTask



                                    start
                                                                             AndGenerator
                                PlanningTask




                       Return

    74 / 75
                     Fault injection pattern

   A Software
Component for Plan                                                interrupted failed stop
 Management in
    Robotics                                                           P3d::Track
  Sylvain Joyeux


Appendix                                                              FilterGenerator
                        replace the real task
                        with a fake one
                                                                   blocked failed stop
                                                    Nav::MoveTo
                        when the real task is                           P3d::Track

                        properly interrupted,
                        forward its stop event to                  blocked failed stop

                        the wanted fault event                        P3d::TrackPath




                                                                                       Sequen


    75 / 75

								
To top