Documents
Resources
Learning Center
Upload
Plans & pricing Sign in
Sign Out
Your Federal Quarterly Tax Payments are due April 15th Get Help Now >>

Monte Carlo Hidden Markov Models (PowerPoint)

VIEWS: 11 PAGES: 60

									Statistical Learning in Robotics
 State-of-the-Art, Challenges and Opportunities


                                Sebastian Thrun
                                 Carnegie Mellon University
   Robotics
Research Today


                                     Estimation and
                                      Learning In
                                        Robotics


                                                      7 Open Problems




Sebastian Thrun   Carnegie Mellon University                ICML   July 10-12, 2002
Robotics Yesterday




 Sebastian Thrun   Carnegie Mellon University   ICML   July 10-12, 2002
Robotics Today




 Sebastian Thrun   Carnegie Mellon University   ICML   July 10-12, 2002
Robotics Tomorrow?




                                                             Thanks to
                                                            T. Dietterich
 Sebastian Thrun   Carnegie Mellon University   ICML   July 10-12, 2002
Robotics @ CMU, 1992




 Sebastian Thrun   Carnegie Mellon University   ICML   July 10-12, 2002
Robotics @ CMU, 1994




 Sebastian Thrun   Carnegie Mellon University   ICML   July 10-12, 2002
Robotics @ CMU 1996




                                                With: RWI / iRobot, Hans Nopper
 Sebastian Thrun   Carnegie Mellon University             ICML    July 10-12, 2002
 Robotics @ CMU/UBonn, 1997




with W. Burgard, A.B. Cremers, D. Fox, D. Hähnel, G. Lakemeyer, D. Schulz, W. Steiner
     Sebastian Thrun   Carnegie Mellon University               ICML   July 10-12, 2002
Robotics @ CMU, 1998




   with M. Beetz, M. Bennewitz, W. Burgard, A.B. Cremers, F. Dellaert, D. Fox,
   D. Hähnel, C. Rosenberg, N. Roy, J. Schulte, D. Schulz
 Sebastian Thrun   Carnegie Mellon University                ICML   July 10-12, 2002
   Robotics
Research Today


                                     Estimation and
                                      Learning In
                                        Robotics


                                                      7 Open Problems




Sebastian Thrun   Carnegie Mellon University                ICML   July 10-12, 2002
The Robot Localization Problem




  • Position tracking (error bounded)
  • Global localization (unbounded error)
  • Kidnapping (recovery from failure)

 Sebastian Thrun   Carnegie Mellon University   ICML   July 10-12, 2002
Probabilistic Localization
          p(x0 | m)




       p(z0 | x, m)


       p(x0 | z0, m)




     p(x1|u1,z0,m)




       p(z1 | x, m)
                                                           [Simmons/Koenig 95]
                                                             [Kaelbling et al 96]
                                                              [Burgard et al 96]
 p(x1| ,z1 ,u1,z0,m)
                                                               [Thrun et al 96]

  Sebastian Thrun   Carnegie Mellon University   ICML   July 10-12, 2002
                                                                                      x = state
  Probabilistic Localization                                                          t = time
                                                                                      m = map
                                                                                      z = measurement
                                                      p(xt|xt-1,ut)                   u = control

                                      xt-1
p( xt | z0t , u0t , m)                              ut
             Bayes
                 p( zt | xt , z0t 1 , umt , m) p( xt | z0t 1 , u0t , m)
                                 map 0
                   laser data                                    p(z|x,m)
            Markov
                 p( zt | xt , m) p( xt | z0t 1 , u0t , m)
                 p( zt | xt , m)  p( xt | xt 1, z0t 1, u0t , m) p( xt 1 | z0t 1, u0t , m) dxt 1

                 p( zt | xt , m)  p( xt | xt 1, ut ) p( xt 1 | z0t 1, u0t 1, m) dxt 1
            Markov




                                                                            [Kalman 60, Rabiner 85]
       Sebastian Thrun   Carnegie Mellon University                            ICML    July 10-12, 2002
      What is the Right Representation?



                     Kalman filter                                                Multi-hypothesis
       [Schiele et al. 94], [Weiß et al. 94], [Borenstein 96],        [Weckesser et al. 98], [Jensfelt et al. 99]
                [Gutmann et al. 96, 98], [Arras 98]




                       Histograms
                    (metric, topological)                                                 Particles
[Nourbakhsh et al. 95], [Simmons et al. 95], [Kaelbling et al. 96],             [Kanazawa et al 95] [de Freitas 98]
           [Burgard et al. 96], [Konolige et al. 99]                            [Isard/Blake 98] [Doucet 98]


            Sebastian Thrun   Carnegie Mellon University                                  ICML      July 10-12, 2002
Monte Carlo Localization (MCL)
         p(x0 | m)




       p(z0 | x, m)


      p(x0 | z0, m)




     p(x1|u1,z0,m)




       p(z1 | x, m)


 p(x1| ,z1 ,u1,z0,m)


 Sebastian Thrun   Carnegie Mellon University   ICML   July 10-12, 2002
Monte Carlo Localization (MCL)




                                                With: Wolfram Burgard, Dieter Fox, Frank Dellaert

 Sebastian Thrun   Carnegie Mellon University                            ICML    July 10-12, 2002
Implications for Planning & Control
             MDP Planner                        POMDP Planner




                                                                        With N. Roy
 Sebastian Thrun   Carnegie Mellon University              ICML   July 10-12, 2002
Monte Carlo Localization




                                                                          With:
                                                                          Frank
                                                                          Dellaert

 Sebastian Thrun   Carnegie Mellon University   ICML   July 10-12, 2002
Sebastian Thrun   Carnegie Mellon University   ICML   July 10-12, 2002
Sebastian Thrun   Carnegie Mellon University   ICML   July 10-12, 2002
Learning Maps
aka Simultaneous Localization and Mapping (SLAM)




                      70 m




   Sebastian Thrun   Carnegie Mellon University    ICML   July 10-12, 2002
  Learning Maps
 Localization:
p( xt | z0t , u0t , m)  p( zt | xt , m)  p( xt | xt 1, ut ) p( xt 1 | z0t 1, u0t 1, m) dxt 1


p( xt , mt | z0t , u0t )  p( zt | xt , mt )  p( xt , mt | xt 1, mt 1, ut ) p( xt 1, mt 1 | z0t 1, u0t 1 ) dxt 1dmt 1



    p( xt , m | z0t , u0t )  p( zt | xt , m)  p( xt | xt 1, ut ) p( xt 1, m | z0t 1, u0t 1 ) dxt 1



                          3 dimensions                          106 dimensions


        Sebastian Thrun   Carnegie Mellon University                                        ICML     July 10-12, 2002
Learning Maps with Extended
Kalman Filters


                                                  l1        l21     l1l2       l1lN  l1x  l1 y  l1 
                                                                                                              
                                                  l2        l1l2    l22        l2lN  l2 x  l2 y  l2 
                                                                                                   
                                                                                                              
                   p( xt , m | z1t , u1t )     lN  ,    l1lN     l 2l N     l N  l N x  l N y  l N 
                                                                                        2

                                                 x                                                            
                                                              l1x     l2 x       lN x  x  xy  x 
                                                                                              2
                                                  
                                                  y       l y       l2 y       lN y  xy  y2  y 
                                                           1                                                  
                                                          l       l2        lN  x  y        2 
                                                             1

                                                                                   [Smith, Self, Cheeseman, 1990]

 Sebastian Thrun    Carnegie Mellon University                                         ICML    July 10-12, 2002
Kalman Filter Mapping: O(N2)




  Sebastian Thrun   Carnegie Mellon University   ICML   July 10-12, 2002
Can We Do the Same With
Particle Filters?

                                                                          sample map + pose
      robot poses and maps




                                            p( xt , m | z0t , u0t )

 Sebastian Thrun   Carnegie Mellon University                           ICML   July 10-12, 2002
Mapping: Structured Generative Model
  Landmark

       m1
                        z1      measurement                z3

                s1                   s2               s3            ...          st
robot pose

             control        u2                   u3                        ut

                                            z2                                        zt
       m2
                                                                N
 p(m, s0t | z0t , u0t )  p ( s0t | z0t , u0t )  p(mn | s0t , z0t , u0t )
                                                                n 1

                                                       With K. Murphy, B. Wegbreit and D. Koller
  Sebastian Thrun   Carnegie Mellon University                            ICML   July 10-12, 2002
   Rao-Blackwellized Particle Filters
                                                                         N
        p(m, s0t | z0t , u0t )  p ( s0t | z0t , u0t )  p(mn | s0t , z0t , u0t )
                                                                        n 1




                                                                                     …
robot poses



                                                      landmark n=1   landmark n=2                landmark n=N




                                                                     p ( mn | xt[ i ] , 0t , u0t )


                                                                                     …
                                                      landmark n=1   landmark n=2                landmark n=N


      xt[ i ] ~ p ( xt | z0t , u0t )
                                                                                   [Murphy 99, Montemerlo 02]
       Sebastian Thrun   Carnegie Mellon University                                      ICML     July 10-12, 2002
Ben Wegbreit’s Log-Trick
                                                                                      n4?
                                                                          T
     new particle                                                                                   F
                                           n2?
                                                              F
                                           T

                                                                n3?

                                                            T                 F

                                                     m3,S3
                                                      [i]         [i]




                                                                                      n4?
                                                                                      k
                                                                          T                                   F
     old particle
                                           n2?
                                           k                                                                                n6?
                                                                                                                            k

                                 T                            F                                                   T                      F


                         n1?
                         k                                      n3?
                                                                k                                   n1?
                                                                                                    k 5                                  n3?
                                                                                                                                         k 7

                   T              F                         T                     F           T                    F               T                    F


             m1,S1              m2,S2                m3,S3                    m4,S4 m5,S5                         m6,S6           m7,S7              m8,S8
                   [i]   [i]         [i]       [i]      [i]         [i]           [i]   [i]   [i]       [i]           [i]   [i]    [i]       [i]        [i]   [i]




 Sebastian Thrun             Michael Montemerlo,
                         Carnegie Mellon University Ben Wegbreit, Daphne Koller & Sebastian Thrun
                                                                                                ICML                                               July 10-12, 2002
Advantage of Structured PF Solution

     Kalman: O(N2)                                Rao-B’ PFs: O(MlogN)


      500 features                              Moore’s Theorem: logN  30
                                                  Experimental: M=250


                                                  1,000,000 features

      + global uncertainty, multimodal
      + non-linear systems
      + sampling over data associations
 Sebastian Thrun   Carnegie Mellon University               ICML   July 10-12, 2002
3 Examples


                Particles +
               Kalman filters                   Particles +
                                                 Particles

                              Particles +
                            Point Estimators




 Sebastian Thrun   Carnegie Mellon University          ICML   July 10-12, 2002
Outdoor Mapping (no GPS)




    With Juan Nieto, Jose Guivant, Eduardo Nebot, Univ of Sydney
 Sebastian Thrun   Carnegie Mellon University        ICML   July 10-12, 2002
   With Juan Nieto, Jose Guivant, Eduardo Nebot, Univ of Sydney
Sebastian Thrun   Carnegie Mellon University        ICML   July 10-12, 2002
Tracking Moving Features




                                                 With: Michael Montemerlo
 Sebastian Thrun   Carnegie Mellon University   ICML   July 10-12, 2002
Tracking Moving Entities Through
Map Differencing




 Sebastian Thrun   Carnegie Mellon University   ICML   July 10-12, 2002
Map-Based People Tracking




                                                 With: Michael Montemerlo
 Sebastian Thrun   Carnegie Mellon University   ICML   July 10-12, 2002
Autonomous People Following




                                                 With: Michael Montemerlo
 Sebastian Thrun   Carnegie Mellon University   ICML   July 10-12, 2002
Indoor Mapping




   Map: point estimators (no uncertainty)
   Lazy

    Sebastian Thrun   Carnegie Mellon University   ICML   July 10-12, 2002
Importance of Probabilistic Component




   Non-probabilistic                              Probabilistic, with samples


   Sebastian Thrun   Carnegie Mellon University             ICML   July 10-12, 2002
Multi-Robot Exploration




  DARPA TMR Texas                                DARPA TMR Maryland




                                                       With: Reid Simmons and Dieter Fox
  Sebastian Thrun   Carnegie Mellon University              ICML    July 10-12, 2002
Learning Object Models




 Sebastian Thrun   Carnegie Mellon University   ICML   July 10-12, 2002
Nearly Planar Maps




Idea: Exploit fact that buildings posses many planar surfaces
 Compacter models

 Higher Accuracy

 Good for capturing environmental change




   Sebastian Thrun   Carnegie Mellon University     ICML   July 10-12, 2002
Online EM and Model Selection




          raw data                              mostly planar map




 Sebastian Thrun   Carnegie Mellon University           ICML   July 10-12, 2002
Online EM and Model Selection




    CMU Wean Hall                               Stanford Gates Hall



 Sebastian Thrun   Carnegie Mellon University           ICML   July 10-12, 2002
3D Mapping Result




With: Christian Martin
 Sebastian Thrun   Carnegie Mellon University   ICML   July 10-12, 2002
  Combining Tracking and Mapping




With Dirk Hähnel, Dirk Schulz and Wolfram Burgard
     Sebastian Thrun   Carnegie Mellon University   ICML   July 10-12, 2002
  Combining Tracking and Mapping




With Dirk Hähnel, Dirk Schulz and Wolfram Burgard
     Sebastian Thrun   Carnegie Mellon University   ICML   July 10-12, 2002
Underwater Mapping                                           (with University of Sydney)




With: Hugh Durrant-Whyte, Somajyoti Majunder, Marc de Battista, Steve Scheding
  Sebastian Thrun   Carnegie Mellon University                      ICML   July 10-12, 2002
   Robotics
Research Today


                                     Estimation and
                                      Learning In
                                        Robotics


                                                      7 Open Problems




Sebastian Thrun   Carnegie Mellon University                ICML   July 10-12, 2002
Can We Learn Better Maps?


   Stationary objects and moving
    objects, people
   Motion characteristics, relational knowledge
   Less structured environments (jungle, underwater)
   In real-time




    Sebastian Thrun   Carnegie Mellon University   ICML   July 10-12, 2002
Can We Learn Control?




                       : p( xt | z 0..t , u 0..t )  u t 1

      Not an MDP
      Not discrete or low-dimensional
      Not knowledge-free


    Sebastian Thrun    Carnegie Mellon University       ICML   July 10-12, 2002
How Can We Learn in Context?

Goal of robotics is not …
   mapping

   classification

   clustering

   density estimation

   reward prediction

   …


But simply: Doing the right thing.

  Sebastian Thrun   Carnegie Mellon University   ICML   July 10-12, 2002
How can we exploit Domain
Knowledge in Learning?
Test: Is hypothesis consistent with
   laws of geometry?

   laws of physics?

   conventional wisdom?

   …


Domain knowledge is your friend!
   ILP?

   “Lifelong” learning?


  Sebastian Thrun   Carnegie Mellon University   ICML   July 10-12, 2002
Can we Integrating Learning
and Programming?




Programming                                                                              Learning

                                   prob<int> x = {{10, 0.2}, {11, 0.8}};
                                   prob<int> y = {{20, 0.5}, {21, 0.5}};
                                   prob<int> z = x + y;
                                   prob<double> f = neuroNet(y);

                                                           with Frank Pfenning, CMU



 Sebastian Thrun   Carnegie Mellon University                                     ICML    July 10-12, 2002
What Can We Learn
From Biology?




                                                Courtesy of Bill Skaggs, University of Pittsburgh
 Sebastian Thrun   Carnegie Mellon University                            ICML   July 10-12, 2002
…And Can We Actually Do
Something Useful?


                                                University of Pittsburgh
                                                 School of Nursing
                                                      Prof. Jackie Dunbar-Jacob
                                                      Prof. Sandy Engberg
                                                      Prof. Margo Holm
                                                      Prof. Deb Lewis
                                                      Prof. Judy Matthews
                                                      Prof. Barbara Spier
                                                 School of Medicine
                                                      Prof. Neil Resnick
                                                      Prof. Joan Rogers
                                                 Intelligent Systems
                                                      Prof. Don Chiarulli

                                                University of Pittsburgh
                                                 Computer Science
                                                      Prof. Martha Pollack


                                                Carnegie Mellon University
                                                 Computer Science, Robotics
                                                       Prof. Sebastian Thrun
                                                       Prof. Geoff Gordon
                                                 Human Computer Interaction
                                                       Prof. Sara Kiesler
                                                Financial Support
                                                 National Science Foundation
                                                       $1.4M ITR Grant
                                                       $3.2M ITR Grant

 Sebastian Thrun   Carnegie Mellon University       ICML         July 10-12, 2002
The Nursebot Project




 Sebastian Thrun   Carnegie Mellon University   ICML   July 10-12, 2002
Haptic Interface (In Development)




 Sebastian Thrun   Carnegie Mellon University   ICML   July 10-12, 2002
Wizard of Oz Studies




                                                By Sara Kiesler, Jenn Goetz
 Sebastian Thrun   Carnegie Mellon University         ICML   July 10-12, 2002
Truly Useful….?




 Sebastian Thrun   Carnegie Mellon University   ICML   July 10-12, 2002

								
To top