HMI-talk

Document Sample
HMI-talk Powered By Docstoc
					Using Robots in Autism Therapy:
 A Survey of Ongoing Research


              Marjorie Skubic
               Associate Professor
   Electrical and Computer Engineering Dept.
       Computer Science Dept. (joint apt.)
                   Outline
 Motivation – How I got interested
 Autistic disorders
 A survey of the research
    – Why robots might help
 The field of researchers
 Conclusions
        How I got interested
 Research in Human-Robot Interaction
 Looking for a killer application
 Better – How can we use robots to help people?
 Talks at the IEEE RO-MAN 2005 Workshop
                 Autistic Disorders
   1 of 300 children diagnosed with autism with rates rising
    – 1 of 800 children diagnosed with Down syndrome
    – 1 of 450 children diagnosed with juvenile diabetes
    – 1 of 333 children will develop cancer by age 20
   Diagnosis currently made through behavioral
    observation
    – No blood test or genetic screening is available although there is
      evidence of a genetic link
Autistic Disorders: Characteristics
  Inability to relate to other people
  Little use of eye contact with other people
  Difficulty understanding gestures and facial
   expressions
  Difficulties with verbal & non-verbal
   communication
  Difficulty understanding other’s intentions,
   feelings, and mental states
            Why Use Robots?
 Most children, including children with autism, are
  attracted to robots.
 This natural affinity is exploited, and the robot is
  used as an interactive toy.
 Robots may provide a less threatening
  environment than interacting with people.
    – Robots can provide a repetitive and more predictable
      environment.
    – This “safe” environment can gently push a child with
      autism towards human interaction.
       The Connection to Imitation
   One theory: Autism may be caused by early
    impairments in imitation and shared attention
    (Rogers & Pennington, 1991) (Baron-Cohen, 1995)

   Imitation is a format of communication, a means to
    express interest and engage others in interaction
    (Nadel, 1999)

   Idea: Use a doll-like robot to engage children with
    autism and teach basic imitative interaction skills
     – From: K. Dautenhahn, and A. Billard, Games Children with Autism Can
       Play With Robota, a Humanoid Robotic Doll, Proc. 1st Cambridge
       Workshop on Universal Access and Assistive Technology, 2002
                        Robota




A six-year old autistic boy playing with Robota. He seemed curious
about Robota's head movements and so he touches the doll.
From: K. Dautenhahn, and A. Billard, Games Children with Autism
Can Play With Robota, a Humanoid Robotic Doll, Proc. 1st
Cambridge Workshop on Universal Access and Assistive
Technology, 2002
            Imitation Using Robota




“Robota … allows the child to understand that the doll’s movement
originates from his own movement (sense of agency) and is limited to a
restricted category of movement (enhances intentional action)”
From: J. Nadel, “Early Imitation and a Sense of Agency,” Proc. 4th Intl.
Workshop on Epigenetic Robots, 2004
An autistic child playing “chasing” games with the mobile robot
From: K. Dautenhahn, and A. Billard, Games Children with Autism
Can Play With Robota, a Humanoid Robotic Doll, Proc. 1st
Cambridge Workshop on Universal Access and Assistive
Technology, 2002
   Joint Attention Using Robota




  Robota is controlled via teleoperation by the investigator.

From: B. Robins, P. Dickerson, and K. Dautenhahn, “Robots as
Embodied Beings – Interactionally Sensitive Body Movements
In Interactions Among Autistic Children and a Robot,”
Proc. RO-MAN 2005
                                              Two autistic children:
                                              Note Andy’s gaze at
                                              Jack.


The investigator encourages the children to show each other how
they can interact with the robot.

The robot will not move unless the children show the same movement,
i.e., they must work together.
Andy and Jack touch each other to balance themselves while
each raising a leg.
                         Adam shows no interest in his classmates
                         and usually tries to avoid the rest of the
                         children. But Adam is interested in Robota.




Adam takes Rob’s hand to show him how to interact with Robota.
      Interacting with Keepon




        Keepon is controlled via teleoperation.

From: H. Kozima, C. Nakagawa, and Y. Yasuda, “Interactive
Robots for Communication-Care: A Case Study in Autism
Therapy,” Proc. RO-MAN 2005
Views from Keepon’s camera eyes
                                         Attentive action




                                         Emotive action
Keepon's kinematic mechanism. Two
gimbals are connected by four wires;
the lower gimbal is driven by two
motors. Another motor rotates the
whole inner-structure; yet another
drives the skull downward for bobbing.
          Enabling Interaction




 Eye-contact: Referring    Joint attention: Sharing the
 to each other's mental    perceptual information
 states
Enables people to exchange intention and emotion toward a targ
Emergence of dyadic interaction. Spontaneous
actions to Keepon (left) and actions copied from
others (right).




Emergence of triadic interaction. The child discovers
excitement in Keepon (left) and then looks at the adult
to share the excitement (right).
Using Robots for Autism Diagnosis




          ESRA                             Playtest

From: B. Scassellati, “Quantitative Metrics of Social Response
for Autism Diagnosis,” Proc. RO-MAN 2005
    Autism Diagnosis Methods
   Reaction to the ESRA robot with and
    without the face configuration


                          Can generate facial expressions
                          using 5 servo motors
    Autism Diagnosis Methods
   Measure listening preferences to speech sounds



                              At the press of a button,
                              an audio clip is played.
                              The interaction is logged
                              in non-volatile memory.
      Autism Diagnosis Methods
     Vocal prosody, i.e., how something is said


                                Features F24 vs. F1
                                Mean pitch * energy vs. mean pitch




Separation of two features used in a Bayesian classifier
distinguishes low energy categories (neutral and soothing) from
high energy categories (approval, attention, and prohibition).
    Autism Diagnosis Methods
   Position tracking relative to another person
    Autism Diagnosis Methods
   Gaze direction and focus of attention




             Red – adolescents with autism
             Blue – typical adolescents
                                                        Linear discriminant
                                                        analysis of autistic
                                                        (au) and typical (nc)
                                                        gaze patterns.
                                                        Linear filters F(x)
                                                        are trained to
                                                        reproduce the gaze
                                                        pattern G(x) of each
                                                        individual x and
                                                        then applied to
                                                        predict the gaze
                                                        patterns of any
                                                        other individual.
For example, F(au)*G(self) indicates a filter trained on an individual with
autism and tested on that same individual while F(nc)*G(au) indicates a
filter trained on a control individual and tested on an individual with
autism. The mean performance of this data (y-axis) is a function of the
response percentile of individual pairings. Significant differences (all
p<0.01 for a two-tailed t-test) are seen between the following classes: (1)
F(nc)*G(self), (2) F(au)*G(self), (3) F(nc)* G(other nc), and (4) the three
other conditions.
      University of Sherbrooke
   Project for engineering students:
    – Design a robotic toy for an autistic child
   Educational value
    – Real world problem
    – Students work together in a team
    – Students must first investigate autistic disorders
          University of Sherbrooke




   Pushing Jumbo around the             Rolling game with Roball.
   play area.

From: Michaud, F., Théberge-Turmel, C. (2002), "Mobile robotic toys and
autism", Socially Intelligent Agents - Creating Relationships with
Computers and Robots, Kluwer, pp. 125-132.
          University of Sherbrooke




       Assembling the arms             Girl showing signs of interest
       and tail of C-Pac.              toward Bobus.

From: Michaud, F., Théberge-Turmel, C. (2002), "Mobile robotic toys and
autism", Socially Intelligent Agents - Creating Relationships with
Computers and Robots, Kluwer, pp. 125-132.
     The Field of Researchers
   Francois Michaud
    – University of Sherbrooke, Canada

   Kerstin Dautenhahn & Ben Robbins
    – University of Hertfordshire, UK

   Aude Billard
    – Swiss Federal Institute of Technology (EPFL)

   Jacqueline Nadel
    – French National Centre of Scientific Research
     The Field of Researchers
   Brian Scassellati and Bob Schultz
    – Yale University

   Javier Movellan
    – University of California – San Diego

   Hideki Kozima
    – National Institute of ICT, Japan

   Michio Okada
    – ATR, Kyoto, Japan
             Conclusions
 The use of robots for autism therapy and
  diagnosis is just beginning.
 There is anecdotal evidence that robot
  therapy can help children with autism
 How can we start here at MU with the new
  Thompson Family Center for Autism and
  Neurodevelopmental Disorders?
Maybe the Tiger Kitty




  The iCat by Philips Research
         Acknowledgements
   Thanks to Brian Scassellati, Francois
    Michaud, Ben Robins, and Hideki Kozima
    for helpful discussions.