Dancing Cheek to Cheek

Document Sample
Dancing Cheek to Cheek Powered By Docstoc
					  Dancing Cheek to Cheek:
  Haptic communication between
partner dancers and swing as a finite
           state machine

            Sommer Gentry

           Advisor: Eric Feron
 Readers: Munther Dahleh, Muriel Medard,
              Jeff Shamma
        Why Swing Dancing?
• Swing dancing is a partnered form of movement
  necessitating shared momentum
• Swing is an exemplar of collaboration with
  – communication over a control input channel (touch)
  – leader and follower structure
• Human-human interaction as a template for
  human-machine interaction
  – Direct communication is an encumbrance, especially
    in a haptic (touch) interaction
• Coordination between humans in contact
  – Ballet pax de deux models exist, but these ignore
    communication and coordination, positing pre-
    arranged sequences [Laws, Physics of Dance 2001]
 Swing is a finite state machine
                                  Leader chooses moves and
    Click to play                   communicates them to
                                    the follower: haptic
                                    (touch) communication
                                    with an audible rhythm
                                    is sufficient

         Tuck turn                       Pullthrough


Closed               Right hand                   Crosshand
          Sendout

                                      Underarm turn
          Circle
         Follower’s problem
• Given the lead as an input, decide what
  movement to do
  – Does the limited vocabulary of moves
    contribute to her success by narrowing her
    choices of which move to execute?
  – Does the musical structure of the song they’re
    both listening to contribute?
  – Is visual information indispensable?
           Leader’s problem
• Choreograph each dance to express
  musical meaning in realtime, and give an
  unambiguous lead
  – Do leaders use the musical structure of each
    song to choose movements that are
    appropriate?
  – Do leaders limit the variety of their moves to
    enable followers to decode the lead?
  – Do leaders try to surprise the audience and
    judges?
  – Can we design an automated choreographer
    to create convincing swing dance sequences?
Part I: Haptic communication and
 haptic collaboration in human
 movement studies
 - PHANToM robot dance experiments
 - Schmidt’s law collaboration experiments
Part II: Automated choreography,
 and experiments on musicality in
 actual lead and follow dancing
  - Automated choreography for harmonious yet
 unpredictable dances
 - Conflicting musical messages and their effect
 on leaders and followers
        Haptic collaboration
• Haptic: pertaining to the sense of touch
• Haptic collaboration: multiple agents
  working to complete a task while in
  contact, touching each other or a common
  object or instrument
  – Examples: human-human collaboration in
    partnered dance; human-robot collaboration
    in surgical robotics; robot teams lifting and
    moving an object, human wearing a strength-
    extending robotic suit
             Applications
• In various settings, one could use
  vocabulary-based haptic collaboration
  with a robot or human as the leader or
  follower
  – Training: robot leader and human follower
  – Surgery: human leader and robot follower
  – Haptic Turing Test: both roles, to
    demonstrate understanding of human-human
    interaction strategies
Do followers know the moves?
• Haptic communication absent visual cues
  – Leader leads known moves in unknown sequence
  – Follower can decode vocabulary using only haptic
    signals, as demonstrated by expert dancers in lab
  – In general, haptic sense is far less precise than vision
    and far less information can be transmitted by touch
    than by sight
• Hypothesis: the small vocabulary of known
  rhythmic moves for swing dancing enhances the
  value of haptic collaboration.
   PHANTOM leader
• Human follower "dances"
  with a PHANTOM leader
  – repeatability for multiple trials
  – no visual display of the desired move is given
  – no advance signaling of move transitions is given
• Moves are circular with reversals
• Force feedback control
 [Feygin et al., Haptic guidance, IEEE Haptics 2002]
  • Modified proportional-derivative controller
    on position error and velocity error
PHANTOM leading moves
Classical tracking task : pursuit
• Classical tracking task for evaluating
  human performance: visual pursuit of
  random-appearing inputs [McRuer, Jex, IEEE
  Human Factors in Electronics, 1967]
  – Closed-loop delays between 100 and 350 ms
    reported

                                         track

                                                 +
                                                     -
                                        human
  Delay
  Repeated inputs are learnable
• [Jagacinski and Hah, 1988, J. Experimental
  Psychology: Human Performance]
  – Effective delays of 32 ms are physiologically
    implausible
  – Demonstrates input reconstruction (Open-loop)


                                          track

                                                  +
                                                      -
                                          human
   Delay
New task demands new metrics
• The space between random-appearing
  and repeated inputs includes tracking
  dance moves sequenced, rhythmically,
  from a small vocabulary
• Metrics used:
  – Delay in reversing direction within move
    and between moves (predictable vs.
    unpredictable)
  – Errors in recognizing a direction reversal
                   Experiment setup
• Hypothesis: subjects will   1. two clockwise circles
  dance with the              1. two clockwise circles

  PHANTOM using the
  move vocabulary.
                                  “ Fly me            to the           moon      and let me”


 Performance, in delays       2. two counter-clockwise circles
 and errors, will differ
 because at move
 transitions, subjects can       “ Fly me            to the           moon      and let me”
 not predict; but within a
 move, subjects can use       3. four upper half circles
 open-loop or pre-
 programmed control
                                “ Fly me            to the            moon     and let me”


• Alternatives: subjects      4. four lower half circles
  track in closed-loop
  fashion throughout
                                “ Fly me            to the            moon     and let me”
Quartiles of delay: predictable
 vs. unpredictable reversals
              300
              250
 Delay (ms)
              200
              150
              100
              50




                    Predictable   Unpredictable
                     Reversals      Reversals
    Metric: Move Errors
                Error
Green: move led Red: actual movement
             Not an error
Green: move led Red: actual movement
 Delays, Errors: predictable or not
          Errors, averaged over all subjects

                 Within Move      Move Transition
                 (predictable)    (unpredictable)
   Error rate        10%               1%


• Knowing the move created confidence
  that resulted in a higher error rate
• In both delays and errors, performance
  differed substantially between predictable
  and unpredictable reversals
   Followers know the moves
• Human subjects demonstrably use the
  known vocabulary in interpreting robot
  leader’s haptic signals
Part I: Haptic communication and
 haptic collaboration in human
 movement studies
 - PHANToM robot dance experiments
 - Schmidt’s law collaboration experiments
Part II: Automated choreography,
 and experiments on musicality in
 actual lead and follow dancing
  - Automated choreography for harmonious yet
 unpredictable dances
 - Conflicting musical messages and their effect
 on leaders and followers
             Schmidt’s Law
• Movements exhibit a tradeoff between
  speed and accuracy
  – Faster movements are less accurate
• Schmidt’s law: distance D is normal, and
  standard deviation increases linearly as
  speed σ  K (D/T)
  – K is skill parameter, varies between subjects
• Schmidt’s for a target: T = A + B (D/We)

   start            D                W
    Slope B and skill K related
• In aiming at a target, variation σ is limited
  by target size: choose speed that results in
  σ with 95% of movements inside target
  – approximately σ  0.25 W
  – leads to T  B (D/W) = 4K (D/W)




     -4   -3   -2    -1   0    1     2    3    4




                     Target size
          (must contain 95% of distribution)
       Two-person aiming task
• How can two people (dyad) collaborate on
  an aiming task?
   – Informationally, as if threading a needle
   – Haptically, both moving a single pointer
   – Aerobics class is informational collaboration,
     while partner dance is haptic collaboration
• Informationally, dyads perform no better
  than solo (Mottet et al., J. Exp Psych 2001)
• Haptically, dyads can perform better than
  solo (Reed et al., Int Conf Robotics Automation, 2004)
      Haptic teams do better
• Fundamental speed-accuracy tradeoff is
  surmountable
  – Human partner or robotic assist
• Haptic contact: requires coordination
  – One person:       F = ma
  – Two force inputs: applied forces have many
    outcomes    (F1-F2) = net F= ma
  – Two people informationally can
    communicate / coordinate visually but each
    can independently predict where he is aiming
          Cyclical aiming task
• Cyclical tasks require timing coordination
• Phase shift from harmonic motion to
  inharmonic motion with difficulty
• Harmonic motion recycles energy

   error                    good    good
        pointer
                  targets
     Cyclical dyad aiming expts.
• Five subjects solo, then all ten dyads
  – used dominant hands to aim cyclically
    between two targets of increasing difficulty
  – 30° between targets of 10.6°, 7.5°, 5.3°, 3.8°, 2.7°
     • Practice 60 aims
     • Experiment 120 aims
     • Movement time (T)
       measured between
       extremes of motion
     • Errors are over and
       undershoots (asked
       to keep below 5%)
 Dyads are superior at this task
• Movement times (T) are lower for dyads than
  for solo performers
• Differences
  increase
  with practice
• Slope of this
  line is B
  – Data analysis:
    fit params A
    and B for each
    solo and dyad
    performance, then average
      Endpoint compromise:
     Schmidt’s law for dyads
• It is unknown how individual movement
  accuracy limitations combine during a
  haptic collaborative movement
• Generalizing Schmidt’s law
  – One possibility is Endpoint compromise: each
    person chooses, independently, an endpoint
    and tries to reach it; the resulting movement
    actually ends at the average of the two points
                                             1
  – Endpoint compromise would have σd= σs
                   1                    1      2
     • Thus Kd =     K    and so Bd =     B
                    2 s                  2 s
  Fits compared, and slope ratios
• Slope of tradeoff is         Fits: T in milliseconds
  significantly lower
                                      Solo       Dyad
  for dyads
                            Slope     102.1      76.6
  – 95% confidence
                            Intercept 189.7      90.9
    interval for slope
    ratio, dyads to         R2        0.98       0.93
    solos, is (1.23,1.51)
    and 2  1.414
 [Slope ratio distribution estimated as t-distribution
    using 20 observations, each a slope ratio between a
    dyad and one of the two individuals in that dyad]
    Haptic collaboration helps
• Haptic interaction eases the speed-
  accuracy tradeoff in human movements
  – Energy recycling through the use of
    harmonic motion becomes compatible with
    more difficult tasks
• Improvements were not realized with
  two-person informationally coupled tasks
• One person can predict another’s actions
  (rhythmic coordination) to achieve the
  desired result, at least in this simple case
  – Do dancers do same using move vocabularies?
Part I: Haptic communication and
 haptic collaboration in human
 movement studies
 - PHANToM robot dance experiments
 - Schmidt’s law collaboration experiments
Part II: Automated choreography,
 and experiments on musicality in
 actual lead and follow dancing
  - Automated choreography for harmonious yet
 unpredictable dances
 - Conflicting musical messages and their effect
 on leaders and followers
Musicality: moves match music




   Matt Smiley and Naomi Uyama: Frenesi
 (American Lindy Hop Championships 2001)
      Musicality applications
• Automated choreography
  – Generate dancing screen displays for music
    playing software
  – Interactively choreograph real dances with
    humans in the loop
• Objective judging standards for dance
  contests
  – Investigate role of musicality in dance
• Fault detection: automatic dance
  transcriptions
  Aesthetic judgments objectively
• Score music phrases and dance moves
  – Loud, Exciting, Staccato, Emphasized
 Distance metric : musicality
For single move mi and song phrase si ,
  given scores on attributes indexed by j:
           d(mi, si)=j(mij -sij)2

Objective in choreographing a song S with
 N phrases is to minimize:
          d(M,S) = Nj(mij-sij)2
Continuity constrains transitions




 Closed position      Open position
                                                                           freeze
  sendout
                                                                                          east coast
                                                                      charleston
 tuck turn                                                                               pop to closed
                                      circle                           footwork
                                                                                          promenade
 swingout
                                                                        jockey
                                    crosshand circle
                                                                                            fox trot
 pop turn                                                               Balboa
                                 Open to Closed
Closed to Open                                                                  Closed


open promenade          shorty                 tabby       sugar push freeze          switches

underarm hijack       circle duck        underarm turn     she goes he goes         left side pass

 whip outside turn       whip             20s charleston    knickerbocker           right side pass

whip leader spin       pizza toss         crosshand whip      sugar push              hacksaw

 whip inside turn       eggbeater          get back here    Texas tommy                judo flip
             open footwork                     Open
        Solution: shortest path
• Move continuity restrictions can be
  formulated as a network
                s1    s2    sN
                m1    m1    m1
  (m1-s1   )2

                m2    m2    m2
  S    (m2-s1)2                    T

 (m41-s1)2
                m41   m41   m41
    Automated choreographer
• Automated choreographer chooses a
  sequence of dance moves to suit the
  music’s style and expression
• Automated choreographer must evade
  prediction, because dancers do
  – Military planners face a similar dilemma
• Solution is to combine an optimization
  step with a randomization step
       Optimize - randomize
• Optimization model ranks alternatives
  – Swingout is best, Foxtrot second best, etc.
• Randomize step chooses the rank of the
  next move from a distribution biased
  towards more musical moves
• Discover the distribution over move
  rankings from actual choreography
Rank alternatives via optimization
• Best move is on shortest path (SP)
• To get second best move, delete best move for
  that phrase, then re-optimize
             s1         s2           sN
                m1     m1           m1
  (m1-s1   )2

                m2     m2           m2
  S    (m2-s1)2                              T

 (m41-s1)2
                m41   m41           m41
  SP, randomize steps alternate
                  s1            s2               sN
                  m1            m1               m1

                  m2            m2               m2
      S                                                      T


                  m41          m41               m41

                        Optimization stage 1


Choose move 41 (rank =3)      Optimization stage 2


          Choose move 2 (rank =3)     Optimization stage 3
       AutoChoreo in action
• As a demonstration of the approach, we
  create a visualization of a few rank-biased
  randomized dances
  – Using motion capture data from a Lindy Hop
    couple
  – Rearrange the capture data to show musically
    expressive dances for a few different songs as
    a Windows widget
  – The visualization code is a senior project for
    Lukasz Hall
   Verify choreography model
• Real choreographers evade prediction, and
  model takes random decisions
   – Can’t use standard verification, where model makes
     predictions and we compare predictions to real data
• Stochastic verification: distribution of rank-bias
  in various humans’ choreography
   – is consistent, and distinct from random choreography
• Given actual expert choreography, rank of
  actual move observed from choreo is recorded
• Rank-bias from actual choreographers drives the
  automated choreographer
             Different choreographers have
                     consistent bias
        14
 • Actual choreography is consistently rank-
   12
    biased, using two dances from different
   10
    choreographers
Count




    8

        6– Matt & Naomi : Frenesi
        4
         – Dorry & Sommer : Darling Daughter
        2

        0
         (Kolmogorov-Smirnov test, p=.4144)
             0    5    10        15         20   25   30
                            Rank of moves
Consistent bias is not an artifact of
         ranking system
• Actual choreography is rank-biased differently
  from 2 instances of random choreography (K-S
  test, p<.001)
   – Random choreography (not rank-biased
     randomization) is a valid move sequence chosen
     uniformly over possible moves, then ranked
   – Comparison to random choreography shows that
     consistent rank-bias between actual dancers is not an
     artifact of the model’s ranking system, because
     random sequences have a different rank bias
Part I: Haptic communication and
 haptic collaboration in human
 movement studies
 - PHANToM robot dance experiments
 - Schmidt’s law collaboration experiments
Part II: Automated choreography,
 and experiments on musicality in
 actual lead and follow dancing
  - Automated choreography for harmonious yet
 unpredictable dances
 - Conflicting musical messages and their effect
 on leaders and followers
  Musicality in lead and follow
• Dances choreographed in realtime by leader
• Follower decodes which move the leader
  wants
  – from a shared, known vocabulary of moves
  – using visual, auditory, and haptic cues
• Does the shared musical environment aid in
  the communication and decoding of moves?
  – the beat as a metronome certainly does
  – musical structure and emotional emphasis
    Is music the “third partner”?
   Does shared music assist
 communication between pair?
• Experimental hypothesis:
  Experienced leaders use the cues in
  emotional and structural content of the
  music to choreograph the dance, and
  experienced followers use emotional and
  structural content of the music to
  decipher the leader’s intentions
  – Test by spoofing the shared musical
    environment to conflicting music
        Musicality and dance
• Dancers choose moves to reflect the music
 (Modelling musically meaningful choreography,
 Gentry,Feron 2004)
• Swing dance music consists of 8-beat bars
  composed into phrases of either 4 bars
  (swing) or 6 bars (jump blues)
• Last bar of every phrase is dramatic
  – Sometimes called a break
   Swing vs. blues song structure




texture

                            break



texture
          Experiment setup
• Modified 8 familiar lindy hop songs
  – tempo-matched and bar-aligned
  – left and right channels combined two
    different songs in conflicting combinations
  – Played over wireless Sennheiser headphones
    to display one song to the leader, and a
    second song to the follower
• Ten experienced dancers, most not
  dancing with their regular partners
      More experiment setup
• To discern even small effects of music
  structure misalignment, paired-
  comparison design
  – Partners danced twice in each trial
  – One dance of each trial had the same song in
    both headphones
  – One dance of each trial had a different song
    in each headphone set
• Dancers were asked to identify the mixed-
  song condition (1st or 2nd dance of each
  trial)
          Hypothesis testing
• If hypothesis is true, then both leaders
  and followers should be able to identify
  the mixed-song condition
• If hypothesis is false, then success rate
  should be like guessing: Binomial (.5)
• Results
  – followers correct: 13 of 16 trials (p=.00854)
  – leaders correct: 8 of 16 trials
    Interviews with followers
• Followers all stated that decoding moves
  correctly is of more value than dancing in
  a musically expressive way
• Followers consciously chose to accept the
  move their partner led, and they also
  consciously recognized some moves as
  being contradictory in emotional content
  to the music they heard
        Followers are robust
• The cognitive dissonance that followers
  experienced is only possible because the
  leaders actually chose moves to express
  the music the leaders heard
• Follower performance at decoding moves
  in time to execute them properly must not
  have degraded with conflicting music,
  because leaders could not detect a
  difference
 The exception proves the rule
• Twice in 32 one-minute dances, a
  follower hijacked (backled), to insist on a
  particular move she felt was called for by
  the music
• Hijacks are rare in Lindy Hop, and happen
  most often at dramatic musical moments
  – In both instances during this experiment,
    follower’s hijack signalled to the leader that
    follower must be hearing the same song
A hijacked (backled) move
      Interviews with leaders
• Leaders wanted to succeed at identifying
  the mixed-song condition
• All leaders said that musicality, dancing to
  express the music, is their goal
  – “I danced the heck out of those songs. It’s
    like, if you go back and match it up, I’m like,
    yeah, I’m rocking this song!”
• Most could not identify the mixed-song
  condition because of their followers’ skill
  at decoding the moves even in the
  presence of conflicting music
         Leader interview

• First tells of a style cue
  that let him detect one
  mixed-song condition
  correctly
• Then relates his
  experience of a backled
  move
     Conclusion: separability
• The haptic (touch) communication system
  between leaders and followers does not
  depend critically on musicality
• Thus, rhythmic motion vocabularies hold
  promise for human-robot interaction even
  without music
• Choreography can be considered
  separately from the problem of
  synchronizing actions between leader and
  follower
     Applications: computer graphics
• Red leader: real data
  Green: computer
  generated follower
• Our experiment
  confirms that
  neglecting musical
  meaning should not
  harm the ability of the
  simulated follower to
  decode moves
                            E. Hsu, S. Gentry, J. Popovic, Example-based
                            control of human motion, Eurographics 2004
Part I: Haptic communication and
 haptic collaboration in human
 movement studies
 - PHANToM robot dance experiments
 - Schmidt’s law collaboration experiments
Part II: Automated choreography,
 and experiments on musicality in
 actual lead and follow dancing
  - Automated choreography for harmonious yet
 unpredictable dances
 - Conflicting musical messages and their effect
 on leaders and followers

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:12
posted:2/27/2012
language:English
pages:64