Cartoon Retargeting by wpr1947

VIEWS: 4 PAGES: 132

									Animating (human) motion
            • Presented by:
              – Yoram Atir
              – Simon Adar
Applications of computer animation
•   Movies
•   Advertising
•   Games
•   Simulators
•   …
 General goals of the work presented

- New methods aimed to save
  time/money/skills needed.
- Study motion (texture).
                    Agenda
-   Basic concepts
-   Motion Synthesis/texture using motion capture
-   Physics/Biomechanics Motion Synthesis
-   Cartoon Motion Retargeting.
                 Basic concepts
   •   Animation world (3D)
   •   Skeletal model representation
   •   Model positioning
   •   Keyframes
   •   Motion capture
   •   Frequency bands
   •   Correlations

Basic Concepts
                 3D animation world
     -   (Human) model is animated in Object space
     -   Animated model projected into “global” space
     -   Camera is placed and rotated
     -   Perspective is set
     -   Other…




Basic Concepts
                        Skeletal representation

        - Each model has its
          own Default Pose
        - DOF’s – joint
          angles/translations
          relative to Default Pose
        - Hierarchical (tree)
          skeletal representation
          of model
   Picture from Lecture in Computer Graphics course
           Department of computer science
              University of Washington
Basic Concepts
                 Creating motion

      - Skeletal variations between frames
      - Overall rotation/Translation between frames
      - Correlate.

      General Problem:
       A LOT of work due to the large number of
       DOFS & high frame rate


Basic Concepts
                 Figure positioning
      - Forward kinematics (simplified): Figure
        positioning by joint data specification.

      Problem:
      - Tedious trial and error.




Basic Concepts
                     Figure positioning
      Inverse kinematics (simplified)
      -   Joint data is acquired by solving for the final position
      -   In general, This is an optimization problem with a large system
          of variables and constraints
      -   Problems often are expressed as minimization problems, and
          solved using standard algorithms (gradient decent etc).
      -   Usually, infinite number of possible solutions.
      -   A “good” solution has to be more than “feasible”
      -   Often one is obtained by embedding specific knowledge as
          additional constraints,        and/or
      -   Using Inverse kinematics as a part of a specific solution.



Basic Concepts
     Basic methods for saving labor

                 KeyFrames   Motion capture




Basic Concepts
                          Keyframes
        – Specifying only part of DOFs and frames
        – Computer interpolation between them
             Problem: “smooth” interpolation looks unreal
             There are methods to apply “specific noise”
        – Term has historical roots




Basic Concepts
                     Motion capture
        – Acquired from “live action”
        – Copied onto animated character
             • Problem: Hard to adapt.
             • “Motion Editing” – methods to adapt mocap
        – Done in studios
        – Mocap libraries exist




Basic Concepts
     Keyframing vs. Mocap

                 Advantages   Disadvantages


                 •Control
  Keyframing



      Mocap


Basic Concepts
     Keyframing vs. Mocap

                 Advantages   Disadvantages


                 •Control
  Keyframing     •Intuitive



      Mocap


Basic Concepts
     Keyframing vs. Mocap

                 Advantages   Disadvantages


                 •Control     •Detail hard
  Keyframing     •Intuitive



      Mocap


Basic Concepts
     Keyframing vs. Mocap

                 Advantages   Disadvantages


                 •Control     •Detail hard
  Keyframing     •Intuitive   •Many DOF



      Mocap


Basic Concepts
     Keyframing vs. Mocap

                 Advantages     Disadvantages


                 •Control       •Detail hard
  Keyframing     •Intuitive     •Many DOF


                 •Detail easy
      Mocap


Basic Concepts
     Keyframing vs. Mocap

                 Advantages     Disadvantages


                 •Control       •Detail hard
  Keyframing     •Intuitive     •Many DOF


                 •Detail easy
      Mocap      •All DOF

Basic Concepts
     Keyframing vs. Mocap

                 Advantages     Disadvantages


                 •Control       •Detail hard
  Keyframing     •Intuitive     •Many DOF


                 •Detail easy   •No control
      Mocap      •All DOF

Basic Concepts
     Keyframing vs. Mocap

                 Advantages     Disadvantages


                 •Control       •Detail hard
  Keyframing     •Intuitive     •Many DOF


                 •Detail easy   •No control
      Mocap      •All DOF       •Not intuitive

Basic Concepts
 Keyframe Data vs.
Motion Capture Data
                              Frequency Bands




                 Right flat      Right toe   Left flat   Left toe



Basic Concepts
                 Frequency Bands
   • Simplifies the form of the data
        – Low frequency Variations:
           Large scale motions.
        – Higher frequency variations:
           individual “noise” / Jitter

        Both are important to preserve in order to
         capture the essence of motion


Basic Concepts
                       Correlations
   • Joints angle/translation data is
     related to each other
   • Joint angles are correlated
     over time
   • Correlation “plot” is
        – (somewhat) Specific to the type
          of motion
        – Carries “personality” information
          (style)


Basic Concepts
            More information…
INTRODUCTION TO COMPUTER ANIMATION – Rick parent
http://www.cis.ohio-state.edu/~parent/book/outline.html

Splines
http://www.people.nnov.ru/fractal/splines/Intro.html

Hash Inc - Animation software (Movies, tutorials…)
http://www.hash.com

Google…
                    Agenda
-   Basic concepts
-   Motion Synthesis/texture using motion capture
-   Physics/Biomechanics Motion Synthesis
-   Cartoon Motion Retargeting
     Goal: Motion Capture Assisted
               Animation

  • Create a method that allows an artist low-
    level control of the motion

  • Combine the strengths of keyframe
    animation with those of mocap



Motion Capture Assisted Animation – Pullen/Bregler
      Goal: Motion Capture Assisted
                Animation

  “Sketch” an animation by keyframing
  • Animate only a few degrees of freedom
  • Set few keyframes
  “Enhance” the result with mocap data
  • Synthesize missing degrees of freedom
  • Texture keyframed degrees of freedom



Motion Capture Assisted Animation – Pullen/Bregler
   What is a Motion Texture?

• Every individual’s movement is unique
• Synthetic motion should capture the
  texture
• To “texture” means to add style to a pre-
  existing motion
• Technically, texturing is a special case
  of synthesis
       Goal: Motion Capture Assisted
                 Animation
  Blue = Keyframed
  Purple = Textured/Synthesized




Motion Capture Assisted Animation – Pullen/Bregler
                    How an Animator Works


  • A few degrees of freedom at first

  • Not in detail

  • Fill in detail with more keyframes later


Motion Capture Assisted Animation – Pullen/Bregler
                          The Method in Words

  • Choose degrees of freedom to drive the animation

  • Compare these degrees of freedom from the
    keyframed data to mocap

  • Find similar regions

  • Look at what the rest of the body is doing in those
    regions

  • Put that data onto the keyframed animation
Motion Capture Assisted Animation – Pullen/Bregler
       Choices the Animator
            Must Make

 1.       Which DOF to use as matching angles

 2.       Which DOF to texture, which to synthesize

 3.       Which frequency band to use in matching

 4.       How many frequency bands to use in texturing

 5.       How many matches to keep

 6.       How many best paths to keep
Motion Capture Assisted Animation – Pullen/Bregler
                  Before Beginning:
               Choose Matching Angles
Root x trans          Left Clavicle x           Left Hip x
Root y trans          Left Clavicle y           Left Hip y
Root z trans          Left Clavicle z           Left Hip z
Root x rot            Left Shoulder x           Left Knee x
Root y rot            Left Shoulder y           Left Knee y
Root z rot            Left Shoulder z           Left Knee z
Spine1 x              Left Elbow x              Left Ankle x
Spine1 y              Left Elbow y              Left Ankle y
Spine1 z              Left Elbow z              Left Ankle z
Spine2 x              Left Wrist x              Left Ball x
Spine2 y              Left Wrist y              Left Ball y
Spine2 z              Left Wrist z              Left Ball z
Spine3 x              Right Clavicle x          Right Hip x
Spine3 y              Right Clavicle y          Right Hip y
Spine3 z              Right Clavicle z          Right Hip z
Neck x                Right Shoulder x          Right Knee x
Neck y                Right Shoulder y          Right Knee y
Neck z                Right Shoulder z          Right Knee z
Head x                Right Elbow x             Right Ankle x
Head y                Right Elbow y             Right Ankle y
Head z                Right Elbow z             Right Ankle z
Head Aim x            Right Wrist x             Right Ball x
Head Aim y            Right Wrist y             Right Ball y
Head Aim z            Right Wrist z             Right Ball z

               Time                      Time                   Time
                Matching Angles
               Drive the Synthesis
Root x trans           Left Clavicle x           Left Hip x
Root y trans           Left Clavicle y           Left Hip y
Root z trans           Left Clavicle z           Left Hip z
Root x rot             Left Shoulder x           Left Knee x
Root y rot             Left Shoulder y           Left Knee y
Root z rot             Left Shoulder z           Left Knee z
Spine1 x               Left Elbow x              Left Ankle x
Spine1 y               Left Elbow y              Left Ankle y
Spine1 z               Left Elbow z              Left Ankle z
Spine2 x               Left Wrist x              Left Ball x
Spine2 y               Left Wrist y              Left Ball y
Spine2 z               Left Wrist z              Left Ball z
Spine3 x               Right Clavicle x          Right Hip x
Spine3 y               Right Clavicle y          Right Hip y
Spine3 z               Right Clavicle z          Right Hip z
Neck x                 Right Shoulder x          Right Knee x
Neck y                 Right Shoulder y          Right Knee y
Neck z                 Right Shoulder z          Right Knee z
Head x                 Right Elbow x             Right Ankle x
Head y                 Right Elbow y             Right Ankle y
Head z                 Right Elbow z             Right Ankle z
Head Aim x             Right Wrist x             Right Ball x
Head Aim y             Right Wrist y             Right Ball y
Head Aim z             Right Wrist z             Right Ball z

                Time                      Time                   Time
          Motion Capture Data

Root x trans          Left Clavicle x           Left Hip x
Root y trans          Left Clavicle y           Left Hip y
Root z trans          Left Clavicle z           Left Hip z
Root x rot            Left Shoulder x           Left Knee x
Root y rot            Left Shoulder y           Left Knee y
Root z rot            Left Shoulder z           Left Knee z
Spine1 x              Left Elbow x              Left Ankle x
Spine1 y              Left Elbow y              Left Ankle y
Spine1 z              Left Elbow z              Left Ankle z
Spine2 x              Left Wrist x              Left Ball x
Spine2 y              Left Wrist y              Left Ball y
Spine2 z              Left Wrist z              Left Ball z
Spine3 x              Right Clavicle x          Right Hip x
Spine3 y              Right Clavicle y          Right Hip y
Spine3 z              Right Clavicle z          Right Hip z
Neck x                Right Shoulder x          Right Knee x
Neck y                Right Shoulder y          Right Knee y
Neck z                Right Shoulder z          Right Knee z
Head x                Right Elbow x             Right Ankle x
Head y                Right Elbow y             Right Ankle y
Head z                Right Elbow z             Right Ankle z
Head Aim x            Right Wrist x             Right Ball x
Head Aim y            Right Wrist y             Right Ball y
Head Aim z            Right Wrist z             Right Ball z

               Time                      Time                   Time
                                                 Overview


  Steps in texture/synthesis method

  • Frequency analysis
  • Matching
  • Path finding
  • Joining

Motion Capture Assisted Animation – Pullen/Bregler
                                                     Example


         In the following series of slides:

         Hip angle = matching angle

         Spine angle = angle being synthesized


Motion Capture Assisted Animation – Pullen/Bregler
                            Frequency Analysis:
                             Break into Bands




Motion Capture Assisted Animation – Pullen/Bregler
                                Frequency Analysis

        Band-pass decomposition of matching angles
               Keyframed Data                           Motion Capture Data
Frequency




                          Time
   Motion Capture Assisted Animation – Pullen/Bregler
                                Frequency Analysis

        Chosen low frequency band
               Keyframed Data                           Motion Capture Data
Frequency




                          Time
   Motion Capture Assisted Animation – Pullen/Bregler
         Chosen Low Frequency Band

  Hip angle data (a matching angle)
            Keyframed Data                           Motion Capture Data




Motion Capture Assisted Animation – Pullen/Bregler
                              Making Fragments

  Break where first derivative changes sign
            Keyframed Data                           Motion Capture Data




Motion Capture Assisted Animation – Pullen/Bregler
                               Making Fragments

  Step through fragments one by one
            Keyframed Data                           Motion Capture Data




Motion Capture Assisted Animation – Pullen/Bregler
                                                 Matching




                                Keyframed
                                Fragment


Motion Capture Assisted Animation – Pullen/Bregler
                                                 Matching


                                           Motion Capture Data




                                Keyframed
                                Fragment


Motion Capture Assisted Animation – Pullen/Bregler
                                                 Matching


                                           Motion Capture Data




                                Keyframed
                                Fragment


Motion Capture Assisted Animation – Pullen/Bregler
                        Matching

Compare to all motion capture fragments
     Angle in degrees




                         Keyframed
                         Mocap

                          Time
                        Matching

Resample mocap fragments to be same length
     Angle in degrees




                         Keyframed
                         Mocap

                          Time
                         Matching
Using some metric on all matching angles
and on their first derivatives:
Keep the K closest matches
      Angle in degrees




                          Keyframed
                          Mocap

                           Time
                                                 Matching


                                           Motion Capture Data




                                Keyframed
                                Fragment


Motion Capture Assisted Animation – Pullen/Bregler
                                                 Matching


                                           Motion Capture Data




                                                            Close
                                                            Matches
                                Keyframed
                                Fragment


Motion Capture Assisted Animation – Pullen/Bregler
                                                Matching


                                  Hip Angle (Matching Angle)



                                   Spine Angle (For Synthesis)




Motion Capture Assisted Animation – Pullen/Bregler
    Matching and Synthesis

  Low frequency hip angle data (a matching angle)




                    Spine angle data to be synthesized




Motion Capture Assisted Animation – Pullen/Bregler
    Matching and Synthesis

  Low frequency hip angle data (a matching angle)




                    Spine angle data to be synthesized




Motion Capture Assisted Animation – Pullen/Bregler
    Matching and Synthesis

  Low frequency hip angle data (a matching angle)




                    Spine angle data to be synthesized




Motion Capture Assisted Animation – Pullen/Bregler
    Matching and Synthesis

  Low frequency hip angle data (a matching angle)




                    Spine angle data to be synthesized




Motion Capture Assisted Animation – Pullen/Bregler
    Matching and Synthesis

  Low frequency hip angle data (a matching angle)




                    Spine angle data to be synthesized




Motion Capture Assisted Animation – Pullen/Bregler
    Matching and Synthesis

  Low frequency hip angle data (a matching angle)




                    Spine angle data to be synthesized




Motion Capture Assisted Animation – Pullen/Bregler
                   Possible Synthetic
Angle in degrees
                   Spine Angle Data




                          Time
                      Path Finding
We would like to:
• Use as much consecutive fragments as possible
• Stay as close as possible to best fit
   Angle in degrees




                             Time
                   Path Finding
Angle in degrees




                           Time
                   Path Finding
Angle in degrees




                           Time
                   Path Finding
Angle in degrees




                           Time
Path Finding
   Angle in degrees




                      Time
       Angle in degrees
                          Joining




Time
Enhancing Animations:
Texturing and Synthesis
Keyframed    Not keyframed




 Textured     Synthesized
                                Texturing

  Synthesize upper frequency bands




Motion Capture Assisted Animation – Pullen/Bregler
                 Texturing

Band-pass decomposition of keyframed data
     Frequency




                       Time
                 Texturing

Synthesize upper frequency bands
     Frequency




                       Time
  Walking animations
Texturing and Synthesis
    Keyframed Sketch
  Walking animations
Texturing and Synthesis
  Motion Capture Data
  Two different styles of walk
  Walking animations
Texturing and Synthesis
    Enhanced Animation
    Upper body is synthesized
    Lower body is textured
Otter Animations: Texturing
        Keyframed data
Otter Animations: Texturing
       Textured animation
Dance Animations: Texturing and
          Synthesis
         Lazy Keyframed
             Sketch
Dance Animations: Texturing and
          Synthesis
          Motion Capture
               Data
Dance Animations: Texturing and
          Synthesis
       Enhanced Animation
     Blue = Keyframed
     Purple = Textured/Synthesized
Dance Animations: Texturing

     Keyframed Sketch With
           More Detail
Dance Animations: Texturing
     Textured Animation
    Blue = Keyframed
    Purple = Textured
Summary of the Method
  Sketch +         Frequency
   Mocap           Analysis             Matching
  Keyframed data   Keyframed Data      Matching Angles




   Mocap Data       Mocap Data      Possible Synthetic Data




 Path Finding        Joining
                                         Enhanced
                                         Animation
     Choices the Animator
          Must Make

1.   Which DOF to use as matching angles

2.   Which DOF to texture, which to synthesize

3.   Which frequency band to use in matching

4.   How many frequency bands to use in texturing

5.   How many matches to keep

6.   How many best paths to keep
     Conclusions and
      Applications
• Appropriate for an artist interested in a very
  particular style of motion

• The artist may have a relatively small motion
  capture set of that style

• The artist may want precise control over parts
  of the motion
     Conclusions and
      Further Work


• Direct incorporation of hard constraints

• Fundamental units of motion
     For more info. . .

http://graphics.stanford.edu/~pullen




Special Thanks to:
Reardon Steele, Electronic Arts
                    Agenda
-   Basic concepts
-   Motion Synthesis/texture using motion capture
-   Physics, Biomechanics Motion Synthesis
-   Cartoon Motion Retargeting
                                    Motivation
   • Generate rapid prototyping of realistic
     character motion
   • Avoid simulated human models, that are
     very complex, and don’t always look
     realistic




"C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations
                                          Scope
   • Highly dynamic movement such as
     jumping, kicking, running, and gymnastics.
   • Less energetic motions such as walking or
     reaching will not work well in this
     framework




"C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations
               Overview of the process
           Motion sketch
                                                   Constraint & phase
                                                       detection
      Character description



            Motion DB
                                        Transition pose
                                                                     Momentum control
                                           synthesis
         User interaction




                         Objective functions                Optimization              Animation




"C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations
               Overview of the process
   • The objective is to
     transforms simple
     animations into realistic
     character motion by
     applying laws of physics
     and the biomechanics
     domain
   • The unknowns are:
     values of joint angles and
     parameters of angular
     and linear momentum

"C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations
               Overview of the process
           Motion sketch
                                                   Constraint & phase
                                                       detection
      Character description



            Motion DB
                                        Transition pose
                                                                     Momentum control
                                           synthesis
         User interaction




                         Objective functions                Optimization              Animation




"C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations
      Constraint and stage detection
   • Each input sequence
     has two parts:
         – The part that needs to
           be improved
         – The part that needs to
           kept intacked
   • Automatically extract
     the positional and
     sliding constrains


"C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations
       Positional constraint detection
   • A positional constraint fixes a specific point
     on the character to a stationary location for
     a period of time
   • We need to find if all these points lie on a
     line, plane
   • In an articulated character we find the
     constraints on each body part




"C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations
       Positional constraint detection
   • The algorithm looks
                                                                     Ti xi  xi
     for fixed points (point,
     line, plane)
                                                                  (Ti  I ) xi  0




"C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations
                        Sliding constraints
                                  min   p ,l    Dist (T W p, l )
                                                i
                                                          i   i




"C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations
               Overview of the process
           Motion sketch
                                                   Constraint & phase
                                                       detection
      Character description



            Motion DB
                                        Transition pose
                                                                     Momentum control
                                           synthesis
         User interaction




                         Objective functions                Optimization              Animation




"C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations
            Transition pose generation
   • A transition pose
     separates constrained
     and unconstrained
     stages.
   • Two possibilities:
         – We ask the animator
           to draw the transition
           poses
         – We have an estimator
           to suggest a transition
           pose


"C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations
             Transition Pose Estimator
      • DB contains examples of different motions
      • The input of that DB are the motion
        parameters like: flight distance, flight
        height, takeoff angle, landing angle, spin
        angle..
      • The DB has a simplified representation of
        the transition poses by three COM’s
      • We use IK to obtain the full character’s
        pose from those three COM’s
      • The KNN - K nearest neighbor algorithm
      • The pose estimator predicts the candidate
        pose by interpolating the KNN with the
        weights that describe the similarity to the                          CB
        input.                                      C B  C B  (C A  C A )                  2

                                                                             CA
                                                                                              2




"C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations
               Overview of the process
           Motion sketch
                                                   Constraint & phase
                                                       detection
      Character description



            Motion DB
                                        Transition pose
                                                                     Momentum control
                                           synthesis
         User interaction




                         Objective functions                Optimization              Animation




"C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations
                        Momentum control
   • Transition poses constrain the motion at few key
     points of the animation




   • Dynamic constraints ensure realistic motion of
     each segment
   • Linear and angular momentum give us these
     dynamic constraints

"C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations
      Momentum during unconstrained
         and constrained stages
   • linear momentum -
                                                        dP(q)
     During “flight” the only                                  mg
     force is gravity                                    dt
   • Angular momentum -
     During “flight” there is no                         dL(q )
     change in Angular                                          0
     momentum
                                                          dt
   • During “ground” stage we
     avoid computing the
     momentums and use
     empirical characteristics


"C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations
               Overview of the process
           Motion sketch
                                                   Constraint & phase
                                                       detection
      Character description



            Motion DB
                                        Transition pose
                                                                     Momentum control
                                           synthesis
         User interaction




                         Objective functions                Optimization              Animation




"C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations
                       Objective functions
   • There are three Objective
     functions, the basic idea
     behind them is power
     consumption
         – Minimum mass displacement
         – Minimal velocity of DOFs
         – Static balance




"C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations
               Overview of the process
           Motion sketch
                                                   Constraint & phase
                                                       detection
      Character description



            Motion DB
                                        Transition pose
                                                                     Momentum control
                                           synthesis
         User interaction




                         Objective functions                Optimization              Animation




"C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations
                     Putting it all together
   •   Environment constraints (Ce)
   •   Transition pose constraints (Cp)
   •   Momentum constraints (Cm)
   •   Q are character’s DOFs

                                      C e (Q )  0
                                     
             min  Ei(q ) subject to C p (Q )  0
                       i

                                     C (Q )  0
              Q

                                      m

"C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations
               Overview of the process
           Motion sketch
                                                   Constraint & phase
                                                       detection
      Character description



            Motion DB
                                        Transition pose
                                                                     Momentum control
                                           synthesis
         User interaction




                         Objective functions                Optimization              Animation




"C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations
                              Some Results
   • Wide variety of figures: male, female, child
   • 51 DOFs
   • The body dimensions and mass distribution is taken from
     biomechanics literature
   • In some of the cases the animator selects the body parts to be
     constraints
   • The animator can change relative timing between each phase
   • The optimization was solved by using SNOPT a general
     nonlinearly-constrained optimization package
   • The optimization time depends on the duration of the animation
   • All of the simple animation took less than five minutes to sketch
   • For all examples the synthesis process took less than five
     minutes




"C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations
                                  Broad jump
   • Only 3 keyframes at
     takeoff, peak and
     landing




"C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations
                                       Running
   • The angular
     momentum constraint
     creates a counter-
     body movement by
     the shoulders and
     arms to counteract
     the angular
     momentum generated
     by the legs.
   • Keyframing 7 DOFs

"C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations
                                   Hopscotch
   • Each hop requires 3
     keyframes and has
     fewer than 7 DOFs




"C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations
                                  Handspring
   • There were no
     handstands within the
     DB so the user had to
     modify the result




"C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations
                                      High-bar
   • Two constraints
     stages: the bar and
     ground




"C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations
                                   Karate kick
   • A second synthesis add a keyframe in the
     peak




"C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations
                            Twist jumps




"C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations
                    Agenda
-   Basic concepts
-   Motion Synthesis/texture using motion capture
-   Physics/Biomechanics Motion Synthesis
-   Cartoon Motion Retargeting
              What is Cartoon Capture &
                     Retargeting
   • Cartoon Capture
         – Track the motion From
           2D Animation
         – Represent the motion
           & save
   • Retargeting
         – Translate the motion
           representation to
           another output media



”C.Bregler, L.Loeb, E.Chuang, H.Deshpande; "Turning to the Masters: Motion Capturing Cartoons
             Cartoon motion capture &
             retargeting scheme
Digitized video
                                                Motion                                  Output video
                          Cartoon capture                             retargeting
                                                representation
   Key shapes




Output corresponding
       key shapes




”C.Bregler, L.Loeb, E.Chuang, H.Deshpande; "Turning to the Masters: Motion Capturing Cartoons
              Modeling Cartoon motion
Digitized video
                                                Motion                                  Output video
                          Cartoon capture                             retargeting
                                                representation
   Key shapes




Output corresponding
       key shapes




”C.Bregler, L.Loeb, E.Chuang, H.Deshpande; "Turning to the Masters: Motion Capturing Cartoons
              Modeling Cartoon motion
   • Two types of
     deformations
         – Affine deformation




         – Key shape
           deformation


”C.Bregler, L.Loeb, E.Chuang, H.Deshpande; "Turning to the Masters: Motion Capturing Cartoons
      y
                        Affine Deformation
                                                        si  xi           yi 1
                                                                                     T




                                               x
                                                   a1                a2      dx 
                              V  warp ( , S )                                S
                                                  a3                 a4      dy 

   • Affine parameters                              (t )  [a1 , a2 , a3 , a4 , d x , d y ]



”C.Bregler, L.Loeb, E.Chuang, H.Deshpande; "Turning to the Masters: Motion Capturing Cartoons
                Key-Shape Deformation

                                   a1                a2      dx              
              V  warp ( , S )                                    wk S k 
                                                              dy   k
                                  a3                 a4                        


   • Sk are the key shapes


                                                    w1            w2         w3



”C.Bregler, L.Loeb, E.Chuang, H.Deshpande; "Turning to the Masters: Motion Capturing Cartoons
              Modeling Cartoon motion
   • In total there are 6+K variables that represent
     the motion


                 (t )  [a1 , a2 , a3 , a4 , d x , d y , w1 ,...., wk ]




”C.Bregler, L.Loeb, E.Chuang, H.Deshpande; "Turning to the Masters: Motion Capturing Cartoons
                 Cartoon motion capture
Digitized video
                                                Motion                                  Output video
                          Cartoon capture                             retargeting
                                                representation
   Key shapes




Output corresponding
       key shapes




”C.Bregler, L.Loeb, E.Chuang, H.Deshpande; "Turning to the Masters: Motion Capturing Cartoons
                 Cartoon motion capture

   • contour capture: the
     input is a sequence of
     contours

   • video capture: the
     input is the video
     sequence



”C.Bregler, L.Loeb, E.Chuang, H.Deshpande; "Turning to the Masters: Motion Capturing Cartoons
                             contour capture
                               Err  V  warp( , S1 ,...,Sk )
                                                                                2


   • Two step minimization:
         – Find Affine parameters
                                                                                                2
                                                     a1              a2        dx 
                                                V                                 S
                                                                                dy 
                                 Erraff
                                                    a3               a4           
                                   a1    a2    dx                T 1
                                                     V  S (S  S )
                                                            T
                                  a      a4    dy 
                                   3
         – Find Key-Shape weights
                                                                                                2
                                            a1               a2       dx 
                                 Err  V                                   ( wk  S k )
                                           a3                a4       dy 
                                                                          
   • Iterate

”C.Bregler, L.Loeb, E.Chuang, H.Deshpande; "Turning to the Masters: Motion Capturing Cartoons
                                   Retargeting
Digitized video
                                                Motion                                  Output video
                          Cartoon capture                             retargeting
                                                representation
   Key shapes




Output corresponding
       key shapes




”C.Bregler, L.Loeb, E.Chuang, H.Deshpande; "Turning to the Masters: Motion Capturing Cartoons
                                   Retargeting
   • For each Input key-shape an Output key-
     shape is drawn.




”C.Bregler, L.Loeb, E.Chuang, H.Deshpande; "Turning to the Masters: Motion Capturing Cartoons
                     Retargeting Process


                                     Key shapes Interpolation
                                                                              Retarget
                                                                            Motion capture
                                    Apply Affine transformation
                                       From motion capture



                                         Retargeted media




”C.Bregler, L.Loeb, E.Chuang, H.Deshpande; "Turning to the Masters: Motion Capturing Cartoons
                                     Examples




”C.Bregler, L.Loeb, E.Chuang, H.Deshpande; "Turning to the Masters: Motion Capturing Cartoons
             Additional constrains & post
                     processing
   • Undesirable effects may still appear
   • Determine constraints that force the
     character go through certain position at
     certain time
   • Apply ad-hoc global transformation that
     fulfill these constraints



”C.Bregler, L.Loeb, E.Chuang, H.Deshpande; "Turning to the Masters: Motion Capturing Cartoons
                                Performance
   • Quantative performance wasn’t mentioned
   • The more complex the motion of the
     character is, the more key-shapes are
     needed
   • Many of the animations contain jitter, but
     the overall exaggerated motion dominates



”C.Bregler, L.Loeb, E.Chuang, H.Deshpande; "Turning to the Masters: Motion Capturing Cartoons

								
To top