# Cartoon Retargeting by wpr1947

VIEWS: 4 PAGES: 132

• pg 1
```									Animating (human) motion
• Presented by:
– Yoram Atir
Applications of computer animation
•   Movies
•   Games
•   Simulators
•   …
General goals of the work presented

- New methods aimed to save
time/money/skills needed.
- Study motion (texture).
Agenda
-   Basic concepts
-   Motion Synthesis/texture using motion capture
-   Physics/Biomechanics Motion Synthesis
-   Cartoon Motion Retargeting.
Basic concepts
•   Animation world (3D)
•   Skeletal model representation
•   Model positioning
•   Keyframes
•   Motion capture
•   Frequency bands
•   Correlations

Basic Concepts
3D animation world
-   (Human) model is animated in Object space
-   Animated model projected into “global” space
-   Camera is placed and rotated
-   Perspective is set
-   Other…

Basic Concepts
Skeletal representation

- Each model has its
own Default Pose
- DOF’s – joint
angles/translations
relative to Default Pose
- Hierarchical (tree)
skeletal representation
of model
Picture from Lecture in Computer Graphics course
Department of computer science
University of Washington
Basic Concepts
Creating motion

- Skeletal variations between frames
- Overall rotation/Translation between frames
- Correlate.

General Problem:
A LOT of work due to the large number of
DOFS & high frame rate

Basic Concepts
Figure positioning
- Forward kinematics (simplified): Figure
positioning by joint data specification.

Problem:
- Tedious trial and error.

Basic Concepts
Figure positioning
Inverse kinematics (simplified)
-   Joint data is acquired by solving for the final position
-   In general, This is an optimization problem with a large system
of variables and constraints
-   Problems often are expressed as minimization problems, and
solved using standard algorithms (gradient decent etc).
-   Usually, infinite number of possible solutions.
-   A “good” solution has to be more than “feasible”
-   Often one is obtained by embedding specific knowledge as
-   Using Inverse kinematics as a part of a specific solution.

Basic Concepts
Basic methods for saving labor

KeyFrames   Motion capture

Basic Concepts
Keyframes
– Specifying only part of DOFs and frames
– Computer interpolation between them
Problem: “smooth” interpolation looks unreal
There are methods to apply “specific noise”
– Term has historical roots

Basic Concepts
Motion capture
– Acquired from “live action”
– Copied onto animated character
• “Motion Editing” – methods to adapt mocap
– Done in studios
– Mocap libraries exist

Basic Concepts
Keyframing vs. Mocap

•Control
Keyframing

Mocap

Basic Concepts
Keyframing vs. Mocap

•Control
Keyframing     •Intuitive

Mocap

Basic Concepts
Keyframing vs. Mocap

•Control     •Detail hard
Keyframing     •Intuitive

Mocap

Basic Concepts
Keyframing vs. Mocap

•Control     •Detail hard
Keyframing     •Intuitive   •Many DOF

Mocap

Basic Concepts
Keyframing vs. Mocap

•Control       •Detail hard
Keyframing     •Intuitive     •Many DOF

•Detail easy
Mocap

Basic Concepts
Keyframing vs. Mocap

•Control       •Detail hard
Keyframing     •Intuitive     •Many DOF

•Detail easy
Mocap      •All DOF

Basic Concepts
Keyframing vs. Mocap

•Control       •Detail hard
Keyframing     •Intuitive     •Many DOF

•Detail easy   •No control
Mocap      •All DOF

Basic Concepts
Keyframing vs. Mocap

•Control       •Detail hard
Keyframing     •Intuitive     •Many DOF

•Detail easy   •No control
Mocap      •All DOF       •Not intuitive

Basic Concepts
Keyframe Data vs.
Motion Capture Data
Frequency Bands

Right flat      Right toe   Left flat   Left toe

Basic Concepts
Frequency Bands
• Simplifies the form of the data
– Low frequency Variations:
Large scale motions.
– Higher frequency variations:
individual “noise” / Jitter

Both are important to preserve in order to
capture the essence of motion

Basic Concepts
Correlations
• Joints angle/translation data is
related to each other
• Joint angles are correlated
over time
• Correlation “plot” is
– (somewhat) Specific to the type
of motion
– Carries “personality” information
(style)

Basic Concepts
INTRODUCTION TO COMPUTER ANIMATION – Rick parent
http://www.cis.ohio-state.edu/~parent/book/outline.html

Splines
http://www.people.nnov.ru/fractal/splines/Intro.html

Hash Inc - Animation software (Movies, tutorials…)
http://www.hash.com

Agenda
-   Basic concepts
-   Motion Synthesis/texture using motion capture
-   Physics/Biomechanics Motion Synthesis
-   Cartoon Motion Retargeting
Goal: Motion Capture Assisted
Animation

• Create a method that allows an artist low-
level control of the motion

• Combine the strengths of keyframe
animation with those of mocap

Motion Capture Assisted Animation – Pullen/Bregler
Goal: Motion Capture Assisted
Animation

“Sketch” an animation by keyframing
• Animate only a few degrees of freedom
• Set few keyframes
“Enhance” the result with mocap data
• Synthesize missing degrees of freedom
• Texture keyframed degrees of freedom

Motion Capture Assisted Animation – Pullen/Bregler
What is a Motion Texture?

• Every individual’s movement is unique
• Synthetic motion should capture the
texture
• To “texture” means to add style to a pre-
existing motion
• Technically, texturing is a special case
of synthesis
Goal: Motion Capture Assisted
Animation
Blue = Keyframed
Purple = Textured/Synthesized

Motion Capture Assisted Animation – Pullen/Bregler
How an Animator Works

• A few degrees of freedom at first

• Not in detail

• Fill in detail with more keyframes later

Motion Capture Assisted Animation – Pullen/Bregler
The Method in Words

• Choose degrees of freedom to drive the animation

• Compare these degrees of freedom from the
keyframed data to mocap

• Find similar regions

• Look at what the rest of the body is doing in those
regions

• Put that data onto the keyframed animation
Motion Capture Assisted Animation – Pullen/Bregler
Choices the Animator
Must Make

1.       Which DOF to use as matching angles

2.       Which DOF to texture, which to synthesize

3.       Which frequency band to use in matching

4.       How many frequency bands to use in texturing

5.       How many matches to keep

6.       How many best paths to keep
Motion Capture Assisted Animation – Pullen/Bregler
Before Beginning:
Choose Matching Angles
Root x trans          Left Clavicle x           Left Hip x
Root y trans          Left Clavicle y           Left Hip y
Root z trans          Left Clavicle z           Left Hip z
Root x rot            Left Shoulder x           Left Knee x
Root y rot            Left Shoulder y           Left Knee y
Root z rot            Left Shoulder z           Left Knee z
Spine1 x              Left Elbow x              Left Ankle x
Spine1 y              Left Elbow y              Left Ankle y
Spine1 z              Left Elbow z              Left Ankle z
Spine2 x              Left Wrist x              Left Ball x
Spine2 y              Left Wrist y              Left Ball y
Spine2 z              Left Wrist z              Left Ball z
Spine3 x              Right Clavicle x          Right Hip x
Spine3 y              Right Clavicle y          Right Hip y
Spine3 z              Right Clavicle z          Right Hip z
Neck x                Right Shoulder x          Right Knee x
Neck y                Right Shoulder y          Right Knee y
Neck z                Right Shoulder z          Right Knee z
Head x                Right Elbow x             Right Ankle x
Head y                Right Elbow y             Right Ankle y
Head z                Right Elbow z             Right Ankle z
Head Aim x            Right Wrist x             Right Ball x
Head Aim y            Right Wrist y             Right Ball y
Head Aim z            Right Wrist z             Right Ball z

Time                      Time                   Time
Matching Angles
Drive the Synthesis
Root x trans           Left Clavicle x           Left Hip x
Root y trans           Left Clavicle y           Left Hip y
Root z trans           Left Clavicle z           Left Hip z
Root x rot             Left Shoulder x           Left Knee x
Root y rot             Left Shoulder y           Left Knee y
Root z rot             Left Shoulder z           Left Knee z
Spine1 x               Left Elbow x              Left Ankle x
Spine1 y               Left Elbow y              Left Ankle y
Spine1 z               Left Elbow z              Left Ankle z
Spine2 x               Left Wrist x              Left Ball x
Spine2 y               Left Wrist y              Left Ball y
Spine2 z               Left Wrist z              Left Ball z
Spine3 x               Right Clavicle x          Right Hip x
Spine3 y               Right Clavicle y          Right Hip y
Spine3 z               Right Clavicle z          Right Hip z
Neck x                 Right Shoulder x          Right Knee x
Neck y                 Right Shoulder y          Right Knee y
Neck z                 Right Shoulder z          Right Knee z
Head x                 Right Elbow x             Right Ankle x
Head y                 Right Elbow y             Right Ankle y
Head z                 Right Elbow z             Right Ankle z
Head Aim x             Right Wrist x             Right Ball x
Head Aim y             Right Wrist y             Right Ball y
Head Aim z             Right Wrist z             Right Ball z

Time                      Time                   Time
Motion Capture Data

Root x trans          Left Clavicle x           Left Hip x
Root y trans          Left Clavicle y           Left Hip y
Root z trans          Left Clavicle z           Left Hip z
Root x rot            Left Shoulder x           Left Knee x
Root y rot            Left Shoulder y           Left Knee y
Root z rot            Left Shoulder z           Left Knee z
Spine1 x              Left Elbow x              Left Ankle x
Spine1 y              Left Elbow y              Left Ankle y
Spine1 z              Left Elbow z              Left Ankle z
Spine2 x              Left Wrist x              Left Ball x
Spine2 y              Left Wrist y              Left Ball y
Spine2 z              Left Wrist z              Left Ball z
Spine3 x              Right Clavicle x          Right Hip x
Spine3 y              Right Clavicle y          Right Hip y
Spine3 z              Right Clavicle z          Right Hip z
Neck x                Right Shoulder x          Right Knee x
Neck y                Right Shoulder y          Right Knee y
Neck z                Right Shoulder z          Right Knee z
Head x                Right Elbow x             Right Ankle x
Head y                Right Elbow y             Right Ankle y
Head z                Right Elbow z             Right Ankle z
Head Aim x            Right Wrist x             Right Ball x
Head Aim y            Right Wrist y             Right Ball y
Head Aim z            Right Wrist z             Right Ball z

Time                      Time                   Time
Overview

Steps in texture/synthesis method

• Frequency analysis
• Matching
• Path finding
• Joining

Motion Capture Assisted Animation – Pullen/Bregler
Example

In the following series of slides:

Hip angle = matching angle

Spine angle = angle being synthesized

Motion Capture Assisted Animation – Pullen/Bregler
Frequency Analysis:
Break into Bands

Motion Capture Assisted Animation – Pullen/Bregler
Frequency Analysis

Band-pass decomposition of matching angles
Keyframed Data                           Motion Capture Data
Frequency

Time
Motion Capture Assisted Animation – Pullen/Bregler
Frequency Analysis

Chosen low frequency band
Keyframed Data                           Motion Capture Data
Frequency

Time
Motion Capture Assisted Animation – Pullen/Bregler
Chosen Low Frequency Band

Hip angle data (a matching angle)
Keyframed Data                           Motion Capture Data

Motion Capture Assisted Animation – Pullen/Bregler
Making Fragments

Break where first derivative changes sign
Keyframed Data                           Motion Capture Data

Motion Capture Assisted Animation – Pullen/Bregler
Making Fragments

Step through fragments one by one
Keyframed Data                           Motion Capture Data

Motion Capture Assisted Animation – Pullen/Bregler
Matching

Keyframed
Fragment

Motion Capture Assisted Animation – Pullen/Bregler
Matching

Motion Capture Data

Keyframed
Fragment

Motion Capture Assisted Animation – Pullen/Bregler
Matching

Motion Capture Data

Keyframed
Fragment

Motion Capture Assisted Animation – Pullen/Bregler
Matching

Compare to all motion capture fragments
Angle in degrees

Keyframed
Mocap

Time
Matching

Resample mocap fragments to be same length
Angle in degrees

Keyframed
Mocap

Time
Matching
Using some metric on all matching angles
and on their first derivatives:
Keep the K closest matches
Angle in degrees

Keyframed
Mocap

Time
Matching

Motion Capture Data

Keyframed
Fragment

Motion Capture Assisted Animation – Pullen/Bregler
Matching

Motion Capture Data

Close
Matches
Keyframed
Fragment

Motion Capture Assisted Animation – Pullen/Bregler
Matching

Hip Angle (Matching Angle)

Spine Angle (For Synthesis)

Motion Capture Assisted Animation – Pullen/Bregler
Matching and Synthesis

Low frequency hip angle data (a matching angle)

Spine angle data to be synthesized

Motion Capture Assisted Animation – Pullen/Bregler
Matching and Synthesis

Low frequency hip angle data (a matching angle)

Spine angle data to be synthesized

Motion Capture Assisted Animation – Pullen/Bregler
Matching and Synthesis

Low frequency hip angle data (a matching angle)

Spine angle data to be synthesized

Motion Capture Assisted Animation – Pullen/Bregler
Matching and Synthesis

Low frequency hip angle data (a matching angle)

Spine angle data to be synthesized

Motion Capture Assisted Animation – Pullen/Bregler
Matching and Synthesis

Low frequency hip angle data (a matching angle)

Spine angle data to be synthesized

Motion Capture Assisted Animation – Pullen/Bregler
Matching and Synthesis

Low frequency hip angle data (a matching angle)

Spine angle data to be synthesized

Motion Capture Assisted Animation – Pullen/Bregler
Possible Synthetic
Angle in degrees
Spine Angle Data

Time
Path Finding
We would like to:
• Use as much consecutive fragments as possible
• Stay as close as possible to best fit
Angle in degrees

Time
Path Finding
Angle in degrees

Time
Path Finding
Angle in degrees

Time
Path Finding
Angle in degrees

Time
Path Finding
Angle in degrees

Time
Angle in degrees
Joining

Time
Enhancing Animations:
Texturing and Synthesis
Keyframed    Not keyframed

Textured     Synthesized
Texturing

Synthesize upper frequency bands

Motion Capture Assisted Animation – Pullen/Bregler
Texturing

Band-pass decomposition of keyframed data
Frequency

Time
Texturing

Synthesize upper frequency bands
Frequency

Time
Walking animations
Texturing and Synthesis
Keyframed Sketch
Walking animations
Texturing and Synthesis
Motion Capture Data
Two different styles of walk
Walking animations
Texturing and Synthesis
Enhanced Animation
Upper body is synthesized
Lower body is textured
Otter Animations: Texturing
Keyframed data
Otter Animations: Texturing
Textured animation
Dance Animations: Texturing and
Synthesis
Lazy Keyframed
Sketch
Dance Animations: Texturing and
Synthesis
Motion Capture
Data
Dance Animations: Texturing and
Synthesis
Enhanced Animation
Blue = Keyframed
Purple = Textured/Synthesized
Dance Animations: Texturing

Keyframed Sketch With
More Detail
Dance Animations: Texturing
Textured Animation
Blue = Keyframed
Purple = Textured
Summary of the Method
Sketch +         Frequency
Mocap           Analysis             Matching
Keyframed data   Keyframed Data      Matching Angles

Mocap Data       Mocap Data      Possible Synthetic Data

Path Finding        Joining
Enhanced
Animation
Choices the Animator
Must Make

1.   Which DOF to use as matching angles

2.   Which DOF to texture, which to synthesize

3.   Which frequency band to use in matching

4.   How many frequency bands to use in texturing

5.   How many matches to keep

6.   How many best paths to keep
Conclusions and
Applications
• Appropriate for an artist interested in a very
particular style of motion

• The artist may have a relatively small motion
capture set of that style

• The artist may want precise control over parts
of the motion
Conclusions and
Further Work

• Direct incorporation of hard constraints

• Fundamental units of motion

http://graphics.stanford.edu/~pullen

Special Thanks to:
Reardon Steele, Electronic Arts
Agenda
-   Basic concepts
-   Motion Synthesis/texture using motion capture
-   Physics, Biomechanics Motion Synthesis
-   Cartoon Motion Retargeting
Motivation
• Generate rapid prototyping of realistic
character motion
• Avoid simulated human models, that are
very complex, and don’t always look
realistic

"C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations
Scope
• Highly dynamic movement such as
jumping, kicking, running, and gymnastics.
• Less energetic motions such as walking or
reaching will not work well in this
framework

"C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations
Overview of the process
Motion sketch
Constraint & phase
detection
Character description

Motion DB
Transition pose
Momentum control
synthesis
User interaction

Objective functions                Optimization              Animation

"C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations
Overview of the process
• The objective is to
transforms simple
animations into realistic
character motion by
applying laws of physics
and the biomechanics
domain
• The unknowns are:
values of joint angles and
parameters of angular
and linear momentum

"C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations
Overview of the process
Motion sketch
Constraint & phase
detection
Character description

Motion DB
Transition pose
Momentum control
synthesis
User interaction

Objective functions                Optimization              Animation

"C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations
Constraint and stage detection
• Each input sequence
has two parts:
– The part that needs to
be improved
– The part that needs to
kept intacked
• Automatically extract
the positional and
sliding constrains

"C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations
Positional constraint detection
• A positional constraint fixes a specific point
on the character to a stationary location for
a period of time
• We need to find if all these points lie on a
line, plane
• In an articulated character we find the
constraints on each body part

"C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations
Positional constraint detection
• The algorithm looks
Ti xi  xi
for fixed points (point,
line, plane)
(Ti  I ) xi  0

"C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations
Sliding constraints
min   p ,l    Dist (T W p, l )
i
i   i

"C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations
Overview of the process
Motion sketch
Constraint & phase
detection
Character description

Motion DB
Transition pose
Momentum control
synthesis
User interaction

Objective functions                Optimization              Animation

"C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations
Transition pose generation
• A transition pose
separates constrained
and unconstrained
stages.
• Two possibilities:
to draw the transition
poses
– We have an estimator
to suggest a transition
pose

"C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations
Transition Pose Estimator
• DB contains examples of different motions
• The input of that DB are the motion
parameters like: flight distance, flight
height, takeoff angle, landing angle, spin
angle..
• The DB has a simplified representation of
the transition poses by three COM’s
• We use IK to obtain the full character’s
pose from those three COM’s
• The KNN - K nearest neighbor algorithm
• The pose estimator predicts the candidate
pose by interpolating the KNN with the
weights that describe the similarity to the                          CB
input.                                      C B  C B  (C A  C A )                  2

CA
2

"C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations
Overview of the process
Motion sketch
Constraint & phase
detection
Character description

Motion DB
Transition pose
Momentum control
synthesis
User interaction

Objective functions                Optimization              Animation

"C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations
Momentum control
• Transition poses constrain the motion at few key
points of the animation

• Dynamic constraints ensure realistic motion of
each segment
• Linear and angular momentum give us these
dynamic constraints

"C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations
Momentum during unconstrained
and constrained stages
• linear momentum -
dP(q)
During “flight” the only                                  mg
force is gravity                                    dt
• Angular momentum -
During “flight” there is no                         dL(q )
change in Angular                                          0
momentum
dt
• During “ground” stage we
avoid computing the
momentums and use
empirical characteristics

"C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations
Overview of the process
Motion sketch
Constraint & phase
detection
Character description

Motion DB
Transition pose
Momentum control
synthesis
User interaction

Objective functions                Optimization              Animation

"C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations
Objective functions
• There are three Objective
functions, the basic idea
behind them is power
consumption
– Minimum mass displacement
– Minimal velocity of DOFs
– Static balance

"C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations
Overview of the process
Motion sketch
Constraint & phase
detection
Character description

Motion DB
Transition pose
Momentum control
synthesis
User interaction

Objective functions                Optimization              Animation

"C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations
Putting it all together
•   Environment constraints (Ce)
•   Transition pose constraints (Cp)
•   Momentum constraints (Cm)
•   Q are character’s DOFs

 C e (Q )  0

min  Ei(q ) subject to C p (Q )  0
i

C (Q )  0
Q

 m

"C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations
Overview of the process
Motion sketch
Constraint & phase
detection
Character description

Motion DB
Transition pose
Momentum control
synthesis
User interaction

Objective functions                Optimization              Animation

"C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations
Some Results
• Wide variety of figures: male, female, child
• 51 DOFs
• The body dimensions and mass distribution is taken from
biomechanics literature
• In some of the cases the animator selects the body parts to be
constraints
• The animator can change relative timing between each phase
• The optimization was solved by using SNOPT a general
nonlinearly-constrained optimization package
• The optimization time depends on the duration of the animation
• All of the simple animation took less than five minutes to sketch
• For all examples the synthesis process took less than five
minutes

"C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations
• Only 3 keyframes at
takeoff, peak and
landing

"C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations
Running
• The angular
momentum constraint
creates a counter-
body movement by
the shoulders and
arms to counteract
the angular
momentum generated
by the legs.
• Keyframing 7 DOFs

"C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations
Hopscotch
• Each hop requires 3
keyframes and has
fewer than 7 DOFs

"C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations
Handspring
• There were no
handstands within the
DB so the user had to
modify the result

"C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations
High-bar
• Two constraints
stages: the bar and
ground

"C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations
Karate kick
• A second synthesis add a keyframe in the
peak

"C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations
Twist jumps

"C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations
Agenda
-   Basic concepts
-   Motion Synthesis/texture using motion capture
-   Physics/Biomechanics Motion Synthesis
-   Cartoon Motion Retargeting
What is Cartoon Capture &
Retargeting
• Cartoon Capture
– Track the motion From
2D Animation
– Represent the motion
& save
• Retargeting
– Translate the motion
representation to
another output media

”C.Bregler, L.Loeb, E.Chuang, H.Deshpande; "Turning to the Masters: Motion Capturing Cartoons
Cartoon motion capture &
retargeting scheme
Digitized video
Motion                                  Output video
Cartoon capture                             retargeting
representation
Key shapes

Output corresponding
key shapes

”C.Bregler, L.Loeb, E.Chuang, H.Deshpande; "Turning to the Masters: Motion Capturing Cartoons
Modeling Cartoon motion
Digitized video
Motion                                  Output video
Cartoon capture                             retargeting
representation
Key shapes

Output corresponding
key shapes

”C.Bregler, L.Loeb, E.Chuang, H.Deshpande; "Turning to the Masters: Motion Capturing Cartoons
Modeling Cartoon motion
• Two types of
deformations
– Affine deformation

– Key shape
deformation

”C.Bregler, L.Loeb, E.Chuang, H.Deshpande; "Turning to the Masters: Motion Capturing Cartoons
y
Affine Deformation
si  xi           yi 1
T

x
 a1                a2      dx 
V  warp ( , S )                                S
a3                 a4      dy 

• Affine parameters                              (t )  [a1 , a2 , a3 , a4 , d x , d y ]

”C.Bregler, L.Loeb, E.Chuang, H.Deshpande; "Turning to the Masters: Motion Capturing Cartoons
Key-Shape Deformation

 a1                a2      dx              
V  warp ( , S )                                    wk S k 
dy   k
a3                 a4                        

• Sk are the key shapes

w1            w2         w3

”C.Bregler, L.Loeb, E.Chuang, H.Deshpande; "Turning to the Masters: Motion Capturing Cartoons
Modeling Cartoon motion
• In total there are 6+K variables that represent
the motion

 (t )  [a1 , a2 , a3 , a4 , d x , d y , w1 ,...., wk ]

”C.Bregler, L.Loeb, E.Chuang, H.Deshpande; "Turning to the Masters: Motion Capturing Cartoons
Cartoon motion capture
Digitized video
Motion                                  Output video
Cartoon capture                             retargeting
representation
Key shapes

Output corresponding
key shapes

”C.Bregler, L.Loeb, E.Chuang, H.Deshpande; "Turning to the Masters: Motion Capturing Cartoons
Cartoon motion capture

• contour capture: the
input is a sequence of
contours

• video capture: the
input is the video
sequence

”C.Bregler, L.Loeb, E.Chuang, H.Deshpande; "Turning to the Masters: Motion Capturing Cartoons
contour capture
Err  V  warp( , S1 ,...,Sk )
2

• Two step minimization:
– Find Affine parameters
2
 a1              a2        dx 
 V                                 S
dy 
Erraff
a3               a4           
 a1    a2    dx                T 1
  V  S (S  S )
T
a      a4    dy 
 3
– Find Key-Shape weights
2
 a1               a2       dx 
Err  V                                   ( wk  S k )
a3                a4       dy 

• Iterate

”C.Bregler, L.Loeb, E.Chuang, H.Deshpande; "Turning to the Masters: Motion Capturing Cartoons
Retargeting
Digitized video
Motion                                  Output video
Cartoon capture                             retargeting
representation
Key shapes

Output corresponding
key shapes

”C.Bregler, L.Loeb, E.Chuang, H.Deshpande; "Turning to the Masters: Motion Capturing Cartoons
Retargeting
• For each Input key-shape an Output key-
shape is drawn.

”C.Bregler, L.Loeb, E.Chuang, H.Deshpande; "Turning to the Masters: Motion Capturing Cartoons
Retargeting Process

Key shapes Interpolation
Retarget
Motion capture
Apply Affine transformation
From motion capture

Retargeted media

”C.Bregler, L.Loeb, E.Chuang, H.Deshpande; "Turning to the Masters: Motion Capturing Cartoons
Examples

”C.Bregler, L.Loeb, E.Chuang, H.Deshpande; "Turning to the Masters: Motion Capturing Cartoons
processing
• Undesirable effects may still appear
• Determine constraints that force the
character go through certain position at
certain time
• Apply ad-hoc global transformation that
fulfill these constraints

”C.Bregler, L.Loeb, E.Chuang, H.Deshpande; "Turning to the Masters: Motion Capturing Cartoons
Performance
• Quantative performance wasn’t mentioned
• The more complex the motion of the
character is, the more key-shapes are
needed
• Many of the animations contain jitter, but
the overall exaggerated motion dominates

”C.Bregler, L.Loeb, E.Chuang, H.Deshpande; "Turning to the Masters: Motion Capturing Cartoons

```
To top