Estimating Human Shape and Pose from a Single Image

					Estimating Human Shape and
  Pose from a Single Image
                  Peng Guan
  Alex Weiss Alexandru Balan   Michael J. Black

               Brown University
        Department of Computer Science

                 ICCV’ 2009
Body shape and pose from 1 image?
What others do                What we do
• Estimating 3D human         • Estimating both 3D shape
  pose in uncalibrated          and pose in uncalibrated
  monocular imagery             monocular imagery
• Use silhouette in multi-
  camera setting to recover   • Use additional monocular
                                cues including smooth
  3D body shape                 shading
• Most work assumes the
  existence of a known        • Use GrabCut to produce
  background to extract         foreground region
  foreground silhouette
• In previous body models,    • Make height variation
  height is correlated with     concentrated along one
  other shape variations        shape basis vector, which
                                allows “height constrained
                                         Previous Work
  3D pose and shape estimation from multiple, calibrated, cameras

Balan, A., Sigal, L., Black, M. J., Davis, J., Haussecker, H, “Detailed human shape and pose from images”, Proc. IEEE Conf.
on Computer Vision and Pattern Recognition, CVPR, Minneapolis, June 2007
                              SCAPE Body Model

D. Anguelov, P. Srinivasan, D. Koller, S. Thrun, J. Rodgers, and J. Davis. SCAPE: Shape completion and
animation of people. SIGGRAPH, 24(3):408–416, 2005.
Body shape/pose from 1 image: Problems

1. High dimensional body model (shape and
   pose) – initialization problem.
2. Background unknown
3. Single, monocular image
  1. poorly constrained
  2. Shape/Pose ambiguities
4. Silhouette insufficient
Solution 1: Pose Initialization

         Shape: initialized to mean body shape.
               Solution 2: Segmentation

C. Rother, V. Kolmogorov, and A. Blake. “GrabCut”: Interactive foreground extraction using iterated graph cuts.
SIGGRAPH, 23(3):309–314, 2004.
Problem: Pose/Shape ambiguities

  Body shape and pose fitted to a single camera view
Solution 3: Height Preserving Shape Space
Shape space without height preserving
Problem: Silhouette not sufficient
Solution 4: Edge Cues
Problem: Shape not well constrained
Solution 5: Parametric Shape from Shading

M. de la Gorce, N. Paragios and David Fleet. Model-Based Hand Tracking with Texture, Shading and Self-
occlusions. IEEE Conference in Computer Vision and Pattern Recognition (CVPR), Anchorage 2008.
  Shading/Overall Cost function
Shading cost function:

Overall cost function:
Experiment: Lab Images
Experiment: Lab Images
Quantitative Comparison
Experiment: Internet Images
Experiment: Paintings
• Solution to a new problem: Human pose and
  shape estimation from a single image
• Parametric shape from shading for estimating
  human shape from complex images and paintings
• Attribute-constrained body model
• Single point light assumption and simplified
  model of surface reflection
• User assistance for pose initialization
• Minimal clothing for shading
• Financial support: NSF IIS-0812364 and the RI Economic
  Development Corp.
• Peng Guan, Alexander Weiss, Alexandru Balan, Michael Black,
  “Estimating Human Shape and Pose from a Single Image”, Int.
  Conf. on Computer Vision, ICCV, Kyoto, Japan, Sept. 2009

• Alexander Weiss: GrabCut 3D pose initialization
• Alexandru Balan: Height preserving shape space
• David Hirshberg: Projection of model edge
Thank you!

Shared By: