Vision Hand Gesture Recognition by emz20494

VIEWS: 140 PAGES: 26

									         Cindy Song
Sharena Paripatyadar
• Use vision for HCI
• Determine steps necessary to
  incorporate vision in HCI applications
• Examine concerns & implications of such
 In Today’s    world:
  • Many devices with integrated cameras
  • Many personal webcams

 Our   Goal:
  • To understand how to take advantage of these
   one camera systems
  Literature       Idea         Media Player
  Research       Generation     Wizard of Oz

  Literature     Re-evaluate    Media Player
  Research        Approach     Implementation

                 Application     Evaluation
                 User Study
 Freeman   and Roth
 For hand posture
 Creates histograms of
  local orientation
  using feature vectors
  from pixel intensity
 Recognizes 10
  gestures in real time
 Triesch and von der
 Based on Elastic
  Graph Matching
 Extended for skin
  color feature
 Recognizes 12
 Freeman
  Uses one open hand
  to control onscreen
 Real time application
 Hand may not be
  prominent in image
5  participants, various technical
  backgrounds, age 20-27
 Using computer with remote control
 Used alternate monitor to show user
  video captured
   Small set of user-intuitive gestures are
    easy to remember, but need some menu
   Show rationale behind gestures
   Visual feedback to show recognized
    command before execution
   Concerns with:
       Low-light condition
       Camera field of view
       Webcam configuration
       Responsiveness
       Accuracy
   Pros
     Don’t have to search for remote
     Don’t have to touch remote while eating
     No battery to run down

   Cons
     Doesn’t have as many features as remote
     Doesn’t work in dark environments
     More ambiguous than remote, more errors
     possible – know what each button will do
 Skin   Color Training
  • Trained on 20+ images
  • Different lighting & people
  • Uses “Lab” color space
 Calibration
  • Short training based on person’s hand and
    lighting conditions - < 1 sec needed
  • Determines correct lighting & with skin color
  • Learns specific hand features
   Hand Location
    • Determines hand position in image using skin
    • Fill in missing portions of hand
    • Create bounding box
   Finger Region Detection
    • Examine bounding box
    • Find connected regions
    • Remove small regions
 Pattern   Recognition
  • Created set patterns based on 10 gestures
  • Counts number of finger regions for
    gestures 1-5
  • For gestures 6-10, based on number
    regions detected, looks at other patterns
    i.e. for 6 determine ratio of finger width to
     space between fingers
 Gesture   Determination
  • 20 frames needed to recognize the gesture
  • Avoids recognizing accidental gestures
 Complex    Backgrounds
 • First skin color analysis
 • Then find large connected regions of fingers and
 Motion
  • Static gestures & frame by frame analysis
  • Allow for moving camera
  • Gesture determination corrects obscurities or
   out of frame hand positioning
5 participants, various technical
  backgrounds, age 20-27
 Taught users 2-4 gestures
 Quizzed users on gestures learned
 Ran gesture recognition algorithm to
  provide feedback
 Asked several follow up questions
   Useful for learning sign language,
    teaching kids to count
   Instant feedback necessary
   Nice to know how to correct gesture
   Needs high accuracy
   Other applications
    • Some said Media Player application more useful
    • Or use as security system (hand gestures as a
   Of implementation
     Real time is difficult
     Pattern recognition for specific gestures vs.
      technique for all types of gestures
     Complex/moving backgrounds important for
      real world applications

   Of user studies
     Video is valuable avenue for many applications
     Accuracy and responsiveness are important
     In one camera systems, there is a tradeoff
     between convenience and clarity
 Real-time
 More  user studies
 Mobile devices
 Gesture learning
  • i.e. Chinese cultural gestures
 MediaPlayer plug-in

To top