Outline
Computer Vision For Interactive Computer Graphics
Nitin Pande 10/11/2000
Why Computer Vision( CV ) as an Interface? What is Interactive Computer Graphics ( ICG )? Interactive Graphics Vs Real Life CV Algorithms for Applications In ICG: ! Large Object Tracking ! Shape Recognition ! Motion Analysis ! Small Object Tracking Conclusions
Computer Vision as an Interface
Potential to sense
! ! ! !
Interactive Computer Graphics
Enhancement Of 2D Graphics Using Computer Vision as an Interface
body position head orientation direction of gaze pointing commands and gestures
"Machine interaction
# # # #
Unencumbered Enjoyable Engaging Safer….
Real Life vs. ICG
Real Life Problem
!
ICG Problem
!
Unconstrained =>Algorithms ! Complex & Slower ! Unreliable ! Expensive Hardware
Context Based Constraints =>Algorithms
! ! !
Simpler and Faster Inexpensive Hardware Real Time Vision Control
1
Large Object Tracking
Fast CV Algorithms for ICG
Large Object??? Applications
! !
Computer games Machine Control
I(x,y) –Image intensity at (x,y) Image Moments :
Image Moments provides fast, coarse summary global orientation position and size of object.
Faster Calc of Image Mom.
Vertical Horizontal and Diagonal Projections Special Hardware : Artificial Retina Chip
Some Applications
Robot Car Control Using Hand Gestures.
2
Computer Game: Night.
Computer Game: Magic Carpet.
Shape Recognition
Orientation Histograms
Summarize how much of each shape is oriented in each possible direction, independent of position Local Orientation Q(x,y) = arctan[I(x,y) – I(x-1,y), I(x,y) –I(x,y-1)] Euclidian distance as a distance measure.
Orientation more robust to lighting changes than pixel Intensities
V Fast Computation using Conventional / Special Hardware.
Shape Recognition :Two Step Process
TRAIN
RUN
Example Application
3
Problems with Orientation Histograms.
Very Different Gestures may have similar Orientation Histograms.
Solution
$ Chose Vocabulary of gestures that avoids confusing
pairs
Similar Gestures may have different Orientation Histograms.
$ Train for each different version of gesture
Motion Analysis
Fast Optical Flow
Extremely fast estimate of relevant Motion Parameters. Fast Optical Flow Algorithm
Application: Sega’s Decathlete Game
Game context greatly simplifies the visual recognition.
Small Object Tracking
Small Objects Normalized Correlation
Application: TV controlled by hand gestures
Design Problems
•Ease of learning of Control functionalities for humans •Difficulty associated with computer for understanding broad hand gestures within a complex unpredictable visual environ.
4
Conclusion
Synergy b/w Fast, Simple Vision Algos and ICG at system level. ICG an enjoyable & intriguing experience. Advances in Algorithms,Processing Power and Memory will further improve Vision based Interfaces.
5