CMPCD3001
User Interface Design
Lecture 10: Advances in
Interaction
Tutor: Contact:
Andrew Symons E-Mail: a.symons@ljmu.ac.uk
Advances in Interaction
• Whole Body Interaction
• Introduction
• Technology
• Examples
• Challenges
2
Who Body Interaction: Introduction
• What do we mean by Who Body Interaction?
• Going beyond Keyboard and Mouse
– Physical body – motion capture
– Physiological state – pulse, breathing, skin conductivity
– All human senses – including balance, propriocentric
– Cognitive State
– Emotional State – inferred from other measures
• Exploiting the full physical capabilities of humans
– Extending current forms of interaction
– New forms of interaction
– Group, shared and remote interaction
3
Technologies
• Some of technologies for whole body interaction have
been around for a decade or more but they have been
– Expensive in themselves
– Limited to specialist usage
– Require high-end computing support
– Required Specialist software
• For example, at LJMU in Sports Science we have the Caren
Platform we combines motion capture via six or more cameras plus
a moving platform which can sense people’s movement
– http://www.e-motek.com/medical/demo/boat.htm
– http://www.youtube.com/watch?v=q3ZEvx63DNA (video)
– The system can capture dozens of reflector points attached to
the subject’s body for highly accurate motion capture
4
Technologies …
• Much of the emphasis on past work has been on
capturing and analysing physical motion
• Whereas in our approach we are more interested in the
human body being an active, interaction controller of the
system
• So now we are looking at
– Less expensive means of motion capture
– A better understanding of how people move combined with the
task goals they are trying to achieve
– More intelligent software to look for patterns in movement and
patterns between movement and other measurements
5
Technologies … Motion Capture
• As the cost of accelerometers and gyroscopes comes
down in price wireless whole body motion kits are now
available for around £1000-£2000
• Examples include
– Xsens Xbus Kit
http://www.xsens.com/en/products/human_motion/xbus_kit.php
– Eowave Eobody2 Hands Free
http://www.eowave.com/products.php?prod=57
– Wiiremote which can be used with PCs and Macs using Glove
PIE and Wiiremote
– http://carl.kenner.googlepages.com/glovepie
– http://sourceforge.net/projects/darwiin-remote/
6
Technologies … Motion Capture
• The Wiiremote is a good example of how whole body
interaction can be exploited in games
– It contains an accelerometer for orientation sensing
– And a four-point IR camera for position tracking
• It lacks a gyroscope for proper inertial sensing
• The casing is too bulky to use multiple Wii’s
• The accelerometer signal can be quite noisy and
requires filtering
• However the addition of the new Wii Motion device
improves the accuracy of the Wiiremote
– http://uk.wii.ign.com/articles/893/893192p1.html
7
Technologies … Motion Capture
• Another means of motion capture is by image processing
– At the high end work at Stanford University tries to recongise the
whole body and create a 3D mesh in real-time
– https://ccrma.stanford.edu/~stefanoc/Markerless/Markerless.html
– At the lower end we can use ordinary webcams and PC’s to
perform shadow and edge detection
– We have used this approach in the Mixed Reality demonstration
presented at the FACT Centre in 2006
– http://www.hci-fun.org.uk/hcied07/
– Microsoft Project Natal
• http://www.xbox.com/en-US/live/projectnatal/
• Follow the links above to see video demonstrations
8
Examples
• At the higher end motion capture is used in
– The Film and Animation industries e.g. Industrial Light and Magic
and Pixar
– Military Training – USC ICT
– Sports Science
– Health
• assessing the susceptibility of older people to fall over whilst walking
• Assessing the therapy needs of people with conditions like cerebral palsy
• In our work we are more interested
– Lower tech devices with networks of multiple sensors
– The software framework to manage these devices
9
Examples
• As an example of low-tech devices Johnny Chung Lee
http://johnnylee.net/projects/wii/ shows how the Wii
remote can be programmed to provide new kinds of
interactions such as
– Finger tip tracking
– Interactive Whiteboard
– And Head tracker
• Using multiple Wii remotes these examples could be
extended to provide full body motion tracking at low cost
10
Challenges
• There remain several challenges to the
further use of Whole Body Interaction
– In HCI – understanding how people make sense of
body movements to system responses
– In Bio-mechanics – understanding the capabilities
and limitations of people moving and acting in virtual,
motion-capture environments
– In Software – building intelligent systems that can
integrate and learn people’s movements and
physiological states
11