Gestural Interfaces for the Tablet PC Simon Fraser University March 3, 2006
Andy van Dam
Brown University
Gestural Interfaces for Tablet PC: MathPad2
• Mathematical sketching – combine handwritten math and freeform – – –
diagrams graphing uses MATLAB® as underlying math engine math expression recognizer
• Diagrams animate according to
associated math expression(s)
• Fully gestural interface for editing – expressions can be deleted, edited, and –
re-recognized modeless operation
Simon Fraser University March 3, 2006 2
Immersive Virtual Reality in Scientific Visualization
Gestural Interfaces for Tablet PC: ChemPad
• Organic chemistry
– requires complex 3D visualization, – physical models are clunky – hand-drawn notation is lingua franca,
but error-prone, messy, & slow even at intro level
• Traditional WIMP chemistry drawing programs exist but
– significantly slower than sketching – used for papers and presentations, not for notes – 3D programs only used by experts
• ChemPad allows – molecules to be quickly sketched – 3D visualization from hand-drawn elements – visualization of chemistry concepts
Immersive Virtual Reality in Scientific Visualization Simon Fraser University March 3, 2006 3
Gestural Interfaces for Tablet PC: ChemPad
•
Molecule Sketching
– –
uses fast, unistroke characters and symbols can be learned quickly (about 5 minutes) based on molecular mechanics gives feedback on common beginner errors more reliable than plastic ball-andstick kits stereochemistry naming with CIP orderings space filling and dot renderings newman projections with energy changes over rotations
Simon Fraser University March 3, 2006 4
•
3D Model Generation
– – –
•
Chemistry Visualizations Include
– – –
Immersive Virtual Reality in Scientific Visualization
Gestural Interfaces for Tablet PC: ChemPad
•
Last spring’s user study
– – –
opened a lab of ChemPad-equipped tablets to Orgo students at Brown helped students learn skills for 3D molecule visualization generated a large amount of praise and some concise criticism from students public free release user studies in three schools accepting traditional chemistry form input in addition to quick note-taking style tools for exploring reactions molecular dynamics simulations
Simon Fraser University March 3, 2006 5
•
Plans for this year include
– – – – –
Immersive Virtual Reality in Scientific Visualization
Gestural Interfaces for Tablet PC: Fluid Inking
• Gestures discovered as command shortcuts – regular mnemonic/iconic structure – similar to command-keys • Modeless integration of gestures & ink – short sequences of simple gestures – pre-fix flicks – terminal punctuation • Enhanced interactions with widgets – gestures for pre-selecting small widgets – gestures for increasing button functionality
taps, sliding, crossing gestures
Immersive Virtual Reality in Scientific Visualization
Simon Fraser University March 3, 2006
6
“To Infinity and Beyond…”
Immersive Virtual Reality in Scientific Visualization
Simon Fraser University March 3, 2006
7