Interaction Techniques for Virtual Environments

Document Sample
Interaction Techniques for Virtual Environments Powered By Docstoc
					3D Interaction Techniques for
    Virtual Environments
        Doug A. Bowman
        Edited by C. Song
      Technique Classification
      by metaphor
                           VE manipulation techniques
                              Exocentric metaphor
                                              World-In-Miniature
                                              Scaled-world grab
                                Egocentric metaphor
                                      Virtual Hand metaphor
                                              "Classical" virtual hand
                                              Go-Go
                                              Indirect, stretch Go-Go

                                      Virtual Pointer metaphor
                                              Ray-casting
                                              Aperture
                                              Flashlight
                                              Image plane
(C) 2005 Doug Bowman, Virginia Tech                                      2
    Manipulation metaphors I

      Ray casting
            little effort required
            Exact positioning and orienting very difficult
             (lever arm effect)




(C) 2005 Doug Bowman, Virginia Tech                      3
    5.4.2 Interacting by Pointing
       Selection process
            When the vector defined by the direction of pointing
             intersects a virtual object, the user can select it by triggering
             event that confirms the selection.
            Examples of triggers are buttons and voice command
       A number of pointing techniques have been reported
            How the pointing direction is defined
            By the shape of the selection volume
       Pointing is a powerful selection technique
       Pointing, however, is generally a very poor positioning
        (Manipulation) technique
            Rotation can be effectively accomplished only about one axis.
            Expressive 6-DOF manipulation is therefore impossible

(C) 2005 Doug Bowman, Virginia Tech                                         4
    5.4.2 Interacting by Pointing

      Simple ray-casting technique
      Two-handed pointing
      Flashlight / Aperture techniques
      Image-plane techniques
      Fishing-reel techniques


(C) 2005 Doug Bowman, Virginia Tech       5
    Ray Casting (I)
       The user points at objects with a virtual ray
        that defines the direction of pointing.
            Attached to the virtual hand controlled by a 6-DOF
             sensor in immersive environment.
            Attached to a 3D widget controlled by a mouse in a
             desktop 3D environment.
       Pointing direction
            p() = h +  P, where P = the direction of user’s
             virtual hand, h= 3D position of the virtual hand
            More than one object can be intersected by the
             line p(), and only one closest to the user should
             be selected.

(C) 2005 Doug Bowman, Virginia Tech                           6
    Ray Casting (II)
       The shape of the ray
            Short line segment; Figure 5.4
            Infinitely long virtual ray; better visual feedback
       Virtual ray casting is a very powerful selection
        technique
            Except when a very high precision of selection is
             required; selecting small or faraway objects
            At close range, ray-casting is perhaps the most
             simple and efficient selection technique.


(C) 2005 Doug Bowman, Virginia Tech                                7
    Two-Handed Pointing
       Two handed technique: one hand specifies the origin
        of the virtual ray, while the other hand specifies where
        the ray is pointing to
       p() = hl +  ( hr – hl ) , where hr and hl = 3D position
        of the right and left virtual hands respectively.
       Disadvantage
            Both hands must be tracked.
       Advantage
            Allow for richer and more effective pointing interaction
            Curve the virtual pointer by twisting the hand slightly.
            Fig. 5.5 참조.


(C) 2005 Doug Bowman, Virginia Tech                                     8
    Flashlight and Aperture Techniques (I)
       Spotlight or flashlight technique
            It replaces the virtual ray with a conic selection
             volume, with the apex of the cone at the input
             device; Fig. 5.6
            Objects that fall within this selection cone can be
             selected.
                  Easy selection of small objects even when they are
                   located far from the user
            Disambiguation
                  When more than one object falls into the spot light.
            Disadvantages:
                  When selection of small objects or tightly grouped
                   objects
(C) 2005 Doug Bowman, Virginia Tech                                       9
    Flashlight and Aperture Techniques (II)
       Aperture Technique
            Modification of the flashlight technique and improve it.
            Selection line P()
             p() = e +  (h – e ) , e = virtual viewpoint
                                     h = hand
            The user can interactively control the spread angle of the
             selection volume simply by bringing the hand sensor closer
             or moving it father away;
                  Figure 5.6
            Simplify selection of virtual objects by using the orientation of
             the pointer around a central avis as an additional
             disambiguation metric.
                  Figure 5.7b
            Used easily in both desktop and immersive VEs.

(C) 2005 Doug Bowman, Virginia Tech                                       10
    Image-Plane Techniques
       The user selects and manipulates 3D objects by
        touching and manipulating their 2D projections on a
        virtual image plane located in front of the user.
            Figure 5.8
       Sticky-finger technique
            Figure 5.8
       Head-crusher technique
            Another Image-plane tech. With a data glove devices
            With two fingers, his thumb and index fingers
       Allows the user to modify the orientation of 3D objects,
        but their distance from the user can not be directly
        controlled.
            Mine’s scaled-world grab, Pierce’s Voodoo Doll technique

(C) 2005 Doug Bowman, Virginia Tech                                     11
    Fishing-Reel Technique

      The ray direction is controlled by the
       spatial movements of user’s hand,
      While distance is controlled by other
       means.
            Controlling the length of the virtual ray.
            A simple mechanical slider or a pair of
             buttons added to the tracking device

(C) 2005 Doug Bowman, Virginia Tech                       12
    Manipulation metaphors II

       Simple virtual hand
             Natural but limited




(C) 2005 Doug Bowman, Virginia Tech   13
 5.4.3 Direct Manipulation: Virtual Hand Techniques

       3D cursor
            3D model of human hand
            Semitransparent volumetric cursor
       Selection
            The user intersects 3D cursor with the target of selection
       Trigger technique : to pick it up
            Button press, voice command, hand gesture
            The object is attached to the virtual hand and can be easily
             translated and rotated within V.E
       Release
            The user release it with another trigger.


(C) 2005 Doug Bowman, Virginia Tech                                       14
    Virtual Hand Interaction Techniques

      Simple (“Classical” ) virtual hand
       technique
      Go-Go technique
      Indirect Go-Go




(C) 2005 Doug Bowman, Virginia Tech         15
    Simple Virtual Hand (I)
       Direct mapping :
          transfer functions or control-display gain function
          pv = αpr , Rv = Rr (Eq. 5.4)            p  p
                                                   v   r



          pr Rr are the position and orientation(3*3 matrix) of
          the user’s real hand.
          pv , Rv are the corresponding position and
          orientation of the virtual hand
          α is a scaling ratio to match the scales of the real
          and virtual coordinate systems.


(C) 2005 Doug Bowman, Virginia Tech                            16
    Simple Virtual Hand (II)

       Scaling rotation
          It is useful to “scale” 3D device rotationsp similar to
                                                    p 
                                                    v   r


          the way we scale translations


       Fundamental problem
          In order to select objects located further away, the
          user must employ a travel technique. Inconvenient
          and increase the complexity of the 3D UI.


(C) 2005 Doug Bowman, Virginia Tech                                  17
    Go-Go Interaction Technique
 While the user’s real hand is
 close to the user (the distance
 to the hand is smaller than
 threshold D), the mapping
 is one to one, the movement
 of the virtual hand correspond
 to the real hand movements

  As the user extends her hand
  beyond D, the mapping becomes nonlinear and the virtual arm
  “grows”, thus permitting the user to access and manipulate
  remote objects. C1 continuity is ensured.
(C) 2005 Doug Bowman, Virginia Tech                    18
    Go-Go IT
       Different mapping function can be used to achieve a
        different control-display gain between the real and
        virtual hands.
       Advantages:
            Provide direct, seamless, 6-DOF object manipulation both
             close to the user and at a distance.
       Disadvantages:
            As the distance increases, the technique maps small
             movements of the user’s hand into large movements of the
             virtual hand, which complicates precise positioning at a
             distance.



(C) 2005 Doug Bowman, Virginia Tech                                     19
    5.4.5 Combining Techniques

       Aggregation of techniques
            Mechanism for choosing the desired manipulation
             technique from a limited set of possible options
       Technique integration
            Combining techniques in which the interface
             switches transparently between interaction
             techniques depending on the current task context.
            Selection과 manipulation 이 반복 적용되므로, 각
             mode에서 최선의 방법들을 선택하여 적용한다.


(C) 2005 Doug Bowman, Virginia Tech                         20
    HOMER technique
          Hand-Centered
          Object    Manipulation
          Extending
          Ray-Casting
                                      Time
       Select: ray-casting
       Manipulate: hand




(C) 2005 Doug Bowman, Virginia Tech    21
    Manipulation metaphors III

       HOMER (ray-casting + arm-extension)
             Easy selection & manipulation
             Expressive over range of distances
             Hard to move objects away from you




(C) 2005 Doug Bowman, Virginia Tech                22
    HOMER implementation
     Requires torso position t
     Upon selection, detach virtual hand from
      tracker, move v. hand to object position in
      world CS, and attach object to v. hand (w/out
      moving object)
     Get physical hand position h and distance
      dh = dist(h, t)
     Get object position o and distance
      do = dist(o, t)
(C) 2005 Doug Bowman, Virginia Tech                   23
    HOMER implementation (cont.)
  Each frame:
         Copy hand tracker matrix to v. hand matrix (to
          set orientation)
         Get physical hand position hcurr and distance:
          dh-curr = dist(hcurr, t)
         V. hand distance             do 
                                      dvh  dhcurr   
                                                      dh               hcurr  t
                                                                thcurr 
         Normalize torso-hand vector                                      hcurr  t
         V. hand position vh = t + dvh*(thcurr)
                            
(C) 2005 Doug Bowman, Virginia Tech                                             24
                                                 
    Manipulation metaphors IV

      World-in-miniature (WIM)
            All manipulation in reach
            Doesn’t scale well, indirect
      Scaled-world grab
            Easy, natural manipulation
            User discomfort with use


(C) 2005 Doug Bowman, Virginia Tech         25
    5.4.4 World-in-miniature (WIM)
   “Dollhouse” world held
    in user’s hand
   Miniature objects can
    be manipulated directly
   Moving miniature
    objects affects full-
    scale objects


(C) 2005 Doug Bowman, Virginia Tech   26
    World-in-Miniature (WIM)
       Provides the user with miniature handheld model of VE, which is
        an exact copy of the VE at a small scale.
       Careful use of back face culling techniques. Only the “inside” of
        the walls of the room model should be rendered.
       WIM combine navigation with manipulation.
       Although WIM works well for small and medium-sized
        environments, using WIM in a very large environment would
        require an extreme scale factor, resulting in very small object
        copies in the WIM. This would make accurate selection and
        manipulation extremely difficult.
       It has been successfully used in 3D interfaces for Augmented
        reality, in desktop 3D Uis. In fact, it can be considered a 3D
        generalization of the traditional overview maps that are often
        used in 3D games.


(C) 2005 Doug Bowman, Virginia Tech                                   27
    WIM implementation
                                               Root

                Root                  head    hand    room

    head       hand       room               WIM room     table
                                              (scaled)
                          table
                                             table copy



(C) 2005 Doug Bowman, Virginia Tech                          28
    Scale-world Grab
       Based on HOMER
       Selection : Image-plane selection tech. is used.
       Manipulation :
            Scale down the entire VE around the user’s virtual viewpoint
             switching into manipulation mode
            In HOMER, scaling the user’s hand motion
            The scaling coefficient s = Dv / Do where Dv is the distance
             from the virtual viewpoint to the virtual hand, and Do is the
             distance from the virtual viewpoint to the selected object.
       Well for operations at a distance but not effective
        when the user wants to pick up an object located
        within arm’s reach and move it farther away.

(C) 2005 Doug Bowman, Virginia Tech                                    29
    Voodoo Dolls
       To overcome the scaling approach such as HOMER, scaled-
        World grab when the user needs to move local objects farther
        away.
       Image-plane and WIM with a pair of pinch gloves
       Manipulate virtual objects indirectly using miniature handheld
        copies of objects called “dolls”; Figure 5.13
       Selecting the target object with an image-plane technique, which
        creates the dolls representing the target objects and places them
        in the user’s hand.
       The doll in her non-dominant hand : the corresponding virtual
        object does not move when the user moves this doll.
       To start manipulation, the user simply passes the doll into
        dominant hand.



(C) 2005 Doug Bowman, Virginia Tech                                   30
    Voodoo Dolls
       Advantages:
            Powerful IT allowing to perform some
             sophisticated tasks, such as the manipulation of
             moving, animated objects
            Key idea is very universal and important insight
                  Separating functionality depending on the dominant and
                   non-dominant hands
       Disadvantages
            Increases H/W demand
            Direct application to a desktop might be difficult.


(C) 2005 Doug Bowman, Virginia Tech                                    31
      Technique Classification by metaphor
                         VE manipulation techniques
                            Exocentric metaphor
                                              World-In-Miniature
                                              Scaled-world grab
                              Egocentric metaphor
                                      Virtual Hand metaphor
                                              "Classical" virtual hand
                                              Go-Go
                                              Indirect, stretch Go-Go

                                      Virtual Pointer metaphor
                                              Ray-casting
                                              Aperture
                                              Flashlight
                                              Image plane

(C) 2005 Doug Bowman, Virginia Tech                                      32
      Technique Classification
      by components
                                                           attach to hand
                                                           attach to gaze
                                      Object Attachment    hand moves to object
                                                           object moves to hand
                                                           user/object scaling

                                                           no control
                                                           1-to-N hand to object motion
                                      Object Position      maintain body-hand relation
                                                           other hand mappings
              Manipulation                                 indirect control

                                                           no control
                                                           1-to-N hand to object rotation
                                      Object Orientation   other hand mappings
                                                           indirect control

                                      Feedback             graphical
                                                           force/tactile
                                                           audio
(C) 2005 Doug Bowman, Virginia Tech                                                         33
    Evaluation: positioning task

       Ray casting effective if the object is
        repositioned at constant distance
       Scaling techniques (HOMER, scaled world
        grab) difficult in outward positioning of objects:
        e.g. pick an object located within reach and
        move it far away
       If outward positioning is not needed then
        scaling techniques might be effective

(C) 2005 Doug Bowman, Virginia Tech                    34
    Evaluation: orientation task

      Setting precise orientation can be very
       difficult
      Shape of objects is important
      Orienting at-a-distance harder than
       positioning at-a-distance
      Techniques should be hand-centered

(C) 2005 Doug Bowman, Virginia Tech              35
    Manipulation notes

      No universally best technique
      Constraints and reduced DOFs
      Naturalism not always desirable
      If VE is not based in the real, design it so
       manipulation is optimized


(C) 2005 Doug Bowman, Virginia Tech             36
    Manipulation enhancements

      Constraints
      2-handed manipulation
      Haptic feedback
      Multi-modal manipulation




(C) 2005 Doug Bowman, Virginia Tech   37
    Implementation issues for
    manipulation techniques
      Integration with selection technique
      Disable selection and selection feedback
       while manipulating
      What happens upon release?




(C) 2005 Doug Bowman, Virginia Tech           38
    SIGGRAPH 2001 for Manipulation

       참고할 것.




(C) 2005 Doug Bowman, Virginia Tech   39
    Common manipulation techniques

      Simple virtual hand
      HOMER
      Scaled-world grab
      World-in-miniature




(C) 2005 Doug Bowman, Virginia Tech   40
    Simple virtual hand technique
   Attach object to virtual hand,            Root
    by making object a child of
                                      head   hand     building
    the hand (w/out moving
    object)
   On release, reattach object to            Root
    world (w/out moving object)
                                      head   hand
   Also applies to Go-Go (and
    other arm-extension                              building

    techniques) and ray-casting
(C) 2005 Doug Bowman, Virginia Tech                      41
    HOMER technique
    Hand-Centered                                              Time
    Object
    Manipulation
    Extending
    Ray-Casting
     Select: ray-casting
     Manipulate: hand
            1.0 m                                      2.0 m
       0.3 m                                  0.6 m


 torso physical                       torso     physical
        hand                                     hand

(C) 2005 Doug Bowman, Virginia Tech                            42
    HOMER implementation
    Requires torso position t
    Upon selection, detach virtual hand from tracker,
     move v. hand to object position in world CS, and
     attach object to v. hand (w/out moving object)
    Get physical hand position h and distance
       dh = dist(h, t)
    Get object position o and distance do = dist(o, t)


(C) 2005 Doug Bowman, Virginia Tech                43
    HOMER implementation (cont.)
   Each frame:
         Copy hand tracker matrix to v. hand matrix (to set
          orientation)
         Get physical hand position hcurr and distance:
          dh-curr = dist(hcurr, t)
         V. hand distanced  d       do 
                                     
                                      vh   h curr   d 
                                                      h
         Normalize torso-hand vector th  hcurr  t
         V. hand position vh = t + dvh*(thcurr) hcurr  t
                                           curr




(C) 2005 Doug Bowman, Virginia Tech                            44
    Scaled-world grab technique
Often used w/ occlusion
At selection, scale user
 up (or world down) so that
 v. hand is actually
 touching selected object
User doesn’t notice a
 change in the image until
 he moves
(C) 2005 Doug Bowman, Virginia Tech   45
    Scaled-world grab implementation
   At selection:
        Get world CS distance from eye to hand deh
        Get world CS distance from eye to object deo
        Scale user (entire user subtree) uniformly by deo /
         deh
        Ensure that eye remains in same position
        Attach selected object to v. hand (w/out moving
         object)
   At release:
        Re-attach object to world (w/out moving object)
        Scale user uniformly by deh / deo
        Ensure that eye remains in same position
(C) 2005 Doug Bowman, Virginia Tech                            46
   World-in-miniature (WIM)
   technique
  “Dollhouse” world held
   in user’s hand
  Miniature objects can
   be manipulated directly
  Moving miniature
   objects affects full-
   scale objects
   Can also be used for
       navigation
(C) 2005 Doug Bowman, Virginia Tech   47
    WIM implementation
                                               Root

                Root                  head    hand    room

    head       hand       room               WIM room     table
                                              (scaled)
                          table
                                             table copy



(C) 2005 Doug Bowman, Virginia Tech                          48
    WIM implementation (cont.)

      On selection:
            Determine which full-scale object
             corresponds to the selected miniature
             object
            Attach miniature object to v. hand (w/out
             moving object)
      Each frame:
            Copy local position matrix of miniature
             object to corresponding full-scale object
(C) 2005 Doug Bowman, Virginia Tech                      49
    Implementation issues for
    manipulation techniques
      Integration with selection technique
      Disable selection and selection feedback
       while manipulating
      What happens upon release?




(C) 2005 Doug Bowman, Virginia Tech           50

				
DOCUMENT INFO