Developing Mixed Reality Boundaries
Boriana Koleva, Holger Schnädelbach, Steve Benford and Chris Greenhalgh
School of Computer Science & Information Technology
The University of Nottingham
Nottingham NG8 1BB, UK
+44 115 951 4203
{bnk, hms sdb, cmg}@cs.nott.ac.uk
ABSTRACT environment. The physical and virtual cameras and
Our work in the field of mixed reality has been concerned projections are aligned so that the images appear to be the
with the development of the mixed reality boundary reverse sides of a common boundary.
approach. In contrast to other approaches that focus on physical environment virtual
superimposing the virtual and the physical environment, the
video
spaces on either side of the boundary are adjacent but
audio
remain distinct. In this position statement we describe the
development of mixed reality boundaries, including the projected on to live video
screen or wall texture map
basic idea, the properties that can be associated with them graphics
and demonstrators. We also discuss our latest work on a
particular type of boundary that establishes the illusion that audio
virtual and physical worlds are joined together and that
users can physically cross from one to the other. Figure 1: a simple mixed reality boundary (from [1])
Keywords This approach has been further developed in [2] by
Mixed Reality Boundaries, Virtual Environments, Tele- identifying a set of properties that can be associated with
presence, Augmented Reality mixed reality boundaries. These properties are grouped into
the three categories of permeability (the extent to which a
MIXED REALITY BOUNDARY APPROACH
boundary allows information and objects to pass across it,
Our work in mixed reality has been focused on developing
including its visibility, audibility and solidity), situation (its
mixed reality boundaries. These represent a specific
spatial relationship to the connected spaces, including its
approach to mixed reality that involves creating transparent
location, alignment, mobility and segmentation), and
windows between physical and virtual environments so that
dynamics (the temporal characteristics of the boundary,
occupants of each can communicate with the other [1]. In
including its lifetime and configurability).
contrast to other approaches that focus on superimposing
the two environments on top of one another (e.g., There have been a number of demonstrations of the mixed
augmented reality typically overlays a virtual environment reality boundary approach. In the Internet Foyer a
on top of a physical environment), the spaces on either side boundary was used to link a 3D visualisation of an
of the boundary are adjacent, but remain distinct. Other organisation’s home pages on the Web (a virtual foyer) to
distinguishing features of this approach are that equal its physical foyer, creating a common entry point into its
weight is placed on the physical and virtual spaces and that physical and electronic manifestations [1]. A second
multiple boundaries might be used to join together many demonstration involved a poet performing as an avatar in a
different physical and virtual environments into a larger virtual world communicating with a physical audience [2].
mixed reality structure. TRAVERSABLE BOUNDARIES
Figure 1 shows how a simple mixed reality boundary can be By examining the utility of boundaries with different
created. On the left is a physical environment and on the property configurations, the idea of traversable boundaries
right a virtual environment. An image of the virtual arose. Traversable boundaries establish the illusion that
environment is projected into the physical environment and virtual and physical worlds are joined together and that
an image of the physical environment captured from a video users can physically cross from one to the other [3].
camera is displayed as a live video texture within the virtual Traversable boundaries provide a solution to one of the
weaknesses of current immersive technologies, namely that
users clearly do not leave their current environment when
they enter a remote environment. The fact that they remain
firmly and visibly present within their local physical space
can be problematic for two reasons. First, the participant’s
own illusion of remote presence may be destroyed by
distractions from the local physical space. Second, contains a second video texture showing the view from a
observers of the interaction can clearly see that camera mounted on this proxy.
participating users have not gone anywhere. This is a physical environment virtual environment
particular problem if the interaction is being staged at least
in part for the benefit of these observers or if the observers video
B
physical antechamber
virtual antechamber
may themselves become participants at a later date. C C audio Cv Cv
Further motivation for traversable interfaces is provided by
work on mixed reality. Paul Milgram has classified mixed
reality technologies according to a ‘virtuality continuum’ graphics
A Dp Dp D D
[4]. At the extremes of this continuum we find purely audio
physical and purely virtual environments with augmented
reality and augmented virtuality in between them. Figure 3: creating a traversable boundary
Traversable interfaces provide a mechanism for people to
dynamically relocate themselves along this continuum. At Our design accommodates the perspectives of users who
one moment they may be primarily located in augmented traverse the interface (labeled C and D in the diagram) and
reality, with a view into an adjoining virtual environment. also observers who are present in the connected physical
They may then traverse the interface and find themselves and virtual worlds (A and B), an important consideration
primarily located within an augmented virtuality, with a for performance and entertainment applications.
view back into a physical environment. Traversal allows CONCLUSIONS
people to move back and forward between primarily real This paper has presented an overview of our work on mixed
and primarily virtual environments, repositioning reality boundaries, including the basic concept, the property
themselves along the virtuality continuum, according to set and the development of traversable boundaries. Our
their interest and whether they want the physical or virtual future plans involve developing and evaluating real
to be their primary focus. applications of traversable boundaries and examining how
The task of creating traversable boundaries can be broken the transition between the real and the virtual is affected by
down into two issues, how to allow participants to enter the the design of the two environments with respect to space,
remote environment and how to create the illusion that they materials and objects.
have left their current environment when doing so. The We are also interested in being involved in a discussion
former can be realised with existing technologies. with designers of other approaches to mixed reality, such as
Immersive displays such as HMD’s allow a user in a augmented virtuality, in order to better understand how
physical environment to enter a virtual world. Access to a these approaches fit in our framework and what
remote physical environment from the virtual can be improvements might be needed.
achieved by taking control of a remote physical proxy such
ACKNOWLEDGMENTS
as a robot. We propose that the illusion of leaving the
We thank the ESPRIT IV I3 programme for supporting this
current local environment may be achieved by using non-
work through the eRENA project and the EPSRC for their
solid projection surfaces so that the user can appear to
support through PhD studentship awards.
directly step into and through the image of the remote
environment. This is simple in the virtual space as a non- REFERENCES
solid texture can be used to display the view of the physical, 1. Benford, S., Greenhalgh, C., Reynard, G., Brown, C.,
enabling avatars to pass though it. The development of and Koleva, B., Understanding and Constructing Shared
physical non-solid projection surfaces has been the most Spaces with Mixed Reality Boundaries, ACM ToCHI, 5
challenging part. We have constructed four such surfaces – (3), 1998, ACM Press, pp. 185-223
fabric curtain, water curtain, sliding door and flip-up 2. Koleva, B., Benford, S. and Greenhalgh, C, The
screen. Properties of Mixed Reality Boundaries, Proc.
Figure 2 shows how these various techniques can be ECSCW’99, Copenhagen, September 1999, Kluwer
integrated to create a traversable boundary. On the left is a 3. Koleva, B., Schnädelbach, H., Benford, S. and
physical environment containing a non-solid projection Greenhalgh, C., Traversable Interfaces Between Real
surface onto which is projected a view of the remote virtual and Virtual Worlds, to be published in proc. CHI 2000,
environment. Behind this is an antechamber containing the April 2000
immersive technology required to become embodied within
the virtual environment. On the right is a virtual 4. Milgram, P. and Kishino, F., A Taxonomy of Mixed
environment containing the video view into the physical Reality Visual Displays, IEICE Transactions on
environment. Behind this is a virtual antechamber that Information Systems, Vol E77-D (12), Dec. 1
contains controls for a remote physical proxy and that also