Illusion of reality
It is asserted that experiences in Virtual Reality present a special character in
relation to experiences obtained with traditional media (Witmer & Singer, 1998;
Lombard & Ditton, 1997).
This assertion is associated with the idea that virtual reality systems (in particular
multisensory, interactive interfaces) are stronger means than traditional media
(books, pictures, movies) for producing an illusion of non-mediation or illusion of
Multisensoriality and interactivity are arguments for accounting for the qualitative
shift from the realm of traditional representation to the realm of a full illusion of
reality (Dihn, et al., 1999; Schubert, et al., 2001). The role of immersion as an
enhancing factor for the illusion of reality is more controversial (Stevens, et al.,
But it is not clear what does it mean to undergo an illusion of reality , and in
particular what do users believe whenever they undergo an illusion of reality .
Clarifying this notion has both theoretical and pragmatic consequences.
Difficulties in the definition of standards for the measurement and improvement of
virtual reality systems might in fact depend on the fuzziness of the notions that
are adopted for characterizing successful and unsuccessful experiences in virtual
A paradigmatic example can be found in (Cho, et al., 2003): scenes of an
undersea world are presented to the subjects of an experiment directed to
individuate factors enhancing the sense of Presence as an illusion of reality. Four
questions are addressed to subjects of the experiment, concerning the level of
realism of the objects and of the entire environment, the sense of presence in the
environment, the capacity of self-localization and localization of objects. It seems
difficult to understand what does it mean for a subject which sits at a table or
even stands into an immersive system to feel present in an undersea world
without feeling wet, gasping for lack of air or asking himself how comes that he is
no more in a safe, dry, air-conditioned lab.
The “official concept of Presence” suggests that users can be only partly
unaware of the mediated condition and only partly deceived (International
Society for Presence Research. (2000). The Concept of Presence: Explication
Statement. Retrieved <03/2007> from http://ispr.info/).
What does it mean to be deluded only in part about reality? Two meanings can
be suggested for this question: one can be only partly deluded because delusion
is a matter of degrees, or one can be only partly deluded in the sense that
delusion comes and goes.
If partial delusion is a matter of degrees, what do users experiencing the
undersea world believe? Do they believe themselves experiencing a visual sea
but not a physical sea? Do they believe that the underworld sea can be seen
without being wet? Do they believe that what they experience is what they would
see if they could experience the undersea world in a sub-marine laboratory? Do
they build narratives that justify their beliefs? This hypothesis is pregnant of
consequences for the study of belief, imagination and making believe. Also it is a
suggestive field for studying the capacity of human beings of dealing with
perceptions and beliefs that are in conflict with previous experiences and beliefs
(the experience of being wet when observing the real undersea world, the beliefs
that one must enter a sub-marine ship in order to visit the undersea world) and
relative revision of beliefs or acquisition of new beliefs.
A different idea is suggested by classic approaches to Presence as illusion of
reality (Slater, 2000, 2003; Gerrig, 1993; Kim & Biocca, 1997): in Slater’s
presence counter, for instance, Presence is measured on the basis of the
number of shifts from the sense of being there in the virtual environment and the
sense of being here in the real lab. Delusion is a state that comes and goes and
partial delusion is a full delusion which is followed by an interruption in delusion.
But then, the experience an undersea world should raise panic reactions each
time full delusion is achieved. It seems reasonable to advance the necessity
considering about what is really believed by users, even in the case of
interrupted delusional states, because the simple interruption of delusion does
not justify the absence of reaction that should be present if users would believe
that what they are experiencing is real.
Another problem that emerges from questionnaires for measuring presence as
illusion of reality or illusion of non-mediation is the nature of the sense of reality
and its cognitive bases. Questionnaires ask subjects to compare the sense of
being there in the computer generated environment and the awareness of real
surroundings (Dihn, et al., 1999) and the awareness of the real world can be
evaluated through reactions to items such as: “The virtual world seemed more
realistic than the real world” or “The television generated world was more real or
present for me compared to the real world” (Schubert, et al., 2001; Kim & Biocca,
Two objections can be raised against the notion of illusion of reality as it is
expressed by these questionnaire items.
First, if an illusion of reality implies taking the fictional world (computer generated
or television generated) for real, how can it be compared with the real world?
Wasn’t the fictional world deceptively taken for the real one? Asking to compare
the true is admitting that the fictional world is never really taken for the real one
and that users are always aware of experiencing a representation, whatever
realistic and engaging this experience can be.
Second, which are the characteristics of the experience that make subjects
consider a world more real or more realistic than another one? Questionnaire
items suggest that the fact of being an experience of the real world is not a
sufficient, not a necessary condition for appearing more real than another one.
The presence of items directed to assess the location of attention also suggests
that attention is considered as a palatable mental function for giving substance to
the notion of illusion of reality (Slater, 2000; Schubert, et al., 2001). But t he fact
that attention is directed toward the computer generated environment, or in
general toward the fictional world described in a book, shown in a picture or in a
movie, cannot be takes as an equivalent or as evidence for the fact that users
deceptively take the fictional world for real. Should we then introduce the
existence of a special sensation beyond attention, a sort of sense of reality that is
independent of experiencing the real world? The existence of such a sensation
must be proved, in order to avoid an unjustified multiplication of mental entities
(as the sense of richness that would be experienced each time one sees a rich
person or an expensive object, the sense of beauty, the sense of ugliness).
A further unclear issue is represented by the relationship between illusion of
reality and the successful or unsuccessful, the satisfactory or unsatisfactory
quality of the experience in virtual reality (see Lombard & Ditton, 1997).
Literature on Presence takes the illusion of reality or of non-mediation as a goal
for virtual reality systems but it is not made explicit in what way the fact of being
aware of the presence of a medium and of the fictional nature of the virtual world
would disrupt the experience, limit the performances and the expression of
adequate emotional, cognitive, motor and perceptual behaviors on the side of the
users. In other words, it is not clear why the illusion of reality should be pursued
as an objective in the design of virtual reality experiences.
On the other hand it is not made explicit in what way the fact of being fooled
constitutes a guarantee of the fact that the user will comply with the aims of the
designer of the experience.
It is not trivial that deception will be necessary in order to produce engagement,
alert attention or enhance enjoyment, as shown by the success of existing video
games and also of movies where deception is far from being complete. As a
mater of fact since long no one runs away from movie projection because of a
state of deception about the reality of the projected contents (as underlined by
Stoffregen, et al., 2003).
Conversely, it could be advanced that awareness (of mediation, of experiencing
a fictional and not the real world) plays positive effects at least at two levels: it
enhances the level of believability of the experience and lowers undesired side-
effects of deception, such as gullibility and unethical persuasion, excessive trust
(a risk which does not seem to be taken into account when the positive role of
presence on persuasion is affirmed, as in Kim & Biocca, 1997).
The fact of being aware that they are dealing a fictional world lowers in fact
users’ and audience’s expectations and reduces the risk of expectations’
frustration, which constitute a major threat for believability (Garau, 2003).
Contrary to what could appear, awareness of the vehicle of the experience can
then be exploited in enhancing believability. This is possible if we consider users’
behaviours in response to virtual environments (including appropriate behaviours
that fulfil the aims the experience has been designed for) as depending both on
the contents and on the context of the experience. The context of the experience
(the presence of the medium) should hence not be hided or made transparent.
The users of a medium should be made aware of the characteristics of the
medium, in terms of the possibilities the medium offers to designers (possibilities
of design and manipulation of the contents) and of the possibilities it offers to
users (quantity and quality of sensory stimulation, possibility of interacting, of
modifying the environment, vs. passive interactions).
The awareness of mediation is particularly important when dealing with sensible
classes of users in order to avoid any type of abuse of trust or of confusion
between what is possible and allowed in the real world and what is possible and
allowed in virtual worlds. If we think of the wide diffusion of new powerful media
for producing new experiences and of the impact that they promise to represent
in our lives (commerce, education, training, immersive games, therapy,
rehabilitation) we can see the importance of accompanying this diffusion with a
responsible consideration of the nature of the proposed experiences. An
ethically-driven development of new media technologies for virtual reality should
include the awareness of interacting with virtual worlds and of being capable of
distinguishing between real and virtual objects and entities as structural
characteristics of our interaction with virtual worlds.
The mediated character of the experience should not be considered as a side-
effect that could be overcome by technological development, but a necessary
step (Bardy, 2002; Stoffregen, 2003) linked to the structure of sensory modalities
and by the structure of cognitive processes (Cutting, 1999). Rather than seeking
for the production of illusions of non-mediation and of illusions of reality, I
contend that designer’s efforts should be devoted to the production of
experiences that are believable, convincing, enjoying, effective, without being
Naturalization of the concept of illusion of reality
Illusion of reality. Ethical issues
Illusion of reality. Cognitive issues
Naturalization of the concept of illusion of reality
The term ‘illusion of reality’ is employed for referring to a state experienced by
users of traditional and new media technologies. In spite of the large diffusion of
the term, the phenomenological contents and the psychological nature of the
illusion of reality are not univocally described and at least four different uses of
the notion of ‘illusion of reality’ can be individuated: a. illusion of reality as
believability of a fictional content in terms of the dramatic effects produced,
illusion of reality as deception, in the form of b. hoax or fraud, whenever users or
spectators are cheated about the nature of the experience by taking a fictional
content for a real one, in the form of an c. illusion of non-mediation, whenever a
mediated experience is taken for a non-mediated one, and in the form of d.
illusion of transportation or being there, whenever the cognitive awareness of
experiencing a fictional world is associated with special proprioceptive
experiences of self-motion and localization in representations.
a. Illusion of reality as believability
Illusion of reality as believability in animated movies, cartoons and comics
Illusion of reality as believability in interactive media and VR
The first use of the notion of illusion of reality is equated to believability and
makes reference to the effects of employing perceptually and psychologically
realistic representations of objects, events and characters in mediated contexts,
in the sense of sharing structural similarities with objects, events, and, people of
the real world. The similarities are compatible with the representation of objects,
events and characters that do not exist in the real world, hence they are to not be
confounded with photorealism, but are conceived as key factors for producing
believable experiences (especially psychological realism and the production of
rich personalities are considered to positively affect believability [Thomas &
Johnston, 1984; Reilly, 1992]). In spite of the fact that creators of such
representations can identify their objective with the production of an illusion of
reality [Thomas & Johnston, 1984], the use of realistic representations is aimed
at producing esthetic and dramatic effects and not at cheating spectators . The
psychological complexity and the emotional richness introduced by Disney
animators in the world of cartoons, for instance, has the effect of inducing Disney
spectators to sympathize with characters, resent a wide range of emotions in
correspondence with the emotions that are resented by the characters on the
screen, but, no matter how gifted, Disney animators cannot make spectators
believe that Disney characters are alive or real. The problem of deluding movies’
spectators and fiction’s audiences in general has been largely debated in relation
to the so-called “paradox of fiction” [Radford, 1975]: how can we be moved by
characters if we don’t believe in their existence? The debate shows a substantial
accord on the fact that audiences and readers do not hold existence beliefs
about fictional contents, as testified by the fact that responses to fictional
contents are similar but never identical to responses to corresponding non-
fictional contents. The difference in reactions testifies that audiences and readers
are aware of the fictional nature of the experience, hence that they do not have
existence beliefs when they experience fictional contents [Radford, 1975;
Stoffregen, 1997; Walton, 1978, 2001; Currie, 1990].
It is also largely accepted by exponents of different representational domains -
such as animation cinema, virtual reality and comics - that the production of
dramatic and emotional effects does not properly depend upon realism as
external similarity between fiction and reality but rather upon some familiarity with
certain contents and aspects of the representation [Thomas & Johnston, 1984;
Reilly, 1992; McCloud, 1993]. As stated by [Bordwell, 1985], in fact, narration is a
matter of active construction, both perceptual and cognitive, on the side of the
spectator. For this activity, spectators exploit their perceptual innate structures,
their experience with the real world and also their experience with other movies
(cinema basic techniques, genres, physics and other general laws - such as
cartoon physics). The cues provided by a movie can hence make spectators
more or less immediately familiar with the fictional environment, in case they fulfill
familiar knowledge and expectations or not. Familiarity hence constitutes the
natural basis of realism and can be used for explaining the preference for familiar
representations on the side of public and the production of believable rather than
non-believable representations. It can be hypothesized that familiarity lowers the
effort that the spectator must do for constructing the narration, in analogy with the
use of stylized, exaggerated and simplified facial expressions (techniques in use
in animated movies, cartoons and comics [Thomas & Johnston, 1984; Reilly,
1992; McCloud, 1993]) that help the characters’ emotions identification thus
reducing the spectator’s cognitive effort. It can also be hypothesized that
believability depends upon the fulfillment of spectators’ expectations acquired in
the experience with the real world, but also in the experience with fictional ones.
The fulfillment of cartoon physics laws can hence fulfill the expectations of
spectators that are acquainted with Warner’s cartoons and give rise to believable
Illusion of reality as believability in animated movies,
cartoons and comics
Disney: simplification and exaggeration
Warner: cartoon physics and characters’ oddity
Comics: cartoon style and closure
Disney: simplification and exaggeration
Thomas and Johnson (1981) is one of the most referred texts for believability, at least in the
domain of animation and especially of characters animation. In addition to provide a
characterization of the notion of believability which is widely adopted even in the domain of VR,
for instance in the domain of interactive drama, the book describes the techniques that are
responsible for the production of believable characters and the general approach to the problem
of believability that lays behind these techniques and can be extended to other domains (the
Since the very Preface, believability is characterized as follows:
“Disney animation makes audiences really believe in those characters, whose adventures and
misfortunes make people laugh – and even cry. There is a special ingredient in our type of
animation that produce drawings that appear to think and make decisions and act of their own
volition; it is what creates the illusion of life.” (p. 9)
Thomas and Johnson go back to the early story of Disney animation and illustrate the successive
steps that have conduced Disney animators to detach themselves from the 20s trend of short
gags played by characters with no personality, individuality and in general mental states
(emotions, desires, beliefs, motivations, intentions) to the realization of a new idea: stories that
are not bound to gags and easy laugh but that are able to attract the interest of the audience for a
long time and even to move audiences as Hollywood films do and characters that appear to think,
to have emotions, to act in reason of their own mental states and not simply in reaction to some
external event. These two ingredients require the development of new techniques in order to be
realized and give rise to a new pattern of believability. According to the authors, characters that
appear to think, have emotions and act of their own and only characters with these characteristics
can be considered as believable because these characteristics confer drawings with an illusion of
The first ingredient, the story, is much more directed to produce involvement of the audience
rather than believability. Disney’s approach to involvement consists in displaying stories that
contain elements that are well recognizable and familiar to the audience. This is a form of realism
as simulation, in the sense that certain aspects of the real world, people and events are imitated
in order to create a sense of empathy and to raise the same emotions that they raise in real life.
Realism is then considered as a crucial ingredient for empathy and empathy a crucial condition
“We involve the audiences in our films the same way [as in our own lives]. We start with
something they know and like. This can be either an idea or character, as long as it is familiar and
appealing. It can be a situation everyone has experienced, an emotional reaction universally
shared, a facet of someone’s personality easily recognized, or any combination of these. But
there must be something that is known and understood if the film is to achieve audience
involvement.” (p. 19)
The second ingredient concerns the realism of the characters. It seems that since the ‘20s Disney
asked for more character realism in two senses: physical aspect of characters and other entities
and mental states. The physical aspect for instance concerns entities weight, characters anatomy
(the presence of bones and muscles). In animation special techniques for suggesting weight and
the presence of bones and muscles must be invented and mastered.
“Everyone knew that it was necessary to get a feeling of weight in the characters and their props
if ever they were to be convincing, but just drawing a figure large has nothing to do with how
heavy he is. A weather balloon is quite large. The animators sensed that the key to the illusion of
weight lies in the timing and how far a character moved and how fluid the action was, but it was
not until they were able to study live action films that the solution finally was found.”
The solution is provided in this case by the imitation of real action and real anatomy.
But Disney’s realism is not a full simulation, a form of hyperrealism or photorealism of the kind
displayed by other animation films such as Final Fantasy. This is not only because of
technological limitation, but lays on an idea of realism and believability as added to the idea of
caricature, simplification and exaggeration. A character is realistic because it has emotions,
personality, because his actions respond to inner volitions and motivations and not only to
external situations. These are characteristics of real human beings, but in animation they are
treated following special techniques that give rise to caricature effects that are indicated to be as
important as realism as simulation for the final effect of entertainment. This is the reason why
Thomas and Johnson speak of caricature realism.
“When Walt asked for realism, he wanted a caricature of realism. One artist analyzed it correctly
when he said, “I don’t think he meant “realism”. I think he meant something that was more
convincing, that made a bigger contact with people, and he just said “realism” because “real”
things do.” (p. 65-66)
“If an animator’s drawings finally reflected a more natural way of moving, Walt would be likely to
say, “[…] We oughta be looking for entertaining ways of doing things. We don’t want to get
straight, y’know – we are not copying nature!” “Caricature” and “exaggeration” were two favorite
words to stimulate the animator’s approach to his scene.” (p. 37)
Two principles of animation are both related to realism and caricature and illustrate how
caricature with exaggeration and simplification can serve to evoke actions and emotions of real
people without fully copying real people:
“When a fixed shape is moved about on the paper from one drawing to the next, there is a
marked rigidity that is emphasized by the movement. In real life, this occurs only with the most
rigid shapes, such as chairs and dishes and pans. Anything composed of living flesh, no matter
how bony, will show considerable movement within its shape in progressing through an action.
[…] The squashed position can depict the form either flattened out by great pressure or bunched
up and pushed together. The stretched position always shows the same form in a very extended
condition. […] Immediately the animators tried to outdo each other in making drawings with more
and more squash and stretch, pushing those principles to the very limits of solid draftsmanship:
eyes squinted shut and eyes popped open; the sunken cheeks of an “inhale” were radically
different from the ballooned cheeks of a blowing action; a mouth chewing on a straw was first
shown far below the nose, and then it actually was compressed up beyond the nose (which
changed shape as well) in showing the chewing action.” (p. 48)
In other words Thomas and Johnson suggest the following schema: imitation of some aspects of
real life situations for creating empathy and hence involvement, imitation of some aspects of real
people mental life, anatomy and physiology and interaction with things in order to create
believable characters, but also objects and events in relationship with characters, but caricature
and exaggeration that limit realism in order to produce entertainment. The association of realism
for certain aspects and of stretched realism (caricature) for other aspects produces caricature
Warner: cartoon physics and characters’ oddity
Chuck Jones, 1989 from Warner Bros. (the father of Bugs Bunny, between other characters)
states that it is the oddity, the quirk that gives personality to a character and that it is personality
that gives life. So it seems that one should allow regularities to be broken in order to create
characters that are more believable because they have a richer personality; And also a specific
personality; in fact, tics and other oddities are specific of an individual and hence help suggesting
that the character possesses a personality of its own.
The same principle of oddity and non-realism governs the events of Warner’s worlds.
The laws that govern cartoon worlds, and typically Warner cartoons’ worlds, are not the same
than real world laws. They have hence been identified as cartoon physics, and described by
several Warner cartoons’ viewers and film critics.
Events ruled by cartoon physics might appear surprising to first-viewers and non-expert cartoon
viewers (such as children). They hence require some little experience and some consideration
about the fictive and cartoon nature of the depicted worlds in order to become believable.
Each cartoon universe has its own specific laws, anime laws being different from Warner-like
cartoons [O’Donnell, 1994; Gould, 1993, Cholodenko, 2006].
Comics: cartoon style and closure
Norman McCloud (McCloud, 1993) develops the problem of realism is comics. Comics constitute
a suggestive domain for realism because they are structurally related to caricature.
McCloud describes a sort of decreasing line of realism in images, from photos to cartoons: in
comics we mainly find simplified, abstract images that are further and further from the “real” face
of the photo, which is realistic and detailed; the limit is the cartoon.
In spite of the use of the two mechanisms of simplification and abstraction, comics are very
acceptable to our eyes, and they seem just as real as photos; in fact, we respond to a cartoon as
much or even more than to a realistic face. Why?
According to McCloud, this is because cartooning is a form of amplification through simplification:
- Focus: much more than eliminating details, abstraction through
cartooning is a way of focusing on specific details, of stripping down an image to its essential
The fact of simplifying images and characters toward a purpose
is not applied to cartoons only, but it constitutes a tool for any storytelling medium. Cartooning is
not a way of drawing, is a way of seeing. This is exemplified by the movie “Wizard of Oz”.
- Universality: the more cartoony a face is, the more general it is. This
fact enhances self-identification. Self-identification is important because the identification with the
character of a story is a sure indicator of the fact that audience is involved. This also explains why
backgrounds tend to be more realistic than characters (in addition to a pragmatic reason
connected to the production of a lot of drawings in animation).
Faces have a special status in perception: human beings see
faces even in abstract drawings once few conditions are fulfilled.
- Concepts: Cartoon style de-emphasize the perceptual appearance of
things in favor of their form or of some concept: it allows to express concepts.
McCloud hence draws the following lines in order to describe the transition from realism to
- photo cartoon
- complex simple
- realistic iconic
- objective subjective
- specific universal
Another interesting mechanism described which is employed in comics but also in cinema is
closure (in cinema see the use of ellipse): recognize the whole when perceiving the parts.
Independently from the cognitive mechanisms implicated, McCloud hints to the role of
collaboration between the drawer and the reader and to the reader’s imagination, hence to the
reader’s previous experiences. For this reason, the artist needs to know what are the
assumptions of the reader in order to foresee what will work and what will not produce closure.
While in electronic media closure is often involuntary and unperceived, comics are particularly
exemplary of closure because they structurally present a fracture of time and space between one
image and the following (gutter): only closure allows the reader to connect the images. In a
sense, comics is closure.
Illusion of reality as believability in VR and interactive media
The Oz project
The Oz project
The Oz project is a project developed at Carnegie Mellon University during the ‘90s in order to
individuate the characteristics that are responsible for the production of believable characters in
dramas and in particular with the aim of producing believable interactive dramas (dramas in
Two elements are present in Oz research: story and characters, hence interactive story and
interactive characters, with the objective of creating a new form of art and entertainment. These
characteristics give rise to Oz worlds that include: a simulated physical environment, automated
agents, a user interface and a planner.
Narration has the function of making the events, actions and reactions of characters
comprehensible. Agents are more comprehensible if their behavior is structured into narratives.
Believable character is a notion from the Arts, and it means
“a character that produces the illusion of life and that permits the suspension of disbelief.” And “ it
is our view that believability will not arise from copying reality.” (Bates, 1994, p.1) Bates consider
that this is a dream which is common to theater, cinema, animation, IA and even robotics (see
Bledsoc, 1986). The domain of animation is of particular interest because of the practical
requirement of producing hundreds of thousands of drawings which force animators to use
extremely simple, non-realistic images and to seek and abstract precisely what is crucial.
Animation and storytelling media in general are taken as a model.
“Also, we are not necessarily interested in lifelike or realistic agents. We don’t really care if our
characters only have 4 fingers or are talking animals. The arts have always abstracted away from
reality and created characters that were “larger than life” and that were much more interesting
than any mere “real” people or animals we migjt know. By believable agents we mean interactive
versions of these abstract characters that have been so successful in non-interactive media.”
(Reilly, Bates, 1995, p. 1)
“The qualities of traditional characters that produce suspension of disbelief in viewers are
understood informally by artists…” (Reilly, Bates, 1995, p. 1)
Contrarily to traditional characters, interactive agents will have to interact between them and with
“Emphasizing believability leads us in new directions. The arts suggest that the most crucial
elements to imbue agents with strong personalities that permeate their actions…in addition to
personality there are several other aspects of characters important for believable social behavior ”
such as show to have emotional reactions, have dynamic social relationships that affect the
agents behavior, appear competent at working toward multiple goals simultaneously and have
robust behavior. (Reilly, Bates, 1995, p. 1)
Reilly, 1996 asserts that artists know how to create believable characters although they do not
have provably correct methods for doing so. They use the term believability as a synonym of a
character that works, that seems to be alive and that the audience has emotions for or about; not
necessarily intelligent or rational as AI, not necessarily realistic, but with a strong personality. It
turns out that sometimes being more realistic can decrease believability. In fact, the state of the
art in computer animation can make a mostly realistic human face, but not a completely realistic
one. This close but not quiet face is very disturbing to watch at because people are so well
adapted to watching human faces. From the standpoint of believability it is better to go with the
less realistic characters which meet the audience’s expectations than to go with the more realistic
ones that don’t. Additionally, powerful artistic techniques such as simplification and exaggeration
rely on altering reality for more effective characters in the attempt to communicate the essential
Basically, following the Oz view, a believable character is a character which looks alive
(consistently with Johnson and Thomas, 1982), whose actions make sense, who allows the
audience to suspend disbelief. Believability is conceived as equivalent to the illusion of life or
illusion of reality and to suspension of disbelief.
“One of the keys to an effective virtual world is for the user to suspend disbelief. That is, the user
must be able to imagine that the world portrayed is real, without being jarred out of this belief by
the world’s behavior” (Bates, Loyall, Reilly, 1991b, p. 1)
This is not the same thing as realism. For example, Bugs Bunny is a believable character but not
a realistic one. (Loyall, Bates, 1997) underlines that believability is not coincident with realism
because the best path to believability almost always involves careful, artistically inspired
abstraction, retaining only those aspects of the agent that are essential to express its personality
and its role in the world of which is part. Nevertheless,
“While full realism is rarely appropriate, we have found, as others before us, that fine details can
have a great influence on whether a creature seems alive. The use of the eyes, the timing of
pauses in speech, an awareness of body position and personal space, are each examples of
these important details.” (Loyall, Bates, 1997, p.1)
Characters that look alive are rich characters. Above all, rich in personality. In a sense, realism is
respected at this level. The requirements indicated for believable characters are the following:
- social relationships
- illusion of life in aspect and behavior
They are what we expect on the basis of what we know about people.
The first aim is to look for broad characters, even if shallow. Broad agents have goals and goal-
directed reactive behaviors, they present some language abilities and some memory and
inferential capacities. Bates, Loyall, Reilly, 1992 describe broad even if shallow characters as
presenting goal directed behavior, emotions, language, perception, reactivity, social
behavior.Goals and goal-directed reactive behaviors seem to be very relevant for providing an
illusion of life. They hence constitute a third type of agent in comparison to AI entities and to
embodied artifacts. Loyall, Bates 1997 describe the analogies and differences between the three
types of agents: broad agents implement ideas of parallelism, situated activity and reactivity with
Bates, 1994 insists on the role of emotions for believable characters intended as engaging,
apparently living creatures. If the character does not react emotionally to events, if it doesn’t care,
neither do users and audience. Additionally, an emotionless entity is perceived as a machine.
Emotions must be appropriately timed and clearly expressed. Realism just concerns the fact of
having emotions, not the reproduction of all the ways emotions are expressed: the clear
expression of emotions require exaggeration and suitable timing because normal timing is too
Reilly, 1995 is dedicated to how to create believable characters, especially at the level of
emotional behavior. He suggests that a first trick is to lower the expectations of the users by the
creation of characters that are different from normal adult humans, such as kids and animals or
even better aliens. In fact in this case, the user has fewer expectations.
Bates, 1994 also suggests that an element that might be useful for attracting audience and users
is the presence of oddities, quirk. In fact, it seem that a bug in the program “Edge of intentions”
had created a repetitive movement for one of the characters which looked like a tic; as a result,
the character did appear more interesting and real.
The Oz project has inspired other developments.
Riedl’s (Riedl, 2005) contribution consists in a system for evaluating the perception character’s
believability in terms of the perception of characters’ intentionality in stories created by automated
story generators. The more characters’ intentionality can be understood, the more the characters
and hence the story are believable. In fat, intentionality is onsidered as a fundamental factor for
As for what concerns the characterization of believability, Riedl (2005) re-proposes the concepts
introduced by Thomas and Johnson and re-edited in interactive drama by the members of the Oz
project in the framework of the problem of automated story generation and storytelling. He
indicates the success of storytelling in the capacity of having emotional and educational impact
on the audience and indicated understandability and believability as the two conditions for
success. Characters’ believability is successively suggested as a story’s component that can
influence both understandability and believability of the story. Following the characterization of
believability as audience’s willing to suspend disbelief and character’s believability as coincident
with the illusion of life, Riedl (2005) focuses on intentionality between the other factors considered
relevant for creating the illusion of life: personality, emotion, physiology and physiological
movement. Character’s intentionality refers to the existence of a relationship between the actions
and behaviors of the characters on one side and the beliefs, desires, intentions and motivations
on the other, that makes the actions and behaviors of the character appear natural and rational.
Two elements are hence present in character’s intentionality: on one side the coherence between
the actions of the character and the coherence between the character’s actions and the situations
presented by the story; on the other side, the fact that the characters are perceived to act upon
their intentions, beliefs, desires and motivations. The first condition is important for character’s
believability because it limits the risk for the character to appear schizophrenic, but the second
condition is very relevant too for creating the illusion of life. The procedure for testing storytelling
generators is the following: The process relies on the fact that a reader/viewer’s perception of
character intentionality can be compared to a QUEST representation of the story because
QUEST is a validated model of human question-answering in the context of stories. the
procedure for evaluating story generation systems is to compare an instance of a QUEST model
of a specific generated story to subject comprehension of narrative structure for that story. In
QUEST, “why” questions inquire about character goals, intentions, and motivations. The general
idea behind the process described here is that a better story generation system (presumably the
test condition) will result in stories whose structures better support human perception of character
intentionality. The better the structure of the generated story, the better a QUEST representation
of that story will predict reader/viewer question-answering.
Three considerations can be advanced:
1) The relationship with dramatic arts creates a sort of connection between believability
and success or positive appreciation of the opera: if an agent or character is believable then the
character will be a success, and viceversa: if the character has success it must be that it is
believable (see Mateas, 1995). But this connection might simply be external and artificial: can a
measure of success constitute a measure of believability? A character can be unsuccessful for
other reasons than the fact that it is not believable? A non believable character can be
Also, when exaggeration in the emotions and timing is presented as a useful ingredient
for identifying the character, recognizing it, this is not necessarily related to believability unless it
is not affirmed that it serves to give rise to a specific personality and that specific personality is a
necessary ingredient for believability.
2) The minimal conditions for a believable character are not investigated in a systematic
way. Only intuitive characteristics are enlisted, for none of which is defined its necessary or
sufficient role or the reason why it should be more necessary than others or if there are other
conditions. How can we know that the enlisted conditions are the only relevant ones? That they
are all relevant?
The principle which is indicated for choosing them and not others is that they are
characteristic of people, that we know that people have these characteristics and that for
consequence we expect to find these characteristics in artificial or virtual characters and agents.
The respect of expectations about human beings constitutes a general principle. But after that the
enlisted expectations to be respected are just chosen on the basis of intuition, and not guided by
another principle. This is why one cannot be sure that each of them is necessary and that the all
of them is sufficient for producing believable characters.
An extreme example: Tamagotchi. Tamagotchi are digital pets, typically included in a
handheld egg-shaped computer, created by A. Maita in 1996 for a commercial company.
Tamagotchi can be fed with unhealthy or healthy food, can be rendered more happy by “playing
with them”, their environment can be cleaned up from excrements and their status (happiness,
hunger, discipline, weight, age) can be checked. They evolve dependently on how well the user
takes care of them. Users care about their digital pet, they spend their time with it and do the right
actions for keeping it “in life” or getting better. Are Tamagotchis believable characters? They are
not realistic in their physical aspect, they are far from being human or animal-like; they are neither
rich in personality, emotions, social relationship, etc. A causal relation seems to subsist between
their state and the actions of the user, but no free actions; can the fact of being in need of food,
attentions, cleaning be considered a sort of will, desire? The problem of the intentional stance
arises and, if we think to the enormous success Tamagotchis have see in the ‘90s, it seems that
human beings are very liberal in attributing mental properties to very simple entities. It is true that
new further releases of new versions of Tamagotchi have included more and more features such
as forms of social relationship and actions of the Tamagotchi toward the user (gifts). Anyway
Tamagotchi represent an ideal case study for the minimal conditions of believability.
3) The position about suspension of disbelief and illusion of reality raises some
objections: how can the audience take a fictional or virtual world as real? Spectator know all the
time that they are assisting a projection and users cannot ignore they are practicing on a VR
environment. They know the difference between a movie and VR because they do not try to
interact with the characters of a movie, while they know they can do it with virtual agents. There is
hence a cognitive awareness of the fact of interacting with a real or a virtual world and even of the
type of participation which is permitted or forbidden. Additionally, emotional responses to real and
fictional worlds can be similar but they are never identical, or spectators would run away from the
projection of horror movies and they would be unable to enjoy murders on the screen. Finally, we
never imagine the real world to be the real world, we make no effort or take any particular attitude
in order to consider it as real. Nor we have a “sense of reality”; we can sense that something
does not look or feel as real things do. It is the same as for coherence: it is only when coherence
is violated that we notice coherence, not when it is respected. But what is the sense of reality,
sense of coherence, etc.? Do we need to have a sense of for everything?
The suspension of disbelief can concern certain beliefs (as the existence of monsters)
and allow the audience and users to enjoy representations that do not fulfill their expectations
issued form the experience with the real world. But suspension of disbelief does not necessarily
concern the belief that the world experienced is real rather than fictional or virtual.
The notion of suspension of disbelief is nevertheless an intriguing one since it suggests
that the user or audience put something in pause when assisting the spectacle or interacting with
the virtual world and that the user and audience assume a certain attitude toward their
experience. Hence we should ask what do we wan to obtain from the audience or user: that they
sit at their place, participate with attention, emotion and eventually action and relevant action?
At Mira-Lab the problem of believability is especially addressed in the frame-work of Mixed or
Augmented Reality (MR and AR), for instance in the frame-work of LifePlus project
(Papagiannakis, 2002, 2004, 2005, 2005; Magnenat-Thalmann, 2004, 2006). It is asserted that
MR creates a cyber-real space that presents characteristics of believability and presence which
are different from experiences in purely virtual spaces and that believability and presence are
both necessary in order to make experiences in the virtual environment to be real. As we have
seen, conveying believable experiences in fact is considered the same than conveying
compelling real experiences. But unlike VR experiences, experiences in MR are not imitations of
a parallel real world, since the real world is part of the augmented experience. Virtual elements of
the experience with LifePlus are nevertheless imitations of real entities. The application in fact
includes 5 realistic virtual humans that interact between them and manipulate objects following a
scenario; the virtual humans evolve in the real context of a tavern in Pompeii ruins or in a lab
maquette of the same tavern. The realistic aspect of virtual humans include clothes, facial
expressions intended to be accorded with the personality of each single character and linguistic
exchanges. The user witnesses the representation and has the possibility of modifying his
position and orientation.
Papagiannakis (2002) describes the following objectives: realism, immersion, interaction in order
to give rise to an experience of transportation (presence?) in fictional and historical spaces.
Papagiannakis (2002) also indicates some aspects that must be included in the virtual simulation
of life (human life) in order to give rise to realistic (hyperrealistic) entities: hair simulation, cloth
animation, skin rendering, Artificial life methods for behavioral animation of virtual characters,
realistic facial emotion expression, intentionality, desires, beliefs, actions that can be read by the
Papagiannakis (2005) asserts that the introduction of realistic human-like virtual characters and
narration (virtual real-time story-telling) in a MR environment increases the sense of presence but
not necessarily believability and that believability depends more on the interaction that the users
are in the condition to establish with the virtual characters. Two directions for enhancing
believability are indicated: more social awareness on the side of virtual characters and new
channels of interactivity between the users and the characters. These assertions are not based
on theoretical assumptions or experimental evidence.
Kim (2004) and Magnenat-Thalmann (2005) define believability as referred to what is considered
as the major goal of VR systems: make the users feel that the generated experience is from the
real world (in this sense a believable experience is also defined a real experience). This
characterization of believability is a sub-class of the notion of illusion of reality. Virtual entities are
believable when one can be deluded that they are creatures of the real world. It raises the
following objection: can a user believe that a virtual monster comes from the real world, when he
perfectly knows monster do not exist in the real world? This characterization seems to be bound
to cases in which believability concerns virtual worlds that are copies of the real world, in which
the objective is to reproduce parts of the real world, hence to virtual worlds that do not contain
any new element or experience.
Factors enhancing believability are discussed in relationship to the level of immersion, to the type
of presentation and to the degree of interactivity.
Immersion is indicated as a pre-condition for believability: the user can better take an experience
for real if he is totally immersed into the virtual environment.
A realistic type of presentation (as real as the real world) is indicated as to increase believability.
Two types of realism are distinguished: sensory and perceptual realism (and believability).
Realism in the sensory channel concerns the external aspects of the character’s behavior and
perceptual realism consists in the presence of emotions, personality, intents. In fact, it is asserted
that realism in the sensory channel is not a sufficient condition for believability: in addition to
provide realism for the sensory channels, the contents of the depicted world must be believable.
Realism in the sensory channel seems also to be non-necessary for believability, as exemplified
by believable stories in text books with no images. Suspension of disbelief and hence of
believability in this case depends on the contents (semantics) of the virtual world, and in
particular: on the consistency between the contents of the virtual world and the expectations of
the user and, when virtual characters are at stake, on the presence of emotional, personalized
and goal-oriented elements.
Furthermore, a realistic presentation can concern each sensory channel, and also multisensory
perception; when multisensory perception is at stake two issues must be considered: the
consistency between sensory modalities and the possibility that the activity of one sensory
modality substitutes information for other sensory modalities. VR research has traditionally
searched for realism in the sense of the reproduction of the physical aspect of the real world, but
recently it has been objected that attention should be paid to the characteristics of perception too;
this is particularly evident in the case of motion perception.
Interaction is considered as a factor that enhances believability in normal cases. The timing of
interaction must respect normal conditions of response to action. Nevertheless, even in this
domain, it is not enough to achieve sensory realism in the interaction; also the contents of the
experience (story) must be believable.
When virtual characters and agents are concerned, personality and emotions must be defined
and coordinated in order to achieve a believable result. Egges (2002, 2003, 2004) address the
role of emotions, mood and personality in the construction of believable agents and characters.
Psychological studies are taken as references for types of emotions and types of personality.
Additionally, movements, even not directly connected to action must be attentively reproduced.
Egges (2004, 2005) argues that realistic characters should display idle motion even when no
action is planned (posture change, little movements related to breathe activity or eye blink) and
that motion should affect several joints. The justification lays in the fact that in nature there exists
no motionless character and that in natural conditions gestures affect several muscles and
several joints. It is also suggested that idle motions can be differenced into types and associated
with different personalities.
In summary, believability is indicated as to highly depend on the consistency between the
contents of the virtual world and the user’s expectations, both at the level of the story and of the
aspect of the environment and characters; on the presence of characters with emotions,
personality and intentions (in accord with the user’s expectations about human-like characters);
on realistic ways of presentation (again in accord with the user’s expectations about the aspect
and behavior of human beings); on realistic interaction (for what concerns timing but also the
type of reactions of the virtual agent following the contents of the experience); on physical
immersion. All these conditions should make the experience with the virtual world so similar to
that with the real world as to be taken as a real experience. A strong accent is posed upon
realism as simulation as a necessary condition for believability, even if objections and limitations
of this assertion are presented here and there: the fact that stories in books are believable without
displaying photorealism, the fact that even for producing realistic simulations of the real world the
structure of human perception could count as much or even more than the physical aspect of the
simulated parts of the world.
This characterization of believability nevertheless seems to concern only a special condition of
believability: the believability of virtual worlds that are identical to the real world, but what about
virtual worlds that contain elements that are not part of the real world?
b. c. d. Illusion of reality as deception
Designers of virtual reality experiences have three declared objectives: producing
the illusion that what is experienced is real, producing an illusion of non-
mediation and producing an illusion of being in the virtual environment. The three
objectives are referred to as ‘illusion of reality’ or ‘illusion of presence’ (or shortly,
as ‘presence’) and have in common the will of deceiving the user as for what
concerns the origin of his experience [International Society for Presence
Research - ISPR at http://www.temple.edu/ispr/].
In spite of their mixing in the notion of presence, these three aspects of deception
present different characteristics and can subsist separated: a mediated fictional
content can be taken for a real content, without any mistake concerning the
mediated context of the experience; a mediated experience can be taken for a
non-mediated one and still be recognized as having a fictional content; and
perceptual experiences of self-motion and of location in representations are
consistent with the cognitive awareness both of fiction and of mediation.
b. Deception as hoax or fraud
Illusion of reality as hoax
The first form of deception, the misrepresentation of a fictional content for a real
one, occurs whenever people are made believe that the fictional content they
experience through a medium (books, TV, movies, internet, virtual reality) is the
record of some real event (like a documentary). This form of deception gives rise
to different cases of hoax (usually humoristic or critic tricks were fiction events
are told for real ones) or even fraud (depending on whether the responsible of
the hoax has a benefit from his trickery).
Hoax is made possible by the fact that mediated contents can be both recorded
real events (such as it is the case for journalistic photographs, radio and
television documentaries) and fictions (such as it is the case for fiction movies,
fiction radio dramas or fiction virtual reality) and by the fact that both fictional and
real contents can be proposed by one and the same medium, such as
photographs, radio, TV or even newspapers. Usually, when fiction is inserted in a
non-fiction communication context, special advices and perceptual cues frame
each type of content. Journalists of RTBF broadcast announcing Flemish
secession provoked panic in Belgian population. However, at the beginning of
the broadcast a message was shown telling: “This is perhaps not a fiction” and
“This is a fiction”, was shown an hour after that [Bilefsky, 2006]. Audiences were
also deceived by emission “Alternative 3” when a group of UK Anglia television
journalists usually commenting documentary real events commented a totally
fictional event: the project of abandoning dying planet earth. No warning or
perceptual indication was present that could have made audiences distinguish
“Alternative 3” from previously released documentaries, thus producing a case of
Illusion of reality as hoax
Not only fraud, but also hoax perpetrated through communication media such as radio and
television is subject to rules and prevention because it is considered as unethical.
This is one of the domains of media-ethics (with moral dilemmas, conflict of interest, plagiarism,
sources, censorship, credibility) and is related to trustful sources and manipulation of evidence
(such as photographs) [Steele & Black, 1999].
The Radio-Television News directors Association states in its code of ethics that “Professional
electronic journalists should operate as trustees of the public, seek the truth, report it fairly and
with integrity and independence, and stand accountable for their actions.” [RTNDA, 2000]. In the
paragraph concerning truth it is stated that professional electronic journalists should continuously
seek truth and should not report anything known to be false. The dissemination of accurate
information is also the principal aim of the code of ethics of the National Association of Science
writers [NASW, 2006].
The National Press Photographers Association code of ethics also points to the duty for
documentary photographers of truthfully informing public. This includes not “to alter the content of
a photograph in any way (electronically or in the darkroom) that deceives the public.” [ASNE,
Beyond journalism, ethics also concerns fiction. Fiction writing for instance is subject to rules of
no-harming and rules of no-hoaxing [Hansen, 2007]. The main problem is represented by not
selling for truth what is fiction.
A world-famous attempt to produce an illusion of reality by the mean of a traditional medium is
constituted by the broadcast Welles’ War of the Worlds, played by Orson Welles and diffused in
the framework of the series Mercury Theater on the Air. At some extend the attempt worked, and
a certain number of American people resulted convinced of being under Martian attack and
reacted with panic. The broadcast became an enormous success for CBS who delivered it and
for Orson Welles. In 1938 American listeners were not used to broadcasts such as “The war of
the worlds” where a fiction was delivered in the form of newsflashes interrupting a music program
for relating the dramatic news of Martian invasion. Newsflashes were hence trusted.
Nevertheless, CBS escaped punishment because the fictional nature of the performance was
reminded through all the broadcast (even if with large holes, for instance, between minute 12 and
minute 40 of the broadcast). In the mean time, CBS was formerly invited to avoid the “we interrupt
this program" device in fictional contexts.
Since then, US TV broadcasts featuring realistic news bulletins post messages to inform
audiences of the fictional nature of the spectacle.
Many other examples of hoax and gullibility are reported:
- The Moon hoax: perpetrated in 1835 by The New York Sun, claiming
that men had been seen on the Moon through telescopes
- the autopsy of the Roswell alien and hundreds of flying saucers
- the Loch Ness monster pictures,
- the Cottingley Fairies pictures,
- the Belgian Secession hoax,
- UK Anglia Television Alternative 3
c. Deception as illusion of non-mediation
Perceptual objections to the illusion of non-mediation
The second form of deception consists in taking a mediated experience for a
non-mediated one. The fictional object then gives rise to what [Dennett, 1991]
refers to as a ‘strong hallucination’: “a hallucination of an apparently concrete and
persisting three-dimensional object in the real world – as contrasted to flashes,
geometric distortions, auras, afterimages, fleeting phantom-limb experiences,
and other anomalous sensations. A strong hallucination would be, say, a ghost
that talked back, that permitted you to touch it, that resisted with a sense of
solidity, that cast a shadow, that was visible from any angle so that you might
walk around it and see what its backed looked like” [Dennett, 1991, p. 7].
[Dennett, 1991] denies the possibility of experiencing strong hallucinations on the
basis of considerations concerning the dynamic nature of perception: how the
world feels depends on how the subject decides to move [see also O’Regan &
Noe, 2001; Casati & Pasquinelli, 2005]. In order to produce strong hallucinations,
it would hence be necessary to calculate all the possible sensations that could
arise in response to all the possible actions of the user, and to propose them in
Other considerations about the nature of perception, and in particular about the
fact that what we perceive is a complex pattern of stimulation, speak against the
possibility of being totally mistaken about the mediated or non-mediated nature of
a perceptual experience. According to the ecological view of perception
[Stoffregen, et al, 1997; Stoffregen, 20003], the stimulus condition (ambient
array) is specified in a different way by parcels of the real world and by
depictions. In fact is supposed that the relationship between a parcel of reality
and the ambient array that it structures is a 1:1 relation [Gibson, 1966]. Hence,
only identical parcels of reality can produce the same ambient array and give rise
to the same pattern of sensory stimulation. As other parts of the real world
simulator systems and cinema then structure the ambient energy fields in a
special manner that is specific to them and that is necessarily different from the
way ambient energy fields are structured by the simulated things. Since, in the
case of mediated experiences, both the contents and the context (the medium
and its characteristics) of the experience contribute to specify its perceptual
characteristics, mediated and non-mediated experiences are by necessity
Not only the contingent limits of technology, but also the necessary nature of
perception would hence prevent the possibility of being totally mistaken about the
mediated or non-mediated nature of a perceptual experience. These perceptual
objections also show the continuity between traditional and new media as for
what concerns the problem of producing an illusion of reality. Multimodal
stimulation in fact still brings information about the presence of the medium and
interactivity encounters the problem of calculating the responses to all the
possible actions performed by users.
Perceptual objections to the illusion of non-mediation
Perceptual objections to the illusion of non-mediation: the multimodal nature of
Perceptual objections to the illusion of non-mediation: the dynamic nature of
Perceptual objections to the illusion of non-mediation: the multimodal
nature of perception
Stoffregen (2003) affirms that believability consists in the (accurate) perception of realism and not
in the illusion of reality: we must distinguish between the perception of reality (inaccurate or
illusory perception that the simulator is the simulated) and the perception of realism (accurate
perception of resemblance between simulator and simulated).
Awareness of a realistic simulation need not imply belief in the reality of that which is simulated.
Moreover, the literal illusion of realism is likely to be very rare. If sensory stimulation produced by
a simulator were exactly identical to the sensory stimulation produced by the simulated thing,
then we could speak of complete sensory fidelity; but it is affirmed that 100% accuracy is
impossible for technology.
It is argued in fact that simulations and the things that they simulate always give rise to different
patterns of sensory stimulation.
First, simulators are real devices as simulated things are. As the other parts of the real world
simulator systems structure the ambient energy fields in a special manner that is specific to them
and that is necessarily different from the way ambient energy fields are structured by the
simulated things. In fact is supposed that the relationship between a parcel of reality and the
ambient array that it structures is a 1:1 relation. Hence, only identical parcels of reality can
produce the same ambient array and give rise to the same pattern of sensory stimulation. In
particular, it is stated that simulators cannot actually give rise to the multisensory complexity of
the simulated parcels of reality, in particular when vestibular and somatosensory arrays are
concerned. The possibility that not all the details of the array are perceived and hence that the
perceptual systems can tolerate many types of non-identity between ambient arrays in virtue of its
psychophysical limitations is suggested by Hochberg (1986). But Stoffregen (2003) considers this
suggestion more plausible for optic and acoustic simulation than for haptic and inertial simulation
where the magnitude of the non-identity between simulator and simulated is greater. It is affirmed
that users can pick up the differences or non-identities between the ambient array produced by
the simulated and the ambient array produced by the simulator even for very advanced flight
simulators. The user is not necessarily aware of the difference, but reactions such as sickness
which are not present in the simulated situation but only in the simulator condition would indicate
the fact that some difference is perceived. For consequence, a fully faithful simulation may be
impossible for most situations. Hence full realism as simulation is a chimere.
If this is true, then it would be inappropriate to expect a simulator to fool
the user; it would also be inappropriate to use subjective reports of “realism” as a basis for
Second, perception is selective and depends on exploration. Simulators can be taken for the
simulated thing only if the possibility of exploration is suitably reduced. In fact, exploration has a
role in disambiguating information that can be used by the perceptual system in order to
distinguish the simulator and the simulated. But restrictions to exploration also alter the sensory
stimulation in a way that informs the subject that ihe is not in the simulated situation, but in the
Third, the user has prior knowledge that he is experiencing a simulator. The same is true in the
case of cinema and might explain why spectators do not run away in panic during horror movies.
In fact, they did at the beginning of the century, when they were not acquainted with cinema: they
run away or fire at the sight of trains or bears (Stoffregen, 1997, 1998; Shapiro, 1992). Stoffregen
(1998, 2003) suggests that the awareness of fiction or simulation does not necessarily lay in
cognition but maybe in perceptual processes.
Some objections can be raised: the flight simulator is not realistic for someone who has piloted an
aircraft, because the two ambient arrays are compared. But what for a non-experienced user? Of
course, flight simulators are designed for training and create expert users, so the point made by
Stoffregen (2003) is quite destructive. But only when simulation of experiences with the real world
is concerned. When none of such experience is directly evoked, or only parcels of the experience
with the real world are evoked (only those parcels that can be faithfully rendered) the argument of
Stoffregen (2003) is no more valid, even in the case of simulation (at least in the case of limited
simulation). Additionally, believability is not necessarily a matter of perceiving the virtual reality as
being identical to a parcel of the real world, in other words, it is not necessarily a matter of
simulation. An expert flyer can consider the flight simulator as non-believable because it is a flight
simulator, and he, the pilot, is an expert in aircraft: the aim is to create a device which responds to
all the expectations that are raised by a real simulator. But in other cases, one can be satisfied in
creating a device that responds only to some of the expectations raised by the simulated parcel of
reality or even a device that creates by itself a number of expectations that are different from
those inherited from the experience with the real world. VR systems should anyway explore
applications different from simulation and/or adopt special narrative strategies that justify
limitations in the explorative possibilities and in the sensory stimulation.
It is suggested by Stoffregen (2003) that simulators may best be evaluated in terms of behavioral
outcomes (e.g., changes in performance), rather than on the basis of subjective experience. What
is evaluated in this case is the effectiveness of the simulator and not believability or other
subjective feeling or judgment.
Riccio (1995) has introduced two terms for evaluating simulators in terms of their fidelity to the
simulated condition: experiential fidelity and action fidelity. Experiential fidelity is the extent to
which the simulator gives rise to an experience of being there, hence to the classic definition of
Presence, and to an illusion of reality: the illusion that the simulator is the simulated and is real.
Stoffregen (2003) asserts the importance of distinguishing the two types of realism that are at
stake: realism as resemblance between simulator and simulated and realism as illusion of reality.
In fact, a representation such as a picture can resemble to the represented parcel of reality and
hence give rise to perception of realism, but the representation is not perceived as real
(perception or illusion of reality).
This distinction seems not be respected in Presence questionnaires that display the following
type of question: “how real does this seem?”; a better question in the case of illusion of reality
would be: “is this a simulation or is this reality?”.
Stoffregen (2003) nevertheless fails to notice that the second concept (those of illusion of reality)
is also broader than the first one since it can be adopted in fictional and virtual systems with no
simulation of parcels of the real world.
Stoffregen (2003) proposes to leave the perception of reality and to concentrate on the perception
of realism, that is, on the resemblance between the simulator and the simulated. He also cites the
case of fictional entertainment, in which audiences are induced to suspend disbelief, that is, to
find the representation realistic even if not real. Suspension of disbelief is hence related to the
perception of realism and not with the perception of reality (Goffmann, 1974).
But, as we have seen, perception of reality is a broader concept than perception of realism
because it applies also to non-simulation virtual systems and to non-simulation fictional
entertainments: if we adopt the suggestion of evaluating virtual systems in terms of perception of
realism we have no mean to evaluate non-simulation virtual systems and fictional entertainments.
It is true that the notion of illusion of reality is problematic: we suggest that audiences and users
are not fooled, as affirmed by Stoffregen (2003) even in fictional and virtual cases with no
simulation: users and audiences are always aware of the fact that what they are perceiving or
reading about is not the real world but a fictional or virtual one that can nevertheless be more or
less believable. Since the fictional or virtual world is not necessarily a simulation of the real one,
the judgment of believability cannot be coincident with the perception of realism. We suggest that
the judgment of believability has something in common with the perception of reality because it is
not an illusion even if it is a subjective judgment and because it is associated with behavioral and
emotional responses that are similar to those that one would present if the situation were real
(even if not similar to situations that can be experienced in the real world); the judgment of
believability can also include perception of realism when the fictional or virtual situation simulates
parcels of the real world. But it is exactly because audiences and users are not fooled about the
reality of the objects of their experience that they can find the experience to be believable even if
it does not display situations that simulate parcels of the real world.
Stoffregen (2003) proposes to evaluate simulators in terms of action fidelity. Action fidelity is
defined as the transfer of performance from the simulator and the simulated system. Action
fidelity exists when performance in the simulator transfers to behavior in the simulated system (or
vice versa). Action fidelity is hence measured via the evaluation of task performance: time to
completion a task, variance in performance across trials, etc.
What is more important is that the transfer of performance is not necessarily related to high
realism in the sense of resemblance with parcels of the world. Action fidelity is hence not a
measure of the perception of realism, no more than it is a measure of the perception of reality.
Moreover, sometimes, Stoffregen (2003) affirms, in certain cases departures from stimulus fidelity
might produce enhancement of action fidelity. Action fidelity is hence a pragmatic measure, which
can be used with simulators even if they are not perfect simulations of aircraft or other for what
concerns the sensory stimuli they display and even if they do not fool the users in the sense of
appearing real things.
The transfer of performance could be one of the behavioral consequences of believability, at least
in the cases where virtual or fictional environments present some form of realism as simulation
and are designed with the aim of training. The idea of measuring actions and reactions that are
similar to those one would experience if the experience were true (with the limitations and
characteristics of a special game of make-believe) is not so far from the idea of transfer of
performances. One can in fact think to the performances in the real world and the performances
in a game of make believe as a sort of transfer of performances.
Perceptual objections to the illusion of non-mediation: the dynamic nature
In his prelude to the explanation of consciousness, Daniel Dennett inquires about the possibility
of experiencing “strong hallucinations”:
“By a strong hallucination I mean a hallucination of an apparently concrete and persisting three-
dimensional object in the real world – as contrasted to flashes, geometric distortions, auras,
afterimages, fleeting phantom-limb experiences, and other anomalous sensations. A strong
hallucination would be, say, a ghost that talked back, that permitted you to touch it, that resisted
with a sense of solidity, that cast a shadow, that was visible from any angle so that you might
walk around it and see what its backed looked like .” [Dennett, 1991, p. 7]
The problem of strong hallucinations is relied by Dennett to the skeptical mental experiment of the
brain in a vat: is it possible that we are just brains in a vat, suitably stimulated by an evil scientist
in order to experience the sensation of living in a world of objects?
A third kind of experience which presents us with the possibility of experiencing the sensations of
objects in absence of real objects is represented by Virtual Reality systems (VRs). VRs are ….
“The state of the art is impressive: electronically rigged gloves that provide a convincing interface
for ‘manipulating’ virtual objects, and head-mounted visual displays that permit you to explore
virtual environments of considerable complexity. The limitations of these systems are apparent,
however, and they bear out my point: it is only by various combinations of physical replicas and
schematizations (a relatively coarse-grained representations) that robust illusions can be
sustained. And even at their best, they are experiences of virtual surreality, not something that
you might mistake for the real thing for more than a moment.” [Dennett, 1991, p. 6-7]
As we can see from this quotation, Dennett’s answer to the three questions is a radical ‘no’:
strong hallucinations cannot exist, we are not brains in a vat and VR systems will never fool us in
such a way that we can take a virtual object for a real one. This is not a purely contingent
impossibility, ascribed to the present limits of technology, but sounds like a permanent one, an
impossibility in principle of creating believable sensations of non-real objects (objects that do not
exist in the real world as material things) by means of whatever developed interfaces. Unless
(since there is a clause in Dennett’s assertion) we are capable of producing “ physical replicas and
schematizations (a relatively coarse-grained representations)” of the real world and objects. But
again, a replica of the world, even a replica of a simple coin, is far beyond human technology.
Dennett’s argument against the believability of virtual objects is grounded on a specific vision of
human perception: how the world feels depends, according to Dennett, on how the subject
decides to move.
An evil scientist or a VRs designer should thus be able to provide their subject with dynamic
stimuli, that is, with stimuli that represent the feedback to the actions of the brain-user.
There’s the rub. Since calculating the proper feedback implies the capability of taking into
account all the possible actions of the brain-subject and all the possible sensations that arise in
response to each action or the capability of calculating, generating and proposing the proper
feedback in real time. The first operation requires the possibility of storing an enormous amount
of information; the second operation is, according to Dennett, intractable on even the fastest
Only the world or a replica of the world can store such an amount of information as it is implied by
the dynamic character of human perception; but creating a replica of the world out of nothing but
mere stimulation is beyond human technology.
“Throw a skeptic a dubious coin, and in a second or two of hefting, scratching, ringing, tasting and
just plain looking at how the sun glints at its surface, the skeptic will consume more bits of
information than a Cray supercomputer can organize in a year. Making a real but counterfeit coin
is child’s play; making a simulated coin out of nothing but organized nerve stimulations is beyond
human technology now and probably forever.” [Dennett, 1991, p. 6]
d. Deception as illusion of transportation
Naturalization of the illusion of transportation: intersensory illusions
Can one experience an illusion of transportation in the virtual world, of being in
the virtual world, without being mistaken about the virtual nature of this world and
eventually without loosing the awareness of mediation? It seems possible to
naturalize some of the phenomena that are referred to as ‘illusion transportation’
or of ‘being there’ by suggesting that certain experiences trigger illusions of
special kinds, such as proprioceptive illusions that extend the boundaries of the
subject’s body and proprioceptive illusions of self-motion. It seems possible to
produce such illusions in the form of visual bias upon proprioception induced by
mirrors, artificial body parts or prisms [Welch & Warren, 1981; Nielsen, 1963;
Holmes, 2004; Ramachandran, et al., 2005; Gibson, 1933; Hay, et al., 1963;
Tastevin, 1937; Botvinik & Cohen, 1998; Pavani, et al., 2000]). These
phenomena are far from equating the massive experience of transportation
described by cyberpunk writers and seek out by VR designers, but they are
compatible with both VR technology and human perceptual mechanisms. The
experience of proprioceptive illusions is also compatible with the cognitive
awareness that the second world one feels to move in is a world of imagination.
Contrary to what could appear, awareness of the nature of the experience can be
exploited for enhancing believability. The fact of being aware that they are
dealing a fictional world lowers in fact users’ and audience’s expectations and
reduces the risk of expectations’ frustration, which constitute a major threat for
believability (Garau, 2003). The context of the experience (the presence of the
medium, its fictional nature) should hence not be hided or made transparent. At
the opposite, users should be made aware of the characteristics of the medium,
in terms of the possibilities the medium offers to users (quantity and quality of
sensory stimulation, possibility of interacting, of modifying the environment vs.
passive interactions) and in terms of the specificity of the interaction with virtual
worlds. As in the case of other experiences with worlds of fiction, users are asked
to activate their imaginative functions in relation to perceptual representations
that act as props for imagination [Walton, 2001] and that can change in response
to the player’s actions (which is not the case for traditional, non-interactive media
[Casati & Pasquinelli, 2005]).
Naturalization of the illusion of transportation: intersensory illusions
The illusion of reality as illusion of transportation or illusion of presence or even
as illusion of non-mediation could be explained by suggesting that certain
experiences (both in mediated and in non-mediated conditions) trigger illusions of
special kinds, such as: proprioceptive illusions that extend the boundaries of the
subject’s body and proprioceptive illusions of self-motion. These illusions are
particularly related to the components of the illusion of reality that are referred to
as “illusion of transportation” and “illusion of presence in the virtual environment”.
Other illusions can concern the reciprocal bias of vision, audition, haptic
perception and the capacity of producing illusory perceptual outcomes. They are
not related to the sense of transportation or of movement but rather to the
perception of special qualities of virtual objects.
The individuation of these perceptual illusions, the fulfilment of the conditions for
their production and the evaluation of their presence and strength is aimed at
producing a naturalization of the notion of illusion of reality in its different
components. The production of illusions and their evaluation does not exhaust
the tasks of producing believable fictional and virtual experiences and of
evaluating them in terms of believability. Believability is a broader concept that
goes beyond the purely perceptual level and includes perceptual and symbolic
The study of proprioceptive and motion illusions produced by different sensory
modalities produces a naturalization of the illusion of reality because it provides
scientific bases, documented in cognitive sciences literature on perception, for
the sensations of being there and moving in a virtual environment and for the
identification of one’s own avatar.
These illusions are mainly related to intermodal bias. In a situation of conflict
between information provided to different sensory modalities (vision and touch,
vision and proprioception, vision and audition, audition and touch, audition and
proprioception) illusions are produced that consist in a bias (more or less
reciprocal) of one sensory modality upon the other. Vision has been considered
for long the dominant sensory modality in case of conflict [Rock & Victor, 1964]
because more precise than the others. After contrary evidence, the precision
hypothesis has been retained [Welch & Warren, 1981] but not the necessary
dominance of vision. More recently, precision has been substituted with minor
variance [Ernst, et al., 2004]. The more precise, less variant information biases
the less reliant one. Since vision is usually more precise and reliable for spatial
location it normally dominates other modalities for this kind of task. In case of
three sensory modalities involved (audition, vision, proprioception), majority
(accord between two modalities in disaccord with the third one) is counter
intuitively not trusted [Bedford, 2007]. This is an important datum for situations
such as cinema and VR where stimulus co-location is not granted.
In addition to psychological studies on intermodal bias and illusions, the
relationship between vision, audition and proprioception is confirmed by the
neurophysiology of multimodal perception [Stein & Meredith, 1993]: overlaps of
proprioceptive and visual brain areas and the existence of multimodal
proprioceptive and visual neurons are described at the level of the superior
colliculus of the cat. Aural perception is involved in the same multimodal
systems, thus suggesting the possibility of auditory-visual interactions (see
Believability & sound) but also of auditory-proprioceptive interactions. These
interactions concern sound localization biased by proprioception, but also touch
and proprioception biased by acoustic perception.
Four main types of illusions are described that can be exploited for producing the
sensation of being in a certain environment, of moving in this environment, of
moving following a certain pattern, of being identified with one’s own avatar and
of proprioceptively feeling the sensations the avatar is supposed to feel. They are
mainly described as visual capture upon proprioception because vision biases
proprioceptive outcomes, but audition to can be exploited in this sense
(especially for producing self-movement illusions):
Proprioceptive illusions induced by prisms
Proprioceptive illusions induced by artificial body parts
Self-movement illusions (illusions of vection)
Knowledge of these illusions has not only a theoretical effect on virtual reality
(the understanding of the perceptual bases of experiences in virtual reality) but a
pragmatic one too. The knowledge of the constraints that produce such illusions
in natural conditions can in fact be used as guidelines for producing the same
effects in virtual reality, hence for enhancing the so-called illusion of reality in
specific domains. Reciprocally, the evaluation of the presence of illusions can be
done in objective ways by measuring the strength of the robustness of the illusion
and the degree of surprise that is experienced when the illusion becomes aware
(intersubjective and intrasubjective robustness and surprise being the main
structural characters of illusions).
One the constraints that seems to be necessary in the case of mirror illusions is
the fact of moving. Other constraints concern the cognitive assumption of unity
(mirror illusions with different pattern of movement between vision and
proprioception). Other illusions (rubber hand illusions) are facilitated by an initial
synchronization and match between vision and proprioception, or by the
combination of visual and aural stimuli (vection illusions) hence by multimodal
VR systems that provide the possibility of moving and multimodal stimuli are
especially suited for producing the proprioceptive illusions described, hence the
naturalized “sense of transportation and presence”. At the opposite of passive
media, such as cinema, a correspondence can be created between the user’s
and the avatar’s movements or between the user’s movements and perceptual
outcomes. In cinema, the movements of the camera do not correspond to the
movements of the spectator’s head and the movement of objects on scene is
independent of any action of the spectator. It is hence easier to produce certain
proprioceptive illusions induced by vision in VR rather than in cinema or other
passive media. This fact could explain why VR is considered to produce a
greater illusion of reality and a greater sense of transportation than passive
media, and why this greater power of presence is attributed as a matter of fact to
the possibility and degree of motor interaction. In any case, the possibility of
producing proprioceptive illusions cannot be confounded with an illusion of
In the case of multimodal media with haptic feedback sound can play an
additional role by influencing not only vision but also haptic perception.
How much realistic must visual or other avatars-representations must be for
producing proprioceptive illusions?
Realism is only relative, because rubber hands but also other non-human-like
artefacts are effective in producing proprioceptive illusions, and because the
seen movement in mirror experiments does not necessarily match performed
movement. In this latter case, however, a cognitive assumption of unity (common
source between visual and proprioceptive stimulus) is made on the side of the
subject of the experiment who believes that he sees his hand in a mirror. In
addition to the understanding of perceptual constraints for producing
proprioceptive illusions of de-location, cognitive constraints must also be
individuated that can let subject to make an assumption of unity (in the case of
sound and images in cinema, for instance, it seems that spectators can attribute
the two to one and the same source because this source is cognitively identified
as fictional rather than real, see Believability & sound).
These considerations suggest the opportunity of making the point about the
different roles that different perceptual modalities play in the believability of
mediated, fictional or virtual experiences. This role in fact is not necessarily
cohincident and cannot be fully understood by the simple reference to realism, to
the idea that adding stimuli for different sensory modalities enhances the realism
of the experience. The case of sound is paradigmatic.
In cinema, for instance, sound is attributed the functions of creating atmosphere,
suggesting time and space localization, commenting visual scenes, creating
emotional moods (see Believability & sound). The capacity of suggesting a
certain time and space location for the narration is different from an illusion of de-
location, because the spectator is not necessarily transported by the sound in old
Scotland, but he just becomes aware (thank to experience and acquaintance with
filmic techniques) that the narration develops in old Scotland. The integration of
sound and images does not depend on co-location of visual and aural stimuli or
on common source, because the source of sound is rarely coincident with the
screen. This integration is not necessarily a perceptual illusion either.
In addition to completing or modifying information gathered though vision, sound
can also help directing the attention toward the visual stimulus it matches with,
thus minimizing distractions provoked by the environment. Matched multimodal
stimuli have in fact the effect of enhancing neural responses [Stein & Meredith,
Auditory stimuli hence do not necessarily add realism or presence, even when
combined with visual ones in unitary percepts, but can help
- directing attention,
- enhancing neural responses
- and providing new information about the virtual or fictional objects’ and events
represented. In general, auditory stimuli provide additional information to visual
one which can complete it, be redundant or even be in contradiction with it.
Auditory information can for instance extend the experienced space beyond the
visual one thus suggesting fictional entities that are not in sight (extension of the
perceivable virtual or fictional space [Riecke, et al., 2005]). This possibility is
often used for guiding behaviours, such as when listening at the sound produced
by a car while driving for detecting eventual engine troubles. Adding engine
sounds to car simulators is hence certainly functional when the driver is
supposed to acquire the capacity of making such a judgment or to exert this
capacity in the virtual setting. However, the inference from functionality in relation
to a task to the supposition that this sound will certainly add realism and that it is
required for realism because it exists in real life situations [Riecke, et al., 2005] is
not authorized. Movies spectators pass unnoticed the absence of most of natural
sounds in movies and are even disturbed by the presence of as many sounds as
they would hear in analogous real life conditions (at least according to sound
designers, see Believability & Sound).
Sounds play another important role, which is not strictly related to perceptual
identification or recognition of objects’ and events’ characteristics:
- they provide instructions to users/spectators about how to feel and what to
think about what they experience. This function is much more in use in cinema
than in VR (the emotional role of sounds is recognized by VR designers, see
[Riecke, et al., 2005] but it is not interpreted as a form of instruction).
Finally, sound can be used to
- produce illusions (through intermodal bias) in the visual and proprioceptive
modalities (in analogy with vision) and auditory-induced illusions can be
considered as another element for the naturalization of the notion of illusion of
reality. Sound stimuli can be considered to produce “acoustic avatars” that match
VR users’ movements, such as in the case of the sound of footsteps that starts
when the user moves through a space and stop when he stops. It should be
ascertained if sound avatars are able to produce proprioeptive illusions
analogous to the rubber hand illusion produced by vision.
[Nielsen, 1963] describes the following experiment with mirrors and substitution of the
experimental subjects’ visible hand with another subject’s hand. The subjects follow a straight line
with their hand in full sight; on some of the trials, a mirror is introduced, unbeknownst to the
subjects, so that they see another person’s hand (that they believe to be their own): the subjects
continue to have the sensation that the seen hand is their own, but they also feel as if they had
lost control over its movement.
[Holmes, et al., 2004] describes a battery of experiments that show the effects of proprioceptive
illusions provoked by the use of mirrors on reaching performances: reaching performances are
affected when the visual position of the reaching hand is altered by the use of a mirror reflecting
the non-reaching hand. The experiments show the presence of a proprioceptive illusion of
position of the reaching hand (visual capture) and the authors hence ascribe the error in the
reaching task to the proprioceptive illusion of position.
The presence of discrepancies between proprioception and the visual position of the hand have
disruptive effects on reaching tasks (measured in terms of lowered speed and of accuracy, that
is, of the distance between the finger tip and the target to be reached) even in virtual reality
contexts [Durlach, et al., 2005]. This fact confirms the general consideration about the importance
of coherence for adaptive behaviours and the disruptive effects of discrepancies upon
performance [Stein & Meredith, 1993].
This fact seems also to suggest that visual capture phenomena (proprioceptive illusions induced
by vision) and of so-called illusion of reality in general might play a negative role on
performances, rather than a positive one, by confusing the subject and by disrupting or adaptive
behaviours (an effect of violations of coherence in general) .
Mirror illusions are exploited for the treatment of pathologies such as the phantom limb, stroke
and hemiplegia and spatial neglect (the success with pathologies other than phantom limbs is still
uncertain) [Holmes, 2004; Altschuler, et al., 1999; Sathian, et al., 2000. Ramachandran, et al.,
[Ramachandran & Hirstein, 1998; Ramachandran & Rogers-Ramachandran, 1996;
Ramachandran & Blakeslee, 1998; Ramachandran, et al., 1995] describe a simple device, the
virtual reality box, or mirror box as being effective in reducing the painful palsy of the phantom
limb, by visually inducing an illusory sensation of movement and position.
The device is composed of a box with two holes in the frontal part of it and a mirror inserted in the
middle. The patient who suffers from phantom palsy (which provokes pain) of his phantom hand
(a quite common phenomenon in subjects who have been amputated after experiencing a palsy
of the affected arm) inserts his hands in the holes. Naturally he really inserts just the normal
hand, but, since he can feel his phantom hand’s position and movement, he has the feeling of
having inserted the phantom limb too. The mirror allows the patient to visualize the phantom hand
that he can only feel: it is the reflection of his normal hand. The patient is asked to open his real
hand and move it, and at the same time to ‘try’ to open his phantom hand. In this way he
provokes the vision of the phantom hand as moving normally and opening against the palsy.
Proprioceptive illusions induced by prisms
Other experiments have been conduced with prisms.
A classic experiment on visual capture upon proprioception (a prism is fixed on the subjects’ head
which displaces the visual position of the arm) is described by [Hay, et al., 1963].
More recently, [Mon-Williams, et al., 1997] have confirmed these results.
[Gibson, 1933] describes the following illusion: a subject moves his hand along a straight surface
while looking through a prism that causes the surface to look curved; he feels it to be curved as
Proprioceptive illusions induced by artificial body parts
Another group of proprioceptive illusions with visual capture has been demonstrated by the use of
artificial body parts.
[Tastevin, 1937] describes the attribution of sensations to an artificial finger when the hand of the
subject is not visible. This illusion, known under the name of “fake hand illusion” or “rubber hand
illusion” has been recently re-demonstrated by [Botvinik & Cohen, 1998] and [Pavani, et al.,
2000]. Others studies on artificial substitutes are: [Austen, et al., 2003; Austen, et al., 2001].
When a rubber hand is brushed in contemporary with the brushing of the hidden hand of a
subject, the subject feels the rubber hand to belong to his body.
The same sense of ownership is obtained in the same conditions (vision + haptic perception
provoked by stroking) when the rubber hand is substituted by a neutral object: shoe or table.
Visual realism is hence not required. This fact suggests that ownership (measured as the
displacement of the sensed position of the real, stroked or brushed hand in relation to the artificial
object) can happen also for avatars that do not represent life-like entities [Armel &
Ramachandran, 2003]. [Armel & Ramachandran, 2003] suggest that this illusion is due to a
Bayesian mechanism of learning of the association between visual and tactile stimuli that are
synchronously matched: any object can be self-attributed whether the statistical correlations
between visual and tactile sensations are strong enough. [Tsakiris & Haggard, ] challenge this
model and even the possibility of producing proprioceptive illusions just by synchronization: both
the aspect and the position of the artificial object seem to count (rubber hand simulates the
opposite hand than the one which is stroked).
It is also possible to visually induce the proprioceptive sensation that the first finger is stimulated
while the fifth one is by projecting the image of a hand which first finger is stimulated. This visual
bias over proprioception only happens when the tactile stimulation of the fifth finger and the
stimulation seen on the screen upon the first finger are in phase. Time matching seems hence to
be relevant in addition to the presence of real tactile stimulation [Schaefer, et al., 2005]. In
conclusion it is possible to provide stimuli for a body part that it is not actually stimulated when
some body part is and when there is consistency in the timing of the two stimuli.
Another variance of the rubber hand illusion can be obtained with tactile and proprioceptive
modalities only, without vision: a blindfolded subject touching with one hand a rubber hand and
synchronously being touched on his other hand, appropriates the rubber hand as his own and
feels like touching his own hand [Ehrsson, et al., 2005].
Illusions of vection
As a matter of fact, proprioception biases audition as shown by the existence of audiogyral (when
the whole body rotates, the auditory target is heard to displace in the opposite direction to rotation
during acceleration, to come back during constant velocity and to displace in the sense of the
rotation during deceleration) and audiogravital illusions (auditory localization of an auditory
stimulus influenced by linear acceleration; this phenomenon can be considered as an illusion or
as correct perception in a different environmental frame-work, in any case it confirms the
influence of proprioception upon auditory perception) [DiZio, et al., 2001].
Also, auditory biases upon proprioception exist. An example is represented by self-motion
illusions produced by the rotation of auditory or visual stimuli (vection) [Riecke, et al., 2005]. The
illusion induced by vision is frequently experienced in natural conditions (the start of a train seen
from a stationary train which is perceived as ego-movement is a classic example of vection).
Audition-induced self-motion sensations are less compelling than vision-induced ones and cannot
be produced in more than 25-60% of bindfolded participants [Lackner, 1977; Larsson, et al.,
2004]. Increase of these effects can be produced by the combination of visual and auditory cues,
and even by the combination of visual cues with vibration [Riecke, et al., 2005]. Auditory cues
must be specific for obtaining this effect, that is, they should match visual cues (spatialized
sounds that can be attributed to the same source than the visual images). It seems that also top-
down processes play a role in inducing vection. The identification of a source of auditory stimuli
that can be consistently interpreted with self-movement enhances vection produced by auditory
stimulation [Riecke, et al., 2005].
In case of devices directed to propose users with experiences of locomotion and in general of
ego-motion in VR, different procedures can be adopted, from treadmills to head-mounted displays
and free walking to the production of an illusion of self-motion through visual and eventually
tactile (vibration) or acoustic stimuli. [Riecke, et al., 2005] affirm that no one of these alternative
procedures is capable to produce a convincing sensation of self-motion in the virtual environment.
This is an exemplary case in which the evaluation of the user’s performance does not need to
take recourse to the notion of illusion of reality or of presence but only needs to be evaluated
against the production of a convincing sensation of vection. In the case of the procedure of using
visual or other stimuli for producing an illusion of self-motion, the illusion of self-motion constitutes
both the target and the behaviour to be evaluated in order to measure the efficiency of the VR
device. The experience that is intended to produce and that must be evaluated is not a not-well-
defined illusion of reality or presence or realism, but a well-known illusion of vection. In this
particular case of devices that propose users with experiences of self-motion, the degree of
illusion of reality can hence be naturalized in terms of the strength of the illusion of self-motion (in
certain cases illusion of vection) which is produced by that certain device (a strength that can be
measured in terms of robustness, at the intersubjective and intrasubjective level, and in terms of
surprise when discovering that no self-motion occurred (for instance by presenting conflicting
information that concerns the static condition of the user).
If it is true that “In the long run, a deeper understanding of any potential causal relations between
presence and the effectiveness of a simulation for a given task or goal (here: self-motion
perception) would be rather helpful for optimizing VR simulations from a perceptual point of view”
[Riecke, et al., 2005, p. 9], then the naturalization of the notion of presence is not a secondary
task for VR designers.
In the perspective proposed here, the comparison of presence tests and of vection evaluation and
the search for causal relations between the two [Riecke, et al., 2005] seems rather odd because
presence is naturalized in terms of this same task. On the contrary, the subjective evaluation of
vection could be doubled with an objective evaluation of the robustness of the illusion and of the
level of surprise produced by the illusion’s awareness. Thus adding objective rather than
Another consideration from vection illusions concerns the importance of topdown processes in
the proposition of believable experiences in general but also, as it seems, of perceptual illusions.
Top-down knowledge is demonstrated to play a role even in other illusions, such as the golf ball
illusion (a weight illusion that only can be evoked for expert golf players). In the case of vection
induced by acoustic stimulation the appropriate use of sounds corresponds to an appropriate use
of the narrative elements of the experience in order to influence the perceptual ones.
The co-occurrence of tactile and auditory stimuli results in combination of the two, with relative
modification of the single perception of each of the unimodal stimuli, in spite of the fact that
subjects to experiments are instructed to ignore the presence of background stimuli [Ernst, et al.,
2004]. This attitude is attributed to the fact that matching between co-occurring multimodal stimuli
is a common experience in everyday life (as when knocking at a door produces auditory signals
that match the number and strength of the actions of knocking as it is perceived through the
proprioceptive modality and trough the tactile one).
“This automatic combination likely results from the fact that matching between co-occurring
multimodal signals is very consistent across our everyday experience, so that the CNS can learn
to co-register sets of redundant sensory signals and identify every single set as elicited by the
same unique event or stimulus.” [Ernst, et al., 2004, p. 7].
Illusion of reality. Ethical issues
In addition to cognitive and pragmatic considerations, distinguishing the different
aspects of the notion of illusion of reality is important for ethical reasons. Illusion
of reality as believability and illusion of reality as proprioceptive illusions of self-
motion in the virtual world appear in fact non only feasible but also ethically
neutral notions (the illusion of non-mediation on the contrary is apparently not
feasible), while the obliteration between fiction and reality can be opposed to
The notion of illusion of reality, whether it is not naturalized though the reference
to perceptual illusions and whether it is not considered as a local, specific
phenomenon one can be or become cognitively aware of, presents ethical
problems connected with the fact of cheating spectators and users and to make
them believe that a fictional or virtual world is not a fiction or even not a
A paradigmatic example of the infraction to the ethical principle of keeping reality
clearly distinguished from fiction and imagination is represented by certain uses
that are presently made of Second Life, a largely visited on-line virtual world.
The term ‘virtual reality’ itself can suggest the wrong idea of a hybrid form of
experience which is neither the experience of the real world, nor the experience
of a fictional world. Virtual reality techniques provide perceptual stimulation that
responds, more or less in real time, to the user’s actions. Whether perceptual
stimulation and action are associated with some, even minimal or abstract
activity of imagination, perceptual experience becomes a fictional one too, where
the user pretends a certain world which he creates from his imagination following
the props that are provided by the system itself and that work as instructions for
imagining [Walton, 1981]. These techniques and these possibilities of action and
perception can be exploited for a large number of uses related to the
manipulation of perceptual, motor and symbolic representations, ranging from
learning, to operating or communicating at distance, to designing and
prototyping, to narrating and creating stories. Of course, all these operations can
produce secondary effects on the real user and on the real world. But all these
operations are done upon representations and within fictional worlds.
The banner of the Second Life web page announces: “Your world. Your
imagination”. Second Life (SL) is a world of imagination which can be accessed
via the net. Tools are provided to users of SL for creating an avatar of the user’s
himself that will be driven by the user to explore representations of lands (for
free) or even to build objects on these lands and exchange objects by the mean
of money (not for free). As for games like Monopoly, money and buildings are
just make-believe ones and the representations that are manipulated by players
are props for driving and bounding their imagination. At the opposite of structured
games, the appearance of each spot of digital space in SL depends on the
esthetics of the spot’s owner and a variety of environments can be encountered.
Avatars are guided by players as children do with puppets in games of make-
believe or as adults do with other paper, electronic and on-line games of make-
believe (World of Warlock, Dungeons and Dragons, Lord of the Rings on-line).
Like puppets and at the opposite of various forms of Tamagotchi, events happen
to avatars only when players command actions: players cannot just see what
happens to their avatar and they must be connected to SL in order to make the
avatar appear somewhere in the SL world. There is no score or specific objective
to be achieved. The player is hence free to act in relationship to the represented
environment and to modify it. As multi-players games, the avatar encounters
other avatars (other puppets) and dialogue and communication can be
established, together with the exchange of objects. This social dimension creates
boundaries to individual imagination.
By the fact that multiple users can furnish their spot in SL as they prefer, create
objects, personalize their avatar and choose how to make it interact (or not
interact) with other avatars, SL could appear as a collection of home pages
where anyone can present the contents he desires. This appearance is
reinforced by the range and diversity of uses that are actually made of SL:
- SL is used for performing make-believe individual or social games: creating
worlds with characters, environment, actions and events, pretending together (as
in other make believe games) at distance (as other on-line make-believe games
with or without perceptual cues). These actions and events include commercials
transactions in Linden dollars and firms selling and advertising digital
representations of objects for SL avatars and SL places (from representations of
t-shirts to representations of buildings)
- SL is used for virtual conferences and staff meetings through SL avatars of
firms really implanted in the real world that use SL in substitution of other tools
for virtual conferencing
- SL is used as a space for commercial advertising and selling real objects and
not representations of objects: objects that will be used by the player and not by
the avatar (as it happens in any medium)
- SL is used for different types of simulation related to design, prototyping and
relative evaluations of usability, in analogy with other digital design and
prototyping tools and with other online tools for working
- SL is used by politicians and religious communities for expressing their ideas
(as TV and other passive media) or for discussing with audiences (as a real
debate) with the aim of convincing players - not avatars - to adhere to their ideas
- SL is used for gambling, with real money won or lost , since Linden dollars can
be exchanged with real dollars. This fact makes any money transaction in SL a
transaction of money in the real world
- SL is maybe used for illegal exchanges such as, exchanges of real
pornographic photos or contacts within terrorist organizations (as the net is in
- SL is used for educational purposes, for instance by letting people take a
course of some distant university (as for other hundreds of on-line courses on the
net), interact as in a class or even interact with avatars representing dead
famous people (as if a professor of history of science dressed like Galileo for
answering questions from audience).
In spite of the multiplication of uses and places, what makes of SL a unitary world
(of fiction) is the presence of a certain number of features that are common to the
entire SL space and that are independent from the specific uses that can be
done of it:
- a specific language of programming though which digital representations
- the accessibility with one and the same avatar to any place in SL
- the presence of one and the same frame: the SL web page, to which one
connects and disconnects at the beginning and at the end of his session of
- the specific possibilities of action and perception: for instance, no haptic
perception, limited auditory perception, movement of the avatars through simple
commands of the hand/fingers, creation of objects via symbolic representations
(language of programming) or via simple gestures
- the appearance of the SL world:
o The visual appearance of SL for instance is not more cartoonist
than the one of other on-line games, such as World of Warcraft and Lord of the
o As cartoons, Second Life has its own physics, which is different
from real world physics: no gravity, no weather, no respiration, possibility of
- the use of one and he same monetary value: the Linden dollar, whciih
can be exchanged with real dollars.
Are there (ethically) legitimate and non-legitimate uses of SL?
SL world internal unity and the fact the at present 8 million users have subscribed
to SL (with picks of only 40000 users actually connected in the same time to SL)
makes SL interesting for those, politicians, firms, churches, educational
institutions who want to join players rather than avatars thus introducing a
different group of uses than pretending. The multiplication of real uses of SL
creates a risk of obliterating the distinction between fictional worlds and the real
world, hence creates a risk of hoax and gullibility. This risk is essentially
produced by the fact that uses that have an effect in the real life and are directed
to the life of the player beyond the connection to SL (commercial advertising for
products that are sold in the real world, religious messages and political
campaigns for elections in the real world, educational tools for learning at
distance, casinos where real money can be won and lost) are not framed and
clearly distinguished from uses that are directed to the avatar and that are related
to the pretending-fictional activity (selling and buying products for avatars or for
furnishing the SL world, creating fictional religions and fictional political structures
for SL citizens, etc.).
When users puppet-master their virtual avatars to buy clothes in SL
representations of shops, to ‘build houses’ (create digital representations of
houses) to live in SL, and invite other avatars to meet them, these events affect
the aspect of the SL virtual world and are circumscribed to a make-believe game
played with digital representations rather than with dolls or paper. As in any other
game of make-believe a player can express his deep feelings and true thoughts
through his own avatar, but he can also just imagine and lie without affecting the
game. One can make his avatar look and behave as much as possible as the
real player behaves in the real world, or he can puppet-master a penguin-like
representation that buys or destroys expensive objects, rapes or kills other
avatars, flies or uses tele-transportation. All this happens in a fictional world and
has the same value (and deserves the same ethical considerations) than other
activities of individual and collective imagination.
But what happens when the same avatar attends Stanford University lessons?
Who is learning: the avatar or the player? While educational structures can be
created for avatars, Stanford and other universities that have implanted their
digital representation in SL do not intend to address avatars and to play a game
of make-believe: the intend to provide courses at distance for real individuals (SL
players), as they do with other tools for on-line teaching. What if a SL avatar
enters Stanford University building for attending a lesson and is shot in the head
by another avatar playing a game of destruction? Who is killed? The avatar, of
course. But who is learning? The player. No perceptual frame, no symbolic
advice can set the distinction, within SL world, between reality (learning,
attending a lesson) and fiction. The same conceptual confusion characterizes
political and religious proselytism and all the other uses of SL that are intended to
affect the real life of players.
We have seen that in the case of traditional media, journalistic, fictional and
commercial uses are framed in order to prevent ethical side-effects of conceptual
confusion, and in particular hoax and fraud. Peter Weir’s movie The Truman
show [Weir, ] is a satire of the effects of the obliteration of any distinction
between fiction and reality, with a man acting in a fictional world he takes for the
real one and actors advertising for commercial products while plying their role. As
VR and SL, traditional media such as radio, cinema and TV are versatile media
that can propose news, interviews, documentaries and fiction. But frames exist
that have the scope of providing identifiable boundaries for each of these uses: a
film is declared as that, it has titles at the beginning and at the end showing the
list of people that made them, the actors that interpreted them. Whether a film is
interrupted by commercial spots, the spots are interpreted by different actors and
frames (undertitles, intertitles) separate them from the movie. Documentaries
have no or few actors mentioned in the titles and are known to be records of true
events. These frames and rules can be insufficient and can deserve satire and
ethical attention, but at least they do exist. In SL all this is mixed up and no frame
indicates the distinction between fiction, uses that affect the player and reality,
and commercial advertising.
The door is hence opened in SL for two types of ethical considerations and risks.
The first consideration concerns the problem of trust and the relative risk
concerns the quality of the engagement of the subjects that are involved in non-
game activities. This problem is common to all on-line real commerce and
education, the commerce of real objects at distance and the education at
distance. An ethical attitude assumed both by providers and users of real
services and specific rules for the control of quality could solve the problem. But
in addition to that, SL presents the specific character of being also a fictional
place. Whether a real task is taken for fictional it can be underestimated, both on
the side of the providers and on the side of the users of the service.
The second consideration is related to the ethical risks of taking fictional contents
for real ones. Users are invited to believe that at least certain actions in the
virtual world will affect their lives, hence that these actions do not happen in a
virtual world of game and fiction, but in the real world. The risk is hence the one
of loosing the capacity of distinguishing between reality and fiction, and be the
prey of hoax and fraud. This second problem cannot be solved by personal
ethics alone, an honest attitude on the side of providers and users of services,
because it originates in a conceptual confusion. Specific rules should hence be
introduced against conceptual confusion that will make the separation between
different uses of SL (and of other VR tools) clear.
Are there (ethically) legitimate and non-legitimate external interventions upon
Terrorism and the exchange of pedophilic or violent material (pictures of children,
of rapes) are actions that are subject to legal interventions and ethical principles.
These phenomena are common throughout the net in general and other media.
What is specific of virtual reality and of SL is the possibility of making the same
experiences with one’s own avatar and with other avatars in a purely fictional
world. At the opposite of the situation where real people are involved, it is hard to
consider even terrorism upon virtual buildings or the rape of a children-like avatar
as criminal or unethical behaviors against society or children, as it is not a crime
the fact of imagining in one’s own head to do the same things or to write them
down in a book or to make a film on them. Ethical objections can be raised
because of the public rather than private (one’s own mind) activity of imagination
which is involved in SL and in other VR games that are free and on-line.
Censorship could thus be invoked for limiting the access to this kind of
representations, as it is the case for other works of fiction. At the end of any
episode of the series Alfred Hitchcock presents featuring a successful crime,
Alfred Hitchcock had the use (imposed by censorship) of declaring that the
criminal had finally be caught and duly punished, thus correcting the idea that
could emerge from the story that crime can be unpunished. In the same way, the
representation of acts of violence and of sex between adult-like and children-like
avatars can be considered as offending, disgusting, anti-educative for the
spectators that assist to these representations (even if films representing horrible
rapes such as The accused have been produced with an educational aim:
showing a terrible reality and its consequences).
But it is important to avoid confusion about what is censored and why. The
particular emotional reactions that are triggered by the simple thought of violence
upon children should not serve as a justification for obliterating the difference
between reality and imagination, because, as we have seen, this confusion has
ethical consequences too. It is a fact that sexual behaviors encounter greater
taboos than death. A SL user who creates an avatar resembling to his boss, and
kills him every evening when back home from his office can be rather seen as a
sympathetic guy trying to evacuate his frustrations through imagination. This is
never the case for rape, pedophilia and sexual violence in general. But, as a
matter of fact, the only difference between the two cases lies in our different
reaction toward sexual behaviors and desires, and not in the real rather than
fictional context of the behavior itself. It is hence important for journalists who
comment events in SL world (the column about SL is becoming more and more
diffused in newspapers throughout the world ) not to increase the confusion
between reality and imagination by mixing, for instance, the diffusion of
pornographic photos of real children and the diffusion of pornographic digital
representations of children or even representations of sex with children-like
avatars. Of course one can comment that it is quite depressing to see people
reproduce in fictional worlds the worst behaviors one can assist to in the real
world. But this is one of the functions of imagination, too: letting desires and fears
that - for ethical reasons - cannot be realized in the real world, free of expressing
in dreams and games.
Virtual reality is a form of representation and a mean for proposing experiences,
(Biocca, Kim, Levy, 1995; Kim & Biocca, 1997; Lessiter, et al., 2001; Lombard &
Ditton, 1997, 2000; Loomis, 1992; Riva, et al., 2003; Steuer, 1995; Vorderer, et
As many types of artworks in the Western aesthetic tradition, ranging from
paintings, tragedies and even landscape gardens, virtual worlds and virtual
reality are “intentionally created and structured in order to produce a certain kind
of effect” (Freeland, 1996, p. 207). The intended effects can be as different as
catharsis, aesthetic distance, free play of imagination, depending on the
particular aesthetic perspective adopted and on the specific artwork concerned
(Freeland, 1996, p. 2007).
It seems hard to consider all the products of virtual reality applications as
aesthetic products. Flight simulators, training in general and tele-operations are
operational situations with no aesthetic aim. But when we consider all these
situations as creations or intentional and structured products, then the
components of creation (in the sense of a intentional product that consists in an
original inedited situation) is quite evident.
Such creations can be considered as aesthetic products once aesthetics is set
free from the reference to beauty and even to art as a form of self-expression
and considered as the outcomes of intentional creation.
We can thus consider experiences with pictures, landscape gardens, books,
movies, virtual worlds and even distant worlds by means of tele-operation
systems as parts of a continuum of products intentionally created and structured
in order to produce a certain effect, a certain type of experience.
Appropriate actions and perceptual or emotional behavioural responses (the
desired effect) provide a metrics for evaluating the effects of an experience in
virtual reality. The definition of virtual reality as an esthetic object intentionally
created to produce a certain kind of effect has thus operational effects in terms of
indications for measuring the effects of experiences in virtual reality (without
taking recourse to unclear notions, such as presence, and to limited notions,
such as realism).
An example can help illustrating the point. Let us imagine a pair of virtual dice
and an interaction by the mean of a haptic interface that allows for natural
complex movements of the hands. A scale of adequateness of the actions and
reactions of the users can be drawn in which spontaneously taking the dice in the
palm of the hand and throwing them with a lateral movement stands at the
highest level of adequacy, while exploring the contour of the dice with the first
finger is less appropriate, and not spontaneously exploring the dice at all can be
considered as low adequacy.
If the virtual object is different from any real object (in the sense that the aim of
the creators is not to simulate any real objects, that the actions and sensations
that are made possible by the material interface are significantly different from
the actions that are normally performed in the real world and that the coded
information presents significant differences from any existing object) the task of
individuating the adequate actions and of distinguishing them from non-adequate
ones becomes more difficult.
Taking recourse to the aims and intentions of the creators helps solving the riddle
if we can clarify how users have access to the intentions of the creators.
Users can gain this knowledge from information explicitly stated by the creators
(declarations of the kind: “this is a flight simulator, or a surgical simulator”).
A significant amount of information about the appropriate actions can be also
gained from perceptual hints in a way that can be assimilated to an affordance
pattern (as described by Gibson, 1979, and ecological perception approach in
general) or to a sensorimotor connection (O’Regan and Noe, 2001;
Innate or acquired through experience, the connections between action and
perception provide non-symbolic information concerning the actions that can be
more or less appropriate to a certain pattern of perception.
In the mean time, virtual reality is different from traditional representational media
because virtual reality systems (and in particular interactive interfaces) allow
action from the user (Casati & Pasquinelli, 2005; Pasquinelli, 2003).
The psychology of (motor) action and the psychology of social interaction and of
interaction in general must thus be added as significant fields of study in the
domain of the sciences of the mind-body. This fact marks the specificity of VR in
respect to more traditional media (Casati & Pasquinelli, 2005; Pasquinelli, 2003).
The possibility of action entails two consequences: the possibility of using of one
more of the human capabilities, notably, motor performances (in addition to
perceptual competences and symbolic, emotional, imaginative and generally
cognitive capacities), and the possibility of interacting.
Interaction implies the possibility of modifying the contents of the representation,
which is quite a new opportunity if compared with the “non-interactive” fruition of
cinema, but not in respect to interactive narrative media that do not necessarily
allow motor action and perception (Casati & Pasquinelli, 2005; Pasquinelli,
A user that interacts with a certain fictional or virtual environment modifies the
environment and perceives the effects of his actions on the environment and not
only a static product, as it happens in more passive interactions (such as the
experience of a movie).
This fact could suggest the erroneous consideration that virtual reality systems
are incomparable with movies (which are representations) and that virtual worlds
are forms of reality because of the fact that users do interact and produce
modifications as they do in the real world. In other words, the possibility of motor
action and interaction with modification could obliterate the representational
nature of virtual worlds.
In spite of the fact that one can interact with and modify virtual worlds, fictional or
virtual worlds are human intentional creations aimed at producing a certain effect
in other subjects and not real worlds. The real world produces a number of
effects on subjects but is not aimed at it. For this reason it makes sense to
measure the effects of virtual reality systems and of movies in terms of the
intentions and aims of the specific application, which does not make sense for
the real world (a stomach is not built for digestion, even if digestion is a function
of the stomach; flies are not created for being eaten by frogs).
The erroneous consideration about the nature of virtual worlds is reinforced by
the technological commonalities shared by virtual reality and tele-operation, in
which case, the distant world is the real one.
But it is not the virtual world which is made real by this association, it is on the
contrary the real world which is perceived through a medium which is made in
some way fictional.
The fact of introducing a medium (a tele-operation system, a telescope) between
the observer and parcels of the real world makes the experience partly artificial,
because both the real world and the medium (causally) influence the appearance
of the perceptual outcome. The perceptual experience of the sky seen through a
telescope (causally) depends on the state of affairs (the sky) and on the
intentions of the creator of the medium (seeing the stars that are far away as if
they were next to he observer). In this sense any experience which is made
through a medium partly (causally) depends on the intended effect of the creator
of the medium and can be considered in analogy with experiences with virtual
reality systems and in analogy with experiences with movies.
The difference lies in the fact that in the experience with movies and with virtual
worlds both the medium and the state of affairs are the product of the intentions
of some creator in search of a particular effect for his audience and users, while
in the case of the telescope and of tele-operation systems the state of affairs
which is perceived through the medium is not an intentional product of some
In spite of its label, then, virtual reality is not a form of reality but a representation.
But can an adult take a virtual world for the real one?
(Dennett, 1991) considers that producing a massive deception (a strong
hallucination) requires more than reproducing the features of the external world)
What is required, in virtue of the dynamic nature of perception, is the
reproduction of all the possible interactions between the bodily possibilities of a
human adult and the real world (sensorimotor interactions). Sensorimotor
interactions are so variable and numerous that strong hallucinations or complete
deception are unlikely to be created by virtual reality systems.
But what for a child? Let us imagine an evil scientist putting a newborn in a room
which is furnished in order to convey haptic, sound and visual stimuli by the
mean of suitable interfaces that are directed by digital code. The virtual world so
created can be very simple and respect the present development of technology.
Of course, only certain types of sensations can be produced, sensations that
depend on mechanical stimulation of some kind (including photostimulation and
acoustic stimulation), but not sensations that depend on molecules entering in
direct contact with receptors, such as in the case of smell and taste. In order to
produce smells and taste the digital code should command the synthesis of
molecules. In this case the synthesized molecules could be considered as the
interface between the code and the subject’s receptors, as haptic, video and
sound interface are.
We know that a newborn presents a high level of cerebral plasticity, even if this
plasticity is not infinite but is determined by previous selection (Edelman, 1987).
Additional selection operates on the neural structure of the child’s brain in accord
to the conditions the child encounters in his environment: the environment which
is intentionally created by the evil scientist. The baby won’t presumably develop a
certain number of functions that we encounter in normal children but the fact of
being stimulated in a way that respects the hardware of his body will allow some
development of action and perception. The world of the room, including code and
interfaces, will be his world. He will notice regularities in the effects of his actions
(if the aim of the creator is to produce coherence), he will hence be able to
regulate his actions and make plans in his world. If he is not alone and if he is
suitably exposed to a real or virtual adult introducing language, the child will also
have the possibility of sharing his world and communicate with another. These
are basic conditions for objectivity (Davidson, 1984). We can hence assume that
- whatever the condition that occupies the top of the causal chain (atoms or
digital code) - a world in the sense of an objective pole can be created which is
If it is indifferent whether we place material atoms or digital code at the beginning
of the causal chain, material components that are part of the real world are
nevertheless necessary in order to vehicle information to the sensory systems.
Since animals (and humans) have evolved in a material world, their senses are
adapted to respond to material stimulations from the real world, whatever their
form at the top of the causal chain. The real, material world is hence present in
any virtual experience if we want to vehicle coded information.
Even if the top of the causal chain is not a material world, the interfaces are
material, because we are dealing with a bodily entity which presents some
specific characteristics that have been determined by a long run of interaction
(selection) with a material world. Even in the case of the famous brain in a vat
(Dennett, 1991, 1978; Putnam, 1982), material stimulations and devices for
stimulation are necessary in order to produce the desired effect. The difference
with haptic, video and sound interfaces is constituted by the point of the nerve to
which they convey their stimulation.
In spite of the possibility of disjointing objectivity and reality adults experiencing
virtual worlds are not fooled by virtual worlds at the point of taking them for real
(Stoffregen, at al, 1997, 2003). As shown by the mental experiment of the child
grown in a digital code world, the problem is not represented by the origin in the
causal chain but by perceptual and cognitive considerations that are related to
the development of sensorimotor connections and beliefs and relative
expectations. Normal adult human beings come to the experience with virtual
worlds charged of expectations based on their past experiences. They can
compare the overall perception produced by non-mediated and mediated
experiences and on this basis they distinguish between the two forms of
experience (Stoffregen, et al., 1997, 2003). Additional; cognitive considerations
(the awareness of wearing some display, of being in a laboratory) can barely be
The objection could be raised that illusions resist cognition. The objection could
be retorted and it could be affirmed that it is because illusions resist to knowledge
that movies and virtual reality cannot be considered as illusions, because the
difference of the reactions provoked by the real world and the reactions provoked
by fiction worlds testifies that cognition has an effect. In fact, behavioral
responses to virtual worlds and to work of fiction in general, such as movies, are
never exactly the same (as testified by the debate around the paradox of fiction).
Illusion of reality. Cognitive issues
Whether virtual reality can be considered as a fictional medium (a medium that
has the function o representing fictional truths), theories of fiction and of how
fiction works in generating behavioral and emotional responses can help
understanding experiences in virtual reality and how they should be designed in
order to produce the desired effects in users.
Another related domain of studies which can help individuating the relevant
perceptual and cognitive problems of designing experiences in virtual reality
consists in recent film studies, film studies that range into the cognitive theory of
cinema. As virtual reality, cinema is in fact a special fictional medium: a
perceptual, multisensory medium (at the opposite of virtual reality, cinema is a
passive medium, since spectators do not interact with representations as they
can do in the case of virtual reality).
The cited domains try to answer to the following questions:
1. Why movies and other works of fiction, such as books, radio dramas, plays,
seem so real? Why are we moved?
2. Why movies (and fiction in general) are acceptable and have success in spite
of the differences between the physical structure of the world and the physical
structure of movies (representations of fiction)? What happens when we look at a
movie (experience fiction)?
These questions are expressed in particular in the framework of the debates
about the so called “ paradox of fiction”, fiction and imagination, and about the
nature of cinema and other fictional media.
The paradox of fiction
The paradox of fiction is a classic paradox discussed by philosophers of art and
introduced for the first time by Radford, 1975. The debate shows a substantial
accord on the fact that audiences and readers do not hold existence beliefs
about fictional contents, as testified by the fact that responses to fictional
contents are similar but never identical to responses to corresponding
nonfictional contents. The difference in reactions testifies that audiences and
readers are aware of the fictional nature of the experience, hence that they do
not have existence beliefs when they experience fictional contents (Radford,
1975; Stoffregen, 1997; Walton, 1978, 2001; Currie, 1990).
The paradox is produced by three apparently inconsistent and plausible tenets:
Premise 1: emotional responses to characters and events require us to believe in
the existence of the events or characters
Premise 2: no existence belief is present in the case of the fruition of fiction
fictional events and characters)
Premise 3: we deploy emotional responses in presence of fictional events and
Conclusion: the emotional response to fictional characters and events is
irrational, incoherent, and inconsistent (Radford, 1975).
Radford, 1975 justifies Premise 1 on the basis of the observation that in non-fictional experiences
people that discover that a heart-wrenching news was just a lie are no more heart-wrenched.
Radford’s defense of Premise 2 is a statement against the so-called illusion of reality, the idea
that audience can forget the fictional nature of the experience and take it for real. As a matter of
fact, Radford states, emotional and behavioral responses to fictional events are not the same as
emotional and behavioral responses to real events: the vision of horror movies is associated with
pleasure and spectators do not try to do something for saving their hero.
Premise 3 indirectly affirms that emotional reactions are directly provoked by fictional characters
Three basic strategies have been advanced in order to solve the paradox that
consist in denying one of the premises (Carroll, 1990). Denials of Premise 1 and
of Premise 3 give rise to Thought theories and to Pretend theories respectively,
according to which the fact of believing in the existence of the entities and events
that are only depicted is not a necessary condition for raising emotional
responses. In other words, fictional characters can raise emotional responses
even if they are not taken for real. According to Thought theorists it is sufficient
that certain events are mentally represented, or entertained in thought in order to
raise emotional reactions (Carroll, 1990; Lamarque, 1981; Smith, 1995),
eventually by evoking thoughts about real people and events (Counterpart
theory, Carroll, 1990). The Pretend theory (Walton, 1978, 2001; Currie, 1995)
suggests that what is experienced in fiction is quasi-emotions, that is emotions
that are produced by the fact of pretending. The Illusion theory stems from the
denial of Premise 2 and is defended by Anderson, 1996 and Tan, 1996. Even in
this case, nevertheless, the evidence of differences in reactions makes it difficult
to defend the existence of a cognitive illusion of reality.
It is then useful to distinguish, as Currie, 1995 does, between perceptual and
cognitive illusions in cinema. The term “perceptual illusions” is referred to the
existence of illusions of movement and other visual phenomena in the perception
of cinema that are traditionally treated as illusions (for instance by Anderson,
1996. For a different view: but not by Currie, 1995). Both Anderson and Tan
defend a perceptual illusion theory but are more shaded concerning the cognitive
illusion theory and acknowledge that spectators are aware of the fictional nature
of films. According to Anderson spectators experience a paradox between the
awareness of depiction and the belief in the depicted facts. In a similar vein, Tan
considers belief in reality a matter of degree and suggests that spectators are
simultaneously aware of both the real and constructed nature of film events
because they have at least some belief on the reality of film contents.
Three basic strategies have been advanced in order to solve the paradox: denial of Premise 1,
denial of Premise 2, denial of Premise 3.
o Strategy 1: denial of Premise 1.
Emotional and behavioral responses to work of fictions and to the real world are different.
Emotional responses do not need existence beliefs in order to arise, it is sufficient that certain
events are mentally represented, or entertained in thought in order to raise emotional reactions
(Carroll, 1990; Lamarque, 1981; Smith, 1995). Carroll, 1990 defines this view : Thought theory.
The fact that emotions are raised by thought rather than by real events explains the difference in
the phenomenology of emotions raised by fiction and emotions raised by reality. Audiences and
readers know that what they are seeing or reading is not true of the real world and but that it is
just a representation, then it is not necessary to call the police when seeing a horror movie.
The Counterpart theory states in particular that fiction provokes thoughts about real people and
events and that these people and events are the intentional contents of the emotions in fictional
experiences, not fictional contents directly (Currie, 1990).
o Strategy 2: denial of Premise 2.
Existence beliefs are present in the case of fiction in the sense that audiences and readers
somehow take fiction for reality.
This option (named the illusion theory by Carroll, 1990) is strongly rejected by Radford, 1975,
Walton, 1978, 2001, Currie, 1990 on the basis of evidence about the difference between
reactions to fiction and reactions to real events. The awareness of the fictional nature of the
experience is related to cognitive considerations on the side of audiences. No consideration is
devoted to perceptual awareness of mediation.
The illusion theory is defended by Anderson, 1996 and Tan, 1996. It is useful to distinguish, as
Currie, 1995, between perceptual and cognitive illusions in cinema. In fact, both Anderson and
Tan defend a perceptual illusion theory but are more shaded concerning the cognitive illusion
theory. The term “perceptual illusions” makes reference to illusions of movement and other visual
phenomena in the perception of cinema that are traditionally treated as illusions (for instance by
Anderson, 1996 but not by Currie, 1995). The term “cognitive illusions” makes reference to the
idea of illusion of reality, the idea that the spectator takes the fictional characters and events for
true. As a matter of fact, according to Currie, 1995 no illusion of reality occurs, but spectators’
emotional responses can be explained by referring to the function of simulation. Human minds
simulate other minds and this fact provides an adaptive advantage. Cinema triggers the function
of simulation. In what concerns the cognitive illusion of reality, both Anderson and Tan
acknowledge that spectators are aware of the fictional nature of films. Nevertheless, according to
Anderson spectators live a paradox between the awareness of depiction and the belief in the
depicted facts. In a similar vein, Tan considers belief in reality a matter of degree and suggests
that spectators are simultaneously aware of both the real and constructed nature of film events
because they have at least some belief on the reality of film contents. In this sense film creates a
benign illusion and not a delusion. Tan, 1995 describes cognitive illusions as illusion of diegetic
effect because the spectator has the illusion of being present in the film environment (in analogy
with Allen, 196 projective illusion).
o Strategy 3: denial of Premise 3.
As a matter of fact, Radford acknowledges that reactions to works of fictional are not the same
than reactions to real events in his defense of Premise 2. The difference in reactions testifies in
fact for Radford that audiences and readers are aware of the fictional nature of the experience
and that they do not have existence beliefs.
Walton, 1978, 2001 affirms that what is experienced in fiction is quasi-emotions (pretend theory).
Quasi-emotions are not provoked by existence beliefs (acceptance of Premise 1 that only
existence beliefs provoke real emotions) but by beliefs that are related to an attitude of make
believe based on the consideration of what is fictionally true according to the work of fiction, and
not true in the real world. In short, in analogy with children make-believe games, the fruition of
works of fiction makes people pretend to take some events of the work of fiction for true and also
pretend to feel related emotions. As in children playing make believe games, audiences and
readers are aware that it is only a game and they are not really afraid, for instance when seeing a
horror movie, but only quasi-afraid. Carroll, 1990 defines this view the pretend theory and objects
to Walton and other representatives of the Pretend theory, such as Currie, that in the case of
fiction audiences do not choose of pretending or not pretending: they are compelled to feel fear
by certain spectacles and not by others. By this fact, reactions to fiction cannot be considered in
analogy to make believe games. Additionally, spectators are not aware of having the intention to
pretend. As Carroll, Hartz, 1999 and Novitz, 1987 object to Walton that the phenomenology of the
reactions to fiction is no the same than the phenomenology of make believe games.
Against these objections, Neill 1991 underlies that in Walton’s pretend theory audiences do not
pretend to have feelings, but only that the experienced feeling is fear.
Other objections concern quasi-emotions as not necessary constructs: on one side, reactions to
fiction include reactions that do not depend upon the existence of beliefs (such as startle
reactions provoked by visual stimuli), and in the other side reactions could be provoked by
unconscious beliefs activated by visual stimuli (Hartz, 1999).
Whatever the strategy for dealing with the problem of emotional reactions to
works of fiction, Premise 2 is widely adopted. In other words, it is widely accepted
in the philosophy of arts that reactions to works of fiction are not the same than
reactions in presence of real events corresponding to the ones depicted in the
work of fiction, and that readers and audiences are aware of the special nature of
the experience which is mediated by fictional media: an experience with fictional
worlds, characters and events.
We can conclude that in the domain of film theory the notion of illusion of reality
as a massive cognitive illusion is largely refused on the basis of the evidence of
the difference in reactions towards fictional worlds and the real world. Audiences’
behavior and emotions are hence appropriate both to the contents of the
experience with fiction (for instance being afraid when viewing a horror movie)
and also to the context of the experience (the fact of fiction and the type of
medium through which it is presented). This double appropriateness explains
both the similarity with the phenomenology of emotions and behaviors raised by
events of the real world and events depicted in the fictional world and their
difference. The responsibility for similarity can be assigned to a similarity
between the contents of the real world and the contents of thought. The
responsibility for the difference can be assigned to the cognitive awareness of
the fictional nature of the experience and of the characteristics of the medium
through which the experience is proposed. The awareness of the medium hence
modulates the emotional and behavioral responses.
A hypothesis can be added to this consideration, which fills some gaps both in
the Pretend and in the Thought theories: the appropriate responses hypothesis.
According to this hypothesis, audiences and readers reactions are both adequate
to the contents and the context of the experience, hence to the contents that are
depicted in the fictional world (contents that are suitable for provoking fear in the
interaction with the real world) and to the context (a fictional and not real world,
presented by the mean of a specific medium which allows certain reactions and
not others). Audiences and readers are hence aware of the fictional nature of the
experience and they react appropriately, as children do in make-believe games.
This explains both the similarity with the phenomenology of emotions and
behaviors raised by events of the real world and events depicted in the fictional
world and their difference. The responsibility for similarity can be assigned to a
similarity between the contents of the real world and the contents of thought. The
responsibility for the difference implies the awareness on the side of audiences
and readers of the fictional nature of the experience and of the characteristics of
the medium through which the experience is proposed.
The appropriate responses hypothesis also accounts for differences in the
perceptual nature of fictional experiences in relation to the perceptual nature of
experiences with the real world. Differencing between fictional and real worlds is
not only a matter of representation, thought and cognitive awareness, but also of
different specification in the stimulus condition. This is the reason why a dog, a
non-thinking organism, does not bark to dogs seen at the screen, but barks to
real dogs and to dogs in a theater play.
A corollary of the appropriate responses hypothesis can explain why the
following two situations raise emotions with different phenomenology: first
situation, one imagines Jason Vorhees penetrating the room. In this case no fear
is resented even if one imagines himself in a dangerous situation. Second
situation: one sees Jason Vorhees penetrating the room of some characters in
Friday the 13th and is scared and pleased in the same time. Something more
than mere imagination must be considered in order to explain the presence of
emotional reactions. The corollary states that the fictional world and the
experience must be believable, so as to engage audiences and readers to deploy
appropriate reactions as it happens in children games of make believe, where the
game is not just imagined but taken quite seriously.
It is not necessary to go as far as to suggest that in the second situation one
takes the fiction for true of the real world. If this was the case, emotions and
behaviors would be the same as for real events, which is not the case. As stated
by the pretend theory and by Radford, 1974 in Premise 2, fictions raises
emotions, but these emotions are not the same than emotions raised by real
Another corollary of the appropriate responses hypothesis consists in the idea
that what is raised by fictional experiences is emotions that are modulated
(amputated in some of their components or modified) in reason of the
characteristics of the context. Some aspects of the emotion of fear and relative
behaviors (call the police, escape from the theater) would be simply modulated or
even stopped by the awareness of the fact that this is only a game. The analogy
with make-believe games still holds, since once a child has chosen to play a fear-
game he cannot really decide to feel fearful or to stop feeling fearful.
Nevertheless, the recourse to the postulation of new mental entities such as
quasi-emotions is no more necessary.
As in the case of Pretend theories, in the appropriate responses hypothesis the
analogy with children make believe games makes certain aspects of adult’s
interaction with fiction quite perspicuous: the fictional world is not taken for real,
but it is taken seriously enough to justify the presence of behavioral responses
that are appropriate to the contents of the world and to the context, or the
material means through which the fictional world is made present (mud for mud
pies, cinema for movies, virtual reality systems for virtual worlds).
Conversely to the Pretend theory, nevertheless, the appropriate responses
hypothesis does not imply that adults pretend their emotions: adults can resent
true emotions that are modulated in relationship to the contents and to the
context of the experience. The same is true for children playing make believe
The appropriate responses hypothesis is compatible with the two Premises of the
Paradox of fiction that are criticized by Thought and Pretend theories, since it is
compatible with the assertion that only existence beliefs can raise complete
emotional responses, non modulated by the context, and it is compatible with the
assertion that we are moved, even if emotions are not the same than the ones
deployed with the real world. Premise 2 is a necessary condition for the
appropriate responses hypothesis.
A mean for testing the hypothesis of the double adequacy to contents and
context consists in comparing reactions to real events, corresponding fictional
events mediated by a certain medium (such as video) and fictional events
mediated by another type of medium (such as interactive VR systems). It can be
hypothesized that the three conditions will differ at least for the perceptual and
motor responses that will be elicited. The difference in perceptual and motor
behaviors cannot be ascribed to the contents of the experience, but only to its
context consisting in the perceptual and motor possibilities that are made
available by the medium.
As a matter of fact, new interactive media constitute a great opportunity for
understanding the problems related to the Paradox of fiction, and in particular for
the appropriate responses hypothesis. Interactive intersensory virtual reality
systems put users in the condition of manifesting multiple motor and perceptual
responses in addition to emotional behaviors typical of passive media. Virtual
reality systems emphasize the difference in reactions deployed with different
media possibilities and stress the analogy with make believe games. As in the
case of make believe games, virtual reality users deploy actions that take in
account both the contents of the proposed experience and the motor possibilities
that are allowed by the particular device which mediates the interaction. Virtual
reality users are in fact called to act, and not only to perceive and resent
emotions. The appropriate responses hypothesis aims at going beyond the
explanation of the emotional behaviors raised by traditional media by considering
these behaviors as part of the cognitive, emotional, perceptual and motor
responses that are raised by fiction and by considering fictional worlds as
composed of contents and contexts that both count for explaining the type of
See: Believability & Expectations
Fiction and imagination
Walton  sustains that fiction and representation can be understood by taking recourse to
the activity of imagination as it is exemplified in children make believe games. The notion and
activity of make believe hence constitute the core of Walton approach to the explanation of both
the importance of representative or fictional art in our lives (aesthetic point of view) and of the
ontology (mental and non-mental) which is implied by the fruition of this kind of human products
(metaphysical point of view).
“The experience of being “caught up in a story,” emotionally involved in the world of a novel or
play or painting, is central in the appreciation of much fiction, and explaining the nature of this
experience is an important task for the aesthetician. It is extraordinarily tempting to suppose that
when one is caught up in a story, one loses touch with reality, temporarily, and actually believes
in fiction. The reader of Anna Karenina abandons himself to the novel and is convinced,
momentarily and partially at least, of Anna’s existence and of the truth of what the novel says
about her. Otherwise why would he be moved by her predicament? Why would one be even
interested enough to bother reading the novel? Yet it also seems that the normal appreciator
does not (of course!) really believe in the fiction. [Walton, 2001, p. 6]
Imagination is enriched and sustained by the presence of real things that work as props. Real
things can cover different functions in imagination: prompting imagination (natural or artificial
things can induce people to imagine), constituting objects of imagination (in presence of a real
think which stands for something else, one does not simply imagine this something else, but
imagines the real thing, the prop, to be something else) and creating fictional truths (a thing that
has been declared in a certain game of make-believe to stand for something else, makes it
fictionally rue that this something is present else each time the thing is present, even if the thing is
not actually perceived and the something else is not actually imagined). Natural things can evoke
imagining but that artificial things are more suitable for prompting imagination towards
Props + principles of generation of rules (in the present game, a certain thing/prop stands for a
certain something else) generate fictional truths, even in absence of imagination. However, those
who do not imagine in presence of the prop are not playing the game, since they are not following
the rule. Fictionality and fictional propositions are hence, in a certain sense, prescriptions to start
a certain activity of imagination.
“[…] representational works of art generally are props in a game of make-believe” [p. 51]
Even in absence of props, of material things (such as it is the case for daydreams), fiction is not
equivalent to imagining and viceversa, since fiction can exist without imagining and imagining
without fiction. Fiction is what is to be imagined, the prescription and rule to imagine. Hence
fiction exists only when a rule is created that contains a prescription to imagine.
Works of art such as paintings, books, films are special props because they are intended to be
so. Representational works of art are hence intended and have the function of functioning as
props in games of make believe. One can not play the game and rather analyze a work of art with
the intent of understanding the author, etc.
Since works of art have this function it is not necessary to re-establish the rule in front of any work
Abstract paintings and works of art are different from figurative art because they are props for an
activity of imagining which is internal to the work of art, while figurative art evokes imagining that
goes beyond the work of art and points to people and things (even fictional ones) that are
different from the work of art.
“Representations are things […] possessing the social function of serving as props in games of
make-believe, although they also prompt imaginings and are sometimes objects of them as well.
A prop is something which, by virtue of conditional principles of generation, mandates imaginings.
Propositions whose imaginings are mandated are fictional, and the fact that a given proposition is
fictional is a fictional truth. Fictional worlds are associated with collections of fictional truths; what
is fictional is fictional in a given world – the world of a game of make-believe, for example, or that
of a representational work of art.” [p. 69]
Currie, 2005 debates Walton’s idea that imagination is a response to the fictional and that
imagination is to be understood as a form of make-believe which is prompted by the presence of
Currie acknowledges that fictional works are guides to imagining and that the guided imaginative
exploration of a work of fiction whatever deserves to be called a game of make-believe, but he
objects to Walton that the role of props is more controversial: in narrative, non-perceptual fiction
props cannot be individuated Props seem to be bound to perceptual media.
Currie asks which kind of imagination is in cause in the case of fiction, for three main forms of
imagination exist according to him: the capacity to form mental images (imagery for different
modalities, such as motor imagery), the capacity to consider different alternatives and make plans
for contingencies (a form of propositional imagining), the capacity of seeing things that are not
According to Currie imagery is not a necessary response to fiction, even if it can occur. In
particular, in theater and film, visual and auditory imagery pays no role because one has direct
access to relevant images and sounds. Vision and visual imagery are mutually exclusive, and so
Currie suggests that the appropriate form of imagination is propositional imagining, and in
particular a sub-class of propositional imagining that has believe-like features. In fact,
propositional imagination exploits knowledge and inferences that are analogous to the ones that
can be drawn from belief. Analogous but not identical, because in the case of fiction certain
events (that would appear as bizarre in reality) can be accepted in virtue of the consideration of
reasons of genre, style, and in general in virtue of the consideration of the fictional nature of the
experience. For this reason, Currie defines his approach to fiction: simulation-inspired theory of
fiction. Simulation theory of fiction is accepted by Nichols and Stich, 2003 and finds evidence in
the study of children make-believe games (Harris, 2000).
Currie underlines that theories of pretence cannot rely only on beliefs but also on motivation and
desire (as expectations).
Walton has suggested that what is involved in imagination, hence in fiction, is not real emotions
but quasi-emotions. Currie & Ravenscroft, 2002 and Lamarque, 1981 for instance object that
fiction raises true emotions.
The nature of cinema
Cinema presents us with different forms of imagination. Bordwell, 1985 describes
audiences as continuously formulating hypotheses and expectations about the
past and present. This activity has the effect of constructing a narration. It hence
serves a different purpose than the activity of imagination which audiences
perform when they make-believe.
In the case of perceptual media, and in particular in the case of multisensory
media, the activity in one sensory modality activates other sensory modalities. It
is not sure that what happens even in these cases is a form of imagery, in the
sense that it is not sure that representations are necessarily involved. It could
simply be the case that connections between sensory modalities are activated
and that this activation does not give rise to representational activity. This would
not be a form of imagination, then. But expectations of a perceptual and motor
nature are raised, however. Even if vision and visual imagery are mutual
exclusive, vision can raise imagery or at least expectations in other sensory
Perceptual cues as prescriptions for imagination
When one sees a movie or interacts with a virtual environment he is not simply
imaging, he is also perceiving. Walton acknowledges the role of perception
through the acknowledgment of the role of real, material things in imagination.
In the case of cinema, spectators perceive real things: moving images and
sounds projected on a screen and released in a theater. These real things are in
the same time props for imagination.
Different sensory modalities might present a different relation with imagination.
Think to sound effects in cinema, that can evoke the image of objects different
from the ones used for producing the sound effect itself. Or to haptic sensations
obtained through force-feedback.
In any case, the experience is so construed as to prompt imagination, both in the
sense of hinting and directing hypotheses and in the sense of evoking the
imagination of perceived objects.
The screen and theater are reminders or generators of a general rule. This
general rule is not necessarily the following one: “imagine that the images and
sounds you perceive are not representations but are people, things, spaces that
look and sound that way”. This rule seems to be specific of a certain style of
filming and of imaging, the so-called Hollywood style (Bordwell, 1985). Other
styles are possible and actually exist, such as film art, where the spectator is kept
aware of the fictional nature of the spectacle through various hints (the
manipulation of the relationship of images and sound, the interrogation of the
spectator by the actor). The frame (the screen, the theater) enounces a very
general rule: ‘follow the rules that the spectacle represents in an indirect way and
that you might know by direct declarations of the authors, other direct information
that you gathered before the spectacle begins or indirect information you
gathered from long acquaintance with this type of spectacle. Direct and indirect
information (even very general information gathered from the experience with the
real world) creates expectations that are employed by spectators in order to
understand a movie, in the sense of re-constructing the narration by perceptual
and cognitive means (Bordwell, 1985).
Realism can be understood as a term which identifies a certain group of
styles/pescriptions for imagination. One prescription can be: “imagine that you do
not see pictures of people but real people”, “imagine that you do not see
drawings of people but real entities with that funny aspect”. Or maybe: “imagine
what it would be like if these pictures or drawing were real, if what you do
perceive could happen (have happened, actually happens, will happen). This
prescription can be identified with the objective of creating the so-called illusion
of reality, in the sense that spectators are asked to imagine that what they
perceive is not a representation but real, material things, or better: what it would
be like if what they perceive occurred of entities of the real world. This rule works
with different styles of production and different levels of realism in the sense of
the similarity with real things: analogic recording of real actors, digital
masterization with special effects, drawing with different levels of caricature. In
the case of cartoons, the techniques of simplification and exaggeration can in
some cases indicate to the spectator not to focus on the apparent nature of the
represented character (a baby animal who looses his mother) but on the
emotions and psychological states he lives (Thomas & Johnston, 1985). What is
prescribed is hence to imagine the sorrows and pain which is represented and to
imagine what it would be like if someone, some human being, would undergo
similar events and analogous pains. A different prescription states: “Imagine that
the images you see are not the outcome of montage and camera views and
setting and acting, but that we have filmed scenes of real life”. Let us add another
instruction: “this is not a simple documentary of something that happened in the
past, but a record of something which is happening right now”. Are there any
ethical problems with these instructions? The answer is no, because they are
instructions to imagine, instructions to play a game. Ethical problems arise only if
the make-believe nature of the game is hidden. It is not the type of instruction
that represents a problem for the ethics of the spectacle, it is the reminders of the
fact of playing a game with certain rules (the screen, the theater). Ethical
problems arise when, for different reasons, the fictional or gaming nature of the
activity is made uncertain. It would have been the case for Orson Welles’
performance of the “War of the worlds” (1938) thank to the scarce use of 1938
audiences to broadcasts featuring apparently real news bulletins, if it wasn’t for
quite constant reminders to the reality of actor Orson Welles playing a H. G.
Welles’s famous novel. It has been the case for the broadcast Alternative 3
diffused by UK Anglia Television, where the abandon of Earth is represented as
a real project in response to pressing planetary dangers. In this case in fact, no
reminder is made of the fictional nature of the images that are presented in the
framework of a well-known documentary, with no use of actors for presenting
them but of the documentarists that have presented all the previous
appointments of the broadcast.
As in the case of cinema, in the case of virtual reality and interactive interfaces,
the screen, or whatever other device that works as interface for presenting the
user with perceptual stimuli and symbolic information, is a reminder or generator
of general prescriptions. The type of general prescriptions depend on the type of
medium: “imagine what it is like to interact with an object that corresponds to the
imagine you are seeing, to the sound you are hearing, or to the haptic sensation
you are experiencing, and do it, following the limits prescribed by the devices you
are wearing. Remember that these devices allow you to perform the x…y set of
Props can be represented by images, sounds, haptic stimuli, depending on the
specific characteristics of the interface.
What is important in the design of virtual reality and other interactive interfaces is
both to design efficient props and to clearly express the rules. Rules include the
prescriptions and the possibilities of interaction that are given to the user. It is
rare that the user can perform any action he is used to perform in the real world.
The limits and left possibilities must be clear in order to give proper course to
imagination and relative action. Prescriptions too must be clear: “Imagine what it
would be like if this was a surgical instrument that cuts in this way but not in that
one and if you were to operate a heart which behaves this and that way”. As it
can be evinced from the example, prescriptions include the aims of the
application (an application for virtual surgery) and the characteristics of the
device, in particular in relation to the differences between the virtual device and
real devices. In fact, surgeons can have expectations relative to surgical
instruments that are not fully respected by the virtual device.
The same thing is true of children games of make believe, when sand stands for
flour and sand-cakes must be carefully cut into slices. In presence of solid
motivations, big efforts can be made for adjusting expectations gathered from the
experience with real things to the characteristics of the props one has at disposal
for playing his game of make-believe. Things can be more difficult when users of
interactive media are asked to act in relation to props, in addition to imagine. In
this case, props must comply to higher requirements than props for children
make-believe games, such as sand for cakes. The way for specifying these
requirements is indicated by the aim of the application and the characteristics of
the interface and that are constrained by the laws that relate perception to action
in normal circumstances. I.e., an instrument for virtual surgery will present stimuli
that are suitable for accomplishing the desired/possible set of surgical actions
and that do not provoke major disturbances in the performance. Ideally, the
stimuli should be designed by a team composed of a virtual reality designer, a
surgeon (for detailing the necessary actions), and an expert in action-perception
(for individuating perceptual disturbances and perceptual enhancements for
proper actions). But another exert is necessary in order to warrant that the rules
are clearly communicated to the user, by symbolic or perceptual means.
It might seem that the case of perceptual and interactive media is different from
cinema, radio and other perceptive but non-interactive media, because
interactive media prescribe users not only to imagine, but also to materially act
through movement. It should be remembered that this is also the case for
children make-believe games. If make-believe games are the reference for
understanding the functioning of fictional media, including interactive media such
as virtual reality systems, then cinema and other passive media, with their
interdict to act except to the rule of make-believe games: imaging and acting in
Imagination and different forms of expectations
There are at least two different imaginative functions in the fruition of
representations. Perceptual hints are present in both forms of imagination, but
have different effects.
The first form of imagination is related to the functions of making hypotheses.
Bordwell, 1985 states that cinema and other narrative media users actively
reconstruct the narration by formulating hypotheses about the future
(expectations, previsions) and about the fact (motivations, causes). This
“imaginative” activity is sustained and directed by the hints that are mostly
intentionally provided by the creators of the experience, through language and
In the case of perceptual media (and at the opposite of non-perceptual media,
such as narrative books) imagination is hence sustained and directed by
perception. In the case of haptic feedback in virtual reality, once the object has
been identified as a square-shaped one, the user will more or less consciously
make previsions about the fact that a certain movement will produce a related
sensation (sensorimotor expectation). We can consider sensorimotor and other
forms of expectations and previsions sub-classes of the imaginative function.
Another imaginative function is to evoke other, associated sensations or
representations. Walton asserts that natural things can evoke imagining but that
artificial things are more suitable for prompting imagination towards
predetermined directions. In this case, representations are supposed to evoke
past sensations and the image or internal representation of perceived objects.
The fact of evoking objects perceived in the past can be considered as a
“realistic” effect. This effect of imagination and association is different from an
illusion of reality, because the representation is correctly perceived as a material
device (a filmic image, a picture, a mechanic arm) with proper perceptual
characteristics and because the user is aware of the fact that he is in presence of
a material device, a medium that proposes a certain experience. The experience
can prompt imagination also in directions that are different from past experienced
objects. This might be the case for abstract representations evoking emotions
rather than perceptual representations.
But also for representations that provide sensations never experienced before.
In the specific case of artificial things that intentionally evoke imagination, the
accord between the imaginative activity which is evoked and the imaginative
activity intended to evoke by the artificial representation is indicative that the
artificial representation works or not as intended. The past experience evoked
can be as general as the one of a square object, or of an object in general, or as
specific as the experience of a pair of dice. The measure of the efficiency of the
device in this evocative function can be refined by including the degree of
certainty and the lapse of time which is necessary in order to evoke the desired
Cinema and illusion
The constructive nature of narration
The mesh with cognitive and perceptual mechanisms
Cinema and illusion
Currie, 1995 defines cinema as a representational medium which essence is visual (because
other sensory modalities are not as necessary as vision) and which images can perform two
(representing) functions: photographic function (as in a documentary) and fictional function (a
fictional story is presented). In the second case, the cinematic images function pictorially or non-
pictorially for informing spectators about what is true in the fiction (images pictorially represent a
certain real actor, but they function pictorially in presenting fiction since non real things cannot be
pictorially represented). The fictional character of a film depends on the intention of the film-
maker of making spectators imagining certain things. The photographic and the fictional function
cannot be conflated because in the fiction film what is intended to be represented is not real, it is
Realism in cinema: cinema is claimed to be a realistic medium for three different reasons:
- transparency: a film is not a mere representation of the world but a reproduction of the
- likeness: the experience of a film is like the experience of the real world
- illusionism: films engender illusions of reality (“illusion of the presentness and reality of
the fictional events”, p. 21 or illusion of movement). This is a widely diffused view about
- social realism: the film portrays significant social interactions
Illusionism: Currie distinguishes between two types of illusionism.
- cognitive illusionism: “The claim that film causes cognitive illusions is this: film
watching, in some systematic way, and as part of the normal process of the viewer’s
engagement, causes the viewer to have the false belief that the fictional characters and
events represented are real.” (p. 22) Currie supports this view with quotations from film
critics. Two versions exist.
o Weak version of cognitive illusionism: the film is taken for the representation of
real events (as a documentary)
o Strong version: the film makes the viewer belief that he is actually watching
real events. Even when this view is not explicitly asserted by its supporters, it is
implicit in the identification of the camera with the viewer’s eyes. This version is
prevailing. It is opened to two objections:
Reactions of film viewers are different from reactions of people who
believe in the reality of the depicted events [Walton, 2001]. Evidence is
Whether it was asserted that viewers’ beliefs are of lower
intensity or partial, this move wouldn’t solve the difficulties with
the illusionist view, since viewers do not behave as someone
who suspects that what he sees is true
The viewer has not the same point of view of the camera, hence there
is no psychological identification
- perceptual illusionism: perceptual illusions are produced by the vision of films that are
not necessarily entertained at the cognitive level (the coexistence of perceptual illusion
and cognitive awareness being permitted by cognitive impenetrability). In Currie
examples of cognitive vs. perceptual illusion, however, the Mueller-Lyer illusion can be
both a cognitive illusion whether the perceiver is not informed about the real length of
the lines, or a perceptual phenomenon, whether the perceiver is cognitively aware of
the discrepancy. An example of perceptual illusion is the illusion of movement: the idea
that static images are perceived as moving by the viewer and hence that cinematic
motion is an illusion. Currie denies that there is such an illusion. Whether for a thing
movement is to be in a certain place at a certain moment and in another place at
another moment, then images (the image of a man) on the screen actually move from
one point to one other of the screen. It is hence correct to perceive cinematic images
as moving images. This characterization of movement requires re-identification. Re-
identification does not depend on the sameness of the causal antecedent of the image
(re-identification is possible even with cartoon images) but is a response-dependent
characteristic of entities that, in normal conditions, evoke a re-identification response in
perceivers (response-dependence is characteristic of secondary qualities such as
colors). The refusal of the idea that cinematic movement is illusory is thus based upon
the characterization of movement as a response-dependent property of things. Once
accepted a response-dependent characterization of movement, cinematic movement
cannot be illusory because images are perceived as moving by normal viewers in
normal conditions: for response-dependent properties, the possibility of error is simply
ruled out, and the perceiver is always correct. It can be objected that any illusion and
the very notion of illusion itself can be ruled out by characterizing any object property in
response-dependent terms: whether length is not a metric, intrinsic property but an
extrinsic property of being experienced as longer than, even the Mueller-Lyer illusion is
no more an illusion. Currie answers this objection since he is not disposed (as
ecologists) to abandon the notion of illusion for Mueller-Lyer-like phenomena. He hence
states that in the case of the Mueller-Lyer figure, the illusion suggests that whether the
perceiver was to measure in traditional metrics the line he would find a certain length
which is not confirmed by actually measuring them. In the case of cinematic movement,
no check can deny that the images of re-identifiable objects that are seen on the
screen are moving from one point to one other of the screen because where the image
is seen on the screen there is illumination. In other words, illusions do not resist
independent checks, while cinematic motion does. This answer partially corrects the
purely response-dependent view of movement and introduces a specific feature of
cinematic movement, because it grounds cinematic movement (and not movement in
general) on the presence of illumination on the screen. At the opposite of cinematic
motion, for instance, the apparent movement of light points in the dark is an illusion,
because it does not resist independent check: no illumination can be found where the
image is seen. Whether response-dependent or not, cinematic movement (and not of
movement in general) depends on the real presence of illumination at some point. As
movement in general, cinematic movement is detected when an object is re-identified
in a different place. But this response would not be enough to rule out the possibility of
error, hence of illusion, as shown by other illusions of movement. The condition for
genuine movement is that not only the image of an object is re-identified at different
places on the screen, but that illumination is really at that place on the screen.
o The view of illusions that makes reference to the effects of the recourse to
independent check, shares analogies with a coherentist view of illusions in the
sense that illusions are errors as inconsistencies between two or more
synchronic or even diachronic perceptual events or between a perceptual
event and a cognitive information that concern objects that are identified as
one and the same by the subject of the illusion.
Transparency: transparency is a modified version of the presentation thesis.
- The presentation thesis claims that photographs and films present the world rather than
representing it. Even if this was true, films would present viewers with the real world of
actors, sets, etc., but could not present fictional characters and events that do not really
exist. “Representations extend our epistemic access to things in the worlds; if they are
reliable, representations give us information about things when those things are not
directly accessible to us.” (p. 49). Representations are different from devices that help
the access to details of things (such as lenses) because the last ones help seeing the
things themselves, not accessing information in absence of the thing. They hence
present and not represent.
- The transparency thesis sustained by Walton affirms that seeing a photograph is
different than seeing a painting because a photograph can represent only things that
really exist, while paintings are not limited to the representation of reality and because
when we see a photograph we see the thing through it. Currie acknowledges the
difference between photographs and other images but he ascribe this difference to the
different ways though which the representation is produced, not to the fact that
photographs are not representations. The resemblance between the image and reality
is characterized in terms of counterfactual dependence between the visual properties of
the world and the visual properties of the representation (the same dependence stands
between the visual scene and the perception of the scene). In the case of painting, but
not of photographs, counterfactual dependence is mediated by the beliefs of the artist.
This is because the way photographs are made cannot take into account the way the
world looks like to the artist as ordinary seeing does not take into account others’
beliefs. But seeing a photograph is substantially different than ordinary seeing. One
reason is that seeing is perspectival and provides egocentric inforla
- The transparency thesis sustained by Walton
- The constructive nature of narration
Bordwell, 1985 presents a constructivist approach to narration (adoption of constructivist
approach to cognition and perception in general; for perception, he makes reference to the view
represented by Helmholtz, Gregory, Gombrich: perceiving as drawing inference, possibility of
Bordwell’s approach is strongly centered on cognitivism. At two levels: high mental operations
described by cognitivism, such as the use of heuristics, and the perceptual level. For the
perceptual level Bordwell adopts the Gregory-Gombrich view of perception and visual art:
perception is active in the sense that it is an activity drawn upon inferences; perceptual inferences
can be of two types: top-down and bottom-up, in the firs case inferences are not as mandatory as
in the second case, in which little choice is left to the interpretation of the visual cues. In any
case, visual cues are always open to interpretation and the data that are reconstructed through
inference from cues are not the same than the uninterpreted reality. Inferences are drawn from
hypotheses that are more or less enracinated into the structure of the body rather than acquired
and, in accord with Gombrich, the acquisition o a certain type of knowledge and expectations can
be facilitated by the presence of a certain structure (reading perspective is more easily learnt than
reading language). As illusions, visual art draws upon the typical knowledge, expectations and
inferences that are stimulated by certain cues and shared between most of individuals.
In the same time Bordwell insists that cinema (and the interpretation of visual arts in general) is
not only a matter of perception: it is a matter of reconstruction, formulation and validation of
hypotheses both at the perceptual and at the higher cognitive level. The filling in of gaps
constitute a paradigmatic example of the activity of the spectator both at the perceptual (gaps
between shots) and at the higher narrative level, the level at which the spectator reconstructs the
fabula from the syuzet (broadly speaking: the world and events that it is possible to reconstruct
from the represented events and the represented events; when the represented events recount or
find their causes in past events that are not represented, those past events are part of the fabula,
as the characters that are never shown in the syuzet but that can be presumed from it).
While a strong cognitive activity seems to be necessary on the side of the spectator (and
demonstrated by the fact that when recounting a film spectators make reference to events of the
fabula that are not directly represented thus showing their activity of reconstruction of a complex
narrative world), the refusal of ecological, direct theories of perception in favor of the excusive
adoption of classic, cognitivist theories of perception, is justified only as a matter of principle: the
spectator is active.
In the cognitivist approach, expectations play their role only in the disambiguation of the stimulus.
It could be objected that expectations shape or are compared with non-ambiguous stimuli that are
directly picked up not in order to interpret bare cues but in order to perform a continuous control
upon coherence, novelty, discrepancy. In this way epistemic organisms are always ready to learn
the existence of a novel object or to revise their beliefs. When, for some reason, these
possibilities are not at disposal, novelty (the unfulfillment of expectations) can be experienced as
a violation of coherence.
This suggestion requires an appropriate theory of expectations. It seems to go into the direction
of creating bridges between opposed theories of perception by preserving the mechanisms
described by indirect theories of perception and by preserving the direct nature of perceptual
picking up of information. The mechanisms described by indirect theories of perception are
mechanisms of control and not of interpretation. It can be suggested that the existence and
application of control mechanisms is not the same for all creatures in all situations (an
assumption which is proper to indirect theories) but are more present in higher creatures which
perform epistemic activities (they have beliefs) and in the course of epistemic activities. In this
way it is possible to accept the existence of expectations and control mechanisms for coherence
without considering perceptual activity as based upon internal representations.
Bordwell’s description of the activity of spectators seem to be compatible with this modification
of cognitive theories of perception: what counts for it is the role of expectations, not the fact that
perception is indirect and based upon interpretation or reconstruction from ambiguous
information. Interpretation and reconstruction work at a higher level and expectations based upon
knowledge or sensorimotor laws are at work at every level in order to perform a check for
Three existing theories of narration are presented:
- mimetic theories of narration: theories that conceive of narration as a form of showing
(presentation of a spectacle); vision is the model for mimetic theories of narration: a visual object
is presented to the eye of the spectator. Bordwell draws the history of mimetic theories back to
Greek theater (theatron = seeing place; in Greek theater a specific space is marked off as
fictional and specific attention is devoted to the place occupied by the spectator, the eye; at the
opposite of parade, circular staging and other forms of representations). Mimetic theories of
narration are applied even to literature: novel as a picture, problems of staging (in analogy with
theater) and point of view. In cinema (see Pudovkin: Film technique, 1926 for the first formulation)
the mimetic theory states that the story events are presented through the vision of an imaginary,
invisible witness; as the camera moves, the witness shifts his attention from one detail to one
other; the invisible observer is hence incarnated by the camera. The camera can be assumed to
embody the narrator’s or the director’s point of view; on the other hand, the spectator is the ideal
viewer. From the technical point of view, camera and microphones are for instance placed where
the eyes and ears of the ideal possible spectator should be.In any case, the witness is not an
ordinary perceiver, but an ideal observer who can see things that are not normally available in
real life. The theory was initially used to valorize a kind of non-theatrical, distanced way of
narration, a way of narration which is similar to the immersion in the flux of life, that is, a way of
narration that ideally mimics the perceptual experience of the world . Bazin: cinema should
present the spectator with the same field of vision which is granted by the perception of the real
world: long takes, staging in depth; the viewer will operate cuts, as he does during the visual
encounter with the real world. Another aspect which is involved in the mimetic approach and in
the idea of the verisimilitude of perception, consists in the fact that the invisible observer is
supposed to observe an autonomous world, a world that could be known independently from the
film. Bordwell criticizes the mimetic theory because it is unable to account for the staging of action
(the theory only focuses on the camera and on space, with no attention for the disposition of the
elements in the scene previous to the intervention of camera and for the mise en scène). Also,
the mimetic theory takes the verisimilitude of perception as congenital to film, as the basis of style
and narration; while, according to Bordwell, the invisible observer and the verisimilitude of
perception are figures of style, are effects a of a particular style of narration. A related issue
consists in the self-conciousness of the spectator:
- Diegetic theories of narration: theories that conceive of narration as a form of telling. The term
diegesis indicates the fictional world of the story. The main theorists of diegesis are French
structuralists (Barthes, Benveniste, Genette, Metz, …): language is the master system for
understanding all types of expressions and every narration depends upon the construction and
interpretation of codes. Film analysts are hence called to unfold the filmaker’s intentions (the
enunciation) disguised in the story, in the diegesis. As in the case of mimetic theories, the
spectator is the victim of an illusion: the natural appearance of the story, of the diegesis, positions
the spectator in a passive place. Bordwell opposes a constructivist theory of narration where the
spectator actively participates to the construction of the story, is not a passive eye fooled by
perspective, editing, narrative point of view who takes the filmed world for an objective one.
- Constructivist/Cognitivist theory of narration: according to Bordwell, understanding the story
which is narrated is a form of activity and construction in which the spectator is active with both
his perceptual and cognitive systems. A theory of narration is hence grounded in a general theory
of perception and cognition. Bordwell accepts the Constructivist view of classic cognitive sciences
(Gregory, Helmholtz): perceiving is drawing nonconscious inferences based on knowledge and
testing hypotheses. So it is film viewing and story understanding: “The artwork is necessarily
incomplete, needing to be unified and fleshed out by the active participation of the viewer. ” (p. 32)
The artwork acts in two main ways on the perceiver’s activity: it provides hints that trigger
perceptual and cognitive reactions (the application of schemata, the formulation of hypotheses,
inferences, expectations, the revision of hypotheses, inferences, expectations) and hints that
constrain, limit the possible actions of the perceiver. The activity of the spectator is hence the
same which the perceiver deploys in everyday life, in the natural, non-mediated perception of the
world. In particular cinema manipulates perceptual deficiencies and capacities of the visual
system (the incapacity to follow rapidly changing light intensities which gives rise to the
impression of steady light or flicker fusion; the capacity of inferring movement from interrupted
inputs when the gap between them is not too large, which gives rise to apparent motion). Cinema
is hence considered by Borwell as a mean for illusions (in the sense described by Gregory) which
is based on our tendency to draw wrong inferences from perceptual data concerning the real
state of affairs of the world. This is because the artwork is incomplete, it presents gaps where the
inferential capacities (and related possible errors) of the spectator can be exercised, information
is missing. In addition to perceptual capacities, cinema triggers and constrains cognitive activity
based on prior knowledge and experiences. “To make sense of a narrative film, the viewer must
do more than perceive movement, construe images and sounds as presenting a three-
dimensional world, and understand oral or written language. The viewer must take as a cognitive
goal the construction of a more or less intelligible story.” (p. 33) Assigning meaning, constructing
the story implies certain activities. The first one consists in assigning coherence, in constructing
unity: the spectator seeks for elements that can be combined in a coherent unit and on this basis
he formulates hypotheses that are in accord with the unity. The second step consists in the
application of schemata to the artwork; schemata constrain the contents of the hypotheses that
can be emitted on the basis of the re-constructed unity. Schemata can derive from the experience
with the real world, with other artworks and with other films. Schemata constrain the assumptions,
expectations, hypotheses that are formulated step by step by the spectator during a film and that,
step by step are confirmed or disconfirmed by the film. Again, this activity is favored by the
presence of hints, gaps, patterns (in the story, in the way of narration, in the style) that have the
effect of encouraging story-constructing activities. Some classes of schemata can be described
(Hastie): prototype schemata (such as prototypes of lovers and bank robbers that help
understanding individual characters in film like Bonnie and Clyde), template schemata (for
instance, experiments on recalling and re-telling stories indicates a tendency to normalize deviant
stories following certain common parameters: occidental viewers of films tend to recall a story as
presenting some characteristic features, such as clear causalty, discriminable events, identifiable
places, even when in reality the story presents omission in these fundamental characters; a
certain degree of toleration of ambiguities allows spectators to enjoy more or less deviant stories),
procedural schemata (procedures through which the spectator justifies elements of the film and
applies template and prototype schemata following certain rules; include the procedures that the
spectator must put in place when the film does not follow template schemata, in order to search
for information and frame hypotheses. Justifications or motivations can be related to parts of the
story –compositional motivations in Bordwell’s taxonomy-, experience with the real world –realistic
motivation- or with other artworks –transtextual motivation, as it is the case for the reference to
genre-, artistic motivation related to the consideration of the form and structure of the artwork).
Bordwell describes the cognitive activities that are associated with the application of schemata.
The first one consists in the position of assumptions; the assumptions that are described by
Bordwell are basic assumptions that can be considered as related to commonsense knowledge,
such as the assumption that objects persist in space and time even when they go unperceived or
that individual personality persists. Even if he does not recall the notion of surprise, Bordwell
asserts that these assumptions normally go unnoticed unless they are violated by the film
(Obscur objet du desir). The second type of cognitive activity consists in the formulation of
inferences, and in particular of inductive inferences that explain causes of actions in film. Finally,
the most interesting activity is the formulation of hypotheses or expectations about future events
(Sternberg). Sternberg describes different forms of hypotheses: curiosity hypotheses (past
events), suspense hypotheses (future events), hypotheses with different degree of probability and
of exclusivity (more or less choices). Hypotheses are tested at different levels: at the micro-level
of step by step action or at specific junctures related to specific events, at the macro-level of the
entire film. When hypotheses are disconfirmed, expectations must be readjusted; Sternberg
affirms that there is a certain resistance in expectations revision and that expectations and
assumptions based on schemata can be retained even when faced to disconfirming evidence.
o A similar insistence upon the constructivist activity of the viewer, the presence of
gaps and the related necessity of filling-in activities based on knowledge, experience, perceptual
capacities, can be found in McCloud’account of Comic art.
o Bordwell’s theory is composed of perceptual constructivism and cognitive
constructivism. It does not seem a necessity to accept both components when accepting one of
the two. For instance, one can accept the cognitive constructivism without necessarily adopting
an indirect view of perception (the view represented by Helmholtz-Gregory).
o Bordwell presents the spectator as having a goal: the goal of making sense of
creating a meaningful story out of the material which is presented.
o Bordwell highlights the importance of unity, coherence and the fact that even
coherence is a matter of construction. He also introduces a consideration about the capacity of
tolerating a certain amount of ambiguity and a certain degree of frustration of the expectations
that are based on the application of schemata, in particular of template schemata. It would be
interesting to know which is the degree of tolerance and how it is established.
o It is not clear at which moment the disconfirmation of hypotheses can produce
disruption of believability and not simple readjusting of hypotheses or belief revision or resistance
to evidence. In any case Sternberg and Bordwell accept the view from cognitive sciences that
affirms that cognitive agents tend to keep their expectations unless there is strong evidence
“For example, beginning a story with "Once upon a time" can be seen as a parallel to the overt
narrational address that initiates most classical films (not just via the credit sequence but by
various self-conscious expository devices; see Bordwell, Staiger, & Thompson 1985, pp. 24-29).
Such overt marking of the tale's fictional status might at first blush seem culturally specific. Yet
there are comparable formulas in languages from Albanian to Serbo-Croatian, and there are
functional equivalents in Navajo ("At the time when men and animals were all the same and
spoke the same language. . ."), in the Tiv tribe of Nigeria ("I can tell lies too!"), and among the
Bandi of northwest Liberia ("Let's throw stories!") (Pellowski 1977). In any culture, it would seem,
the story must be "framed" by conventional markers; otherwise, it may be mistaken for reportage.
The self-conscious opening that frames the story may well be a pragmatic universal, like
politeness formulas (Brown & Levinson 1987). The point is that we should not let a justifiable
resistance to biologism block us from a rich and comprehensive explanation of how filmmaking
and film viewing, like other cultural activities, build upon acquired skills and innate capacities.”
The framing of the story recalls the idea that the user/spectator/listener is aware of the fact of
assisting a representation which is not reporting or reality.
The mesh with perceptual and cognitive mechanisms
Cutting, 1992 asserts that filmmakers and movies match our perceptual and cognitive
dispositions and mesh with the human visual system. Hollywood style, for instance, subordinate
the presentation of the story to the narrative and narrative coherence, hence if narrative works, no
attention is posed to other structural aspects such as cuts and the audience does not become
self-aware. There are also technical escamotages that allow the avoidance of self-awareness,
such as the fact of placing the action in a space (the action space) which is different from the
personal space of the spectator. In general, the filmmaker plays attention at do not making the
perceptual differences between real world and film perception and to make the camera stay
transparent to the story.
Stoffregen, 1997 discusses the notion of illusion of reality in cinema, as asserted for instance by
Anderson, 1986, who, characterizes the perception of films as illusory because spectators
perceive the events depicted in films as real and not as depictions of reality. Anderson is besides
aware that spectators do not run away when the film features a horrible monster.
As Gibson, 1979, Anderson recognizes that no one is completely fooled by films, and that film
perception provides two inconsistent sets of information. For Gibson, 1979, in fact, the illusion of
reality is a myth, because a depiction is perceived as such; but depictions are also perceived as
conveying information about something else: depiction and films are hence paradoxes.
Stoffregen, 1997 suggests that the paradox can be solved by stating that what is directly
perceived is a state or events depicted on a surface. The fact that we differentiate depictions from
reality is proved by the fact that we differently react to the two, not running away from a movie
monster and not calling the police for a fictional murder. The ambient array is specified in a
different way by a parcel of the real world and a depiction. Stoffregen, 1997 cites Hochberg,
1996. Stoffregen focuses on the structure of the stimulus and on perception and leaves behind
But the problem of the “illusion of reality”, of the power of films to move spectators and to seem
real concerns other dramatic arts as theater representations that cannot be considered as
depictions as films and pictures because the representation is directly conveyed with no medium
(no surface). Are perceptual hints enough to differentiate between theater and reality?
Additionally, the problem of the “reality” of a depiction or of a representation is not simply a
problem of the resemblance between reality and representation: fictions can produce the so-
called illusion of reality or move the audience even if they do not resemble to parcels of the real
world; as it happens in the case of monster movies.
The problem of the illusion of reality is hence broader than a problem of depictions and broader
than a problem of simulation of parcels of reality.
It seems widely accepted that no one is really fooled by depictions or other fictions. The example
of monster movies is especially significant because it exemplifies two objections to the idea that
spectators are fooled: spectators do not present exactly the same reactions they would have in
presence of a real monster and there are no monsters in the real world: no one can hence believe
that what he is perceiving is the real world and not a fictional one. Nevertheless, monster movies
and films in general, depictions, dramatic arts and also virtual reality provoke emotional and
behavioral reactions in spectators and users and this emotional reactions present some affinity
with the reactions one would have if the fictional events were true.
The perception of the difference between fiction and reality explains the fact that spectators and
users are not fooled, that the emotional and behavioral responses to fictions are not the same
than to reality; but it does not explain the affinity. Furthermore, purely perceptual explanations of
the awareness of the fictional character of a fiction might not be sufficient when all types of fiction
are taken into account.
Stoffregen, 1997 cites the example of his sister’s dog who is able as human beings to make the
difference between a dog in a film and a dog in the real world (better: to learn the difference,
because the dog in question, as probably human beings in the early years of cinema, barks to the
first dog he sees in TV): Stoffregen sister’s dog barks to real dogs but not to dog in TV. We can
hypothesize that there are there differences in the stimulus situation that are sufficient to produce
different behavioral answers. These differences in the stimulus situation and in the behavioral
answer need not to be aware; in fact, even dogs seem to make the difference and present
different reactions. I am still assuming the chauvinist position according to which dogs and other
animals cannot be considered to be aware of something because they cannot be considered to
represent something, but just to directly react to stimuli. The great number of different reactions
they present is not due to intelligence in the human sense as thought or capacity to represent but
just to the corresponding great number of different stimulus situations. A depicted dog and a real
dog structure the ambient array in different ways and hence give rise, according to Stoffregen, to
different (correct) perceptions and correspondent (correct) behavioral reactions.
But Stoffregen sister’s dog would probably bark at a dog representing a dog in a theater picture.
The actor-dog in fact cannot be differenced on the basis of perceptual conditions, but only on the
basis of cognitive considerations. In this case one has to be aware of the difference between
reality and theater, aware in cognitive, thoughtful way ad not only aware in a sub-personal,
reactive way (which is not awareness, but picking up of perceptual differences and behavioral
If we are to understand the “illusion of reality” in cinema and the capacity of films of moving the
audience, we must take into account theater as we take into account depictions, hence we must
take into account aware, cognitive processes as much s we should take into account perceptual
Stoffregen, 1997 affirms that the understanding of films requires to understand the relations
between films and other kinds of storytelling media. He considers that the common problem
between these media is the one of involvement (as opposed to perception) and that involvement
concerns the story or diegesis which is clearly perceived as fictional.
Another characteristics of films that should not be overlooked is their intentional character as a
human product: as stated by Ittelson, 1996, Anderson, 1996 and by Gibson, 1979 (see
Stoffregen, 1997), films are the purposeful product of the intention of the film-maker; Anderson,
1996 describes films as programs or sets of instructions for what the film-maker wants his
audience to see. This is probably more suitable as a description of classic Hollywood movies,
where the spectator is heavily directed and the point of view of the film director is imposed over
the spectator, than for instance, for Nouvelle Vague French movies (see Bazin). But we can
imagine that the set of instructions is more or less flexible. Nevertheless, also the intentionality of
the viewer is concerned: Gombrich, 1972, for instance, cites the fact that human beings are
capable of seeing castles in the clouds or faces in the fire, that is, to treat as depictions natural
forms that have no artistic intention. More in general, we can say that human beings are capable
of seeing as and not only of seeing: they can see the meaning in addition to pick up a stimulus
and react (see Wittgenstein, 1958). Stoffregen, 1997 adds the case of natural reflecting surfaces
such as water that do not seem to imply any form of intentionality, and hence can be perceived
even by animals; he suggests that a theory of depiction should take into account all these
If we accept a chauvinist view of representations, not only an animal cannot see a face in the fire,
but it cannot see the resemblance between a parcel of the world and that parcel reflected in
water: it can just perceive both and react of each of them in a way that suits to his behavioral
structure (see Davidson, 2004). Additionally, films are not necessary a matter of resemblance
with parcels of the real world, and Anderson’s statement about the nature of films as sets of
instructions about what should be seen respects this condition.