Docstoc

VIRTUAL REALITY AS AN AESTHETIC PRODUCT AND NOT AN ILLUSION

Document Sample
VIRTUAL REALITY AS AN AESTHETIC PRODUCT AND NOT AN ILLUSION Powered By Docstoc
					7/10/2011




Illusion of reality




It is asserted that experiences in Virtual Reality present a special character in

relation to experiences obtained with traditional media (Witmer & Singer, 1998;

Lombard & Ditton, 1997).

This assertion is associated with the idea that virtual reality systems (in particular

multisensory, interactive interfaces) are stronger means than traditional media

(books, pictures, movies) for producing an illusion of non-mediation or illusion of

reality.




Multisensoriality and interactivity are arguments for accounting for the qualitative

shift from the realm of traditional representation to the realm of a full illusion of

reality (Dihn, et al., 1999; Schubert, et al., 2001). The role of immersion as an

enhancing factor for the illusion of reality is more controversial (Stevens, et al.,

2002).




But it is not clear what does it mean to undergo an illusion of reality , and in

particular what do users believe whenever they undergo an illusion of reality .

Clarifying this notion has both theoretical and pragmatic consequences.



                                                                                    1
Difficulties in the definition of standards for the measurement and improvement of

virtual reality systems might in fact depend on the fuzziness of the notions that

are adopted for characterizing successful and unsuccessful experiences in virtual

reality.




A paradigmatic example can be found in (Cho, et al., 2003): scenes of an

undersea world are presented to the subjects of an experiment directed to

individuate factors enhancing the sense of Presence as an illusion of reality. Four

questions are addressed to subjects of the experiment, concerning the level of

realism of the objects and of the entire environment, the sense of presence in the

environment, the capacity of self-localization and localization of objects. It seems

difficult to understand what does it mean for a subject which sits at a table or

even stands into an immersive system to feel present in an undersea world

without feeling wet, gasping for lack of air or asking himself how comes that he is

no more in a safe, dry, air-conditioned lab.




The “official concept of Presence” suggests that users can be only partly

unaware of the mediated condition and only partly deceived (International

Society for Presence Research. (2000). The Concept of Presence: Explication

Statement. Retrieved <03/2007> from http://ispr.info/).




                                                                                  2
What does it mean to be deluded only in part about reality? Two meanings can

be suggested for this question: one can be only partly deluded because delusion

is a matter of degrees, or one can be only partly deluded in the sense that

delusion comes and goes.




If partial delusion is a matter of degrees, what do users experiencing the

undersea world believe? Do they believe themselves experiencing a visual sea

but not a physical sea? Do they believe that the underworld sea can be seen

without being wet? Do they believe that what they experience is what they would

see if they could experience the undersea world in a sub-marine laboratory? Do

they build narratives that justify their beliefs? This hypothesis is pregnant of

consequences for the study of belief, imagination and making believe. Also it is a

suggestive field for studying the capacity of human beings of dealing with

perceptions and beliefs that are in conflict with previous experiences and beliefs

(the experience of being wet when observing the real undersea world, the beliefs

that one must enter a sub-marine ship in order to visit the undersea world) and

relative revision of beliefs or acquisition of new beliefs.




A different idea is suggested by classic approaches to Presence as illusion of

reality (Slater, 2000, 2003; Gerrig, 1993; Kim & Biocca, 1997): in Slater’s




                                                                                3
presence counter, for instance, Presence is measured on the basis of the

number of shifts from the sense of being there in the virtual environment and the

sense of being here in the real lab. Delusion is a state that comes and goes and

partial delusion is a full delusion which is followed by an interruption in delusion.

But then, the experience an undersea world should raise panic reactions each

time full delusion is achieved. It seems reasonable to advance the necessity

considering about what is really believed by users, even in the case of

interrupted delusional states, because the simple interruption of delusion does

not justify the absence of reaction that should be present if users would believe

that what they are experiencing is real.




Another problem that emerges from questionnaires for measuring presence as

illusion of reality or illusion of non-mediation is the nature of the sense of reality

and its cognitive bases. Questionnaires ask subjects to compare the sense of

being there in the computer generated environment and the awareness of real

surroundings (Dihn, et al., 1999) and the awareness of the real world can be

evaluated through reactions to items such as: “The virtual world seemed more

realistic than the real world” or “The television generated world was more real or




                                                                                        4
present for me compared to the real world” (Schubert, et al., 2001; Kim & Biocca,

1997).

Two objections can be raised against the notion of illusion of reality as it is

expressed by these questionnaire items.




First, if an illusion of reality implies taking the fictional world (computer generated

or television generated) for real, how can it be compared with the real world?

Wasn’t the fictional world deceptively taken for the real one? Asking to compare

the true is admitting that the fictional world is never really taken for the real one

and that users are always aware of experiencing a representation, whatever

realistic and engaging this experience can be.

Second, which are the characteristics of the experience that make subjects

consider a world more real or more realistic than another one? Questionnaire

items suggest that the fact of being an experience of the real world is not a

sufficient, not a necessary condition for appearing more real than another one.

The presence of items directed to assess the location of attention also suggests

that attention is considered as a palatable mental function for giving substance to

the notion of illusion of reality (Slater, 2000; Schubert, et al., 2001). But t he fact

that attention is directed toward the computer generated environment, or in

general toward the fictional world described in a book, shown in a picture or in a




                                                                                     5
movie, cannot be takes as an equivalent or as evidence for the fact that users

deceptively take the fictional world for real. Should we then introduce the

existence of a special sensation beyond attention, a sort of sense of reality that is

independent of experiencing the real world? The existence of such a sensation

must be proved, in order to avoid an unjustified multiplication of mental entities

(as the sense of richness that would be experienced each time one sees a rich

person or an expensive object, the sense of beauty, the sense of ugliness).




A further unclear issue is represented by the relationship between illusion of

reality and the successful or unsuccessful, the satisfactory or unsatisfactory

quality of the experience in virtual reality (see Lombard & Ditton, 1997).



Literature on Presence takes the illusion of reality or of non-mediation as a goal

for virtual reality systems but it is not made explicit in what way the fact of being

aware of the presence of a medium and of the fictional nature of the virtual world

would disrupt the experience, limit the performances and the expression of

adequate emotional, cognitive, motor and perceptual behaviors on the side of the

users. In other words, it is not clear why the illusion of reality should be pursued

as an objective in the design of virtual reality experiences.




                                                                                   6
On the other hand it is not made explicit in what way the fact of being fooled

constitutes a guarantee of the fact that the user will comply with the aims of the

designer of the experience.

It is not trivial that deception will be necessary in order to produce engagement,

alert attention or enhance enjoyment, as shown by the success of existing video

games and also of movies where deception is far from being complete. As a

mater of fact since long no one runs away from movie projection because of a

state of deception about the reality of the projected contents (as underlined by

Stoffregen, et al., 2003).




Conversely, it could be advanced that awareness (of mediation, of experiencing

a fictional and not the real world) plays positive effects at least at two levels: it

enhances the level of believability of the experience and lowers undesired side-

effects of deception, such as gullibility and unethical persuasion, excessive trust

(a risk which does not seem to be taken into account when the positive role of

presence on persuasion is affirmed, as in Kim & Biocca, 1997).




The fact of being aware that they are dealing a fictional world lowers in fact

users’ and audience’s expectations and reduces the risk of expectations’

frustration, which constitute a major threat for believability (Garau, 2003).




                                                                                   7
Contrary to what could appear, awareness of the vehicle of the experience can

then be exploited in enhancing believability. This is possible if we consider users’

behaviours in response to virtual environments (including appropriate behaviours

that fulfil the aims the experience has been designed for) as depending both on

the contents and on the context of the experience. The context of the experience

(the presence of the medium) should hence not be hided or made transparent.

The users of a medium should be made aware of the characteristics of the

medium, in terms of the possibilities the medium offers to designers (possibilities

of design and manipulation of the contents) and of the possibilities it offers to

users (quantity and quality of sensory stimulation, possibility of interacting, of

modifying the environment, vs. passive interactions).




The awareness of mediation is particularly important when dealing with sensible

classes of users in order to avoid any type of abuse of trust or of confusion

between what is possible and allowed in the real world and what is possible and

allowed in virtual worlds. If we think of the wide diffusion of new powerful media

for producing new experiences and of the impact that they promise to represent

in our lives (commerce, education, training, immersive games, therapy,

rehabilitation) we can see the importance of accompanying this diffusion with a

responsible consideration of the nature of the proposed experiences. An




                                                                                  8
ethically-driven development of new media technologies for virtual reality should

include the awareness of interacting with virtual worlds and of being capable of

distinguishing between real and virtual objects and entities as structural

characteristics of our interaction with virtual worlds.




The mediated character of the experience should not be considered as a side-

effect that could be overcome by technological development, but a necessary

step (Bardy, 2002; Stoffregen, 2003) linked to the structure of sensory modalities

and by the structure of cognitive processes (Cutting, 1999). Rather than seeking

for the production of illusions of non-mediation and of illusions of reality, I

contend that designer’s efforts should be devoted to the production of

experiences that are believable, convincing, enjoying, effective, without being

thereby unaware.




Naturalization of the concept of illusion of reality

Illusion of reality. Ethical issues


Illusion             of               reality.            Cognitive        issues




                                                                                9
Naturalization of the concept of illusion of reality



The term ‘illusion of reality’ is employed for referring to a state experienced by

users of traditional and new media technologies. In spite of the large diffusion of

the term, the phenomenological contents and the psychological nature of the

illusion of reality are not univocally described and at least four different uses of

the notion of ‘illusion of reality’ can be individuated: a. illusion of reality as

believability of a fictional content in terms of the dramatic effects produced,

illusion of reality as deception, in the form of b. hoax or fraud, whenever users or

spectators are cheated about the nature of the experience by taking a fictional

content for a real one, in the form of an c. illusion of non-mediation, whenever a

mediated experience is taken for a non-mediated one, and in the form of d.

illusion of transportation or being there, whenever the cognitive awareness of

experiencing a fictional world is associated with special proprioceptive

experiences of self-motion and localization in representations.




                                                                                 10
a. Illusion of reality as believability


Illusion of reality as believability in animated movies, cartoons and comics

Illusion of reality as believability in interactive media and VR




The first use of the notion of illusion of reality is equated to believability and

makes reference to the effects of employing perceptually and psychologically

realistic representations of objects, events and characters in mediated contexts,

in the sense of sharing structural similarities with objects, events, and, people of

the real world. The similarities are compatible with the representation of objects,

events and characters that do not exist in the real world, hence they are to not be

confounded with photorealism, but are conceived as key factors for producing

believable experiences (especially psychological realism and the production of

rich personalities are considered to positively affect believability [Thomas &

Johnston, 1984; Reilly, 1992]). In spite of the fact that creators of such

representations can identify their objective with the production of an illusion of

reality [Thomas & Johnston, 1984], the use of realistic representations is aimed

at producing esthetic and dramatic effects and not at cheating spectators . The

psychological complexity and the emotional richness introduced by Disney

animators in the world of cartoons, for instance, has the effect of inducing Disney



                                                                                 11
spectators to sympathize with characters, resent a wide range of emotions in

correspondence with the emotions that are resented by the characters on the

screen, but, no matter how gifted, Disney animators cannot make spectators

believe that Disney characters are alive or real. The problem of deluding movies’

spectators and fiction’s audiences in general has been largely debated in relation

to the so-called “paradox of fiction” [Radford, 1975]: how can we be moved by

characters if we don’t believe in their existence? The debate shows a substantial

accord on the fact that audiences and readers do not hold existence beliefs

about fictional contents, as testified by the fact that responses to fictional

contents are similar but never identical to responses to corresponding non-

fictional contents. The difference in reactions testifies that audiences and readers

are aware of the fictional nature of the experience, hence that they do not have

existence beliefs when they experience fictional contents [Radford, 1975;

Stoffregen, 1997; Walton, 1978, 2001; Currie, 1990].

It is also largely accepted by exponents of different representational domains -

such as animation cinema, virtual reality and comics - that the production of

dramatic and emotional effects does not properly depend upon realism as

external similarity between fiction and reality but rather upon some familiarity with

certain contents and aspects of the representation [Thomas & Johnston, 1984;

Reilly, 1992; McCloud, 1993]. As stated by [Bordwell, 1985], in fact, narration is a




                                                                                  12
matter of active construction, both perceptual and cognitive, on the side of the

spectator. For this activity, spectators exploit their perceptual innate structures,

their experience with the real world and also their experience with other movies

(cinema basic techniques, genres, physics and other general laws - such as

cartoon physics). The cues provided by a movie can hence make spectators

more or less immediately familiar with the fictional environment, in case they fulfill

familiar knowledge and expectations or not. Familiarity hence constitutes the

natural basis of realism and can be used for explaining the preference for familiar

representations on the side of public and the production of believable rather than

non-believable representations. It can be hypothesized that familiarity lowers the

effort that the spectator must do for constructing the narration, in analogy with the

use of stylized, exaggerated and simplified facial expressions (techniques in use

in animated movies, cartoons and comics [Thomas & Johnston, 1984; Reilly,

1992; McCloud, 1993]) that help the characters’ emotions identification thus

reducing the    spectator’s cognitive effort. It can also be hypothesized that

believability depends upon the fulfillment of spectators’ expectations acquired in

the experience with the real world, but also in the experience with fictional ones.

The fulfillment of cartoon physics laws can hence fulfill the expectations of

spectators that are acquainted with Warner’s cartoons and give rise to believable

experiences.




                                                                                   13
Illusion of reality as believability in animated movies,

cartoons and comics



Disney: simplification and exaggeration


Warner: cartoon physics and characters’ oddity


Comics: cartoon style and closure




                                                      14
Disney: simplification and exaggeration



Thomas and Johnson (1981) is one of the most referred texts for believability, at least in the

domain of animation and especially of characters animation. In addition to provide a

characterization of the notion of believability which is widely adopted even in the domain of VR,

for instance in the domain of interactive drama, the book describes the techniques that are

responsible for the production of believable characters and the general approach to the problem

of believability that lays behind these techniques and can be extended to other domains (the

“special ingredient”).

Since the very Preface, believability is characterized as follows:

“Disney animation makes audiences really believe in those characters, whose adventures and

misfortunes make people laugh – and even cry. There is a special ingredient in our type of

animation that produce drawings that appear to think and make decisions and act of their own

volition; it is what creates the illusion of life.” (p. 9)

Thomas and Johnson go back to the early story of Disney animation and illustrate the successive

steps that have conduced Disney animators to detach themselves from the 20s trend of short

gags played by characters with no personality, individuality and in general mental states

(emotions, desires, beliefs, motivations, intentions) to the realization of a new idea: stories that

are not bound to gags and easy laugh but that are able to attract the interest of the audience for a

long time and even to move audiences as Hollywood films do and characters that appear to think,

to have emotions, to act in reason of their own mental states and not simply in reaction to some

external event. These two ingredients require the development of new techniques in order to be

realized and give rise to a new pattern of believability. According to the authors, characters that

appear to think, have emotions and act of their own and only characters with these characteristics



                                                                                                 15
can be considered as believable because these characteristics confer drawings with an illusion of

life.




The first ingredient, the story, is much more directed to produce involvement of the audience

rather than believability. Disney’s approach to involvement consists in displaying stories that

contain elements that are well recognizable and familiar to the audience. This is a form of realism

as simulation, in the sense that certain aspects of the real world, people and events are imitated

in order to create a sense of empathy and to raise the same emotions that they raise in real life.

Realism is then considered as a crucial ingredient for empathy and empathy a crucial condition

for involvement.

“We involve the audiences in our films the same way [as in our own lives]. We start with

something they know and like. This can be either an idea or character, as long as it is familiar and

appealing. It can be a situation everyone has experienced, an emotional reaction universally

shared, a facet of someone’s personality easily recognized, or any combination of these. But

there must be something that is known and understood if the film is to achieve audience

involvement.” (p. 19)



The second ingredient concerns the realism of the characters. It seems that since the ‘20s Disney

asked for more character realism in two senses: physical aspect of characters and other entities

and mental states. The physical aspect for instance concerns entities weight, characters anatomy

(the presence of bones and muscles). In animation special techniques for suggesting weight and

the presence of bones and muscles must be invented and mastered.

“Everyone knew that it was necessary to get a feeling of weight in the characters and their props

if ever they were to be convincing, but just drawing a figure large has nothing to do with how

heavy he is. A weather balloon is quite large. The animators sensed that the key to the illusion of


                                                                                                 16
weight lies in the timing and how far a character moved and how fluid the action was, but it was

not until they were able to study live action films that the solution finally was found.”

The solution is provided in this case by the imitation of real action and real anatomy.




But Disney’s realism is not a full simulation, a form of hyperrealism or photorealism of the kind

displayed by other animation films such as Final Fantasy. This is not only because of

technological limitation, but lays on an idea of realism and believability as added to the idea of

caricature, simplification and exaggeration. A character is realistic because it has emotions,

personality, because his actions respond to inner volitions and motivations and not only to

external situations. These are characteristics of real human beings, but in animation they are

treated following special techniques that give rise to caricature effects that are indicated to be as

important as realism as simulation for the final effect of entertainment. This is the reason why

Thomas and Johnson speak of caricature realism.

“When Walt asked for realism, he wanted a caricature of realism. One artist analyzed it correctly

when he said, “I don’t think he meant “realism”. I think he meant something that was more

convincing, that made a bigger contact with people, and he just said “realism” because “real”

things do.” (p. 65-66)

“If an animator’s drawings finally reflected a more natural way of moving, Walt would be likely to

say, “[…] We oughta be looking for entertaining ways of doing things. We don’t want to get

straight, y’know – we are not copying nature!” “Caricature” and “exaggeration” were two favorite

words to stimulate the animator’s approach to his scene.” (p. 37)



Two principles of animation are both related to realism and caricature and illustrate how

caricature with exaggeration and simplification can serve to evoke actions and emotions of real

people without fully copying real people:



                                                                                                  17
“When a fixed shape is moved about on the paper from one drawing to the next, there is a

marked rigidity that is emphasized by the movement. In real life, this occurs only with the most

rigid shapes, such as chairs and dishes and pans. Anything composed of living flesh, no matter

how bony, will show considerable movement within its shape in progressing through an action.

[…] The squashed position can depict the form either flattened out by great pressure or bunched

up and pushed together. The stretched position always shows the same form in a very extended

condition. […] Immediately the animators tried to outdo each other in making drawings with more

and more squash and stretch, pushing those principles to the very limits of solid draftsmanship:

eyes squinted shut and eyes popped open; the sunken cheeks of an “inhale” were radically

different from the ballooned cheeks of a blowing action; a mouth chewing on a straw was first

shown far below the nose, and then it actually was compressed up beyond the nose (which

changed shape as well) in showing the chewing action.” (p. 48)



In other words Thomas and Johnson suggest the following schema: imitation of some aspects of

real life situations for creating empathy and hence involvement, imitation of some aspects of real

people mental life, anatomy and physiology and interaction with things in order to create

believable characters, but also objects and events in relationship with characters, but caricature

and exaggeration that limit realism in order to produce entertainment. The association of realism

for certain aspects and of stretched realism (caricature) for other aspects produces caricature

realism.




                                                                                               18
Warner: cartoon physics and characters’ oddity



Chuck Jones, 1989 from Warner Bros. (the father of Bugs Bunny, between other characters)

states that it is the oddity, the quirk that gives personality to a character and that it is personality

that gives life. So it seems that one should allow regularities to be broken in order to create

characters that are more believable because they have a richer personality; And also a specific

personality; in fact, tics and other oddities are specific of an individual and hence help suggesting

that the character possesses a personality of its own.

The same principle of oddity and non-realism governs the events of Warner’s worlds.

The laws that govern cartoon worlds, and typically Warner cartoons’ worlds, are not the same

than real world laws. They have hence been identified as cartoon physics, and described by

several Warner cartoons’ viewers and film critics.

Events ruled by cartoon physics might appear surprising to first-viewers and non-expert cartoon

viewers (such as children). They hence require some little experience and some consideration

about the fictive and cartoon nature of the depicted worlds in order to become believable.

Each cartoon universe has its own specific laws, anime laws being different from Warner-like

cartoons [O’Donnell, 1994; Gould, 1993, Cholodenko, 2006].




                                                                                                     19
    Comics: cartoon style and closure



    Norman McCloud (McCloud, 1993) develops the problem of realism is comics. Comics constitute

    a suggestive domain for realism because they are structurally related to caricature.

    McCloud describes a sort of decreasing line of realism in images, from photos to cartoons: in

    comics we mainly find simplified, abstract images that are further and further from the “real” face

    of the photo, which is realistic and detailed; the limit is the cartoon.

    In spite of the use of the two mechanisms of simplification and abstraction, comics are very

    acceptable to our eyes, and they seem just as real as photos; in fact, we respond to a cartoon as

    much or even more than to a realistic face. Why?




    According to McCloud, this is because cartooning is a form of amplification through simplification:

-                               Focus: much more than eliminating details, abstraction through

    cartooning is a way of focusing on specific details, of stripping down an image to its essential

    meaning.

                                      The fact of simplifying images and characters toward a purpose

    is not applied to cartoons only, but it constitutes a tool for any storytelling medium. Cartooning is

    not a way of drawing, is a way of seeing. This is exemplified by the movie “Wizard of Oz”.

-                               Universality: the more cartoony a face is, the more general it is. This

    fact enhances self-identification. Self-identification is important because the identification with the

    character of a story is a sure indicator of the fact that audience is involved. This also explains why

    backgrounds tend to be more realistic than characters (in addition to a pragmatic reason

    connected to the production of a lot of drawings in animation).




                                                                                                        20
                                    Faces have a special status in perception: human beings see

    faces even in abstract drawings once few conditions are fulfilled.

-                               Concepts: Cartoon style de-emphasize the perceptual appearance of

    things in favor of their form or of some concept: it allows to express concepts.




    McCloud hence draws the following lines in order to describe the transition from realism to

    cartoon style:

    - photo  cartoon

    - complex  simple

    - realistic  iconic

    - objective  subjective

    - specific  universal




    Another interesting mechanism described which is employed in comics but also in cinema is

    closure (in cinema see the use of ellipse): recognize the whole when perceiving the parts.

    Independently from the cognitive mechanisms implicated, McCloud hints to the role of

    collaboration between the drawer and the reader and to the reader’s imagination, hence to the

    reader’s previous experiences. For this reason, the artist needs to know what are the

    assumptions of the reader in order to foresee what will work and what will not produce closure.




    While in electronic media closure is often involuntary and unperceived, comics are particularly

    exemplary of closure because they structurally present a fracture of time and space between one

    image and the following (gutter): only closure allows the reader to connect the images. In a

    sense, comics is closure.




                                                                                                      21
Illusion of reality as believability in VR and interactive media




The Oz project


Augmented reality




                                                                   22
The Oz project



The Oz project is a project developed at Carnegie Mellon University during the ‘90s in order to

individuate the characteristics that are responsible for the production of believable characters in

dramas and in particular with the aim of producing believable interactive dramas (dramas in

interactive media).

Two elements are present in Oz research: story and characters, hence interactive story and

interactive characters, with the objective of creating a new form of art and entertainment. These

characteristics give rise to Oz worlds that include: a simulated physical environment, automated

agents, a user interface and a planner.

Narration has the function of making the events, actions and reactions of characters

comprehensible. Agents are more comprehensible if their behavior is structured into narratives.




Believable character is a notion from the Arts, and it means

“a character that produces the illusion of life and that permits the suspension of disbelief.” And “ it

is our view that believability will not arise from copying reality.” (Bates, 1994, p.1) Bates consider

that this is a dream which is common to theater, cinema, animation, IA and even robotics (see

Bledsoc, 1986). The domain of animation is of particular interest because of the practical

requirement of producing hundreds of thousands of drawings which force animators to use

extremely simple, non-realistic images and to seek and abstract precisely what is crucial.

Animation and storytelling media in general are taken as a model.

“Also, we are not necessarily interested in lifelike or realistic agents. We don’t really care if our

characters only have 4 fingers or are talking animals. The arts have always abstracted away from

reality and created characters that were “larger than life” and that were much more interesting



                                                                                                    23
than any mere “real” people or animals we migjt know. By believable agents we mean interactive

versions of these abstract characters that have been so successful in non-interactive media.”

(Reilly, Bates, 1995, p. 1)

“The qualities of traditional characters that produce suspension of disbelief in viewers are

understood informally by artists…” (Reilly, Bates, 1995, p. 1)

Contrarily to traditional characters, interactive agents will have to interact between them and with

the user.

“Emphasizing believability leads us in new directions. The arts suggest that the most crucial

elements to imbue agents with strong personalities that permeate their actions…in addition to

personality there are several other aspects of characters important for believable social behavior ”

such as show to have emotional reactions, have dynamic social relationships that affect the

agents behavior, appear competent at working toward multiple goals simultaneously and have

robust behavior. (Reilly, Bates, 1995, p. 1)

Reilly, 1996 asserts that artists know how to create believable characters although they do not

have provably correct methods for doing so. They use the term believability as a synonym of a

character that works, that seems to be alive and that the audience has emotions for or about; not

necessarily intelligent or rational as AI, not necessarily realistic, but with a strong personality. It

turns out that sometimes being more realistic can decrease believability. In fact, the state of the

art in computer animation can make a mostly realistic human face, but not a completely realistic

one. This close but not quiet face is very disturbing to watch at because people are so well

adapted to watching human faces. From the standpoint of believability it is better to go with the

less realistic characters which meet the audience’s expectations than to go with the more realistic

ones that don’t. Additionally, powerful artistic techniques such as simplification and exaggeration

rely on altering reality for more effective characters in the attempt to communicate the essential

personality.



                                                                                                    24
    Basically, following the Oz view, a believable character is a character which looks alive

    (consistently with Johnson and Thomas, 1982), whose actions make sense, who allows the

    audience to suspend disbelief. Believability is conceived as equivalent to the illusion of life or

    illusion of reality and to suspension of disbelief.

    “One of the keys to an effective virtual world is for the user to suspend disbelief. That is, the user

    must be able to imagine that the world portrayed is real, without being jarred out of this belief by

    the world’s behavior” (Bates, Loyall, Reilly, 1991b, p. 1)



    This is not the same thing as realism. For example, Bugs Bunny is a believable character but not

    a realistic one. (Loyall, Bates, 1997) underlines that believability is not coincident with realism

    because the best path to believability almost always involves careful, artistically inspired

    abstraction, retaining only those aspects of the agent that are essential to express its personality

    and its role in the world of which is part. Nevertheless,

    “While full realism is rarely appropriate, we have found, as others before us, that fine details can

    have a great influence on whether a creature seems alive. The use of the eyes, the timing of

    pauses in speech, an awareness of body position and personal space, are each examples of

    these important details.” (Loyall, Bates, 1997, p.1)



    Characters that look alive are rich characters. Above all, rich in personality. In a sense, realism is

    respected at this level. The requirements indicated for believable characters are the following:

-                               personality

-                               emotion

-                               self-motivation

-                               change

-                               social relationships



                                                                                                       25
-                              illusion of life in aspect and behavior

    They are what we expect on the basis of what we know about people.




    The first aim is to look for broad characters, even if shallow. Broad agents have goals and goal-

    directed reactive behaviors, they present some language abilities and some memory and

    inferential capacities. Bates, Loyall, Reilly, 1992 describe broad even if shallow characters as

    presenting    goal   directed   behavior,   emotions,   language,     perception,   reactivity,   social

    behavior.Goals and goal-directed reactive behaviors seem to be very relevant for providing an

    illusion of life. They hence constitute a third type of agent in comparison to AI entities and to

    embodied artifacts. Loyall, Bates 1997 describe the analogies and differences between the three

    types of agents: broad agents implement ideas of parallelism, situated activity and reactivity with

    explicit goals.




    Bates, 1994 insists on the role of emotions for believable characters intended as engaging,

    apparently living creatures. If the character does not react emotionally to events, if it doesn’t care,

    neither do users and audience. Additionally, an emotionless entity is perceived as a machine.

    Emotions must be appropriately timed and clearly expressed. Realism just concerns the fact of

    having emotions, not the reproduction of all the ways emotions are expressed: the clear

    expression of emotions require exaggeration and suitable timing because normal timing is too

    quick.




    Reilly, 1995 is dedicated to how to create believable characters, especially at the level of

    emotional behavior. He suggests that a first trick is to lower the expectations of the users by the

    creation of characters that are different from normal adult humans, such as kids and animals or

    even better aliens. In fact in this case, the user has fewer expectations.



                                                                                                         26
Bates, 1994 also suggests that an element that might be useful for attracting audience and users

is the presence of oddities, quirk. In fact, it seem that a bug in the program “Edge of intentions”

had created a repetitive movement for one of the characters which looked like a tic; as a result,

the character did appear more interesting and real.




The Oz project has inspired other developments.

Riedl’s (Riedl, 2005) contribution consists in a system for evaluating the perception character’s

believability in terms of the perception of characters’ intentionality in stories created by automated

story generators. The more characters’ intentionality can be understood, the more the characters

and hence the story are believable. In fat, intentionality is onsidered as a fundamental factor for

characters’ believability.

As for what concerns the characterization of believability, Riedl (2005) re-proposes the concepts

introduced by Thomas and Johnson and re-edited in interactive drama by the members of the Oz

project in the framework of the problem of automated story generation and storytelling. He

indicates the success of storytelling in the capacity of having emotional and educational impact

on the audience and indicated understandability and believability as the two conditions for

success. Characters’ believability is successively suggested as a story’s component that can

influence both understandability and believability of the story. Following the characterization of

believability as audience’s willing to suspend disbelief and character’s believability as coincident

with the illusion of life, Riedl (2005) focuses on intentionality between the other factors considered

relevant for creating the illusion of life: personality, emotion, physiology and physiological

movement. Character’s intentionality refers to the existence of a relationship between the actions

and behaviors of the characters on one side and the beliefs, desires, intentions and motivations

on the other, that makes the actions and behaviors of the character appear natural and rational.



                                                                                                   27
Two elements are hence present in character’s intentionality: on one side the coherence between

the actions of the character and the coherence between the character’s actions and the situations

presented by the story; on the other side, the fact that the characters are perceived to act upon

their intentions, beliefs, desires and motivations. The first condition is important for character’s

believability because it limits the risk for the character to appear schizophrenic, but the second

condition is very relevant too for creating the illusion of life. The procedure for testing storytelling

generators is the following: The process relies on the fact that a reader/viewer’s perception of

character intentionality can be compared to a QUEST representation of the story because

QUEST is a validated model of human question-answering in the context of stories. the

procedure for evaluating story generation systems is to compare an instance of a QUEST model

of a specific generated story to subject comprehension of narrative structure for that story. In

QUEST, “why” questions inquire about character goals, intentions, and motivations. The general

idea behind the process described here is that a better story generation system (presumably the

test condition) will result in stories whose structures better support human perception of character

intentionality. The better the structure of the generated story, the better a QUEST representation

of that story will predict reader/viewer question-answering.




Three considerations can be advanced:

        1) The relationship with dramatic arts creates a sort of connection between believability

and success or positive appreciation of the opera: if an agent or character is believable then the

character will be a success, and viceversa: if the character has success it must be that it is

believable (see Mateas, 1995). But this connection might simply be external and artificial: can a

measure of success constitute a measure of believability? A character can be unsuccessful for




                                                                                                     28
other reasons than the fact that it is not believable? A non believable character can be

successful?

        Also, when exaggeration in the emotions and timing is presented as a useful ingredient

for identifying the character, recognizing it, this is not necessarily related to believability unless it

is not affirmed that it serves to give rise to a specific personality and that specific personality is a

necessary ingredient for believability.

        2) The minimal conditions for a believable character are not investigated in a systematic

way. Only intuitive characteristics are enlisted, for none of which is defined its necessary or

sufficient role or the reason why it should be more necessary than others or if there are other

conditions. How can we know that the enlisted conditions are the only relevant ones? That they

are all relevant?

        The principle which is indicated for choosing them and not others is that they are

characteristic of people, that we know that people have these characteristics and that for

consequence we expect to find these characteristics in artificial or virtual characters and agents.

The respect of expectations about human beings constitutes a general principle. But after that the

enlisted expectations to be respected are just chosen on the basis of intuition, and not guided by

another principle. This is why one cannot be sure that each of them is necessary and that the all

of them is sufficient for producing believable characters.

        An extreme example: Tamagotchi. Tamagotchi are digital pets, typically included in a

handheld egg-shaped computer, created by A. Maita in 1996 for a commercial company.

Tamagotchi can be fed with unhealthy or healthy food, can be rendered more happy by “playing

with them”, their environment can be cleaned up from excrements and their status (happiness,

hunger, discipline, weight, age) can be checked. They evolve dependently on how well the user

takes care of them. Users care about their digital pet, they spend their time with it and do the right

actions for keeping it “in life” or getting better. Are Tamagotchis believable characters? They are



                                                                                                      29
not realistic in their physical aspect, they are far from being human or animal-like; they are neither

rich in personality, emotions, social relationship, etc. A causal relation seems to subsist between

their state and the actions of the user, but no free actions; can the fact of being in need of food,

attentions, cleaning be considered a sort of will, desire? The problem of the intentional stance

arises and, if we think to the enormous success Tamagotchis have see in the ‘90s, it seems that

human beings are very liberal in attributing mental properties to very simple entities. It is true that

new further releases of new versions of Tamagotchi have included more and more features such

as forms of social relationship and actions of the Tamagotchi toward the user (gifts). Anyway

Tamagotchi represent an ideal case study for the minimal conditions of believability.




        3) The position about suspension of disbelief and illusion of reality raises some

objections: how can the audience take a fictional or virtual world as real? Spectator know all the

time that they are assisting a projection and users cannot ignore they are practicing on a VR

environment. They know the difference between a movie and VR because they do not try to

interact with the characters of a movie, while they know they can do it with virtual agents. There is

hence a cognitive awareness of the fact of interacting with a real or a virtual world and even of the

type of participation which is permitted or forbidden. Additionally, emotional responses to real and

fictional worlds can be similar but they are never identical, or spectators would run away from the

projection of horror movies and they would be unable to enjoy murders on the screen. Finally, we

never imagine the real world to be the real world, we make no effort or take any particular attitude

in order to consider it as real. Nor we have a “sense of reality”; we can sense that something

does not look or feel as real things do. It is the same as for coherence: it is only when coherence

is violated that we notice coherence, not when it is respected. But what is the sense of reality,

sense of coherence, etc.? Do we need to have a sense of for everything?




                                                                                                    30
        The suspension of disbelief can concern certain beliefs (as the existence of monsters)

and allow the audience and users to enjoy representations that do not fulfill their expectations

issued form the experience with the real world. But suspension of disbelief does not necessarily

concern the belief that the world experienced is real rather than fictional or virtual.

        The notion of suspension of disbelief is nevertheless an intriguing one since it suggests

that the user or audience put something in pause when assisting the spectacle or interacting with

the virtual world and that the user and audience assume a certain attitude toward their

experience. Hence we should ask what do we wan to obtain from the audience or user: that they

sit at their place, participate with attention, emotion and eventually action and relevant action?




                                                                                                     31
Augmented reality



At Mira-Lab the problem of believability is especially addressed in the frame-work of Mixed or

Augmented Reality (MR and AR), for instance in the frame-work of LifePlus project

(Papagiannakis, 2002, 2004, 2005, 2005; Magnenat-Thalmann, 2004, 2006). It is asserted that

MR creates a cyber-real space that presents characteristics of believability and presence which

are different from experiences in purely virtual spaces and that believability and presence are

both necessary in order to make experiences in the virtual environment to be real. As we have

seen, conveying believable experiences in fact is considered the same than conveying

compelling real experiences. But unlike VR experiences, experiences in MR are not imitations of

a parallel real world, since the real world is part of the augmented experience. Virtual elements of

the experience with LifePlus are nevertheless imitations of real entities. The application in fact

includes 5 realistic virtual humans that interact between them and manipulate objects following a

scenario; the virtual humans evolve in the real context of a tavern in Pompeii ruins or in a lab

maquette of the same tavern. The realistic aspect of virtual humans include clothes, facial

expressions intended to be accorded with the personality of each single character and linguistic

exchanges. The user witnesses the representation and has the possibility of modifying his

position and orientation.




Papagiannakis (2002) describes the following objectives: realism, immersion, interaction in order

to give rise to an experience of transportation (presence?) in fictional and historical spaces.

Papagiannakis (2002) also indicates some aspects that must be included in the virtual simulation

of life (human life) in order to give rise to realistic (hyperrealistic) entities: hair simulation, cloth

animation, skin rendering, Artificial life methods for behavioral animation of virtual characters,



                                                                                                      32
realistic facial emotion expression, intentionality, desires, beliefs, actions that can be read by the

audience.

Papagiannakis (2005) asserts that the introduction of realistic human-like virtual characters and

narration (virtual real-time story-telling) in a MR environment increases the sense of presence but

not necessarily believability and that believability depends more on the interaction that the users

are in the condition to establish with the virtual characters. Two directions for enhancing

believability are indicated: more social awareness on the side of virtual characters and new

channels of interactivity between the users and the characters. These assertions are not based

on theoretical assumptions or experimental evidence.




Kim (2004) and Magnenat-Thalmann (2005) define believability as referred to what is considered

as the major goal of VR systems: make the users feel that the generated experience is from the

real world (in this sense a believable experience is also defined a real experience). This

characterization of believability is a sub-class of the notion of illusion of reality. Virtual entities are

believable when one can be deluded that they are creatures of the real world. It raises the

following objection: can a user believe that a virtual monster comes from the real world, when he

perfectly knows monster do not exist in the real world? This characterization seems to be bound

to cases in which believability concerns virtual worlds that are copies of the real world, in which

the objective is to reproduce parts of the real world, hence to virtual worlds that do not contain

any new element or experience.

Factors enhancing believability are discussed in relationship to the level of immersion, to the type

of presentation and to the degree of interactivity.

Immersion is indicated as a pre-condition for believability: the user can better take an experience

for real if he is totally immersed into the virtual environment.




                                                                                                        33
A realistic type of presentation (as real as the real world) is indicated as to increase believability.

Two types of realism are distinguished: sensory and perceptual realism (and believability).

Realism in the sensory channel concerns the external aspects of the character’s behavior and

perceptual realism consists in the presence of emotions, personality, intents. In fact, it is asserted

that realism in the sensory channel is not a sufficient condition for believability: in addition to

provide realism for the sensory channels, the contents of the depicted world must be believable.

Realism in the sensory channel seems also to be non-necessary for believability, as exemplified

by believable stories in text books with no images. Suspension of disbelief and hence of

believability in this case depends on the contents (semantics) of the virtual world, and in

particular: on the consistency between the contents of the virtual world and the expectations of

the user and, when virtual characters are at stake, on the presence of emotional, personalized

and goal-oriented elements.

Furthermore, a realistic presentation can concern each sensory channel, and also multisensory

perception; when multisensory perception is at stake two issues must be considered: the

consistency between sensory modalities and the possibility that the activity of one sensory

modality substitutes information for other sensory modalities. VR research has traditionally

searched for realism in the sense of the reproduction of the physical aspect of the real world, but

recently it has been objected that attention should be paid to the characteristics of perception too;

this is particularly evident in the case of motion perception.

Interaction is considered as a factor that enhances believability in normal cases. The timing of

interaction must respect normal conditions of response to action. Nevertheless, even in this

domain, it is not enough to achieve sensory realism in the interaction; also the contents of the

experience (story) must be believable.

When virtual characters and agents are concerned, personality and emotions must be defined

and coordinated in order to achieve a believable result. Egges (2002, 2003, 2004) address the



                                                                                                    34
role of emotions, mood and personality in the construction of believable agents and characters.

Psychological studies are taken as references for types of emotions and types of personality.

Additionally, movements, even not directly connected to action must be attentively reproduced.

Egges (2004, 2005) argues that realistic characters should display idle motion even when no

action is planned (posture change, little movements related to breathe activity or eye blink) and

that motion should affect several joints. The justification lays in the fact that in nature there exists

no motionless character and that in natural conditions gestures affect several muscles and

several joints. It is also suggested that idle motions can be differenced into types and associated

with different personalities.

In summary, believability is indicated as to highly depend on the consistency between the

contents of the virtual world and the user’s expectations, both at the level of the story and of the

aspect of the environment and characters; on the presence of characters with emotions,

personality and intentions (in accord with the user’s expectations about human-like characters);

on realistic ways of presentation (again in accord with the user’s expectations about the aspect

and behavior of human beings); on realistic interaction (for what concerns timing but also the

type of reactions of the virtual agent following the contents of the experience); on physical

immersion. All these conditions should make the experience with the virtual world so similar to

that with the real world as to be taken as a real experience. A strong accent is posed upon

realism as simulation as a necessary condition for believability, even if objections and limitations

of this assertion are presented here and there: the fact that stories in books are believable without

displaying photorealism, the fact that even for producing realistic simulations of the real world the

structure of human perception could count as much or even more than the physical aspect of the

simulated parts of the world.




                                                                                                     35
This characterization of believability nevertheless seems to concern only a special condition of

believability: the believability of virtual worlds that are identical to the real world, but what about

virtual worlds that contain elements that are not part of the real world?




                                                                                                    36
b. c. d. Illusion of reality as deception


Designers of virtual reality experiences have three declared objectives: producing

the illusion that what is experienced is real, producing an illusion of non-

mediation and producing an illusion of being in the virtual environment. The three

objectives are referred to as ‘illusion of reality’ or ‘illusion of presence’ (or shortly,

as ‘presence’) and have in common the will of deceiving the user as for what

concerns the origin of his experience [International Society for Presence

Research - ISPR at http://www.temple.edu/ispr/].

In spite of their mixing in the notion of presence, these three aspects of deception

present different characteristics and can subsist separated: a mediated fictional

content can be taken for a real content, without any mistake concerning the

mediated context of the experience; a mediated experience can be taken for a

non-mediated one and still be recognized as having a fictional content; and

perceptual experiences of self-motion and of location in representations are

consistent with the cognitive awareness both of fiction and of mediation.




                                                                                       37
b. Deception as hoax or fraud


Illusion of reality as hoax




The first form of deception, the misrepresentation of a fictional content for a real

one, occurs whenever people are made believe that the fictional content they

experience through a medium (books, TV, movies, internet, virtual reality) is the

record of some real event (like a documentary). This form of deception gives rise

to different cases of hoax (usually humoristic or critic tricks were fiction events

are told for real ones) or even fraud (depending on whether the responsible of

the hoax has a benefit from his trickery).

Hoax is made possible by the fact that mediated contents can be both recorded

real events (such as it is the case for journalistic photographs, radio and

television documentaries) and fictions (such as it is the case for fiction movies,

fiction radio dramas or fiction virtual reality) and by the fact that both fictional and

real contents can be proposed by one and the same medium, such as

photographs, radio, TV or even newspapers. Usually, when fiction is inserted in a

non-fiction communication context, special advices and perceptual cues frame

each type of content. Journalists of RTBF broadcast announcing Flemish

secession provoked panic in Belgian population. However, at the beginning of



                                                                                     38
the broadcast a message was shown telling: “This is perhaps not a fiction” and

“This is a fiction”, was shown an hour after that [Bilefsky, 2006]. Audiences were

also deceived by emission “Alternative 3” when a group of UK Anglia television

journalists usually commenting documentary real events commented a totally

fictional event: the project of abandoning dying planet earth. No warning or

perceptual indication was present that could have made audiences distinguish

“Alternative 3” from previously released documentaries, thus producing a case of

hoax.




                                                                               39
Illusion of reality as hoax



Not only fraud, but also hoax perpetrated through communication media such as radio and

television is subject to rules and prevention because it is considered as unethical.

This is one of the domains of media-ethics (with moral dilemmas, conflict of interest, plagiarism,

sources, censorship, credibility) and is related to trustful sources and manipulation of evidence

(such as photographs) [Steele & Black, 1999].




The Radio-Television News directors Association states in its code of ethics that “Professional

electronic journalists should operate as trustees of the public, seek the truth, report it fairly and

with integrity and independence, and stand accountable for their actions.” [RTNDA, 2000]. In the

paragraph concerning truth it is stated that professional electronic journalists should continuously

seek truth and should not report anything known to be false. The dissemination of accurate

information is also the principal aim of the code of ethics of the National Association of Science

writers [NASW, 2006].

The National Press Photographers Association code of ethics also points to the duty for

documentary photographers of truthfully informing public. This includes not “to alter the content of

a photograph in any way (electronically or in the darkroom) that deceives the public.” [ASNE,

1999]




Beyond journalism, ethics also concerns fiction. Fiction writing for instance is subject to rules of

no-harming and rules of no-hoaxing [Hansen, 2007]. The main problem is represented by not

selling for truth what is fiction.




                                                                                                  40
A world-famous attempt to produce an illusion of reality by the mean of a traditional medium is

constituted by the broadcast Welles’ War of the Worlds, played by Orson Welles and diffused in

the framework of the series Mercury Theater on the Air. At some extend the attempt worked, and

a certain number of American people resulted convinced of being under Martian attack and

reacted with panic. The broadcast became an enormous success for CBS who delivered it and

for Orson Welles. In 1938 American listeners were not used to broadcasts such as “The war of

the worlds” where a fiction was delivered in the form of newsflashes interrupting a music program

for relating the dramatic news of Martian invasion. Newsflashes were hence trusted.

Nevertheless, CBS escaped punishment because the fictional nature of the performance was

reminded through all the broadcast (even if with large holes, for instance, between minute 12 and

minute 40 of the broadcast). In the mean time, CBS was formerly invited to avoid the “we interrupt

this program" device in fictional contexts.

Since then, US TV broadcasts featuring realistic news bulletins post messages to inform

audiences of the fictional nature of the spectacle.




Many other examples of hoax and gullibility are reported:

                       -   The Moon hoax: perpetrated in 1835 by The New York Sun, claiming

                           that men had been seen on the Moon through telescopes

                       -   the autopsy of the Roswell alien and hundreds of flying saucers

                           pictures,

                       -   the Loch Ness monster pictures,

                       -   the Cottingley Fairies pictures,

                       -   the Belgian Secession hoax,




                                                                                               41
-   UK Anglia Television Alternative 3




                                         42
c. Deception as illusion of non-mediation


Perceptual objections to the illusion of non-mediation




The second form of deception consists in taking a mediated experience for a

non-mediated one. The fictional object then gives rise to what [Dennett, 1991]

refers to as a ‘strong hallucination’: “a hallucination of an apparently concrete and

persisting three-dimensional object in the real world – as contrasted to flashes,

geometric distortions, auras, afterimages, fleeting phantom-limb experiences,

and other anomalous sensations. A strong hallucination would be, say, a ghost

that talked back, that permitted you to touch it, that resisted with a sense of

solidity, that cast a shadow, that was visible from any angle so that you might

walk around it and see what its backed looked like” [Dennett, 1991, p. 7].

[Dennett, 1991] denies the possibility of experiencing strong hallucinations on the

basis of considerations concerning the dynamic nature of perception: how the

world feels depends on how the subject decides to move [see also O’Regan &

Noe, 2001; Casati & Pasquinelli, 2005]. In order to produce strong hallucinations,

it would hence be necessary to calculate all the possible sensations that could

arise in response to all the possible actions of the user, and to propose them in

real time.



                                                                                  43
Other considerations about the nature of perception, and in particular about the

fact that what we perceive is a complex pattern of stimulation, speak against the

possibility of being totally mistaken about the mediated or non-mediated nature of

a perceptual experience. According to the ecological view of perception

[Stoffregen, et al, 1997; Stoffregen, 20003], the stimulus condition (ambient

array) is specified in a different way by parcels of the real world and by

depictions. In fact is supposed that the relationship between a parcel of reality

and the ambient array that it structures is a 1:1 relation [Gibson, 1966]. Hence,

only identical parcels of reality can produce the same ambient array and give rise

to the same pattern of sensory stimulation. As other parts of the real world

simulator systems and cinema then structure the ambient energy fields in a

special manner that is specific to them and that is necessarily different from the

way ambient energy fields are structured by the simulated things. Since, in the

case of mediated experiences, both the contents and the context (the medium

and its characteristics) of the experience contribute to specify its perceptual

characteristics, mediated and non-mediated experiences are by necessity

perceptually distinct.

Not only the contingent limits of technology, but also the necessary nature of

perception would hence prevent the possibility of being totally mistaken about the

mediated or non-mediated nature of a perceptual experience. These perceptual




                                                                               44
objections also show the continuity between traditional and new media as for

what concerns the problem of producing an illusion of reality. Multimodal

stimulation in fact still brings information about the presence of the medium and

interactivity encounters the problem of calculating the responses to all the

possible actions performed by users.




                                                                              45
Perceptual objections to the illusion of non-mediation



Perceptual objections to the illusion of non-mediation: the multimodal nature of

perception




Perceptual objections to the illusion of non-mediation: the dynamic nature of

perception




                                                                             46
Perceptual objections to the illusion of non-mediation:                           the multimodal

nature of perception



Stoffregen (2003) affirms that believability consists in the (accurate) perception of realism and not

in the illusion of reality: we must distinguish between the perception of reality (inaccurate or

illusory perception that the simulator is the simulated) and the perception of realism (accurate

perception of resemblance between simulator and simulated).

Awareness of a realistic simulation need not imply belief in the reality of that which is simulated.

Moreover, the literal illusion of realism is likely to be very rare. If sensory stimulation produced by

a simulator were exactly identical to the sensory stimulation produced by the simulated thing,

then we could speak of complete sensory fidelity; but it is affirmed that 100% accuracy is

impossible for technology.




It is argued in fact that simulations and the things that they simulate always give rise to different

patterns of sensory stimulation.

First, simulators are real devices as simulated things are. As the other parts of the real world

simulator systems structure the ambient energy fields in a special manner that is specific to them

and that is necessarily different from the way ambient energy fields are structured by the

simulated things. In fact is supposed that the relationship between a parcel of reality and the

ambient array that it structures is a 1:1 relation. Hence, only identical parcels of reality can

produce the same ambient array and give rise to the same pattern of sensory stimulation. In

particular, it is stated that simulators cannot actually give rise to the multisensory complexity of

the simulated parcels of reality, in particular when vestibular and somatosensory arrays are




                                                                                                    47
concerned. The possibility that not all the details of the array are perceived and hence that the

perceptual systems can tolerate many types of non-identity between ambient arrays in virtue of its

psychophysical limitations is suggested by Hochberg (1986). But Stoffregen (2003) considers this

suggestion more plausible for optic and acoustic simulation than for haptic and inertial simulation

where the magnitude of the non-identity between simulator and simulated is greater. It is affirmed

that users can pick up the differences or non-identities between the ambient array produced by

the simulated and the ambient array produced by the simulator even for very advanced flight

simulators. The user is not necessarily aware of the difference, but reactions such as sickness

which are not present in the simulated situation but only in the simulator condition would indicate

the fact that some difference is perceived. For consequence, a fully faithful simulation may be

impossible for most situations. Hence full realism as simulation is a chimere.




If this is true, then it would be inappropriate to expect a simulator to fool

the user; it would also be inappropriate to use subjective reports of “realism” as a basis for

evaluating simulations.




Second, perception is selective and depends on exploration. Simulators can be taken for the

simulated thing only if the possibility of exploration is suitably reduced. In fact, exploration has a

role in disambiguating information that can be used by the perceptual system in order to

distinguish the simulator and the simulated. But restrictions to exploration also alter the sensory

stimulation in a way that informs the subject that ihe is not in the simulated situation, but in the

simulator.




Third, the user has prior knowledge that he is experiencing a simulator. The same is true in the

case of cinema and might explain why spectators do not run away in panic during horror movies.



                                                                                                   48
In fact, they did at the beginning of the century, when they were not acquainted with cinema: they

run away or fire at the sight of trains or bears (Stoffregen, 1997, 1998; Shapiro, 1992). Stoffregen

(1998, 2003) suggests that the awareness of fiction or simulation does not necessarily lay in

cognition but maybe in perceptual processes.




Some objections can be raised: the flight simulator is not realistic for someone who has piloted an

aircraft, because the two ambient arrays are compared. But what for a non-experienced user? Of

course, flight simulators are designed for training and create expert users, so the point made by

Stoffregen (2003) is quite destructive. But only when simulation of experiences with the real world

is concerned. When none of such experience is directly evoked, or only parcels of the experience

with the real world are evoked (only those parcels that can be faithfully rendered) the argument of

Stoffregen (2003) is no more valid, even in the case of simulation (at least in the case of limited

simulation). Additionally, believability is not necessarily a matter of perceiving the virtual reality as

being identical to a parcel of the real world, in other words, it is not necessarily a matter of

simulation. An expert flyer can consider the flight simulator as non-believable because it is a flight

simulator, and he, the pilot, is an expert in aircraft: the aim is to create a device which responds to

all the expectations that are raised by a real simulator. But in other cases, one can be satisfied in

creating a device that responds only to some of the expectations raised by the simulated parcel of

reality or even a device that creates by itself a number of expectations that are different from

those inherited from the experience with the real world. VR systems should anyway explore

applications different from simulation and/or adopt special narrative strategies that justify

limitations in the explorative possibilities and in the sensory stimulation.




It is suggested by Stoffregen (2003) that simulators may best be evaluated in terms of behavioral

outcomes (e.g., changes in performance), rather than on the basis of subjective experience. What



                                                                                                      49
is evaluated in this case is the effectiveness of the simulator and not believability or other

subjective feeling or judgment.

Riccio (1995) has introduced two terms for evaluating simulators in terms of their fidelity to the

simulated condition: experiential fidelity and action fidelity. Experiential fidelity is the extent to

which the simulator gives rise to an experience of being there, hence to the classic definition of

Presence, and to an illusion of reality: the illusion that the simulator is the simulated and is real.




Stoffregen (2003) asserts the importance of distinguishing the two types of realism that are at

stake: realism as resemblance between simulator and simulated and realism as illusion of reality.

In fact, a representation such as a picture can resemble to the represented parcel of reality and

hence give rise to perception of realism, but the representation is not perceived as real

(perception or illusion of reality).




This distinction seems not be respected in Presence questionnaires that display the following

type of question: “how real does this seem?”; a better question in the case of illusion of reality

would be: “is this a simulation or is this reality?”.




Stoffregen (2003) nevertheless fails to notice that the second concept (those of illusion of reality)

is also broader than the first one since it can be adopted in fictional and virtual systems with no

simulation of parcels of the real world.




Stoffregen (2003) proposes to leave the perception of reality and to concentrate on the perception

of realism, that is, on the resemblance between the simulator and the simulated. He also cites the

case of fictional entertainment, in which audiences are induced to suspend disbelief, that is, to




                                                                                                         50
find the representation realistic even if not real. Suspension of disbelief is hence related to the

perception of realism and not with the perception of reality (Goffmann, 1974).




But, as we have seen, perception of reality is a broader concept than perception of realism

because it applies also to non-simulation virtual systems and to non-simulation fictional

entertainments: if we adopt the suggestion of evaluating virtual systems in terms of perception of

realism we have no mean to evaluate non-simulation virtual systems and fictional entertainments.

It is true that the notion of illusion of reality is problematic: we suggest that audiences and users

are not fooled, as affirmed by Stoffregen (2003) even in fictional and virtual cases with no

simulation: users and audiences are always aware of the fact that what they are perceiving or

reading about is not the real world but a fictional or virtual one that can nevertheless be more or

less believable. Since the fictional or virtual world is not necessarily a simulation of the real one,

the judgment of believability cannot be coincident with the perception of realism. We suggest that

the judgment of believability has something in common with the perception of reality because it is

not an illusion even if it is a subjective judgment and because it is associated with behavioral and

emotional responses that are similar to those that one would present if the situation were real

(even if not similar to situations that can be experienced in the real world); the judgment of

believability can also include perception of realism when the fictional or virtual situation simulates

parcels of the real world. But it is exactly because audiences and users are not fooled about the

reality of the objects of their experience that they can find the experience to be believable even if

it does not display situations that simulate parcels of the real world.




Stoffregen (2003) proposes to evaluate simulators in terms of action fidelity. Action fidelity is

defined as the transfer of performance from the simulator and the simulated system. Action

fidelity exists when performance in the simulator transfers to behavior in the simulated system (or



                                                                                                   51
vice versa). Action fidelity is hence measured via the evaluation of task performance: time to

completion a task, variance in performance across trials, etc.




What is more important is that the transfer of performance is not necessarily related to high

realism in the sense of resemblance with parcels of the world. Action fidelity is hence not a

measure of the perception of realism, no more than it is a measure of the perception of reality.

Moreover, sometimes, Stoffregen (2003) affirms, in certain cases departures from stimulus fidelity

might produce enhancement of action fidelity. Action fidelity is hence a pragmatic measure, which

can be used with simulators even if they are not perfect simulations of aircraft or other for what

concerns the sensory stimuli they display and even if they do not fool the users in the sense of

appearing real things.




The transfer of performance could be one of the behavioral consequences of believability, at least

in the cases where virtual or fictional environments present some form of realism as simulation

and are designed with the aim of training. The idea of measuring actions and reactions that are

similar to those one would experience if the experience were true (with the limitations and

characteristics of a special game of make-believe) is not so far from the idea of transfer of

performances. One can in fact think to the performances in the real world and the performances

in a game of make believe as a sort of transfer of performances.




                                                                                               52
Perceptual objections to the illusion of non-mediation: the dynamic nature

of perception



In his prelude to the explanation of consciousness, Daniel Dennett inquires about the possibility

of experiencing “strong hallucinations”:

“By a strong hallucination I mean a hallucination of an apparently concrete and persisting three-

dimensional object in the real world – as contrasted to flashes, geometric distortions, auras,

afterimages, fleeting phantom-limb experiences, and other anomalous sensations. A strong

hallucination would be, say, a ghost that talked back, that permitted you to touch it, that resisted

with a sense of solidity, that cast a shadow, that was visible from any angle so that you might

walk around it and see what its backed looked like .” [Dennett, 1991, p. 7]



The problem of strong hallucinations is relied by Dennett to the skeptical mental experiment of the

brain in a vat: is it possible that we are just brains in a vat, suitably stimulated by an evil scientist

in order to experience the sensation of living in a world of objects?




A third kind of experience which presents us with the possibility of experiencing the sensations of

objects in absence of real objects is represented by Virtual Reality systems (VRs). VRs are ….

“The state of the art is impressive: electronically rigged gloves that provide a convincing interface

for ‘manipulating’ virtual objects, and head-mounted visual displays that permit you to explore

virtual environments of considerable complexity. The limitations of these systems are apparent,

however, and they bear out my point: it is only by various combinations of physical replicas and

schematizations (a relatively coarse-grained representations) that robust illusions can be




                                                                                                      53
sustained. And even at their best, they are experiences of virtual surreality, not something that

you might mistake for the real thing for more than a moment.” [Dennett, 1991, p. 6-7]



As we can see from this quotation, Dennett’s answer to the three questions is a radical ‘no’:

strong hallucinations cannot exist, we are not brains in a vat and VR systems will never fool us in

such a way that we can take a virtual object for a real one. This is not a purely contingent

impossibility, ascribed to the present limits of technology, but sounds like a permanent one, an

impossibility in principle of creating believable sensations of non-real objects (objects that do not

exist in the real world as material things) by means of whatever developed interfaces. Unless

(since there is a clause in Dennett’s assertion) we are capable of producing “ physical replicas and

schematizations (a relatively coarse-grained representations)” of the real world and objects. But

again, a replica of the world, even a replica of a simple coin, is far beyond human technology.




Dennett’s argument against the believability of virtual objects is grounded on a specific vision of

human perception: how the world feels depends, according to Dennett, on how the subject

decides to move.

An evil scientist or a VRs designer should thus be able to provide their subject with dynamic

stimuli, that is, with stimuli that represent the feedback to the actions of the brain-user.

There’s the rub.    Since calculating the proper feedback implies the capability of taking into

account all the possible actions of the brain-subject and all the possible sensations that arise in

response to each action or the capability of calculating, generating and proposing the proper

feedback in real time. The first operation requires the possibility of storing an enormous amount

of information; the second operation is, according to Dennett, intractable on even the fastest

computer.




                                                                                                  54
Only the world or a replica of the world can store such an amount of information as it is implied by

the dynamic character of human perception; but creating a replica of the world out of nothing but

mere stimulation is beyond human technology.


“Throw a skeptic a dubious coin, and in a second or two of hefting, scratching, ringing, tasting and
just plain looking at how the sun glints at its surface, the skeptic will consume more bits of
information than a Cray supercomputer can organize in a year. Making a real but counterfeit coin
is child’s play; making a simulated coin out of nothing but organized nerve stimulations is beyond
human      technology     now     and     probably     forever.”    [Dennett,     1991,     p.    6]




                                                                                                 55
d. Deception as illusion of transportation


Naturalization of the illusion of transportation: intersensory illusions




Can one experience an illusion of transportation in the virtual world, of being in

the virtual world, without being mistaken about the virtual nature of this world and

eventually without loosing the awareness of mediation? It seems possible to

naturalize some of the phenomena that are referred to as ‘illusion transportation’

or of ‘being there’ by suggesting that certain experiences trigger illusions of

special kinds, such as proprioceptive illusions that extend the boundaries of the

subject’s body and proprioceptive illusions of self-motion. It seems possible to

produce such illusions in the form of visual bias upon proprioception induced by

mirrors, artificial body parts or prisms [Welch & Warren, 1981; Nielsen, 1963;

Holmes, 2004; Ramachandran, et al., 2005; Gibson, 1933; Hay, et al., 1963;

Tastevin, 1937; Botvinik & Cohen, 1998; Pavani, et al., 2000]). These

phenomena are far from equating the massive experience of transportation

described by cyberpunk writers and seek out by VR designers, but they are

compatible with both VR technology and human perceptual mechanisms. The

experience of proprioceptive illusions is also compatible with the cognitive

awareness that the second world one feels to move in is a world of imagination.




                                                                                 56
Contrary to what could appear, awareness of the nature of the experience can be

exploited for enhancing believability. The fact of being aware that they are

dealing a fictional world lowers in fact users’ and audience’s expectations and

reduces the risk of expectations’ frustration, which constitute a major threat for

believability (Garau, 2003). The context of the experience (the presence of the

medium, its fictional nature) should hence not be hided or made transparent. At

the opposite, users should be made aware of the characteristics of the medium,

in terms of the possibilities the medium offers to users (quantity and quality of

sensory stimulation, possibility of interacting, of modifying the environment vs.

passive interactions) and in terms of the specificity of the interaction with virtual

worlds. As in the case of other experiences with worlds of fiction, users are asked

to activate their imaginative functions in relation to perceptual representations

that act as props for imagination [Walton, 2001] and that can change in response

to the player’s actions (which is not the case for traditional, non-interactive media

[Casati & Pasquinelli, 2005]).




                                                                                  57
Naturalization of the illusion of transportation: intersensory illusions


The illusion of reality as illusion of transportation or illusion of presence or even

as illusion of non-mediation could be explained by suggesting that certain

experiences (both in mediated and in non-mediated conditions) trigger illusions of

special kinds, such as: proprioceptive illusions that extend the boundaries of the

subject’s body and proprioceptive illusions of self-motion. These illusions are

particularly related to the components of the illusion of reality that are referred to

as “illusion of transportation” and “illusion of presence in the virtual environment”.

Other illusions can concern the reciprocal bias of vision, audition, haptic

perception and the capacity of producing illusory perceptual outcomes. They are

not related to the sense of transportation or of movement but rather to the

perception of special qualities of virtual objects.

The individuation of these perceptual illusions, the fulfilment of the conditions for

their production and the evaluation of their presence and strength is aimed at

producing a naturalization of the notion of illusion of reality in its different

components. The production of illusions and their evaluation does not exhaust

the tasks of producing believable fictional and virtual experiences and of

evaluating them in terms of believability. Believability is a broader concept that




                                                                                   58
goes beyond the purely perceptual level and includes perceptual and symbolic

expectations.




The study of proprioceptive and motion illusions produced by different sensory

modalities produces a naturalization of the illusion of reality because it provides

scientific bases, documented in cognitive sciences literature on perception, for

the sensations of being there and moving in a virtual environment and for the

identification of one’s own avatar.




These illusions are mainly related to intermodal bias. In a situation of conflict

between information provided to different sensory modalities (vision and touch,

vision and proprioception, vision and audition, audition and touch, audition and

proprioception) illusions are produced that consist in a bias (more or less

reciprocal) of one sensory modality upon the other. Vision has been considered

for long the dominant sensory modality in case of conflict [Rock & Victor, 1964]

because more precise than the others. After contrary evidence, the precision

hypothesis has been retained [Welch & Warren, 1981] but not the necessary

dominance of vision. More recently, precision has been substituted with minor

variance [Ernst, et al., 2004]. The more precise, less variant information biases

the less reliant one. Since vision is usually more precise and reliable for spatial




                                                                                59
location it normally dominates other modalities for this kind of task. In case of

three sensory modalities involved (audition, vision, proprioception), majority

(accord between two modalities in disaccord with the third one) is counter

intuitively not trusted [Bedford, 2007]. This is an important datum for situations

such as cinema and VR where stimulus co-location is not granted.




In addition to psychological studies on intermodal bias and illusions, the

relationship between vision, audition and proprioception is confirmed by the

neurophysiology of multimodal perception [Stein & Meredith, 1993]: overlaps of

proprioceptive and visual brain areas and the existence of multimodal

proprioceptive and visual neurons are described at the level of the superior

colliculus of the cat. Aural perception is involved in the same multimodal

systems, thus suggesting the possibility of auditory-visual interactions (see

Believability & sound) but also of auditory-proprioceptive interactions. These

interactions concern sound localization biased by proprioception, but also touch

and proprioception biased by acoustic perception.




Four main types of illusions are described that can be exploited for producing the

sensation of being in a certain environment, of moving in this environment, of

moving following a certain pattern, of being identified with one’s own avatar and




                                                                               60
of proprioceptively feeling the sensations the avatar is supposed to feel. They are

mainly described as visual capture upon proprioception because vision biases

proprioceptive outcomes, but audition to can be exploited in this sense

(especially for producing self-movement illusions):




Mirror illusions

Proprioceptive illusions induced by prisms

Proprioceptive illusions induced by artificial body parts

Self-movement illusions (illusions of vection)

Auditory-tactile illusions




Knowledge of these illusions has not only a theoretical effect on virtual reality

(the understanding of the perceptual bases of experiences in virtual reality) but a

pragmatic one too. The knowledge of the constraints that produce such illusions

in natural conditions can in fact be used as guidelines for producing the same

effects in virtual reality, hence for enhancing the so-called illusion of reality in

specific domains. Reciprocally, the evaluation of the presence of illusions can be

done in objective ways by measuring the strength of the robustness of the illusion

and the degree of surprise that is experienced when the illusion becomes aware




                                                                                 61
(intersubjective and intrasubjective robustness and surprise being the main

structural characters of illusions).




One the constraints that seems to be necessary in the case of mirror illusions is

the fact of moving. Other constraints concern the cognitive assumption of unity

(mirror illusions with different pattern of movement between vision and

proprioception). Other illusions (rubber hand illusions) are facilitated by an initial

synchronization and match between vision and proprioception, or by the

combination of visual and aural stimuli (vection illusions) hence by multimodal

stimulation.



VR systems that provide the possibility of moving and multimodal stimuli are

especially suited for producing the proprioceptive illusions described, hence the

naturalized “sense of transportation and presence”. At the opposite of passive

media, such as cinema, a correspondence can be created between the user’s

and the avatar’s movements or between the user’s movements and perceptual

outcomes. In cinema, the movements of the camera do not correspond to the

movements of the spectator’s head and the movement of objects on scene is

independent of any action of the spectator. It is hence easier to produce certain

proprioceptive illusions induced by vision in VR rather than in cinema or other




                                                                                   62
passive media. This fact could explain why VR is considered to produce a

greater illusion of reality and a greater sense of transportation than passive

media, and why this greater power of presence is attributed as a matter of fact to

the possibility and degree of motor interaction. In any case, the possibility of

producing proprioceptive illusions cannot be confounded with an illusion of

reality.




In the case of multimodal media with haptic feedback sound can play an

additional role by influencing not only vision but also haptic perception.




How much realistic must visual or other avatars-representations must be for

producing proprioceptive illusions?

Realism is only relative, because rubber hands but also other non-human-like

artefacts are effective in producing proprioceptive illusions, and because the

seen movement in mirror experiments does not necessarily match performed

movement. In this latter case, however, a cognitive assumption of unity (common

source between visual and proprioceptive stimulus) is made on the side of the

subject of the experiment who believes that he sees his hand in a mirror. In

addition   to   the   understanding   of   perceptual constraints    for producing

proprioceptive illusions of de-location, cognitive constraints must also be




                                                                                63
individuated that can let subject to make an assumption of unity (in the case of

sound and images in cinema, for instance, it seems that spectators can attribute

the two to one and the same source because this source is cognitively identified

as fictional rather than real, see Believability & sound).




These considerations suggest the opportunity of making the point about the

different roles that different perceptual modalities play in the believability of

mediated, fictional or virtual experiences. This role in fact is not necessarily

cohincident and cannot be fully understood by the simple reference to realism, to

the idea that adding stimuli for different sensory modalities enhances the realism

of the experience. The case of sound is paradigmatic.




In cinema, for instance, sound is attributed the functions of creating atmosphere,

suggesting time and space localization, commenting visual scenes, creating

emotional moods (see Believability & sound). The capacity of suggesting a

certain time and space location for the narration is different from an illusion of de-

location, because the spectator is not necessarily transported by the sound in old

Scotland, but he just becomes aware (thank to experience and acquaintance with

filmic techniques) that the narration develops in old Scotland. The integration of

sound and images does not depend on co-location of visual and aural stimuli or




                                                                                   64
    on common source, because the source of sound is rarely coincident with the

    screen. This integration is not necessarily a perceptual illusion either.

    In addition to completing or modifying information gathered though vision, sound

    can also help directing the attention toward the visual stimulus it matches with,

    thus minimizing distractions provoked by the environment. Matched multimodal

    stimuli have in fact the effect of enhancing neural responses [Stein & Meredith,

    1993].




    Auditory stimuli hence do not necessarily add realism or presence, even when

    combined with visual ones in unitary percepts, but can help

-      directing attention,

-      enhancing neural responses

-      and providing new information about the virtual or fictional objects’ and events

    represented. In general, auditory stimuli provide additional information to visual

    one which can complete it, be redundant or even be in contradiction with it.

    Auditory information can for instance extend the experienced space beyond the

    visual one thus suggesting fictional entities that are not in sight (extension of the

    perceivable virtual or fictional space [Riecke, et al., 2005]). This possibility is

    often used for guiding behaviours, such as when listening at the sound produced

    by a car while driving for detecting eventual engine troubles. Adding engine




                                                                                      65
    sounds to car simulators is hence certainly functional when the driver is

    supposed to acquire the capacity of making such a judgment or to exert this

    capacity in the virtual setting. However, the inference from functionality in relation

    to a task to the supposition that this sound will certainly add realism and that it is

    required for realism because it exists in real life situations [Riecke, et al., 2005] is

    not authorized. Movies spectators pass unnoticed the absence of most of natural

    sounds in movies and are even disturbed by the presence of as many sounds as

    they would hear in analogous real life conditions (at least according to sound

    designers, see Believability & Sound).

    Sounds play another important role, which is not strictly related to perceptual

    identification or recognition of objects’ and events’ characteristics:

-      they provide instructions to users/spectators about how to feel and what to

    think about what they experience. This function is much more in use in cinema

    than in VR (the emotional role of sounds is recognized by VR designers, see

    [Riecke, et al., 2005] but it is not interpreted as a form of instruction).

    Finally, sound can be used to

-      produce illusions (through intermodal bias) in the visual and proprioceptive

    modalities (in analogy with vision) and auditory-induced illusions can be

    considered as another element for the naturalization of the notion of illusion of

    reality. Sound stimuli can be considered to produce “acoustic avatars” that match




                                                                                         66
VR users’ movements, such as in the case of the sound of footsteps that starts

when the user moves through a space and stop when he stops. It should be

ascertained if sound avatars are able to produce proprioeptive illusions

analogous to the rubber hand illusion produced by vision.




                                                                           67
Mirror illusions



[Nielsen, 1963] describes the following experiment with mirrors and substitution of the

experimental subjects’ visible hand with another subject’s hand. The subjects follow a straight line

with their hand in full sight; on some of the trials, a mirror is introduced, unbeknownst to the

subjects, so that they see another person’s hand (that they believe to be their own): the subjects

continue to have the sensation that the seen hand is their own, but they also feel as if they had

lost control over its movement.




[Holmes, et al., 2004] describes a battery of experiments that show the effects of proprioceptive

illusions provoked by the use of mirrors on reaching performances: reaching performances are

affected when the visual position of the reaching hand is altered by the use of a mirror reflecting

the non-reaching hand. The experiments show the presence of a proprioceptive illusion of

position of the reaching hand (visual capture) and the authors hence ascribe the error in the

reaching task to the proprioceptive illusion of position.

The presence of discrepancies between proprioception and the visual position of the hand have

disruptive effects on reaching tasks (measured in terms of lowered speed and of accuracy, that

is, of the distance between the finger tip and the target to be reached) even in virtual reality

contexts [Durlach, et al., 2005]. This fact confirms the general consideration about the importance

of coherence for adaptive behaviours and the disruptive effects of discrepancies upon

performance [Stein & Meredith, 1993].

This fact seems also to suggest that visual capture phenomena (proprioceptive illusions induced

by vision) and of so-called illusion of reality in general might play a negative role on




                                                                                                 68
performances, rather than a positive one, by confusing the subject and by disrupting or adaptive

behaviours (an effect of violations of coherence in general) .



Mirror illusions are exploited for the treatment of pathologies such as the phantom limb, stroke

and hemiplegia and spatial neglect (the success with pathologies other than phantom limbs is still

uncertain) [Holmes, 2004; Altschuler, et al., 1999; Sathian, et al., 2000. Ramachandran, et al.,

1999].

[Ramachandran      &    Hirstein,   1998;   Ramachandran         &   Rogers-Ramachandran,      1996;

Ramachandran & Blakeslee, 1998; Ramachandran, et al., 1995] describe a simple device, the

virtual reality box, or mirror box as being effective in reducing the painful palsy of the phantom

limb, by visually inducing an illusory sensation of movement and position.

The device is composed of a box with two holes in the frontal part of it and a mirror inserted in the

middle. The patient who suffers from phantom palsy (which provokes pain) of his phantom hand

(a quite common phenomenon in subjects who have been amputated after experiencing a palsy

of the affected arm) inserts his hands in the holes. Naturally he really inserts just the normal

hand, but, since he can feel his phantom hand’s position and movement, he has the feeling of

having inserted the phantom limb too. The mirror allows the patient to visualize the phantom hand

that he can only feel: it is the reflection of his normal hand. The patient is asked to open his real

hand and move it, and at the same time to ‘try’ to open his phantom hand. In this way he

provokes the vision of the phantom hand as moving normally and opening against the palsy.




                                                                                                  69
Proprioceptive illusions induced by prisms



Other experiments have been conduced with prisms.

A classic experiment on visual capture upon proprioception (a prism is fixed on the subjects’ head

which displaces the visual position of the arm) is described by [Hay, et al., 1963].

More recently, [Mon-Williams, et al., 1997] have confirmed these results.

[Gibson, 1933] describes the following illusion: a subject moves his hand along a straight surface

while looking through a prism that causes the surface to look curved; he feels it to be curved as

well.




                                                                                               70
Proprioceptive illusions induced by artificial body parts



Another group of proprioceptive illusions with visual capture has been demonstrated by the use of

artificial body parts.

[Tastevin, 1937] describes the attribution of sensations to an artificial finger when the hand of the

subject is not visible. This illusion, known under the name of “fake hand illusion” or “rubber hand

illusion” has been recently re-demonstrated by [Botvinik & Cohen, 1998] and [Pavani, et al.,

2000]. Others studies on artificial substitutes are: [Austen, et al., 2003; Austen, et al., 2001].

When a rubber hand is brushed in contemporary with the brushing of the hidden hand of a

subject, the subject feels the rubber hand to belong to his body.




The same sense of ownership is obtained in the same conditions (vision + haptic perception

provoked by stroking) when the rubber hand is substituted by a neutral object: shoe or table.

Visual realism is hence not required. This fact suggests that ownership (measured as the

displacement of the sensed position of the real, stroked or brushed hand in relation to the artificial

object) can happen also for avatars that do not represent life-like entities [Armel &

Ramachandran, 2003]. [Armel & Ramachandran, 2003] suggest that this illusion is due to a

Bayesian mechanism of learning of the association between visual and tactile stimuli that are

synchronously matched: any object can be self-attributed whether the statistical correlations

between visual and tactile sensations are strong enough. [Tsakiris & Haggard, ] challenge this

model and even the possibility of producing proprioceptive illusions just by synchronization: both

the aspect and the position of the artificial object seem to count (rubber hand simulates the

opposite hand than the one which is stroked).




                                                                                                   71
It is also possible to visually induce the proprioceptive sensation that the first finger is stimulated

while the fifth one is by projecting the image of a hand which first finger is stimulated. This visual

bias over proprioception only happens when the tactile stimulation of the fifth finger and the

stimulation seen on the screen upon the first finger are in phase. Time matching seems hence to

be relevant in addition to the presence of real tactile stimulation [Schaefer, et al., 2005]. In

conclusion it is possible to provide stimuli for a body part that it is not actually stimulated when

some body part is and when there is consistency in the timing of the two stimuli.




Another variance of the rubber hand illusion can be obtained with tactile and proprioceptive

modalities only, without vision: a blindfolded subject touching with one hand a rubber hand and

synchronously being touched on his other hand, appropriates the rubber hand as his own and

feels like touching his own hand [Ehrsson, et al., 2005].




                                                                                                    72
Illusions of vection



As a matter of fact, proprioception biases audition as shown by the existence of audiogyral (when

the whole body rotates, the auditory target is heard to displace in the opposite direction to rotation

during acceleration, to come back during constant velocity and to displace in the sense of the

rotation during deceleration) and audiogravital illusions (auditory localization of an auditory

stimulus influenced by linear acceleration; this phenomenon can be considered as an illusion or

as correct perception in a different environmental frame-work, in any case it confirms the

influence of proprioception upon auditory perception) [DiZio, et al., 2001].




Also, auditory biases upon proprioception exist. An example is represented by self-motion

illusions produced by the rotation of auditory or visual stimuli (vection) [Riecke, et al., 2005]. The

illusion induced by vision is frequently experienced in natural conditions (the start of a train seen

from a stationary train which is perceived as ego-movement is a classic example of vection).

Audition-induced self-motion sensations are less compelling than vision-induced ones and cannot

be produced in more than 25-60% of bindfolded participants [Lackner, 1977; Larsson, et al.,

2004]. Increase of these effects can be produced by the combination of visual and auditory cues,

and even by the combination of visual cues with vibration [Riecke, et al., 2005]. Auditory cues

must be specific for obtaining this effect, that is, they should match visual cues (spatialized

sounds that can be attributed to the same source than the visual images). It seems that also top-

down processes play a role in inducing vection. The identification of a source of auditory stimuli

that can be consistently interpreted with self-movement enhances vection produced by auditory

stimulation [Riecke, et al., 2005].




                                                                                                   73
In case of devices directed to propose users with experiences of locomotion and in general of

ego-motion in VR, different procedures can be adopted, from treadmills to head-mounted displays

and free walking to the production of an illusion of self-motion through visual and eventually

tactile (vibration) or acoustic stimuli. [Riecke, et al., 2005] affirm that no one of these alternative

procedures is capable to produce a convincing sensation of self-motion in the virtual environment.

This is an exemplary case in which the evaluation of the user’s performance does not need to

take recourse to the notion of illusion of reality or of presence but only needs to be evaluated

against the production of a convincing sensation of vection. In the case of the procedure of using

visual or other stimuli for producing an illusion of self-motion, the illusion of self-motion constitutes

both the target and the behaviour to be evaluated in order to measure the efficiency of the VR

device. The experience that is intended to produce and that must be evaluated is not a not-well-

defined illusion of reality or presence or realism, but a well-known illusion of vection. In this

particular case of devices that propose users with experiences of self-motion, the degree of

illusion of reality can hence be naturalized in terms of the strength of the illusion of self-motion (in

certain cases illusion of vection) which is produced by that certain device (a strength that can be

measured in terms of robustness, at the intersubjective and intrasubjective level, and in terms of

surprise when discovering that no self-motion occurred (for instance by presenting conflicting

information that concerns the static condition of the user).

If it is true that “In the long run, a deeper understanding of any potential causal relations between

presence and the effectiveness of a simulation for a given task or goal (here: self-motion

perception) would be rather helpful for optimizing VR simulations from a perceptual point of view”

[Riecke, et al., 2005, p. 9], then the naturalization of the notion of presence is not a secondary

task for VR designers.

In the perspective proposed here, the comparison of presence tests and of vection evaluation and

the search for causal relations between the two [Riecke, et al., 2005] seems rather odd because



                                                                                                      74
presence is naturalized in terms of this same task. On the contrary, the subjective evaluation of

vection could be doubled with an objective evaluation of the robustness of the illusion and of the

level of surprise produced by the illusion’s awareness. Thus adding objective rather than

subjective measures.




Another consideration from vection illusions concerns the importance of topdown processes in

the proposition of believable experiences in general but also, as it seems, of perceptual illusions.

Top-down knowledge is demonstrated to play a role even in other illusions, such as the golf ball

illusion (a weight illusion that only can be evoked for expert golf players). In the case of vection

induced by acoustic stimulation the appropriate use of sounds corresponds to an appropriate use

of the narrative elements of the experience in order to influence the perceptual ones.




                                                                                                 75
Auditory-tactile illusions



The co-occurrence of tactile and auditory stimuli results in combination of the two, with relative

modification of the single perception of each of the unimodal stimuli, in spite of the fact that

subjects to experiments are instructed to ignore the presence of background stimuli [Ernst, et al.,

2004]. This attitude is attributed to the fact that matching between co-occurring multimodal stimuli

is a common experience in everyday life (as when knocking at a door produces auditory signals

that match the number and strength of the actions of knocking as it is perceived through the

proprioceptive modality and trough the tactile one).

“This automatic combination likely results from the fact that matching between co-occurring

multimodal signals is very consistent across our everyday experience, so that the CNS can learn

to co-register sets of redundant sensory signals and identify every single set as elicited by the

same unique event or stimulus.” [Ernst, et al., 2004, p. 7].




                                                                                                 76
Illusion of reality. Ethical issues



In addition to cognitive and pragmatic considerations, distinguishing the different

aspects of the notion of illusion of reality is important for ethical reasons. Illusion

of reality as believability and illusion of reality as proprioceptive illusions of self-

motion in the virtual world appear in fact non only feasible but also ethically

neutral notions (the illusion of non-mediation on the contrary is apparently not

feasible), while the obliteration between fiction and reality can be opposed to

ethical principles.

The notion of illusion of reality, whether it is not naturalized though the reference

to perceptual illusions and whether it is not considered as a local, specific

phenomenon one can be or become cognitively aware of, presents ethical

problems connected with the fact of cheating spectators and users and to make

them believe that a fictional or virtual world is not a fiction or even not a

representation.




A paradigmatic example of the infraction to the ethical principle of keeping reality

clearly distinguished from fiction and imagination is represented by certain uses

that are presently made of Second Life, a largely visited on-line virtual world.




                                                                                     77
The term ‘virtual reality’ itself can suggest the wrong idea of a hybrid form of

experience which is neither the experience of the real world, nor the experience

of a fictional world. Virtual reality techniques provide perceptual stimulation that

responds, more or less in real time, to the user’s actions. Whether perceptual

stimulation and action are associated with some, even minimal or abstract

activity of imagination, perceptual experience becomes a fictional one too, where

the user pretends a certain world which he creates from his imagination following

the props that are provided by the system itself and that work as instructions for

imagining [Walton, 1981]. These techniques and these possibilities of action and

perception can be exploited for a large number of uses related to the

manipulation of perceptual, motor and symbolic representations, ranging from

learning, to operating or communicating at distance, to designing and

prototyping, to narrating and creating stories. Of course, all these operations can

produce secondary effects on the real user and on the real world. But all these

operations are done upon representations and within fictional worlds.




                                                                                 78
Second Life

The banner of the Second Life web page announces: “Your world. Your

imagination”. Second Life (SL) is a world of imagination which can be accessed

via the net. Tools are provided to users of SL for creating an avatar of the user’s

himself that will be driven by the user to explore representations of lands (for

free) or even to build objects on these lands and exchange objects by the mean

of money (not for free). As for games like Monopoly, money and buildings are

just make-believe ones and the representations that are manipulated by players

are props for driving and bounding their imagination. At the opposite of structured

games, the appearance of each spot of digital space in SL depends on the

esthetics of the spot’s owner and a variety of environments can be encountered.

Avatars are guided by players as children do with puppets in games of make-

believe or as adults do with other paper, electronic and on-line games of make-

believe (World of Warlock, Dungeons and Dragons, Lord of the Rings on-line).

Like puppets and at the opposite of various forms of Tamagotchi, events happen

to avatars only when players command actions: players cannot just see what

happens to their avatar and they must be connected to SL in order to make the

avatar appear somewhere in the SL world. There is no score or specific objective

to be achieved. The player is hence free to act in relationship to the represented

environment and to modify it. As multi-players games, the avatar encounters



                                                                                79
other avatars (other puppets) and dialogue and communication can be

established, together with the exchange of objects. This social dimension creates

boundaries to individual imagination.

By the fact that multiple users can furnish their spot in SL as they prefer, create

objects, personalize their avatar and choose how to make it interact (or not

interact) with other avatars, SL could appear as a collection of home pages

where anyone can present the contents he desires. This appearance is

reinforced by the range and diversity of uses that are actually made of SL:

- SL is used for performing make-believe individual or social games: creating

worlds with characters, environment, actions and events, pretending together (as

in other make believe games) at distance (as other on-line make-believe games

with or without perceptual cues). These actions and events include commercials

transactions in Linden dollars and firms selling and advertising digital

representations of objects for SL avatars and SL places (from representations of

t-shirts to representations of buildings)

- SL is used for virtual conferences and staff meetings through SL avatars of

firms really implanted in the real world that use SL in substitution of other tools

for virtual conferencing




                                                                                80
- SL is used as a space for commercial advertising and selling real objects and

not representations of objects: objects that will be used by the player and not by

the avatar (as it happens in any medium)

- SL is used for different types of simulation related to design, prototyping and

relative evaluations of usability, in analogy with other digital design and

prototyping tools and with other online tools for working

- SL is used by politicians and religious communities for expressing their ideas

(as TV and other passive media) or for discussing with audiences (as a real

debate) with the aim of convincing players - not avatars - to adhere to their ideas

- SL is used for gambling, with real money won or lost [], since Linden dollars can

be exchanged with real dollars. This fact makes any money transaction in SL a

transaction of money in the real world

- SL is maybe used for illegal exchanges such as, exchanges of real

pornographic photos or contacts within terrorist organizations (as the net is in

general)

- SL is used for educational purposes, for instance by letting people take a

course of some distant university (as for other hundreds of on-line courses on the

net), interact as in a class or even interact with avatars representing dead

famous people (as if a professor of history of science dressed like Galileo for

answering questions from audience).




                                                                                 81
    In spite of the multiplication of uses and places, what makes of SL a unitary world

    (of fiction) is the presence of a certain number of features that are common to the

    entire SL space and that are independent from the specific uses that can be

    done of it:

-             a specific language of programming though which digital representations

    are created

-             the accessibility with one and the same avatar to any place in SL

-             the presence of one and the same frame: the SL web page, to which one

    connects and disconnects at the beginning and at the end of his session of

    playing

-             the specific possibilities of action and perception: for instance, no haptic

    perception, limited auditory perception, movement of the avatars through simple

    commands of the hand/fingers, creation of objects via symbolic representations

    (language of programming) or via simple gestures

-             the appearance of the SL world:

o                    The visual appearance of SL for instance is not more cartoonist

    than the one of other on-line games, such as World of Warcraft and Lord of the

    Rings




                                                                                       82
o                   As cartoons, Second Life has its own physics, which is different

    from real world physics: no gravity, no weather, no respiration, possibility of

    flying, tele-transportation

-            the use of one and he same monetary value: the Linden dollar, whciih

    can be exchanged with real dollars.




    Are there (ethically) legitimate and non-legitimate uses of SL?

    SL world internal unity and the fact the at present 8 million users have subscribed

    to SL (with picks of only 40000 users actually connected in the same time to SL)

    makes SL interesting for those, politicians, firms, churches, educational

    institutions who want to join players rather than avatars thus introducing a

    different group of uses than pretending. The multiplication of real uses of SL

    creates a risk of obliterating the distinction between fictional worlds and the real

    world, hence creates a risk of hoax and gullibility. This risk is essentially

    produced by the fact that uses that have an effect in the real life and are directed

    to the life of the player beyond the connection to SL (commercial advertising for

    products that are sold in the real world, religious messages and political

    campaigns for elections in the real world, educational tools for learning at

    distance, casinos where real money can be won and lost) are not framed and

    clearly distinguished from uses that are directed to the avatar and that are related




                                                                                     83
to the pretending-fictional activity (selling and buying products for avatars or for

furnishing the SL world, creating fictional religions and fictional political structures

for SL citizens, etc.).

When users puppet-master their virtual avatars to buy clothes in SL

representations of shops, to ‘build houses’ (create digital representations of

houses) to live in SL, and invite other avatars to meet them, these events affect

the aspect of the SL virtual world and are circumscribed to a make-believe game

played with digital representations rather than with dolls or paper. As in any other

game of make-believe a player can express his deep feelings and true thoughts

through his own avatar, but he can also just imagine and lie without affecting the

game. One can make his avatar look and behave as much as possible as the

real player behaves in the real world, or he can puppet-master a penguin-like

representation that buys or destroys expensive objects, rapes or kills other

avatars, flies or uses tele-transportation. All this happens in a fictional world and

has the same value (and deserves the same ethical considerations) than other

activities of individual and collective imagination.




But what happens when the same avatar attends Stanford University lessons?

Who is learning: the avatar or the player? While educational structures can be

created for avatars, Stanford and other universities that        have implanted their




                                                                                     84
digital representation in SL do not intend to address avatars and to play a game

of make-believe: the intend to provide courses at distance for real individuals (SL

players), as they do with other tools for on-line teaching. What if a SL avatar

enters Stanford University building for attending a lesson and is shot in the head

by another avatar playing a game of destruction? Who is killed? The avatar, of

course. But who is learning? The player. No perceptual frame, no symbolic

advice can set the distinction, within SL world, between reality (learning,

attending a lesson) and fiction. The same conceptual confusion characterizes

political and religious proselytism and all the other uses of SL that are intended to

affect the real life of players.

We have seen that in the case of traditional media, journalistic, fictional and

commercial uses are framed in order to prevent ethical side-effects of conceptual

confusion, and in particular hoax and fraud. Peter Weir’s movie The Truman

show [Weir, ] is a satire of the effects of the obliteration of any distinction

between fiction and reality, with a man acting in a fictional world he takes for the

real one and actors advertising for commercial products while plying their role. As

VR and SL, traditional media such as radio, cinema and TV are versatile media

that can propose news, interviews, documentaries and fiction. But frames exist

that have the scope of providing identifiable boundaries for each of these uses: a

film is declared as that, it has titles at the beginning and at the end showing the




                                                                                  85
list of people that made them, the actors that interpreted them. Whether a film is

interrupted by commercial spots, the spots are interpreted by different actors and

frames (undertitles, intertitles) separate them from the movie. Documentaries

have no or few actors mentioned in the titles and are known to be records of true

events. These frames and rules can be insufficient and can deserve satire and

ethical attention, but at least they do exist. In SL all this is mixed up and no frame

indicates the distinction between fiction, uses that affect the player and reality,

and commercial advertising.




The door is hence opened in SL for two types of ethical considerations and risks.

The first consideration concerns the problem of trust and the relative risk

concerns the quality of the engagement of the subjects that are involved in non-

game activities. This problem is common to all on-line real commerce and

education, the commerce of real objects at distance and the education at

distance. An ethical attitude assumed both by providers and users of real

services and specific rules for the control of quality could solve the problem. But

in addition to that, SL presents the specific character of being also a fictional

place. Whether a real task is taken for fictional it can be underestimated, both on

the side of the providers and on the side of the users of the service.




                                                                                   86
The second consideration is related to the ethical risks of taking fictional contents

for real ones. Users are invited to believe that at least certain actions in the

virtual world will affect their lives, hence that these actions do not happen in a

virtual world of game and fiction, but in the real world. The risk is hence the one

of loosing the capacity of distinguishing between reality and fiction, and be the

prey of hoax and fraud. This second problem cannot be solved by personal

ethics alone, an honest attitude on the side of providers and users of services,

because it originates in a conceptual confusion. Specific rules should hence be

introduced against conceptual confusion that will make the separation between

different uses of SL (and of other VR tools) clear.




Are there (ethically) legitimate and non-legitimate external interventions upon

SL?

Terrorism and the exchange of pedophilic or violent material (pictures of children,

of rapes) are actions that are subject to legal interventions and ethical principles.

These phenomena are common throughout the net in general and other media.

What is specific of virtual reality and of SL is the possibility of making the same

experiences with one’s own avatar and with other avatars in a purely fictional

world. At the opposite of the situation where real people are involved, it is hard to

consider even terrorism upon virtual buildings or the rape of a children-like avatar




                                                                                  87
as criminal or unethical behaviors against society or children, as it is not a crime

the fact of imagining in one’s own head to do the same things or to write them

down in a book or to make a film on them. Ethical objections can be raised

because of the public rather than private (one’s own mind) activity of imagination

which is involved in SL and in other VR games that are free and on-line.

Censorship could thus be invoked for limiting the access to this kind of

representations, as it is the case for other works of fiction. At the end of any

episode of the series Alfred Hitchcock presents featuring a successful crime,

Alfred Hitchcock had the use (imposed by censorship) of declaring that the

criminal had finally be caught and duly punished, thus correcting the idea that

could emerge from the story that crime can be unpunished. In the same way, the

representation of acts of violence and of sex between adult-like and children-like

avatars can be considered as offending, disgusting, anti-educative for the

spectators that assist to these representations (even if films representing horrible

rapes such as The accused have been produced with an educational aim:

showing a terrible reality and its consequences).

But it is important to avoid confusion about what is censored and why. The

particular emotional reactions that are triggered by the simple thought of violence

upon children should not serve as a justification for obliterating the difference

between reality and imagination, because, as we have seen, this confusion has




                                                                                 88
ethical consequences too. It is a fact that sexual behaviors encounter greater

taboos than death. A SL user who creates an avatar resembling to his boss, and

kills him every evening when back home from his office can be rather seen as a

sympathetic guy trying to evacuate his frustrations through imagination. This is

never the case for rape, pedophilia and sexual violence in general. But, as a

matter of fact, the only difference between the two cases lies in our different

reaction toward sexual behaviors and desires, and not in the real rather than

fictional context of the behavior itself. It is hence important for journalists who

comment events in SL world (the column about SL is becoming more and more

diffused in newspapers throughout the world []) not to increase the confusion

between reality and imagination by mixing, for instance, the diffusion of

pornographic photos of real children and the diffusion of pornographic digital

representations of children or even representations of sex with children-like

avatars. Of course one can comment that it is quite depressing to see people

reproduce in fictional worlds the worst behaviors one can assist to in the real

world. But this is one of the functions of imagination, too: letting desires and fears

that - for ethical reasons - cannot be realized in the real world, free of expressing

in dreams and games.




                                                                                   89
Virtual reality




Virtual reality is a form of representation and a mean for proposing experiences,

(Biocca, Kim, Levy, 1995; Kim & Biocca, 1997; Lessiter, et al., 2001; Lombard &

Ditton, 1997, 2000; Loomis, 1992; Riva, et al., 2003; Steuer, 1995; Vorderer, et

al. 2004).




As many types of artworks in the Western aesthetic tradition, ranging from

paintings, tragedies and even landscape gardens, virtual worlds and virtual

reality are “intentionally created and structured in order to produce a certain kind

of effect” (Freeland, 1996, p. 207). The intended effects can be as different as

catharsis, aesthetic distance, free play of imagination, depending on the

particular aesthetic perspective adopted and on the specific artwork concerned

(Freeland, 1996, p. 2007).




It seems hard to consider all the products of virtual reality applications as

aesthetic products. Flight simulators, training in general and tele-operations are

operational situations with no aesthetic aim. But when we consider all these

situations as creations or intentional and structured products, then the




                                                                                 90
components of creation (in the sense of a intentional product that consists in an

original inedited situation) is quite evident.

Such creations can be considered as aesthetic products once aesthetics is set

free from the reference to beauty and even to art as a form of self-expression

and considered as the outcomes of intentional creation.

We can thus consider experiences with pictures, landscape gardens, books,

movies, virtual worlds and even distant worlds by means of tele-operation

systems as parts of a continuum of products intentionally created and structured

in order to produce a certain effect, a certain type of experience.




Appropriate actions and perceptual or emotional behavioural responses (the

desired effect) provide a metrics for evaluating the effects of an experience in

virtual reality. The definition of virtual reality as an esthetic object intentionally

created to produce a certain kind of effect has thus operational effects in terms of

indications for measuring the effects of experiences in virtual reality (without

taking recourse to unclear notions, such as presence, and to limited notions,

such as realism).

An example can help illustrating the point. Let us imagine a pair of virtual dice

and an interaction by the mean of a haptic interface that allows for natural

complex movements of the hands. A scale of adequateness of the actions and




                                                                                   91
reactions of the users can be drawn in which spontaneously taking the dice in the

palm of the hand and throwing them with a lateral movement stands at the

highest level of adequacy, while exploring the contour of the dice with the first

finger is less appropriate, and not spontaneously exploring the dice at all can be

considered as low adequacy.

If the virtual object is different from any real object (in the sense that the aim of

the creators is not to simulate any real objects, that the actions and sensations

that are made possible by the material interface are significantly different from

the actions that are normally performed in the real world and that the coded

information presents significant differences from any existing object) the task of

individuating the adequate actions and of distinguishing them from non-adequate

ones becomes more difficult.

Taking recourse to the aims and intentions of the creators helps solving the riddle

if we can clarify how users have access to the intentions of the creators.

Users can gain this knowledge from information explicitly stated by the creators

(declarations of the kind: “this is a flight simulator, or a surgical simulator”).

A significant amount of information about the appropriate actions can be also

gained from perceptual hints in a way that can be assimilated to an affordance

pattern (as described by Gibson, 1979, and ecological perception approach in




                                                                                     92
general) or to a sensorimotor connection (O’Regan and Noe, 2001;

Castelfranchi, 2003).

Innate or acquired through experience, the connections between action and

perception provide non-symbolic information concerning the actions that can be

more or less appropriate to a certain pattern of perception.




In the mean time, virtual reality is different from traditional representational media

because virtual reality systems (and in particular interactive interfaces) allow

action from the user (Casati & Pasquinelli, 2005; Pasquinelli, 2003).

The psychology of (motor) action and the psychology of social interaction and of

interaction in general must thus be added as significant fields of study in the

domain of the sciences of the mind-body. This fact marks the specificity of VR in

respect to more traditional media (Casati & Pasquinelli, 2005; Pasquinelli, 2003).




The possibility of action entails two consequences: the possibility of using of one

more of the human capabilities, notably, motor performances (in addition to

perceptual competences and symbolic, emotional, imaginative and generally

cognitive capacities), and the possibility of interacting.

Interaction implies the possibility of modifying the contents of the representation,

which is quite a new opportunity if compared with the “non-interactive” fruition of




                                                                                   93
cinema, but not in respect to interactive narrative media that do not necessarily

allow motor action and perception (Casati & Pasquinelli, 2005; Pasquinelli,

2003).




A user that interacts with a certain fictional or virtual environment modifies the

environment and perceives the effects of his actions on the environment and not

only a static product, as it happens in more passive interactions (such as the

experience of a movie).

This fact could suggest the erroneous consideration that virtual reality systems

are incomparable with movies (which are representations) and that virtual worlds

are forms of reality because of the fact that users do interact and produce

modifications as they do in the real world. In other words, the possibility of motor

action and interaction with modification could obliterate the representational

nature of virtual worlds.

In spite of the fact that one can interact with and modify virtual worlds, fictional or

virtual worlds are human intentional creations aimed at producing a certain effect

in other subjects and not real worlds. The real world produces a number of

effects on subjects but is not aimed at it. For this reason it makes sense to

measure the effects of virtual reality systems and of movies in terms of the

intentions and aims of the specific application, which does not make sense for




                                                                                    94
the real world (a stomach is not built for digestion, even if digestion is a function

of the stomach; flies are not created for being eaten by frogs).




The erroneous consideration about the nature of virtual worlds is reinforced by

the technological commonalities shared by virtual reality and tele-operation, in

which case, the distant world is the real one.

But it is not the virtual world which is made real by this association, it is on the

contrary the real world which is perceived through a medium which is made in

some way fictional.

The fact of introducing a medium (a tele-operation system, a telescope) between

the observer and parcels of the real world makes the experience partly artificial,

because both the real world and the medium (causally) influence the appearance

of the perceptual outcome. The perceptual experience of the sky seen through a

telescope (causally) depends on the state of affairs (the sky) and on the

intentions of the creator of the medium (seeing the stars that are far away as if

they were next to he observer). In this sense any experience which is made

through a medium partly (causally) depends on the intended effect of the creator

of the medium and can be considered in analogy with experiences with virtual

reality systems and in analogy with experiences with movies.




                                                                                  95
The difference lies in the fact that in the experience with movies and with virtual

worlds both the medium and the state of affairs are the product of the intentions

of some creator in search of a particular effect for his audience and users, while

in the case of the telescope and of tele-operation systems the state of affairs

which is perceived through the medium is not an intentional product of some

designer.




In spite of its label, then, virtual reality is not a form of reality but a representation.




But can an adult take a virtual world for the real one?

(Dennett, 1991) considers that producing a massive deception (a strong

hallucination) requires more than reproducing the features of the external world)

What is required, in virtue of the dynamic nature of perception, is the

reproduction of all the possible interactions between the bodily possibilities of a

human adult and the real world (sensorimotor interactions). Sensorimotor

interactions are so variable and numerous that strong hallucinations or complete

deception are unlikely to be created by virtual reality systems.




But what for a child? Let us imagine an evil scientist putting a newborn in a room

which is furnished in order to convey haptic, sound and visual stimuli by the




                                                                                        96
mean of suitable interfaces that are directed by digital code. The virtual world so

created can be very simple and respect the present development of technology.

Of course, only certain types of sensations can be produced, sensations that

depend on mechanical stimulation of some kind (including photostimulation and

acoustic stimulation), but not sensations that depend on molecules entering in

direct contact with receptors, such as in the case of smell and taste. In order to

produce smells and taste the digital code should command the synthesis of

molecules. In this case the synthesized molecules could be considered as the

interface between the code and the subject’s receptors, as haptic, video and

sound interface are.

We know that a newborn presents a high level of cerebral plasticity, even if this

plasticity is not infinite but is determined by previous selection (Edelman, 1987).

Additional selection operates on the neural structure of the child’s brain in accord

to the conditions the child encounters in his environment: the environment which

is intentionally created by the evil scientist. The baby won’t presumably develop a

certain number of functions that we encounter in normal children but the fact of

being stimulated in a way that respects the hardware of his body will allow some

development of action and perception. The world of the room, including code and

interfaces, will be his world. He will notice regularities in the effects of his actions

(if the aim of the creator is to produce coherence), he will hence be able to




                                                                                     97
regulate his actions and make plans in his world. If he is not alone and if he is

suitably exposed to a real or virtual adult introducing language, the child will also

have the possibility of sharing his world and communicate with another. These

are basic conditions for objectivity (Davidson, 1984). We can hence assume that

- whatever the condition that occupies the top of the causal chain (atoms or

digital code) - a world in the sense of an objective pole can be created which is

intersubjectively valid.




If it is indifferent whether we place material atoms or digital code at the beginning

of the causal chain, material components that are part of the real world are

nevertheless necessary in order to vehicle information to the sensory systems.

Since animals (and humans) have evolved in a material world, their senses are

adapted to respond to material stimulations from the real world, whatever their

form at the top of the causal chain. The real, material world is hence present in

any virtual experience if we want to vehicle coded information.

Even if the top of the causal chain is not a material world, the interfaces are

material, because we are dealing with a bodily entity which presents some

specific characteristics that have been determined by a long run of interaction

(selection) with a material world. Even in the case of the famous brain in a vat

(Dennett, 1991, 1978; Putnam, 1982), material stimulations and devices for




                                                                                  98
stimulation are necessary in order to produce the desired effect. The difference

with haptic, video and sound interfaces is constituted by the point of the nerve to

which they convey their stimulation.




In spite of the possibility of disjointing objectivity and reality adults experiencing

virtual worlds are not fooled by virtual worlds at the point of taking them for real

(Stoffregen, at al, 1997, 2003). As shown by the mental experiment of the child

grown in a digital code world, the problem is not represented by the origin in the

causal chain but by perceptual and cognitive considerations that are related to

the development of sensorimotor connections and beliefs and relative

expectations. Normal adult human beings come to the experience with virtual

worlds charged of expectations based on their past experiences. They can

compare the overall perception produced by non-mediated and mediated

experiences and on this basis they distinguish between the two forms of

experience (Stoffregen, et al., 1997, 2003). Additional; cognitive considerations

(the awareness of wearing some display, of being in a laboratory) can barely be

suppressed.

The objection could be raised that illusions resist cognition. The objection could

be retorted and it could be affirmed that it is because illusions resist to knowledge

that movies and virtual reality cannot be considered as illusions, because the




                                                                                   99
difference of the reactions provoked by the real world and the reactions provoked

by fiction worlds testifies that cognition has an effect. In fact, behavioral

responses to virtual worlds and to work of fiction in general, such as movies, are

never exactly the same (as testified by the debate around the paradox of fiction).




                                                                               100
Illusion of reality. Cognitive issues



Whether virtual reality can be considered as a fictional medium (a medium that

has the function o representing fictional truths), theories of fiction and of how

fiction works in generating behavioral and emotional responses can help

understanding experiences in virtual reality and how they should be designed in

order to produce the desired effects in users.

Another related domain of studies which can help individuating the relevant

perceptual and cognitive problems of designing experiences in virtual reality

consists in recent film studies, film studies that range into the cognitive theory of

cinema. As virtual reality, cinema is in fact a special fictional medium: a

perceptual, multisensory medium (at the opposite of virtual reality, cinema is a

passive medium, since spectators do not interact with representations as they

can do in the case of virtual reality).




The cited domains try to answer to the following questions:

1. Why movies and other works of fiction, such as books, radio dramas, plays,

seem so real? Why are we moved?

2. Why movies (and fiction in general) are acceptable and have success in spite

of the differences between the physical structure of the world and the physical


                                                                                 101
structure of movies (representations of fiction)? What happens when we look at a

movie (experience fiction)?




These questions are expressed in particular in the framework of the debates

about the so called “ paradox of fiction”, fiction and imagination, and about the

nature of cinema and other fictional media.




                                                                             102
The paradox of fiction



The paradox of fiction is a classic paradox discussed by philosophers of art and

introduced for the first time by Radford, 1975. The debate shows a substantial

accord on the fact that audiences and readers do not hold existence beliefs

about fictional contents, as testified by the fact that responses to fictional

contents are similar but never identical to responses to corresponding

nonfictional contents. The difference in reactions testifies that audiences and

readers are aware of the fictional nature of the experience, hence that they do

not have existence beliefs when they experience fictional contents (Radford,

1975; Stoffregen, 1997; Walton, 1978, 2001; Currie, 1990).

The paradox is produced by three apparently inconsistent and plausible tenets:

Premise 1: emotional responses to characters and events require us to believe in

the existence of the events or characters

Premise 2: no existence belief is present in the case of the fruition of fiction

fictional events and characters)

Premise 3: we deploy emotional responses in presence of fictional events and

characters




                                                                             103
Conclusion: the emotional response to fictional characters and events is

irrational, incoherent, and inconsistent (Radford, 1975).




Radford, 1975 justifies Premise 1 on the basis of the observation that in non-fictional experiences

people that discover that a heart-wrenching news was just a lie are no more heart-wrenched.

Radford’s defense of Premise 2 is a statement against the so-called illusion of reality, the idea

that audience can forget the fictional nature of the experience and take it for real. As a matter of

fact, Radford states, emotional and behavioral responses to fictional events are not the same as

emotional and behavioral responses to real events: the vision of horror movies is associated with

pleasure and spectators do not try to do something for saving their hero.

Premise 3 indirectly affirms that emotional reactions are directly provoked by fictional characters

and events.




Three basic strategies have been advanced in order to solve the paradox that

consist in denying one of the premises (Carroll, 1990). Denials of Premise 1 and

of Premise 3 give rise to Thought theories and to Pretend theories respectively,

according to which the fact of believing in the existence of the entities and events

that are only depicted is not a necessary condition for raising emotional

responses. In other words, fictional characters can raise emotional responses

even if they are not taken for real. According to Thought theorists it is sufficient

that certain events are mentally represented, or entertained in thought in order to

raise emotional reactions (Carroll, 1990; Lamarque, 1981; Smith, 1995),



                                                                                               104
eventually by evoking thoughts about real people and events (Counterpart

theory, Carroll, 1990). The Pretend theory (Walton, 1978, 2001; Currie, 1995)

suggests that what is experienced in fiction is quasi-emotions, that is emotions

that are produced by the fact of pretending. The Illusion theory stems from the

denial of Premise 2 and is defended by Anderson, 1996 and Tan, 1996. Even in

this case, nevertheless, the evidence of differences in reactions makes it difficult

to defend the existence of a cognitive illusion of reality.

It is then useful to distinguish, as Currie, 1995 does, between perceptual and

cognitive illusions in cinema. The term “perceptual illusions” is referred to the

existence of illusions of movement and other visual phenomena in the perception

of cinema that are traditionally treated as illusions (for instance by Anderson,

1996. For a different view: but not by Currie, 1995). Both Anderson and Tan

defend a perceptual illusion theory but are more shaded concerning the cognitive

illusion theory and acknowledge that spectators are aware of the fictional nature

of films. According to Anderson spectators experience a paradox between the

awareness of depiction and the belief in the depicted facts. In a similar vein, Tan

considers belief in reality a matter of degree and suggests that spectators are

simultaneously aware of both the real and constructed nature of film events

because they have at least some belief on the reality of film contents.




                                                                                105
    Three basic strategies have been advanced in order to solve the paradox: denial of Premise 1,

    denial of Premise 2, denial of Premise 3.




o           Strategy 1: denial of Premise 1.

    Emotional and behavioral responses to work of fictions and to the real world are different.

    Emotional responses do not need existence beliefs in order to arise, it is sufficient that certain

    events are mentally represented, or entertained in thought in order to raise emotional reactions

    (Carroll, 1990; Lamarque, 1981; Smith, 1995). Carroll, 1990 defines this view : Thought theory.

    The fact that emotions are raised by thought rather than by real events explains the difference in

    the phenomenology of emotions raised by fiction and emotions raised by reality. Audiences and

    readers know that what they are seeing or reading is not true of the real world and but that it is

    just a representation, then it is not necessary to call the police when seeing a horror movie.

    The Counterpart theory states in particular that fiction provokes thoughts about real people and

    events and that these people and events are the intentional contents of the emotions in fictional

    experiences, not fictional contents directly (Currie, 1990).

o           Strategy 2: denial of Premise 2.

    Existence beliefs are present in the case of fiction in the sense that audiences and readers

    somehow take fiction for reality.

    This option (named the illusion theory by Carroll, 1990) is strongly rejected by Radford, 1975,

    Walton, 1978, 2001, Currie, 1990 on the basis of evidence about the difference between

    reactions to fiction and reactions to real events. The awareness of the fictional nature of the

    experience is related to cognitive considerations on the side of audiences. No consideration is

    devoted to perceptual awareness of mediation.

    The illusion theory is defended by Anderson, 1996 and Tan, 1996. It is useful to distinguish, as

    Currie, 1995, between perceptual and cognitive illusions in cinema. In fact, both Anderson and



                                                                                                     106
    Tan defend a perceptual illusion theory but are more shaded concerning the cognitive illusion

    theory. The term “perceptual illusions” makes reference to illusions of movement and other visual

    phenomena in the perception of cinema that are traditionally treated as illusions (for instance by

    Anderson, 1996 but not by Currie, 1995). The term “cognitive illusions” makes reference to the

    idea of illusion of reality, the idea that the spectator takes the fictional characters and events for

    true. As a matter of fact, according to Currie, 1995 no illusion of reality occurs, but spectators’

    emotional responses can be explained by referring to the function of simulation. Human minds

    simulate other minds and this fact provides an adaptive advantage. Cinema triggers the function

    of simulation. In what concerns the cognitive illusion of reality, both Anderson and Tan

    acknowledge that spectators are aware of the fictional nature of films. Nevertheless, according to

    Anderson spectators live a paradox between the awareness of depiction and the belief in the

    depicted facts. In a similar vein, Tan considers belief in reality a matter of degree and suggests

    that spectators are simultaneously aware of both the real and constructed nature of film events

    because they have at least some belief on the reality of film contents. In this sense film creates a

    benign illusion and not a delusion. Tan, 1995 describes cognitive illusions as illusion of diegetic

    effect because the spectator has the illusion of being present in the film environment (in analogy

    with Allen, 196 projective illusion).

o           Strategy 3: denial of Premise 3.

    As a matter of fact, Radford acknowledges that reactions to works of fictional are not the same

    than reactions to real events in his defense of Premise 2. The difference in reactions testifies in

    fact for Radford that audiences and readers are aware of the fictional nature of the experience

    and that they do not have existence beliefs.

    Walton, 1978, 2001 affirms that what is experienced in fiction is quasi-emotions (pretend theory).

    Quasi-emotions are not provoked by existence beliefs (acceptance of Premise 1 that only

    existence beliefs provoke real emotions) but by beliefs that are related to an attitude of make



                                                                                                     107
believe based on the consideration of what is fictionally true according to the work of fiction, and

not true in the real world. In short, in analogy with children make-believe games, the fruition of

works of fiction makes people pretend to take some events of the work of fiction for true and also

pretend to feel related emotions. As in children playing make believe games, audiences and

readers are aware that it is only a game and they are not really afraid, for instance when seeing a

horror movie, but only quasi-afraid. Carroll, 1990 defines this view the pretend theory and objects

to Walton and other representatives of the Pretend theory, such as Currie, that in the case of

fiction audiences do not choose of pretending or not pretending: they are compelled to feel fear

by certain spectacles and not by others. By this fact, reactions to fiction cannot be considered in

analogy to make believe games. Additionally, spectators are not aware of having the intention to

pretend. As Carroll, Hartz, 1999 and Novitz, 1987 object to Walton that the phenomenology of the

reactions to fiction is no the same than the phenomenology of make believe games.

Against these objections, Neill 1991 underlies that in Walton’s pretend theory audiences do not

pretend to have feelings, but only that the experienced feeling is fear.

Other objections concern quasi-emotions as not necessary constructs: on one side, reactions to

fiction include reactions that do not depend upon the existence of beliefs (such as startle

reactions provoked by visual stimuli), and in the other side reactions could be provoked by

unconscious beliefs activated by visual stimuli (Hartz, 1999).




Whatever the strategy for dealing with the problem of emotional reactions to

works of fiction, Premise 2 is widely adopted. In other words, it is widely accepted

in the philosophy of arts that reactions to works of fiction are not the same than

reactions in presence of real events corresponding to the ones depicted in the




                                                                                               108
work of fiction, and that readers and audiences are aware of the special nature of

the experience which is mediated by fictional media: an experience with fictional

worlds, characters and events.

We can conclude that in the domain of film theory the notion of illusion of reality

as a massive cognitive illusion is largely refused on the basis of the evidence of

the difference in reactions towards fictional worlds and the real world. Audiences’

behavior and emotions are hence appropriate both to the contents of the

experience with fiction (for instance being afraid when viewing a horror movie)

and also to the context of the experience (the fact of fiction and the type of

medium through which it is presented). This double appropriateness explains

both the similarity with the phenomenology of emotions and behaviors raised by

events of the real world and events depicted in the fictional world and their

difference. The responsibility for similarity can be assigned to a similarity

between the contents of the real world and the contents of thought. The

responsibility for the difference can be assigned to the cognitive awareness of

the fictional nature of the experience and of the characteristics of the medium

through which the experience is proposed. The awareness of the medium hence

modulates the emotional and behavioral responses.




                                                                               109
A hypothesis can be added to this consideration, which fills some gaps both in

the Pretend and in the Thought theories: the appropriate responses hypothesis.

According to this hypothesis, audiences and readers reactions are both adequate

to the contents and the context of the experience, hence to the contents that are

depicted in the fictional world (contents that are suitable for provoking fear in the

interaction with the real world) and to the context (a fictional and not real world,

presented by the mean of a specific medium which allows certain reactions and

not others). Audiences and readers are hence aware of the fictional nature of the

experience and they react appropriately, as children do in make-believe games.




This explains both the similarity with the phenomenology of emotions and

behaviors raised by events of the real world and events depicted in the fictional

world and their difference. The responsibility for similarity can be assigned to a

similarity between the contents of the real world and the contents of thought. The

responsibility for the difference implies the awareness on the side of audiences

and readers of the fictional nature of the experience and of the characteristics of

the medium through which the experience is proposed.

The appropriate responses hypothesis also accounts for differences in the

perceptual nature of fictional experiences in relation to the perceptual nature of

experiences with the real world. Differencing between fictional and real worlds is




                                                                                 110
not only a matter of representation, thought and cognitive awareness, but also of

different specification in the stimulus condition. This is the reason why a dog, a

non-thinking organism, does not bark to dogs seen at the screen, but barks to

real dogs and to dogs in a theater play.

A corollary of the appropriate responses hypothesis can explain why the

following two situations raise emotions with different phenomenology: first

situation, one imagines Jason Vorhees penetrating the room. In this case no fear

is resented even if one imagines himself in a dangerous situation. Second

situation: one sees Jason Vorhees penetrating the room of some characters in

Friday the 13th and is scared and pleased in the same time. Something more

than mere imagination must be considered in order to explain the presence of

emotional reactions. The corollary states that the fictional world and the

experience must be believable, so as to engage audiences and readers to deploy

appropriate reactions as it happens in children games of make believe, where the

game is not just imagined but taken quite seriously.

It is not necessary to go as far as to suggest that in the second situation one

takes the fiction for true of the real world. If this was the case, emotions and

behaviors would be the same as for real events, which is not the case. As stated

by the pretend theory and by Radford, 1974 in Premise 2, fictions raises




                                                                              111
emotions, but these emotions are not the same than emotions raised by real

events.

Another corollary of the appropriate responses hypothesis consists in the idea

that what is raised by fictional experiences is emotions that are modulated

(amputated in some of their components or modified) in reason of the

characteristics of the context. Some aspects of the emotion of fear and relative

behaviors (call the police, escape from the theater) would be simply modulated or

even stopped by the awareness of the fact that this is only a game. The analogy

with make-believe games still holds, since once a child has chosen to play a fear-

game he cannot really decide to feel fearful or to stop feeling fearful.

Nevertheless, the recourse to the postulation of new mental entities such as

quasi-emotions is no more necessary.

As in the case of Pretend theories, in the appropriate responses hypothesis the

analogy with children make believe games makes certain aspects of adult’s

interaction with fiction quite perspicuous: the fictional world is not taken for real,

but it is taken seriously enough to justify the presence of behavioral responses

that are appropriate to the contents of the world and to the context, or the

material means through which the fictional world is made present (mud for mud

pies, cinema for movies, virtual reality systems for virtual worlds).




                                                                                  112
Conversely to the Pretend theory, nevertheless, the appropriate responses

hypothesis does not imply that adults pretend their emotions: adults can resent

true emotions that are modulated in relationship to the contents and to the

context of the experience. The same is true for children playing make believe

games.

The appropriate responses hypothesis is compatible with the two Premises of the

Paradox of fiction that are criticized by Thought and Pretend theories, since it is

compatible with the assertion that only existence beliefs can raise complete

emotional responses, non modulated by the context, and it is compatible with the

assertion that we are moved, even if emotions are not the same than the ones

deployed with the real world.       Premise 2 is a necessary condition for the

appropriate responses hypothesis.

A mean for testing the hypothesis of the double adequacy to contents and

context consists in comparing reactions to real events, corresponding fictional

events mediated by a certain medium (such as video) and fictional events

mediated by another type of medium (such as interactive VR systems). It can be

hypothesized that the three conditions will differ at least for the perceptual and

motor responses that will be elicited. The difference in perceptual and motor

behaviors cannot be ascribed to the contents of the experience, but only to its




                                                                               113
context consisting in the perceptual and motor possibilities that are made

available by the medium.

As a matter of fact, new interactive media constitute a great opportunity for

understanding the problems related to the Paradox of fiction, and in particular for

the appropriate responses hypothesis. Interactive intersensory virtual reality

systems put users in the condition of manifesting multiple motor and perceptual

responses in addition to emotional behaviors typical of passive media. Virtual

reality systems emphasize the difference in reactions deployed with different

media possibilities and stress the analogy with make believe games. As in the

case of make believe games, virtual reality users deploy actions that take in

account both the contents of the proposed experience and the motor possibilities

that are allowed by the particular device which mediates the interaction. Virtual

reality users are in fact called to act, and not only to perceive and resent

emotions. The appropriate responses hypothesis aims at going beyond the

explanation of the emotional behaviors raised by traditional media by considering

these behaviors as part of the cognitive, emotional, perceptual and motor

responses that are raised by fiction and by considering fictional worlds as

composed of contents and contexts that both count for explaining the type of

activated response.



See: Believability & Expectations



                                                                               114
Fiction and imagination


Walton [2001] sustains that fiction and representation can be understood by taking recourse to

the activity of imagination as it is exemplified in children make believe games. The notion and

activity of make believe hence constitute the core of Walton approach to the explanation of both

the importance of representative or fictional art in our lives (aesthetic point of view) and of the

ontology (mental and non-mental) which is implied by the fruition of this kind of human products

(metaphysical point of view).

“The experience of being “caught up in a story,” emotionally involved in the world of a novel or

play or painting, is central in the appreciation of much fiction, and explaining the nature of this

experience is an important task for the aesthetician. It is extraordinarily tempting to suppose that

when one is caught up in a story, one loses touch with reality, temporarily, and actually believes

in fiction. The reader of Anna Karenina abandons himself to the novel and is convinced,

momentarily and partially at least, of Anna’s existence and of the truth of what the novel says

about her. Otherwise why would he be moved by her predicament? Why would one be even

interested enough to bother reading the novel? Yet it also seems that the normal appreciator

does not (of course!) really believe in the fiction. [Walton, 2001, p. 6]




Imagination is enriched and sustained by the presence of real things that work as props. Real

things can cover different functions in imagination: prompting imagination (natural or artificial

things can induce people to imagine), constituting objects of imagination (in presence of a real

think which stands for something else, one does not simply imagine this something else, but

imagines the real thing, the prop, to be something else) and creating fictional truths (a thing that

has been declared in a certain game of make-believe to stand for something else, makes it




                                                                                                115
fictionally rue that this something is present else each time the thing is present, even if the thing is

not actually perceived and the something else is not actually imagined). Natural things can evoke

imagining but that artificial things are more suitable for prompting imagination towards

predetermined directions.




Props + principles of generation of rules (in the present game, a certain thing/prop stands for a

certain something else) generate fictional truths, even in absence of imagination. However, those

who do not imagine in presence of the prop are not playing the game, since they are not following

the rule. Fictionality and fictional propositions are hence, in a certain sense, prescriptions to start

a certain activity of imagination.

“[…] representational works of art generally are props in a game of make-believe” [p. 51]




Even in absence of props, of material things (such as it is the case for daydreams), fiction is not

equivalent to imagining and viceversa, since fiction can exist without imagining and imagining

without fiction. Fiction is what is to be imagined, the prescription and rule to imagine. Hence

fiction exists only when a rule is created that contains a prescription to imagine.




Works of art such as paintings, books, films are special props because they are intended to be

so. Representational works of art are hence intended and have the function of functioning as

props in games of make believe. One can not play the game and rather analyze a work of art with

the intent of understanding the author, etc.

Since works of art have this function it is not necessary to re-establish the rule in front of any work

of art.

Abstract paintings and works of art are different from figurative art because they are props for an

activity of imagining which is internal to the work of art, while figurative art evokes imagining that



                                                                                                   116
goes beyond the work of art and points to people and things (even fictional ones) that are

different from the work of art.




“Representations are things […] possessing the social function of serving as props in games of

make-believe, although they also prompt imaginings and are sometimes objects of them as well.

A prop is something which, by virtue of conditional principles of generation, mandates imaginings.

Propositions whose imaginings are mandated are fictional, and the fact that a given proposition is

fictional is a fictional truth. Fictional worlds are associated with collections of fictional truths; what

is fictional is fictional in a given world – the world of a game of make-believe, for example, or that

of a representational work of art.” [p. 69]




                                                                                                     117
Currie, 2005 debates Walton’s idea that imagination is a response to the fictional and that

imagination is to be understood as a form of make-believe which is prompted by the presence of

props.

Currie acknowledges that fictional works are guides to imagining and that the guided imaginative

exploration of a work of fiction whatever deserves to be called a game of make-believe, but he

objects to Walton that the role of props is more controversial: in narrative, non-perceptual fiction

props cannot be individuated Props seem to be bound to perceptual media.




Currie asks which kind of imagination is in cause in the case of fiction, for three main forms of

imagination exist according to him: the capacity to form mental images (imagery for different

modalities, such as motor imagery), the capacity to consider different alternatives and make plans

for contingencies (a form of propositional imagining), the capacity of seeing things that are not

there.

According to Currie imagery is not a necessary response to fiction, even if it can occur. In

particular, in theater and film, visual and auditory imagery pays no role because one has direct

access to relevant images and sounds. Vision and visual imagery are mutually exclusive, and so

on.

Currie suggests that the appropriate form of imagination is propositional imagining, and in

particular a sub-class of propositional imagining that has believe-like features. In fact,

propositional imagination exploits knowledge and inferences that are analogous to the ones that

can be drawn from belief. Analogous but not identical, because in the case of fiction certain

events (that would appear as bizarre in reality) can be accepted in virtue of the consideration of

reasons of genre, style, and in general in virtue of the consideration of the fictional nature of the

experience. For this reason, Currie defines his approach to fiction: simulation-inspired theory of




                                                                                                118
fiction. Simulation theory of fiction is accepted by Nichols and Stich, 2003 and finds evidence in

the study of children make-believe games (Harris, 2000).

Currie underlines that theories of pretence cannot rely only on beliefs but also on motivation and

desire (as expectations).

Walton has suggested that what is involved in imagination, hence in fiction, is not real emotions

but quasi-emotions. Currie & Ravenscroft, 2002 and Lamarque, 1981 for instance object that

fiction raises true emotions.




                                                                                              119
The nature of cinema



Cinema presents us with different forms of imagination. Bordwell, 1985 describes

audiences as continuously formulating hypotheses and expectations about the

past and present. This activity has the effect of constructing a narration. It hence

serves a different purpose than the activity of imagination which audiences

perform when they make-believe.

In the case of perceptual media, and in particular in the case of multisensory

media, the activity in one sensory modality activates other sensory modalities. It

is not sure that what happens even in these cases is a form of imagery, in the

sense that it is not sure that representations are necessarily involved. It could

simply be the case that connections between sensory modalities are activated

and that this activation does not give rise to representational activity. This would

not be a form of imagination, then. But expectations of a perceptual and motor

nature are raised, however. Even if vision and visual imagery are mutual

exclusive, vision can raise imagery or at least expectations in other sensory

modalities.




Perceptual cues as prescriptions for imagination




                                                                                120
When one sees a movie or interacts with a virtual environment he is not simply

imaging, he is also perceiving. Walton acknowledges the role of perception

through the acknowledgment of the role of real, material things in imagination.

In the case of cinema, spectators perceive real things: moving images and

sounds projected on a screen and released in a theater. These real things are in

the same time props for imagination.

Different sensory modalities might present a different relation with imagination.

Think to sound effects in cinema, that can evoke the image of objects different

from the ones used for producing the sound effect itself. Or to haptic sensations

obtained through force-feedback.

In any case, the experience is so construed as to prompt imagination, both in the

sense of hinting and directing hypotheses and in the sense of evoking the

imagination of perceived objects.

The screen and theater are reminders or generators of a general rule. This

general rule is not necessarily the following one: “imagine that the images and

sounds you perceive are not representations but are people, things, spaces that

look and sound that way”. This rule seems to be specific of a certain style of

filming and of imaging, the so-called Hollywood style (Bordwell, 1985). Other

styles are possible and actually exist, such as film art, where the spectator is kept

aware of the fictional nature of the spectacle through various hints (the




                                                                                  121
manipulation of the relationship of images and sound, the interrogation of the

spectator by the actor). The frame (the screen, the theater) enounces a very

general rule: ‘follow the rules that the spectacle represents in an indirect way and

that you might know by direct declarations of the authors, other direct information

that you gathered before the spectacle begins or indirect information you

gathered from long acquaintance with this type of spectacle. Direct and indirect

information (even very general information gathered from the experience with the

real world) creates expectations that are employed by spectators in order to

understand a movie, in the sense of re-constructing the narration by perceptual

and cognitive means (Bordwell, 1985).

Realism can be understood as a term which identifies a certain group of

styles/pescriptions for imagination. One prescription can be: “imagine that you do

not see pictures of people but real people”, “imagine that you do not see

drawings of people but real entities with that funny aspect”. Or maybe: “imagine

what it would be like if these pictures or drawing were real, if what you do

perceive could happen (have happened, actually happens, will happen). This

prescription can be identified with the objective of creating the so-called illusion

of reality, in the sense that spectators are asked to imagine that what they

perceive is not a representation but real, material things, or better: what it would

be like if what they perceive occurred of entities of the real world. This rule works




                                                                                 122
with different styles of production and different levels of realism in the sense of

the similarity with real things: analogic recording of real actors, digital

masterization with special effects, drawing with different levels of caricature. In

the case of cartoons, the techniques of simplification and exaggeration can in

some cases indicate to the spectator not to focus on the apparent nature of the

represented character (a baby animal who looses his mother) but on the

emotions and psychological states he lives (Thomas & Johnston, 1985). What is

prescribed is hence to imagine the sorrows and pain which is represented and to

imagine what it would be like if someone, some human being, would undergo

similar events and analogous pains. A different prescription states: “Imagine that

the images you see are not the outcome of montage and camera views and

setting and acting, but that we have filmed scenes of real life”. Let us add another

instruction: “this is not a simple documentary of something that happened in the

past, but a record of something which is happening right now”. Are there any

ethical problems with these instructions? The answer is no, because they are

instructions to imagine, instructions to play a game. Ethical problems arise only if

the make-believe nature of the game is hidden. It is not the type of instruction

that represents a problem for the ethics of the spectacle, it is the reminders of the

fact of playing a game with certain rules (the screen, the theater). Ethical

problems arise when, for different reasons, the fictional or gaming nature of the




                                                                                 123
activity is made uncertain. It would have been the case for Orson Welles’

performance of the “War of the worlds” (1938) thank to the scarce use of 1938

audiences to broadcasts featuring apparently real news bulletins, if it wasn’t for

quite constant reminders to the reality of actor Orson Welles playing a H. G.

Welles’s famous novel. It has been the case for the broadcast Alternative 3

diffused by UK Anglia Television, where the abandon of Earth is represented as

a real project in response to pressing planetary dangers. In this case in fact, no

reminder is made of the fictional nature of the images that are presented in the

framework of a well-known documentary, with no use of actors for presenting

them but of the documentarists that have presented all the previous

appointments of the broadcast.




Interactive media

As in the case of cinema, in the case of virtual reality and interactive interfaces,

the screen, or whatever other device that works as interface for presenting the

user with perceptual stimuli and symbolic information, is a reminder or generator

of general prescriptions. The type of general prescriptions depend on the type of

medium: “imagine what it is like to interact with an object that corresponds to the

imagine you are seeing, to the sound you are hearing, or to the haptic sensation

you are experiencing, and do it, following the limits prescribed by the devices you




                                                                                124
are wearing. Remember that these devices allow you to perform the x…y set of

movements”.

Props can be represented by images, sounds, haptic stimuli, depending on the

specific characteristics of the interface.

What is important in the design of virtual reality and other interactive interfaces is

both to design efficient props and to clearly express the rules. Rules include the

prescriptions and the possibilities of interaction that are given to the user. It is

rare that the user can perform any action he is used to perform in the real world.

The limits and left possibilities must be clear in order to give proper course to

imagination and relative action. Prescriptions too must be clear: “Imagine what it

would be like if this was a surgical instrument that cuts in this way but not in that

one and if you were to operate a heart which behaves this and that way”. As it

can be evinced from the example, prescriptions include the aims of the

application (an application for virtual surgery) and the characteristics of the

device, in particular in relation to the differences between the virtual device and

real devices. In fact, surgeons can have expectations relative to surgical

instruments that are not fully respected by the virtual device.

The same thing is true of children games of make believe, when sand stands for

flour and sand-cakes must be carefully cut into slices. In presence of solid

motivations, big efforts can be made for adjusting expectations gathered from the




                                                                                  125
experience with real things to the characteristics of the props one has at disposal

for playing his game of make-believe. Things can be more difficult when users of

interactive media are asked to act in relation to props, in addition to imagine. In

this case, props must comply to higher requirements than props for children

make-believe games, such as sand for cakes. The way for specifying these

requirements is indicated by the aim of the application and the characteristics of

the interface and that are constrained by the laws that relate perception to action

in normal circumstances. I.e., an instrument for virtual surgery will present stimuli

that are suitable for accomplishing the desired/possible set of surgical actions

and that do not provoke major disturbances in the performance. Ideally, the

stimuli should be designed by a team composed of a virtual reality designer, a

surgeon (for detailing the necessary actions), and an expert in action-perception

(for individuating perceptual disturbances and perceptual enhancements for

proper actions). But another exert is necessary in order to warrant that the rules

are clearly communicated to the user, by symbolic or perceptual means.




It might seem that the case of perceptual and interactive media is different from

cinema, radio and other perceptive but non-interactive media, because

interactive media prescribe users not only to imagine, but also to materially act

through movement. It should be remembered that this is also the case for



                                                                                 126
children make-believe games. If make-believe games are the reference for

understanding the functioning of fictional media, including interactive media such

as virtual reality systems, then cinema and other passive media, with their

interdict to act except to the rule of make-believe games: imaging and acting in

consequence.




Imagination and different forms of expectations

There are at least two different imaginative functions in the fruition of

representations. Perceptual hints are present in both forms of imagination, but

have different effects.

The first form of imagination is related to the functions of making hypotheses.

Bordwell, 1985 states that cinema and other narrative media users actively

reconstruct   the   narration   by   formulating   hypotheses   about   the   future

(expectations, previsions) and about the fact (motivations, causes). This

“imaginative” activity is sustained and directed by the hints that are mostly

intentionally provided by the creators of the experience, through language and

perception.

In the case of perceptual media (and at the opposite of non-perceptual media,

such as narrative books) imagination is hence sustained and directed by

perception. In the case of haptic feedback in virtual reality, once the object has




                                                                                127
been identified as a square-shaped one, the user will more or less consciously

make previsions about the fact that a certain movement will produce a related

sensation (sensorimotor expectation). We can consider sensorimotor and other

forms of expectations and previsions sub-classes of the imaginative function.

Another imaginative function is to evoke other, associated sensations or

representations. Walton asserts that natural things can evoke imagining but that

artificial   things   are   more   suitable   for   prompting   imagination   towards

predetermined directions. In this case, representations are supposed to evoke

past sensations and the image or internal representation of perceived objects.

The fact of evoking objects perceived in the past can be considered as a

“realistic” effect. This effect of imagination and association is different from an

illusion of reality, because the representation is correctly perceived as a material

device (a filmic image, a picture, a mechanic arm) with proper perceptual

characteristics and because the user is aware of the fact that he is in presence of

a material device, a medium that proposes a certain experience. The experience

can prompt imagination also in directions that are different from past experienced

objects. This might be the case for abstract representations evoking emotions

rather than perceptual representations.

But also for representations that provide sensations never experienced before.




                                                                                  128
In the specific case of artificial things that intentionally evoke imagination, the

accord between the imaginative activity which is evoked and the imaginative

activity intended to evoke by the artificial representation is indicative that the

artificial representation works or not as intended. The past experience evoked

can be as general as the one of a square object, or of an object in general, or as

specific as the experience of a pair of dice. The measure of the efficiency of the

device in this evocative function can be refined by including the degree of

certainty and the lapse of time which is necessary in order to evoke the desired

past experience.



Cinema and illusion

The constructive nature of narration

The mesh with cognitive and perceptual mechanisms




                                                                               129
Cinema and illusion

Currie, 1995 defines cinema as a representational medium which essence is visual (because

other sensory modalities are not as necessary as vision) and which images can perform two

(representing) functions: photographic function (as in a documentary) and fictional function (a

fictional story is presented). In the second case, the cinematic images function pictorially or non-

pictorially for informing spectators about what is true in the fiction (images pictorially represent a

certain real actor, but they function pictorially in presenting fiction since non real things cannot be

pictorially represented). The fictional character of a film depends on the intention of the film-

maker of making spectators imagining certain things. The photographic and the fictional function

cannot be conflated because in the fiction film what is intended to be represented is not real, it is

fictional.




Realism in cinema: cinema is claimed to be a realistic medium for three different reasons:

       -     transparency: a film is not a mere representation of the world but a reproduction of the

             world

       -     likeness: the experience of a film is like the experience of the real world

       -     illusionism: films engender illusions of reality (“illusion of the presentness and reality of

             the fictional events”, p. 21 or illusion of movement). This is a widely diffused view about

             cinema

       -     social realism: the film portrays significant social interactions




Illusionism: Currie distinguishes between two types of illusionism.

       -     cognitive illusionism: “The claim that film causes cognitive illusions is this: film

             watching, in some systematic way, and as part of the normal process of the viewer’s



                                                                                                     130
    engagement, causes the viewer to have the false belief that the fictional characters and

    events represented are real.” (p. 22) Currie supports this view with quotations from film

    critics. Two versions exist.

        o   Weak version of cognitive illusionism: the film is taken for the representation of

            real events (as a documentary)

        o   Strong version: the film makes the viewer belief that he is actually watching

            real events. Even when this view is not explicitly asserted by its supporters, it is

            implicit in the identification of the camera with the viewer’s eyes. This version is

            prevailing. It is opened to two objections:

                    Reactions of film viewers are different from reactions of people who

                     believe in the reality of the depicted events [Walton, 2001]. Evidence is

                     trivial.

                               Whether it was asserted that viewers’ beliefs are of lower

                                intensity or partial, this move wouldn’t solve the difficulties with

                                the illusionist view, since viewers do not behave as someone

                                who suspects that what he sees is true

                    The viewer has not the same point of view of the camera, hence there

                     is no psychological identification

-   perceptual illusionism: perceptual illusions are produced by the vision of films that are

    not necessarily entertained at the cognitive level (the coexistence of perceptual illusion

    and cognitive awareness being permitted by cognitive impenetrability). In Currie

    examples of cognitive vs. perceptual illusion, however, the Mueller-Lyer illusion can be

    both a cognitive illusion whether the perceiver is not informed about the real length of

    the lines, or a perceptual phenomenon, whether the perceiver is cognitively aware of

    the discrepancy. An example of perceptual illusion is the illusion of movement: the idea



                                                                                               131
that static images are perceived as moving by the viewer and hence that cinematic

motion is an illusion. Currie denies that there is such an illusion. Whether for a thing

movement is to be in a certain place at a certain moment and in another place at

another moment, then images (the image of a man) on the screen actually move from

one point to one other of the screen. It is hence correct to perceive cinematic images

as moving images. This characterization of movement requires re-identification. Re-

identification does not depend on the sameness of the causal antecedent of the image

(re-identification is possible even with cartoon images) but is a response-dependent

characteristic of entities that, in normal conditions, evoke a re-identification response in

perceivers (response-dependence is characteristic of secondary qualities such as

colors). The refusal of the idea that cinematic movement is illusory is thus based upon

the characterization of movement as a response-dependent property of things. Once

accepted a response-dependent characterization of movement, cinematic movement

cannot be illusory because images are perceived as moving by normal viewers in

normal conditions: for response-dependent properties, the possibility of error is simply

ruled out, and the perceiver is always correct. It can be objected that any illusion and

the very notion of illusion itself can be ruled out by characterizing any object property in

response-dependent terms: whether length is not a metric, intrinsic property but an

extrinsic property of being experienced as longer than, even the Mueller-Lyer illusion is

no more an illusion. Currie answers this objection since he is not disposed (as

ecologists) to abandon the notion of illusion for Mueller-Lyer-like phenomena. He hence

states that in the case of the Mueller-Lyer figure, the illusion suggests that whether the

perceiver was to measure in traditional metrics the line he would find a certain length

which is not confirmed by actually measuring them. In the case of cinematic movement,

no check can deny that the images of re-identifiable objects that are seen on the



                                                                                       132
          screen are moving from one point to one other of the screen because where the image

          is seen on the screen there is illumination. In other words, illusions do not resist

          independent checks, while cinematic motion does. This answer partially corrects the

          purely response-dependent view of movement and introduces a specific feature of

          cinematic movement, because it grounds cinematic movement (and not movement in

          general) on the presence of illumination on the screen. At the opposite of cinematic

          motion, for instance, the apparent movement of light points in the dark is an illusion,

          because it does not resist independent check: no illumination can be found where the

          image is seen. Whether response-dependent or not, cinematic movement (and not of

          movement in general) depends on the real presence of illumination at some point. As

          movement in general, cinematic movement is detected when an object is re-identified

          in a different place. But this response would not be enough to rule out the possibility of

          error, hence of illusion, as shown by other illusions of movement. The condition for

          genuine movement is that not only the image of an object is re-identified at different

          places on the screen, but that illumination is really at that place on the screen.

              o   The view of illusions that makes reference to the effects of the recourse to

                  independent check, shares analogies with a coherentist view of illusions in the

                  sense that illusions are errors as inconsistencies between two or more

                  synchronic or even diachronic perceptual events or between a perceptual

                  event and a cognitive information that concern objects that are identified as

                  one and the same by the subject of the illusion.




Transparency: transparency is a modified version of the presentation thesis.

      -   The presentation thesis claims that photographs and films present the world rather than

          representing it. Even if this was true, films would present viewers with the real world of



                                                                                               133
    actors, sets, etc., but could not present fictional characters and events that do not really

    exist. “Representations extend our epistemic access to things in the worlds; if they are

    reliable, representations give us information about things when those things are not

    directly accessible to us.” (p. 49). Representations are different from devices that help

    the access to details of things (such as lenses) because the last ones help seeing the

    things themselves, not accessing information in absence of the thing. They hence

    present and not represent.

-   The transparency thesis sustained by Walton affirms that seeing a photograph is

    different than seeing a painting because a photograph can represent only things that

    really exist, while paintings are not limited to the representation of reality and because

    when we see a photograph we see the thing through it. Currie acknowledges the

    difference between photographs and other images but he ascribe this difference to the

    different ways though which the representation is produced, not to the fact that

    photographs are not representations. The resemblance between the image and reality

    is characterized in terms of counterfactual dependence between the visual properties of

    the world and the visual properties of the representation (the same dependence stands

    between the visual scene and the perception of the scene). In the case of painting, but

    not of photographs, counterfactual dependence is mediated by the beliefs of the artist.

    This is because the way photographs are made cannot take into account the way the

    world looks like to the artist as ordinary seeing does not take into account others’

    beliefs. But seeing a photograph is substantially different than ordinary seeing. One

    reason is that seeing is perspectival and provides egocentric inforla




                                                                                           134
-

-   The transparency thesis sustained by Walton




                                                  135
      -   The constructive nature of narration



Bordwell, 1985 presents a constructivist approach to narration (adoption of constructivist

approach to cognition and perception in general; for perception, he makes reference to the view

represented by Helmholtz, Gregory, Gombrich: perceiving as drawing inference, possibility of

error).

Bordwell’s approach is strongly centered on cognitivism. At two levels: high mental operations

described by cognitivism, such as the use of heuristics, and the perceptual level. For the

perceptual level Bordwell adopts the Gregory-Gombrich view of perception and visual art:

perception is active in the sense that it is an activity drawn upon inferences; perceptual inferences

can be of two types: top-down and bottom-up, in the firs case inferences are not as mandatory as

in the second case, in which little choice is left to the interpretation of the visual cues. In any

case, visual cues are always open to interpretation and the data that are reconstructed through

inference from cues are not the same than the uninterpreted reality. Inferences are drawn from

hypotheses that are more or less enracinated into the structure of the body rather than acquired

and, in accord with Gombrich, the acquisition o a certain type of knowledge and expectations can

be facilitated by the presence of a certain structure (reading perspective is more easily learnt than

reading language). As illusions, visual art draws upon the typical knowledge, expectations and

inferences that are stimulated by certain cues and shared between most of individuals.

In the same time Bordwell insists that cinema (and the interpretation of visual arts in general) is

not only a matter of perception: it is a matter of reconstruction, formulation and validation of

hypotheses both at the perceptual and at the higher cognitive level. The filling in of gaps

constitute a paradigmatic example of the activity of the spectator both at the perceptual (gaps

between shots) and at the higher narrative level, the level at which the spectator reconstructs the




                                                                                                136
fabula from the syuzet (broadly speaking: the world and events that it is possible to reconstruct

from the represented events and the represented events; when the represented events recount or

find their causes in past events that are not represented, those past events are part of the fabula,

as the characters that are never shown in the syuzet but that can be presumed from it).




While a strong cognitive activity seems to be necessary on the side of the spectator (and

demonstrated by the fact that when recounting a film spectators make reference to events of the

fabula that are not directly represented thus showing their activity of reconstruction of a complex

narrative world), the refusal of ecological, direct theories of perception in favor of the excusive

adoption of classic, cognitivist theories of perception, is justified only as a matter of principle: the

spectator is active.

In the cognitivist approach, expectations play their role only in the disambiguation of the stimulus.

It could be objected that expectations shape or are compared with non-ambiguous stimuli that are

directly picked up not in order to interpret bare cues but in order to perform a continuous control

upon coherence, novelty, discrepancy. In this way epistemic organisms are always ready to learn

the existence of a novel object or to revise their beliefs. When, for some reason, these

possibilities are not at disposal, novelty (the unfulfillment of expectations) can be experienced as

a violation of coherence.

This suggestion requires an appropriate theory of expectations. It seems to go into the direction

of creating bridges between opposed theories of perception by preserving the mechanisms

described by indirect theories of perception and by preserving the direct nature of perceptual

picking up of information. The mechanisms described by indirect theories of perception are

mechanisms of control and not of interpretation. It can be suggested that the existence and

application of control mechanisms is not the same for all creatures in all situations (an

assumption which is proper to indirect theories) but are more present in higher creatures which



                                                                                                   137
perform epistemic activities (they have beliefs) and in the course of epistemic activities. In this

way it is possible to accept the existence of expectations and control mechanisms for coherence

without considering perceptual activity as based upon internal representations.

Bordwell’s description of the activity of spectators seem to be compatible with this modification

of cognitive theories of perception: what counts for it is the role of expectations, not the fact that

perception is indirect and based upon interpretation or reconstruction from ambiguous

information. Interpretation and reconstruction work at a higher level and expectations based upon

knowledge or sensorimotor laws are at work at every level in order to perform a check for

coherence.




Three existing theories of narration are presented:

- mimetic theories of narration: theories that conceive of narration as a form of showing

(presentation of a spectacle); vision is the model for mimetic theories of narration: a visual object

is presented to the eye of the spectator. Bordwell draws the history of mimetic theories back to

Greek theater (theatron = seeing place; in Greek theater a specific space is marked off as

fictional and specific attention is devoted to the place occupied by the spectator, the eye; at the

opposite of parade, circular staging and other forms of representations). Mimetic theories of

narration are applied even to literature: novel as a picture, problems of staging (in analogy with

theater) and point of view. In cinema (see Pudovkin: Film technique, 1926 for the first formulation)

the mimetic theory states that the story events are presented through the vision of an imaginary,

invisible witness; as the camera moves, the witness shifts his attention from one detail to one

other; the invisible observer is hence incarnated by the camera. The camera can be assumed to

embody the narrator’s or the director’s point of view; on the other hand, the spectator is the ideal

viewer. From the technical point of view, camera and microphones are for instance placed where

the eyes and ears of the ideal possible spectator should be.In any case, the witness is not an



                                                                                                 138
ordinary perceiver, but an ideal observer who can see things that are not normally available in

real life. The theory was initially used to valorize a kind of non-theatrical, distanced way of

narration, a way of narration which is similar to the immersion in the flux of life, that is, a way of

narration that ideally mimics the perceptual experience of the world . Bazin: cinema should

present the spectator with the same field of vision which is granted by the perception of the real

world: long takes, staging in depth; the viewer will operate cuts, as he does during the visual

encounter with the real world. Another aspect which is involved in the mimetic approach and in

the idea of the verisimilitude of perception, consists in the fact that the invisible observer is

supposed to observe an autonomous world, a world that could be known independently from the

film. Bordwell criticizes the mimetic theory because it is unable to account for the staging of action

(the theory only focuses on the camera and on space, with no attention for the disposition of the

elements in the scene previous to the intervention of camera and for the mise en scène). Also,

the mimetic theory takes the verisimilitude of perception as congenital to film, as the basis of style

and narration; while, according to Bordwell, the invisible observer and the verisimilitude of

perception are figures of style, are effects a of a particular style of narration. A related issue

consists in the self-conciousness of the spectator:

- Diegetic theories of narration: theories that conceive of narration as a form of telling. The term

diegesis indicates the fictional world of the story. The main theorists of diegesis are French

structuralists (Barthes, Benveniste, Genette, Metz,       …): language is the master system for

understanding all types of expressions and every narration depends upon the construction and

interpretation of codes. Film analysts are hence called to unfold the filmaker’s intentions (the

enunciation) disguised in the story, in the diegesis. As in the case of mimetic theories, the

spectator is the victim of an illusion: the natural appearance of the story, of the diegesis, positions

the spectator in a passive place. Bordwell opposes a constructivist theory of narration where the




                                                                                                  139
spectator actively participates to the construction of the story, is not a passive eye fooled by

perspective, editing, narrative point of view who takes the filmed world for an objective one.

- Constructivist/Cognitivist theory of narration: according to Bordwell, understanding the story

which is narrated is a form of activity and construction in which the spectator is active with both

his perceptual and cognitive systems. A theory of narration is hence grounded in a general theory

of perception and cognition. Bordwell accepts the Constructivist view of classic cognitive sciences

(Gregory, Helmholtz): perceiving is drawing nonconscious inferences based on knowledge and

testing hypotheses. So it is film viewing and story understanding: “The artwork is necessarily

incomplete, needing to be unified and fleshed out by the active participation of the viewer. ” (p. 32)

The artwork acts in two main ways on the perceiver’s activity: it provides hints that trigger

perceptual and cognitive reactions (the application of schemata, the formulation of hypotheses,

inferences, expectations, the revision of hypotheses, inferences, expectations) and hints that

constrain, limit the possible actions of the perceiver. The activity of the spectator is hence the

same which the perceiver deploys in everyday life, in the natural, non-mediated perception of the

world. In particular cinema manipulates perceptual deficiencies and capacities of the visual

system (the incapacity to follow rapidly changing light intensities which gives rise to the

impression of steady light or flicker fusion; the capacity of inferring movement from interrupted

inputs when the gap between them is not too large, which gives rise to apparent motion). Cinema

is hence considered by Borwell as a mean for illusions (in the sense described by Gregory) which

is based on our tendency to draw wrong inferences from perceptual data concerning the real

state of affairs of the world. This is because the artwork is incomplete, it presents gaps where the

inferential capacities (and related possible errors) of the spectator can be exercised, information

is missing. In addition to perceptual capacities, cinema triggers and constrains cognitive activity

based on prior knowledge and experiences. “To make sense of a narrative film, the viewer must

do more than perceive movement, construe images and sounds as presenting a three-


                                                                                                 140
dimensional world, and understand oral or written language. The viewer must take as a cognitive

goal the construction of a more or less intelligible story.” (p. 33) Assigning meaning, constructing

the story implies certain activities. The first one consists in assigning coherence, in constructing

unity: the spectator seeks for elements that can be combined in a coherent unit and on this basis

he formulates hypotheses that are in accord with the unity. The second step consists in the

application of schemata to the artwork; schemata constrain the contents of the hypotheses that

can be emitted on the basis of the re-constructed unity. Schemata can derive from the experience

with the real world, with other artworks and with other films. Schemata constrain the assumptions,

expectations, hypotheses that are formulated step by step by the spectator during a film and that,

step by step are confirmed or disconfirmed by the film. Again, this activity is favored by the

presence of hints, gaps, patterns (in the story, in the way of narration, in the style) that have the

effect of encouraging story-constructing activities. Some classes of schemata can be described

(Hastie): prototype schemata (such as prototypes of lovers and             bank robbers that help

understanding individual characters in film like Bonnie and Clyde), template schemata (for

instance, experiments on recalling and re-telling stories indicates a tendency to normalize deviant

stories following certain common parameters: occidental viewers of films tend to recall a story as

presenting some characteristic features, such as clear causalty, discriminable events, identifiable

places, even when in reality the story presents omission in these fundamental characters; a

certain degree of toleration of ambiguities allows spectators to enjoy more or less deviant stories),

procedural schemata (procedures through which the spectator justifies elements of the film and

applies template and prototype schemata following certain rules; include the procedures that the

spectator must put in place when the film does not follow template schemata, in order to search

for information and frame hypotheses. Justifications or motivations can be related to parts of the

story –compositional motivations in Bordwell’s taxonomy-, experience with the real world –realistic

motivation- or with other artworks –transtextual motivation, as it is the case for the reference to



                                                                                                141
    genre-, artistic motivation related to the consideration of the form and structure of the artwork).

    Bordwell describes the cognitive activities that are associated with the application of schemata.

    The first one consists in the position of assumptions; the assumptions that are described by

    Bordwell are basic assumptions that can be considered as related to commonsense knowledge,

    such as the assumption that objects persist in space and time even when they go unperceived or

    that individual personality persists. Even if he does not recall the notion of surprise, Bordwell

    asserts that these assumptions normally go unnoticed unless they are violated by the film

    (Obscur objet du desir). The second type of cognitive activity consists in the formulation of

    inferences, and in particular of inductive inferences that explain causes of actions in film. Finally,

    the most interesting activity is the formulation of hypotheses or expectations about future events

    (Sternberg). Sternberg describes different forms of hypotheses: curiosity hypotheses (past

    events), suspense hypotheses (future events), hypotheses with different degree of probability and

    of exclusivity (more or less choices). Hypotheses are tested at different levels: at the micro-level

    of step by step action or at specific junctures related to specific events, at the macro-level of the

    entire film. When hypotheses are disconfirmed, expectations must be readjusted; Sternberg

    affirms that there is a certain resistance in expectations revision and that expectations and

    assumptions based on schemata can be retained even when faced to disconfirming evidence.

o                    A similar insistence upon the constructivist activity of the viewer, the presence of

    gaps and the related necessity of filling-in activities based on knowledge, experience, perceptual

    capacities, can be found in McCloud’account of Comic art.

    o                Bordwell’s theory is composed of perceptual constructivism and cognitive

    constructivism. It does not seem a necessity to accept both components when accepting one of

    the two. For instance, one can accept the cognitive constructivism without necessarily adopting

    an indirect view of perception (the view represented by Helmholtz-Gregory).




                                                                                                     142
o               Bordwell presents the spectator as having a goal: the goal of making sense of

creating a meaningful story out of the material which is presented.

o               Bordwell highlights the importance of unity, coherence and the fact that even

coherence is a matter of construction. He also introduces a consideration about the capacity of

tolerating a certain amount of ambiguity and a certain degree of frustration of the expectations

that are based on the application of schemata, in particular of template schemata. It would be

interesting to know which is the degree of tolerance and how it is established.

o               It is not clear at which moment the disconfirmation of hypotheses can produce

disruption of believability and not simple readjusting of hypotheses or belief revision or resistance

to evidence. In any case Sternberg and Bordwell accept the view from cognitive sciences that

affirms that cognitive agents tend to keep their expectations unless there is strong evidence

against.




“For example, beginning a story with "Once upon a time" can be seen as a parallel to the overt

narrational address that initiates most classical films (not just via the credit sequence but by

various self-conscious expository devices; see Bordwell, Staiger, & Thompson 1985, pp. 24-29).

Such overt marking of the tale's fictional status might at first blush seem culturally specific. Yet

there are comparable formulas in languages from Albanian to Serbo-Croatian, and there are

functional equivalents in Navajo ("At the time when men and animals were all the same and

spoke the same language. . ."), in the Tiv tribe of Nigeria ("I can tell lies too!"), and among the

Bandi of northwest Liberia ("Let's throw stories!") (Pellowski 1977). In any culture, it would seem,

the story must be "framed" by conventional markers; otherwise, it may be mistaken for reportage.

The self-conscious opening that frames the story may well be a pragmatic universal, like

politeness formulas (Brown & Levinson 1987). The point is that we should not let a justifiable




                                                                                                143
resistance to biologism block us from a rich and comprehensive explanation of how filmmaking

and film viewing, like other cultural activities, build upon acquired skills and innate capacities.”




The framing of the story recalls the idea that the user/spectator/listener is aware of the fact of

assisting a representation which is not reporting or reality.




                                                                                                       144
The mesh with perceptual and cognitive mechanisms



Cutting, 1992 asserts that filmmakers and movies match our perceptual and cognitive

dispositions and mesh with the human visual system. Hollywood style, for instance, subordinate

the presentation of the story to the narrative and narrative coherence, hence if narrative works, no

attention is posed to other structural aspects such as cuts and the audience does not become

self-aware. There are also technical escamotages that allow the avoidance of self-awareness,

such as the fact of placing the action in a space (the action space) which is different from the

personal space of the spectator. In general, the filmmaker plays attention at do not making the

perceptual differences between real world and film perception and to make the camera stay

transparent to the story.




Stoffregen, 1997 discusses the notion of illusion of reality in cinema, as asserted for instance by

Anderson, 1986, who, characterizes the perception of films as illusory because spectators

perceive the events depicted in films as real and not as depictions of reality. Anderson is besides

aware that spectators do not run away when the film features a horrible monster.

As Gibson, 1979, Anderson recognizes that no one is completely fooled by films, and that film

perception provides two inconsistent sets of information. For Gibson, 1979, in fact, the illusion of

reality is a myth, because a depiction is perceived as such; but depictions are also perceived as

conveying information about something else: depiction and films are hence paradoxes.

Stoffregen, 1997 suggests that the paradox can be solved by stating that what is directly

perceived is a state or events depicted on a surface. The fact that we differentiate depictions from

reality is proved by the fact that we differently react to the two, not running away from a movie

monster and not calling the police for a fictional murder. The ambient array is specified in a



                                                                                               145
different way by a parcel of the real world and a depiction. Stoffregen, 1997 cites Hochberg,

1996. Stoffregen focuses on the structure of the stimulus and on perception and leaves behind

cognitive considerations.

But the problem of the “illusion of reality”, of the power of films to move spectators and to seem

real concerns other dramatic arts as theater representations that cannot be considered as

depictions as films and pictures because the representation is directly conveyed with no medium

(no surface). Are perceptual hints enough to differentiate between theater and reality?

Additionally, the problem of the “reality” of a depiction or of a representation is not simply a

problem of the resemblance between reality and representation: fictions can produce the so-

called illusion of reality or move the audience even if they do not resemble to parcels of the real

world; as it happens in the case of monster movies.

The problem of the illusion of reality is hence broader than a problem of depictions and broader

than a problem of simulation of parcels of reality.

It seems widely accepted that no one is really fooled by depictions or other fictions. The example

of monster movies is especially significant because it exemplifies two objections to the idea that

spectators are fooled: spectators do not present exactly the same reactions they would have in

presence of a real monster and there are no monsters in the real world: no one can hence believe

that what he is perceiving is the real world and not a fictional one. Nevertheless, monster movies

and films in general, depictions, dramatic arts and also virtual reality provoke emotional and

behavioral reactions in spectators and users and this emotional reactions present some affinity

with the reactions one would have if the fictional events were true.

The perception of the difference between fiction and reality explains the fact that spectators and

users are not fooled, that the emotional and behavioral responses to fictions are not the same

than to reality; but it does not explain the affinity. Furthermore, purely perceptual explanations of




                                                                                                146
the awareness of the fictional character of a fiction might not be sufficient when all types of fiction

are taken into account.

Stoffregen, 1997 cites the example of his sister’s dog who is able as human beings to make the

difference between a dog in a film and a dog in the real world (better: to learn the difference,

because the dog in question, as probably human beings in the early years of cinema, barks to the

first dog he sees in TV): Stoffregen sister’s dog barks to real dogs but not to dog in TV. We can

hypothesize that there are there differences in the stimulus situation that are sufficient to produce

different behavioral answers. These differences in the stimulus situation and in the behavioral

answer need not to be aware; in fact, even dogs seem to make the difference and present

different reactions. I am still assuming the chauvinist position according to which dogs and other

animals cannot be considered to be aware of something because they cannot be considered to

represent something, but just to directly react to stimuli. The great number of different reactions

they present is not due to intelligence in the human sense as thought or capacity to represent but

just to the corresponding great number of different stimulus situations. A depicted dog and a real

dog structure the ambient array in different ways and hence give rise, according to Stoffregen, to

different (correct) perceptions and correspondent (correct) behavioral reactions.

But Stoffregen sister’s dog would probably bark at a dog representing a dog in a theater picture.

The actor-dog in fact cannot be differenced on the basis of perceptual conditions, but only on the

basis of cognitive considerations. In this case one has to be aware of the difference between

reality and theater, aware in cognitive, thoughtful way ad not only aware in a sub-personal,

reactive way (which is not awareness, but picking up of perceptual differences and behavioral

reaction).

If we are to understand the “illusion of reality” in cinema and the capacity of films of moving the

audience, we must take into account theater as we take into account depictions, hence we must




                                                                                                  147
take into account aware, cognitive processes as much s we should take into account perceptual

conditions.

Stoffregen, 1997 affirms that the understanding of films requires to understand the relations

between films and other kinds of storytelling media. He considers that the common problem

between these media is the one of involvement (as opposed to perception) and that involvement

concerns the story or diegesis which is clearly perceived as fictional.




Another characteristics of films that should not be overlooked is their intentional character as a

human product: as stated by Ittelson, 1996, Anderson, 1996 and by Gibson, 1979 (see

Stoffregen, 1997), films are the purposeful product of the intention of the film-maker; Anderson,

1996 describes films as programs or sets of instructions for what the film-maker wants his

audience to see. This is probably more suitable as a description of classic Hollywood movies,

where the spectator is heavily directed and the point of view of the film director is imposed over

the spectator, than for instance, for Nouvelle Vague French movies (see Bazin). But we can

imagine that the set of instructions is more or less flexible. Nevertheless, also the intentionality of

the viewer is concerned: Gombrich, 1972, for instance, cites the fact that human beings are

capable of seeing castles in the clouds or faces in the fire, that is, to treat as depictions natural

forms that have no artistic intention. More in general, we can say that human beings are capable

of seeing as and not only of seeing: they can see the meaning in addition to pick up a stimulus

and react (see Wittgenstein, 1958). Stoffregen, 1997 adds the case of natural reflecting surfaces

such as water that do not seem to imply any form of intentionality, and hence can be perceived

even by animals; he suggests that a theory of depiction should take into account all these

different cases.

If we accept a chauvinist view of representations, not only an animal cannot see a face in the fire,

but it cannot see the resemblance between a parcel of the world and that parcel reflected in



                                                                                                  148
water: it can just perceive both and react of each of them in a way that suits to his behavioral

structure (see Davidson, 2004). Additionally, films are not necessary a matter of resemblance

with parcels of the real world, and Anderson’s statement about the nature of films as sets of

instructions about what should be seen respects this condition.




                                                                                            149

				
DOCUMENT INFO