Pictures as Language

Document Sample
Pictures as Language Powered By Docstoc
					Language and Visualisation
conference november 8–9, 2001, sto ckholm, sweden

as Language
henrik danielsson and bodil jönsson

The purpose of this lecture is to shed light on a potential, and invite
those who are interested to join in and make use of this potential. Here
is the background:
     Starting with the Isaac Project in 1993,Certec has developed a
technological/educational concept based on pictures as language for
people with limited spoken language or none at all.There are a number of
case studies (our own and others) of the interaction between people with
different types of cognitive disabilities and a multiplicity of personal
digital photographs (over 1,000,often well over 10,000).There is also a
growing and increasingly widespread use of digital photos in special
educational facilities,care-giving services for disabled people,and in the
fields of autism and psychiatry.Added to this are the experiences gained
from a picture web board in which people involved in using pictures as
language have been able to share their experiences with one another.
     We can provide examples of what has been done and document the
effects. But still—8 years later—we are unable to carry out sufficiently
qualified and relevant analyses and assessments because we have not
been able to find applicable explanatory models for the effects; neither
have we found relevant fundamental theories supporting what we have
observed. This is profoundly unsatisfactory and that is why we are now
appealing for help.
     If we, together, can identify relevant existing theoretical
foundations, or come up with new useworthy ones, the effects can be at
least twofold: On the one hand, we will better be able to understand and
make use of the results achieved so far. On the other, with good theories
as a basis, excellent prerequisites should exist for inspiration to an even
more effective implementation than what we have hitherto accomplis-
     The article contains a comprehensive description and references to
Certec-related work in this area for the past 8 years, as well as a literature
review that shows how difficult it has been to find a basis in established
scientific research.

henrik danielsson is an engineer and graduate student at
The Swedish Institute for Disability Research, Örebro and Linköping
Universities. His areas of interest are communication and memory aids.

bodil jönsson is a physicist, educator and author in the fields of
science and technology. She is a professor at Certec, Division of
Rehabilitation Engineering Reaserch, Department of Design Sciences,
Lund University, Sweden.

2   •   pictures as language
1. Introduction
Certec has developed a technological/educational concept based on
pictures as language starting with the Isaac Project in 1993. It has
primarily been used for people with developmental disabilities who
have limited or non-existent oral language. The concept has also been
applied to and produced significant results for people with aphasia and
psychiatric disorders. There are now a number of case studies (our own
and others) of the interplay between people with different types of
language disabilities and a multiplicity of personal digital photographs
(over 1,000, often well over 10,000). There is also a growing and
increasingly widespread use of digital photos in the daily activities of
special educational facilities and care-giving organizations for people
with different disabilities.
    The most striking effect of the picture-as-language concept has
been its ability to produce drastic and long-lasting effects on the
learning acquisition of people with cognitive disabilities/mental
retardation/developmental disabilities/learning disabilities. Though
the categories are many, none of them suggest why the modern day
availability of a multiplicity of digital photos can be as effective as it is.
However, if one narrows it down to the language disorder as being a
significant, often the most significant aspect of the disability, one does
not in that way deny the existence of corresponding motor problems (in
walking, hygiene, ability to hold one’s head up, etc.), contact avoidance,
attention disturbances, compulsive behaviors, etc. But this does not
diminish the value of pursuing the thought: What exactly is it in the life
of a developmentally disabled person that has to do with her language
disorder, and how can the language disorder be tackled?
    The dramatic effects of the digital pictures appear logical if the
following chain of thought is true: If you are unable to acquire a
language you will be unable to learn, to any great extent. If the effects of
this have been accumulating since early childhood, it can result in the
developmentally disabled adult being very different from others in the
areas of feelings, knowledge, behavior and relationships. But it is never
too late. In fact, the introduction of pictures as language to people in
their 50s or over can result in a strong, positive change and constructive
process for the people involved.
    In this article, we have compiled the experiences of what can happen
when people acquire pictures as language. The cases we will present are:
    • Changes in the learning acquisition of two adults with develop-
        mental disabilities (now in their 50s) over an 8-year period.
    • An autistic boy’s (now 10 years old) development over a 4-year
    • The language development of an adult aphasic woman (in her
        60s) over a 4-year period.

3   •   pictures as language
    We will also give an account of the experiences of the Picture Web-
Board that Certec has had up and running for 2 years and in so doing
include perspectives that have come from the general dissemination of
the picture-as-language concept.
    For all of these different issues, it has been difficult to find evidence
in established scholarship. The literature review that is included makes
this clear.Work in the field constitutes a mixture of efforts on different
levels that are difficult to structure.What is striking is that they have
different purposes and varying concept formations.

2.Literature review
henrik danielsson
An article search was carried out in the following databases: Medline
(pubmed), PsychInfo and Web of Science. The following keywords were
used: (Aphasia OR autism OR developmental disability OR develop-
mental retardation OR mental retardation OR mental disability OR
mental disorder OR developmental disorder) AND (visual short-term
memory OR visual long-term memory OR visual working memory OR
visual episodic memory OR visual semantic memory OR visual
procedural memory OR spatial short-term memory OR spatial long-
term memory OR spatial working memory OR spatial episodic
memory OR spatial semantic memory OR spatial procedural memory
OR visuo-spatial memory OR iconic memory).After reading 4 000
abstracts, the ones chosen were those that dealt with pictures (both
symbols and photographs), autism and developmental disabilities.

Pictures are better than words for accessing semantics for people with
autism [27, 28]. This indicates that it is relevant to use pictures with this
group even though they have defects in their visual short-term and
long-term memory, especially with nonsense figures [34]. This suggests
that the pictures should be meaningful, just like photographs are,
particularly when they are taken in the immediate surroundings. It
seems as though autistic people look more at the details than at the
whole picture when compared to people without autism. This
phenomenon is called “local bias”. The literature agrees that people
with autism look more at details [e.g., 20, 30, 31, 32, 33, 36], but it is not
certain that this occurs at the cost of looking less at the whole [20, 30,
33]. The tests have involved different tasks such as copying pictures,
numbers, music and linguistic information.

Developmental disabilities
Limited short-term memory in people with developmental disabilities
results in them using other strategies [6, 19]. One explanation as to why

4   •   pictures as language
digital pictures work better than symbols can be that they enable those
involved to use other, easier strategies.
    People with developmental disabilities often fail to spontaneously
use repetition strategies when the task requires verbal [7], visual or
kinesthetic [16] short-term memory. Those who are mildly retarded are
poorer at picture recall than a control group that was matched for age
and IQ [17]. This indicates that pictures (digital photos and symbols)
should not be used with this group. The problem is to find a means of
communication that does not involve verbal, visual or kinesthetic
short-term memory.
    Verbal labeling did not improve visual recognition in people with
severe mental retardation (the severely subnormal), but did so in a
control group without mental retardation. On the other hand, verbal
recall improved in both groups and particularly in those who were
severely retarded [8]. This is interesting since certain symbol systems
make use of some form of picture labeling. Some of the people who use
digital photos have sound labels on them. Does this help the users the
most, or is it primarily for those who are to interpret the pictures?
    There is no difference in recognition of faces for people with
developmental disabilities compared to a control group that was
matched for chronological and mental age [12]. Many digital photos
display the faces of people, which makes this finding interesting.
    People with Down syndrome and others with unspecified develop-
mental disorders have difficulties remembering things and spatial
placement. The differences in memory for spatial placement in people
with moderate retardation depends on the level of cognitive reduction
and not on age or etiology [15]. This is not so good if you are to keep
track of where you have your digital photos when you print them out. I
have not seen anything that suggests that this is a problem for retarded
people or those with autism. They usually have no problem finding the
pictures they are looking for.
   Limitations in short-term visual memory of size, form or order are
characteristic attributes of mentally retarded people (subnormal) [38].
What this means for digital pictures is difficult to interpret. The pictures
have the same size and shape, and the order is often of little significance,
but pictures do contain details that have size, form and internal order.

As was the case with the aphasia search results, those for developmental
disabilities did not provide us with any significant explanatiens for the
phenomenon we have observed. That is what has been the most
interesting result of the article search. I found almost nothing on the use
of photographs with people with autism, developmental disabilities or
aphasia. I had expected to find connections in several areas, in special
education at any rate, because I know that photos were used in that area

5   •   pictures as language
even before digital camera were available. Have I looked in the wrong
places or is there not much written at an academic level? Just because it
is used in a given discipline does not mean that it has a basis in research.
    We have also read material in related disciplines such as research in
cognition, art and communication. The lasting impression is that
although close, it is not close enough to be of use. This does not
necessarily mean that the answers are not out there, but if so, they are
formulated in a way that we do not understand how we can make use of
them. This article is an attempt to get around that by describing what we
do so that you have the opportunity to explain your theories and
methods that fit the description.

3.Case study:Stig and Thomas
bodil jönsson

Stig Nilsson and Thomas Åkesson, the principal characters in this
section, have participated for many years in a day center facility in
Lund, Sweden, called The Pictorium. Göran Plato, their mental
companion, works not only as a professional caregiver but is also an
artist. The most recent material published about The Pictorium reflects
this in its title: Art and Science—A Different Convergence [35].
     What has happened to Stig and Thomas during the years has been
continuously documented [5, 9, 10, 12, 21, 23-28, 39]. They are both
upper-middle-aged and have developed in a way that has, perhaps,
never before been reported—both have gained in stature, in self-
respect, and have improved their thinking and communication abilities.
Thomas has acquired an extensive oral language and Stig has greatly
strengthened his ability to express himself, primarily through pictures.
And we see that after 7 years there is no sign of the development leveling
off; if anything, the opposite.
     Thomas’s language development should be measured day by day—
he has changed so much from the time when he was a person of
extremely few words, having only a couple hundred in his vocabulary,
the pronunciation of which was difficult to understand. The only
evaluation carried out by a speech and language pathologist is from the
spring of 1998 [37]. Three years later, in the spring of 2001, Göran wrote
in an e-mail to me:
     “Thomas is using many new word and sentences. I have never heard
these before: ‘I have to have.’‘Don’t you want?’‘Come over, sit here.’
‘What are we gonna do now, today?’‘Am better now.’‘That’s not right, it’s
wrong.’‘I wanna go home.’ (When something goes wrong or doesn’t suit
him.) ‘Good, you’re nice, you coming tomorrow?’‘Will you get me
tomorrow?’‘Who’s driving tomorrow?’‘Shall I get something?’‘Damn it,

6   •   pictures as language
stop fighting!’‘We need to shop, we’re out of a lot.’‘Do it yourself!’‘I
wanna call someone.’‘Shall we work?’‘Don’t wanna, do it yourself!’
‘What does this say?’
     “Stig is still not saying a lot, but he expresses so much with photos
that you find yourself forgetting every now and then that he doesn’t talk.
Before he had only ‘Iiiig’ (Stig) and ‘Jaaa’ (Yes). New word-like sounds
from Stig (2001):‘Ooooooja; Ja;Va, va va; Na-na-na; Mu, mu, mu’. This
happens when he is the one controlling the bar code scanner and
picture rollers. Stig usually sits at the computer, looks at his pictures and
tries to imitate the sounds I make. He’s using many new words that I
find difficult to understand.”(From Göran Plato’s e-mail)
     All this development started with the Isaac Project and the 15th of
October 1993.An excerpt from What Isaac Taught Us, 1998, [23] can be
found in the Appendix. Now, 3 years later, there is still no leveling off of
the progress, on the contrary. That the Isaac Project found a home at
The Pictorium and was further developed there was no coincidence—
the environment was already so creative. But still, Göran or anyone else
would not have been able to manage with only human support to give
Stig and Thomas what they required to develop their thinking skills.
They also needed tools and artifacts to think with (see Arne Svensk’s
licentiate thesis, Design for Cognitive Assistance [39]). That is how
Certec could assist, could come up with solutions so that Stig and
Thomas and others can think better themselves.
     I was the one who at the time of the Isaac technology creation lobied
for the inclusion of a built-in, digital camera, something quite exotic in
1993. It was obviously a “solution”but to what and why? I asked myself:
“How does one go about talking to others if one does not have the
words?”What proved to have the potential in the Isaac concept were the
photographs with bar codes and accompanying sound that the
participants could manage themselves by using the bar code scanner.
Most of the computer interface is right there in the rooms they use—a
tremendous number of pictures mounted on rollers and on everything
else on which pictures can be placed.

Pictures during and after learning versus
pictures as instructions
As it turned out, the digital camera represented a means that was
infinitely more successful than I ever could have imagined. For I had
only understood the half of it (if even that). To my greate surprise when
I look back on it, was the fact that I believed in the right kind of
instructions, created through the use of personal photos. But it was not
the chains of correctly constructed instructional photos that resulted in
success; it was the chains of photos that were built up while learning, as a
support in learning that made the difference. The pictures help Stig and
Thomas while they are in the process and afterwards they are a

7   •   pictures as language
confirmation that,“Now I know that.”A happy memory, in other words,
rather than a set of instructions for future use. See the Appendix with
the excerpt on how Thomas learned to measure with a yardstick.

The picture and the question rather than
the picture and the answer
A decisive factor in the success of Isaac at The Pictorium was, most
likely, Göran Plato’s artistry.Art and science share many points in
common. One of these is that both raise new questions. But there is a
basic difference in how the questions are used.While science
immediately throws itself into finding answers to the questions and then
making them public, art takes on the role of finding questions,
accentuating them and hanging the questions up for people to see. So
that people can answer themselves—or continue living with the
    This has been quite evident in our different ways of working.At first
The Pictorium and Certec are united in the joy of discovering the new
questions and asking one another:“Do you think that it can actually be
like this?!”But then Certec tries to find the answers, come up with the
technical solutions, etc.While Göran makes use of our answers by
making them over into questions, as it were, that are similar to the
original questions and then holds them up in front of Stig and Thomas.
In what follows there are a number of direct excerpts from Göran’s e-
mail messages to me. Even more can be found in Art and Science [35].
The personal development among the participants of The Pictorium is
closely connected to their interaction with Göran, (See Annamaria
Dahlöf ’s film, Uncommon Friendship [9]) and with his consistent
refusal to live life for anyone else, to decide for anyone else.
    If ever there was a group of people who has been more exposed to
training in following directions and repetitive practice than others, it is
people with developmental disabilities. Learning by transfer.An
expressed stimuli-response situation, an expressed belief in learning
through repetition, combined with assuming guardianship, that is,
others doing things and making decisions for the person involved. This
approach has dominated and dominates still. The pattern is deeply
ingrained even today, in spite of all the good talk of individual
development and the right to self-determination that we have heard for
a long time now.
    Perhaps the pattern will not be broken until one understands the
difference between elevating the questions versus furnishing the
answers. There is no doubt that the people we are involved with need
support, lots of it, in order to see structure and context. Read about
Henrik Person’s black Wednesday as described by Arne Svensk in
Design for Cognitive Assistance. [39]. HP needs help with all the many
different situations that he does not grasp. But is it preaching, nagging,

8   •   pictures as language
reproaching and repetition that he needs? Or a more restricted set of
values in society saying that people like HP really shouldn’t be allowed
to live on their own, shouldn’t be allowed to cut the grass, shouldn’t …?
     There is a polarization between focusing on questions versus
answers in different approaches to education. On the one hand, we find
constructivism, which with its view of learning as a searching and
creative process, places the main emphasis on questions. The
supposition is that people learn by asking questions, searching, creating
and finding support in the world and inside themselves, all in order to
achieve learning, i.e., internal change. On the other hand, there is the
psychology of learning, programmed learning, teaching technology
and similar approaches, in which learning is studied as if it were
independent of content and environment and as if it could primarily be
controlled through situations and instructions arranged in the right
manner by the external world.

Pictures and taking the initiative
One thing is for certain, Göran does not want to make decisions for Stig
and Thomas more than is absolutely necessary. It has resulted in Göran
signaling them with a wave of the hand about things they can do on
their own, at times even waiting them out. But all the new possibilities
have resulted in Stig and Thomas starting to take the initiative entirely
on their own. Some examples:
    “Thomas asked me:‘Plato, you don’t happen to have a bag at home
that I could borrow?’ It’s as if the more information pictures we produce
and the more we create, the greater the new initiatives they start taking
    “Stig didn’t have any bandages left on his arm when he came today,
but suddenly he wanted one. I asked why. He pointed at the picture of
Christer and explained that he had a bandage and so Stig wanted to
have one too. Maybe to get a little attention?”
    “Thomas just got his new computer. He’s decided where he wants it.
The next day he all of a sudden came rushing in and said:‘Plato, Plato,
there aren’t any outlets in the room!’ His next question was:‘When can I
get pictures in my computer?’ He can talk and wonder about all sorts of

An explosion in the ability to observe and in conceptual
developement of how things are related
Closely related to taking the initiative is one’s own attention, judgment,
and desire to try something new.An example among many:“We drive
the same route from the area where Thomas lives every day. Thomas has
seen a motorcycle there that he likes. One day when we drove by he said:
‘Plato, the motorcycle isn’t there today.’ I hadn’t really noticed myself so I
had to turn and look and sure enough, there was no motorcycle!”

9   •   pictures as language
     All the participants have exploded in their ability to see new
contexts. The habit of using pictures (today between 35 000 and 50 000)
has also improved their ability to observe the world around them. It is
most evident with Thomas since he has become so much more verbal at
the same time. Just take the following example (from Göran’s e-mail,
     “When we drove past a neighboring high school, as we do each day
Thomas said:‘Everyone’s started school again.’ I asked him how he
knew and he replied:‘All the cars are here again,’ pointing at the parking
lot.Another day:‘Many are free today. Few cars.’”
     And an example of Christmas logic from Göran:
     “Happy New Year. Everyone here sends their love and thanks for the
Christmas presents.As Thomas said:‘I’ve been very good, got lots of
presents. Bodil’s been very good too, so she also gets many.’”
     “We often watch the film from our Stockholm trip.There is one
sequence I find particularly interesting.Thomas was sitting by a window
and looking at a building.‘Plato,’he said,‘look at all the window-pictures
there are in that building.’He pointed back and forth at the building.I
asked what he meant by ‘there is a picture in each window’.We
videotaped it and I helped Thomas to get a close-up by zooming in on
the window, so he could see what they were doing inside.There was
someone sitting at a computer, someone making a phone call, someone
drinking coffee, etc.He sat for a long time and looked at it.”
     This example shows how we have in a way come full circle. One of
the things we discovered a long time ago was that Thomas could not
clearly see a picture if it did not have a frame. So we started to frame in
all the pictures, of course. That gave Agneta Dyberg-Ek, a special
education teacher in Hörby, Sweden (Abdulkader’s teacher—see the
next case study) sudden insight:“That’s the reason why sitting by a
window has such a great effect on some students. It is as though the
window gives them a frame through which to see the world.And in that
way, one and the same slice of real life becomes easier to handle than the
complex, three-dimensional, constantly changing world.”
     Thomas now saw the windows from the outside as pictures. He even
said so:“there is a picture in each window”. That Göran then had the
opportunity to zoom in with the camera and show what was actually in
the pictures—that was fantastic.

Pictures as help when catastrophe strikes
Like the rest of us, people with developmental disabilities have accidents
and catastrophes, and like the rest of us things usually work out for
them in the end. But the connection isn’t always obvious and they lack
the ability to generalize to the next difficult situation. Göran has
developed a fascinating, effective concept with empty pictures to be
filled in step by step:

10   •   pictures as language
    “Thomas dropped his glasses and they broke.As if that wasn’t
enough, the handle on his briefcase was loose.As a result, the briefcase
got caught between his legs, he fell on the pavement, cut his elbow, hand
and knee. His hand was so swollen that he had to go to the clinic.
    “And Stig had just had his cast removed when he was going to show
a group of visitors a picture. He fell—and his arm swelled up again.
What an unfortunate set of accidents the guys have been through.
    “My recipe: I recreate the events through pictures. Build up how they
happened.What we are going to do.And see to it that they end with a
positive event, when everything is fine again.
    “I’ve started with pictures of the actual injuries.When the fellows
saw them they started laughing, started talking about them with
everybody and explaining or trying to explain. In that way they started
to recreate what had happened and you better believe that they were
very good at doing it all over again in pictures, so that they could see for
themselves what had happened and tell others on their own. The more
they are able to picture-talk about it, the less of a problem it is. Just
imagine how little is needed to increase quality of life for them!
     “To document a negative event in pictures has changed it to a
positive, healing one. It has been a kind of first aid.As soon a Stig has a
pain or thinks about it, he takes out the pictures and explains how it
happened.What we have done. How it will turn out:‘I will be fine!’
     “They have managed their problems on their own. Fantastic! They
have laughed over their accidents. It’s terrific that pictures can heal.”

Pictures and weight watching
Thomas is overweight. For many, many years others have tried now and
then to get him to lose weight: they have reduced the portion sizes, given
him fewer portions, healthier food—and in between a lot of food to
comfort and spoil him, especially from inexperienced staff. The results
have been that Thomas has grown heavier and heavier. He finds it hard
to walk, cannot bend over to tie his shoes, etc.
     It took a while for Göran to understand that the road to success
would be through pictures: pictures to display attractive dishes and to
show Thomas how to go about losing weight. One day Göran sent the
following e-mail:
     “Thomas wants to send some new pictures of himself. His new
weight is 87.5 kilos. He was up to 102. He has really gone all out for this.
Wants to have pictures of appetizing food. Some of your co-workers
from Certec were here yesterday and we invited them to stay for coffee.
But when it was Thomas’s turn to take a piece of cake, he said he doesn’t
eat it anymore.‘I have my own.’
     “He can now put on his socks again.And today he picked out a
picture of food he had eaten before and asked if he could have it again
tomorrow.What progress he has made! I am really touched by it all.

11   •   pictures as language
    “I forgot to tell you about Thomas’s morning coffee break. He was
able to choose between a baguette with marmalade and a cracker with
lettuce, tomato, cucumber and a little slice of low fat cheese. Thomas
chose the cracker.‘That one looks really appetizing. I’ll take it.
Mmmmmmm good,’ he said and laughed.
    “I’ve started planning a diet for him consisting of 1500 calories a
day. In pictures, of course.”

At The Pictorium, language and visualization are inseparable, i.e., the
convergence of language and visualization is obvious from most points of
views. The ones mentioned here are:

     •    Pictures during and after learning versus pictures as instruction
     •    The picture and the question rather than the picture and the
     •    Pictures and taking the initiative
     •    An explosion in the ability to observe and in the development of
          concepts about how things are related
     •    Pictures as help when catastrophe strikes
     •    Pictures and weight watching

You can find many more perspectives and examples from the Pictorium
among the references.

4.Case study:Abdulkader Faraax
bodil jönsson

This story about a ten-year-old boy has been written in close cooperation
with his teacher, Agneta Dyberg-Ek. His real name is being used because
both his parents and Agneta agree that that is the way is should be. He has
the right to be known under his own name, just like all the rest of us.

Abdulkader was born in Sweden in 1991. His parents immigrated to
Sweden from Somalia at the end of the 1980s and have lived in Eslöv
since the beginning of the 1990s. The parents have a loving relationship
with their son and a close physical one with lots of hugging and kissing.
     Abdulkader did not develop like other children. He was extremely
active and restless from an early age. He was diagnosed as having autism
at the age of three. By then, he had already started at daycare and
continued there with different personal assistants until the autumn of
1997. He and his assistants had a room of their own and worked with a
variety of social interaction games, colors and other activities according

12   •   pictures as language
to a schedule. They used sign language in combination with pictograms
and small photographs. The most fun he had was when he was
swinging—something he could do for hours in the gym or outside. He
also liked to be chased and to run around and around. He had, for the
most part, no spoken language.
    When Abdulkader started school as a six-year-old with Agneta
Dyberg-Ek as his teacher, he could dress and undress himself, eat on his
own with good manners and manage his toilet needs. He only had a few
words: names of those closest to him and individual words such as “eat”
and “water”. The family wanted him to have Swedish as his language,
but had always spoken Somalian with him even though they had been
educated in Sweden, having completed upper secondary school.

Pictures as language
Agneta had, and still has, close contact with Certec and started
participating in the Isaac project as early as 1995/96.As a teacher she
has consistently tried, with the aid of personal digital photos, to
understand how Abdulkader and other pupils interpret their (our)
world and what they wonder about. This means that she has
photographed and photographed and photographed in order to use
pictures as language.

Pictures fill the following functions for Abdulkader:
Provide awareness that what is invisible for the moment still exists
It made a big difference when Agneta understood that for Abdulkader it
was as if his parents had disappeared when he was in school.
Correspondingly,Agneta disappeared out of his mind when he was not
in school. If she turned up at his home or in a store, the situation was
entirely wrong. She was supposed to be in school and mamma was
supposed to be at home. It was first by using photos of people, which
they moved between photos of different buildings, that he gradually
began to realize that people do not disappear just because they no
longer can be seen.
    All such insight needs to be continually reinforced. This autumn,
one of Abdulkader’s classmates changed classes. He is still in the same
building, comes by sometimes but is no longer a part of Abdulkader’s
world. Talking about it with him is not enough.Agneta thought that
Abdulkader understood, so she had not photographed the event. It
wasn’t until all the classes were on an outing and Abdulkader started
pushing his former classmate that she realized something was wrong.
Back at school, she and Abdulkader immediately went to the old
classmate in his new classroom, photographed him there and that was
enough to satisfy Abdulkader. He stopped picking on his former
classmate. Now he knows.

13   •   pictures as language
Provide confirmation
Closely connected with this is that Abdulkader often wonders, wonders
intensively about things and people and that is when he needs to see a
picture as confirmation that the listener understands what and who he
means. If there is no picture, such confirmation is impossible. In his
own classroom everything is turned upside down for him, if his teacher
forgets to photograph something new in the environment or if he does
not have access to pictures of those people who are usually there. Not
until he is able to see the person in question in a picture and is able to
confirm for himself that it is the one he was thinking about, can he settle

     Example 1: The first few weeks of the 1997 school term were extremely
     turbulent, but after a month things settled down. A letter about this
     from Agneta to me has been published in What Isaac Taught Us [23].
     The letter tells how things did not start to calm down until pictures
     were taken of almost everything. Suddenly one day, though, all
     reverted to what it had been before—Abdulkader was tremendously
     restless all day long, unable to concentrate or to settle down. When it
     was time for him to go home by taxi, and Agneta accompanied him out
     to the car, she saw that the driver was new.“Is that what you were
     trying to tell me all day, Abdulkader? That there was a new taxi
     driver?”He did not answer.“Shall I go in and get the camera and take
     a picture of him?”No answer. But Agneta fetched the camera anyway,
     took the photo, and then, not until then, when Abdulkader could see
     the photo himself and see that the new driver was also in the picture,
     did he collapse, totally exhausted in the back seat of the taxi. A whole
     day of considerable stress and strain was over.

As an extension of this example, it is worth mentioning that the taxi has
dropped Abdulkader off at the wrong address three times in less than
six months. The last thing he does in school everyday now, before he
goes out to the taxi, is to confirm for the staff with his pictures where he
is going and with whom he will be spending the afternoon and evening.
Then he feels secure. If he is dropped off at the wrong place, though, he
quickly loses all the trust that has been built up. So now the staff has
established the routine that Abdulkader will always have pictures with
him of the building to which he is going. The driver has to get out of the
taxi, open the door and before they take off, ask Abdulkader:“Where are
you going?”When Abdulkader shows the driver his picture, he knows
that he has control over the situation and that he will be dropped off at
the right place. In this case he is using the picture to answer a question.

     Example 2: A new question word after the summer of 2001 has been,
     “Truck?”“What do you mean by ‘truck’?”Agneta wonders. She checks

14   •   pictures as language
     to see if he wants to read about trucks. No, that’s not what he means.
     He continues calling out,“truck, truck”, and is obviously upset. Is he
     afraid of trucks? Has something happened involving a truck? She
     doesn’t understand. The staff at his after school program communicate
     that he is also yelling out “truck”there.
          Days go by. One Monday he shouts “truck”several times in the
     morning. Then the big, red truck drives up to the school, the one that
     comes every Monday with supplies, and that Abdulkader has seen
     through the window for several years. Agneta wonders:“Is that what
     he wanted confirmed, so that we would understand what he was
     thinking of? Or perhaps he was wondering when the truck was going to
     come?”She photographed the truck immediately, printed it out and
     Abdulkader calmed down. Now he has a picture of a truck on his
     Monday schedule. And since then he hasn’t called out “Truck?”.

To see people
Abdulkader finds it difficult to see people even if he is looking right at
them. It is not due to fear of making eye contact, but stems from
something else.With the help of a digital camera, it is possible to make
people visible so that he can see them.We do not know why. It may be
because pictures are two dimensional, that they are framed, that they do
not move [22], etc.
    On one occasion, however, we found out by chance something quite
surprising.Abdulkader had a teacher’s aide for a period of time.When
the aide said:“It’s time for us to go and eat,Abdulkader,” the child did
not react. But one day when the aide had a photo of himself on the lapel
of his jacket and happened to point at the photo, not at himself when he
said:“It’s time for us to go and eat,Abdulkader,” the boy stood up right
away and accompanied him.

Questions with pictures
Abdulkader’s path to being able to ask questions has been long and
slow. It has mostly consisted of one or two word sentences. He has been
able to ask for things that he has seen, however.And he has been able to
choose when presented with two pictures and also use the words.
    Agneta has avoided using pictures to give instructions of the type:
“Now we are going to do this.”She displays a photo and tries to get
Abdulkader to orally express or in some other way show what he is
thinking when he sees it. She does this by showing him the picture and
saying:“What shall we do?”If Abdulkader says what Agneta thinks he is
going to, they do it. But if he says something totally different, she tries to
figure out exactly what he means, shows him other pictures, adds
information, seeks out his confirmation.

15   •   pictures as language
Show how things are related
What does,“Put on your cap!”mean? Abdulkader did not understand,
even if Agneta put the cap on him. It wasn’t until he saw the picture of
himself wearing the cap that he could accept and understand and then
put the cap on.

To prepare
It would be unimaginable to visit a new place with Abdulkader without
preparing him.Agneta has even been able to travel with him and his
classmates to Denmark’s Aquarium when everything has been
photographed in advance, enabling him to obtain confirmation the
entire time from the pictures he has with him.

To give instructions
There are situations in which instructions need to be given. If you are on
a class outing, you can’t just say to Abdulkader:“Sit down. This is where
we are going to eat,” and point at a stone or bench. But it works quite well
to take a picture of a stone or bench and say:“This is where you can sit.”
    One autumn day after school it was time to make preparations for
the winter season. The table at which Abdulkader usually sat and drew
needed to be moved in from a room with no radiator. The situation
turned chaotic—as expected. But the solution was obvious: show him a
picture of the table in its new position with drawing paper and crayons
on it and say:“This is where you can sit and draw,Abdulkader.”And that
solved the problem.

To make apparent through variation. To make the world real.
To make words visible has meant a lot and created trust. Many visits in
many different shops were needed before Abdulkader understood what
“shopping”meant. But now he does and he likes to come along when it
is time to go to the store.Agneta is of the opinion that it is the
multiplicity of his own experiences of reality that remain in the pictures
that make his world, thoughts and words real for him.

Winning back Swedish after the summer of 2001
During the summer of 2001 he spent eight weeks in a totally Swedish-
speaking environment without his parents, who always speak Somalian
with one another and with their children. Nevertheless, it was as though
all the Swedish words had disappeared by the autumn when he returned
to school. The jumping, running around and anxiety that existed with
Abdulkader when he started school four years ago was there again, and
Agneta is trying to understand what happened and has started from the
beginning again.When he is able to see his pictures, though, his own
through the years, he can, nevertheless, answer in Swedish.

16   •   pictures as language
Size of Abdulkader’s vocabulary in words and pictures
At the most,Abdulkader has used a total of 300 to 400 words actively,
primarily in combination with pictures or objects.A day when he is
most talkative, he will produce perhaps 20—30 words. Most are
question words (like “Truck?”in the example above) or words that he
uses to give confirmation. He has spontaneously spoken on a few
occasions. One time when he was sitting at the window and the first
snow was falling he suddenly said:“Go sledding”.And on another
occasion when he had just had his hair cut, he came up to Agneta and
said:“Cut hair”. His parents have related many more similar examples
of spontaneous speech.
    His first year in school (97/98) he had 400 pictures; the second an
additional 700; the third, 2,000, and the fourth, 1,500. He now has a total
of 5 000 pictures. But during an ordinary week at school, he does not use
more than perhaps 50 to 60 of these actively.

Concluding comments
The situations described here are the ones which Agneta and I discuss
in detail and which we think we are able to describe even though we do
not understand them. But there is so much more that has happened and
is happening with Abdulkader that we do not understand and find hard
to even express, that his case offers an excellent illustration of the need
to strive for better models of explanation and better concepts
concerning language and visualization.

17   •   pictures as language
5.Case study:Gun
henrik danielsson

Gun was chief secretary and supervisor of the admissions and
educational affairs office at the Institute of Technology, Lund University,
Sweden. She suffered a stroke on May 4, 1991 resulting in aphasia. It
manifests itself currently in word-finding difficulties, even though she
knows what the words mean. Other changes have included reduced
memory and weakness on one side of her body, but these are less
troublesome at present. Maria Sporre, a speech and language
pathologist, has met Gun on several occasions and described her
aphasia as follows in the summer of 1997:

     Gun’s biggest problem is to find the right word or more accurately,
     “come up with what the word is called”. She experiences certain words
     as being easier and others more difficult to find. Some words are totally
     impossible. There has been an obvious improvement in her
     reacquisition of words. It is quite a strenuous task to learn them when
     each one has to be worked on repeatedly. On the other hand, Gun
     relates that some words just “come to her”, and then it seems to be
     more of an aha! experience than the effects of training.

A detailed description of Gun’s aphasia can be found in Gun Andersson
och hennes väg framåt (Gun Anderson and Her Road Forward), [1] and
Bildligt talat (Figuratively speaking) [11].

Training aids
Gun has used different aids to work on reducing the effects of her
aphasia. Up until the summer of 1997 she primarily used five
approaches. The first four were used between six months and one year,
and the fifth for two years. This was not due to decisions made on the
part of Certec; Gun has used them as long as she has chosen to.

Talfyran is a communication and speech training aid that was
developed as a project at Certec in 1991 and 1992 by a technological
student. Talfyran consists of a microphone, a speaker and buttons for
recording and play.You are able to record four different messages of a
maximum length of eight seconds. Gun used the device to record a
word and then listen to it. She could hear if she said it wrong and could
then correct herself. Gun used Talfyran in 1993 and learned
approximately one word or expression a week.

18   •   pictures as language
MacSnack is a speech training computer program for people with
aphasia that Gun used for six months starting in June 1993. The
program was developed by the Speech Pathology Department at
Malmö General Hospital, Sweden. The program required that someone
help Gun to get started, but then she was able to manage it on her own.

                                                                             One of Gun’s flowerbeds in the
                                                                             multimedia program Gun’s
Overview   Change view   Click on this part to zoom in

Gun’s Garden
Gun’s Garden is a multimedia program, which is both a communication
aid and a training program. It was specially developed for Gun and
contains an overview of her garden at home.All of her flowerbeds are
included and if she clicks on one, she sees a picture of the different
flowers and bushes that are there. She can access the names of the plants
in Swedish and Latin, in text and in sound. She can also find out other
information about when they flower and ideal growing conditions.
    Gun’s main hobby is gardening and she used the program to
practice the names of flowers and plants and as an aid in remembering
them.When she forgets a name, she can easily find it by clicking on the
flowerbed in Gun’s Garden.

Musical training
Even though the ability to speak may be damaged, musical abilities can
still be intact. This can be used for alternative communication. Music
does not have the same direct connection to thought as language does,
but it can still adequately express feelings and moods. This worked
relatively well with Gun, but not to the point that she wanted to continue
using it.

Introduction of digital photographs
Since digital photographs have been so successful in supporting
memory and communication for people with developmental dis-

19   •   pictures as language
abilities, we thought that they could even help Gun. The constellations
of problems involved in the two disabilities are quite different, but both
include memory difficulties.We simply borrowed The Pictorium
concept and adapted it to Gun’s needs. To our great delight, it worked
for her. Over time, Gun’s and The Pictorium’s photo processing system
has changed considerably. Today, Gun uses standard products on the
market. She has a photo scanner that transfers the images to the
computer, the ACDSee thumbnail program for processing images, and
she writes about her pictures in Word. It took about three months before
Gun was able to manage taking pictures on her own and transferring
them over to the computer without the assistance of others.

Pictures as reminders
Gun uses her pictures primarily as reminders. She uses them to remind
herself of people or the name of a person, a difficult word that she finds
in a newspaper (it is interesting to note that she uses the same
technology as the Pictograms at The Pictorium, i.e., having the word
printed on the picture), a place she has been, an event she has
experienced, or something she has seen on TV.

Personal reasons for pictures
What has been described above you might think could be handled by a
                                                                              To remember someones name.
symbol system.With what is about to be described, it would be more
difficult because it is so closely connected to Gun’s personality. She takes
great pleasure in her grandchildren and loves to photograph them. She
thinks it is fun to see how they have grown and developed from photo to
    She takes pictures of plants and flowers too and learns the names of
different species. She has another area of interest: looking at clouds and
finding faces in them. Symbol systems have difficulties dealing with
highly personalized, special areas of interest because they become
unmanageable when there are too many unique words in one category.
    She is also interested in taking pictures of scenic areas near her
home by the sea.                                                              … special interests … more
                                                                              pictures on the next pages
Gun’s writing
Since the autumn of 1995, Gun has written down all the words that have
come back to her in her own “glossary”with a short explanation of what
it was that triggered the return. She was acquiring about a word a day,
with considerable variation, of course, from day to day and even from
month to month.
    By the autumn of 1997, though, Gun stopped writing down new
words because she couldn’t keep up with them. So many words were
coming back to her every day that it took her several hours to write
them and the explanations down.

20   •   pictures as language
                                             Places and events.

Difficult words.

Seen on TV.                 Grandchildren.            Remember the name of a flower.


21   •   bilder som kognitiv assistans
     She has continued, though, to write about her language develop-
ment. The examples below are from reports she has written at Certec
and you can see the improvement.
     From My Diary with Digital Photos [2], 1997:
     January 28, 8 a.m. I was at the clinic to see Doctor Mats Person.
When I was going to leave, I asked if I could take his picture. OK.
It turned our quite well, don’t you think?
     From It Never Ends [4], 1999:
     Friday I got up at 7:30 (I thought that I was going to sleep later) and
for the first time this year I ate breakfast outside. It felt so good and I also
sat on the “new”bench and had breakfast on the “new”table that Ben
had painted. Our garden furniture was brown for many years. I have
                                                                                  The Return of Words [3].
photographed the “new”pieces that now are painted sky blue.
                                                                                  Examples from Gun’s glossary,
Improvements                                                                      June 8
Since Gun started using digital photos, her language has improved in              Dachshund
                                                                                  Have heard, but first heard
many ways:
   • Gun has started reading the newspaper. At first just an occasio-
       nal article but now the whole newspaper.
   • She understands jokes and can even joke herself. This shows that             Ate this evening
       she is quicker at understanding what is said.
                                                                                  June 9
   • She speaks more fluently because she no longer needs as much                  Buttercup
       help in finding the word she intends to say, nor does she need to           Saw in the park
       ask as often what certain words mean.
   • Gun manages to understand spoken conversation and written
       texts faster, which she displays in a number of ways. She has
       finally managed to watch foreign programs on TV because she
       can now read the subtitles. It is hard to get much out of a
       program by just looking at the picture or vice versa, by just
       reading parts of the subtitles. Now she can do both.
   • Many people with aphasia have problems when there are several
       others talking at the same time, or they quite often mix up who
       is speaking. It is also bothersome when there is a lot going on
       around them. This is exactly what happens during coffee breaks
       at Certec. Nowadays, Gun joins in the conversation in a different
       way and understands more, even if there are many speaking at
       the same time.
   • Gun is so secure in speaking that she now always answers the
       telephone. Before she only did it occasionally. This is a very
       significant improvement as it is an indication of stronger self-
   • Gun has held talks on aphasia on her own. Previously, she would
       always ask someone to come along and speak for her or just be
       there as a backup in case she got sidetracked, but now she does it
       on her own.

22   •   pictures as language
     •   Gun uses more precise words for what she wants to express.
         Those things that she could only call “ice”and “green”before are
         now “frost”and “salad”.

6.The Picture WebBoard—indication
of a wider usage
henrik danielsson

The use of digital photographs is on the increase in schools,
rehabilitation, and care services for the disabled. Interest in digital
pictures has spread to many people, at the same time as very little is
being written about it. That is why I established a meeting place on the
Internet using the WebBoard conference system in which all who want
to can participate and share their experiences. I call it the Picture
WebBoard and the address is: It has been a great
success and there are over 300 participants at present.

How it works
The first time you want to gain access to the discussion forum, you need
to establish a user profile by providing your name and e-mail address,
among other things. This is done so that you do not need to write who
you are each time you make a contribution. Inside the WebBoard, you
can choose among the different headings such as,“My Example”,
“Cameras”,“Reference Tips”,“My Pictures”and “Cyber Café”. Under
each heading are contributions that you can read.You can also reply if
you want to comment on what has been said, or write your own
contribution.Additional functions include chatting, searching, and
seeing who has logged in during the day.

Why I think it is valuable
The WebBoard is very good for me as a researcher and for people
working with digital photos. I receive many more descriptions of how
pictures are used than I would be able to collect myself by visiting the
different organizations. I also find out about good educational
programs, literature, cameras and computer programs that process
digital images. Moreover, I have established a large network of people
with experience in the area with whom I can brainstorm. Those whom I
have interviewed think the WebBoard is good, but for other reasons.
They like to read about the experiences of others so that they can gain
new ideas of what they can do and how to go about it. In addition, they
are able to share and receive feedback. It is very stimulating and often
results in the generation of new ideas. If you are just beginning to work

23   •   pictures as language
with digital photos, the WebBoard is an excellent starting point.You will
find many suggestions on what to do and can avoid the mistakes others
have made. It gives you a kind of “kick start”.
    This way of saving knowledge and experience in one place results in
one never needing to start from the beginning, which is the case with
new staff members who have no experience with digital photos.Arne
Svensk investigates the subject of retaining knowledge and conveying it
to different people and contexts in his licentiate thesis [39].

An examination of what has been discussed and how
the pictures have been used (based on [13])

An analysis of the postings in the discussion forum was carried out to
see what kinds of digital pictures usage were of interest. In the
discussion forum there were 757 postings from November 4, 1999 when
the forum started, until December 1, 2000. 156 of them were under the
title “picture usage”.We used knowledge obtained from previous work
with digital pictures to divide the postings into different categories. The
reliability of the division into categories was investigated by having 25
randomly selected postings independently judged by two other
     We also wanted to know actual usage, not only what the persons in
the discussion forum reported. So we carried out an e-mail survey with
questions based on the results of the analysis. The survey was sent to the
268 persons who had given a correct e-mail address of the 296 persons
that had visited the discussion forum.We were only interested in the
answers from persons who had experience of practical use of digital
pictures, so the first question was:“Do you use digital pictures for
persons with disabilities?”Those who answered yes continued with
“Yes/no”questions of the format:“Have you used digital pictures in this
way?”, where in this way consisted of the categories from the analysis of
the postings.
    A statistical analysis (χ2 test “goodness of fit”) was carried out to see
if the discussion matched what was actually done. The answers in the e-
mail survey were also analyzed in the same way to see if they
differentiated from random answers.

The analysis of the postings resulted in nine categories.We list them
here (the number of postings within brackets) along with an example
from real users of how they use digital pictures in that way. 27 postings
were placed in the 10th category “Unrelated to this investigation”. The
categories together with the number of postings are shown in Figure 1.

24   •   pictures as language
                  Documentation                  26
                 Self-awareemess                 25
               Situation of choice               19
                         Planning                17
                        Sequence                 16
               Wishes and dreams                 10
            Stimulation of speech                 6
                         Initiative               6
                           History                4                             Figure 1. Number of postings in
                                                                                the discussion forum.

Documentation (26)
The father of a girl with autism and a developmental disability tells that
they are doing weekly schedules with digital pictures. They save the
schedules and use them as documentation. The girl likes to sit and look
at old schedules by herself.

Creating self-awareness (25)
Pictures have been taken of a person with a developmental disability
when he expresses different feelings. Now he can see what he looks like
if he is happy or angry. It is also much easier for him to tell others how he

Situation of choice (19)
At a group home for persons with cognitive disabilities, they present
leisure activities with digital pictures, video sequences and sound on a
computer.You can go through the alternatives as many times as you like
before you decide. Earlier, when the activities were presented in text and
symbols, some persons often regretted their choice after a month, but
this seldom happens now.

Planning (17)
At a special education school in Sweden they were planning a one-day
trip to Denmark. They put together a description in pictures of what
was going to happen the whole day, and gave it to every pupil. The
description contained pictures of taxi – bus – train – ferry – little
mermaid – walk – zoo – picnic –walk – bus – ferry – train – bus – taxi.
The possibility for the pupils to see what was going to happen made
them feel secure and the trip was a great success.

Many pictures in sequence to tell a story (16)
When sending a digital picture letter to an activity center for persons
with developmental disabilities, showing the backyard of a house with

25   •   pictures as language
an apple tree and rotten apples laying on the ground, we received an
answer in 11 pictures showing an appropriate way to take care of the
apples. The pictures were from their garden and showed all the steps in
taking care of apples.

Wishes and dreams (10)
At an activity center for persons with developmental disabilities, they
have tried to visualize their wishes and dreams and make them easier to
talk about. They can choose among different pictures of things they
want, people they want to meet and places they would like to go.After
that a picture of the person was pasted into the pictures. One showed a
person sitting in a red sports car.

Stimulation of speech (6)
In the same activity center a person started to talk to the pictures. He
hadn’t talked much before, but now he often stands alone in front of a
wall covered with pictures, talking about what is happening in the

Taking initiative (6)
In a home for persons with cognitive disabilities, they had pictures of
the personnel working there. The pictures were placed on different
boards depending on if they worked that day, were going to work that
night or were on vacation. One morning one person got up earlier than
usual and moved a picture of a staff member to the vacation board,
since he didn’t want that person on his schedule.

History (4)
A young man with autism in Denmark had only 2 persons whom he
had known his whole life: his stepmother and stepfather. The
stepmother came to his activity center and told the personnel that she
had cancer, and only had 3 months to live. To help the young man retain

                                                                           Figure 2. Results of the e-mail
                  Documentation               65                           survey.
                 Self-awareemess              36
               Situation of choice            65
                         Planning             68
                        Sequence              61
               Wishes and dreams              34
            Stimulation of speech             37
                         Initiative           37
                           History            35

26   •   pictures as language
his history, a CD with pictures of all the big moments in his life was
made including the stepmother’s voice telling what happened. Now that
his stepmother is dead he often sits with the CD, recalling what has

The result of the reliability of the division into categories was that 88%
(21 of 25) of the postings were in the same category for all judges.
We received 140 answers from the e-mail survey, which means an
answer rate of 54%. 77 of them used digital pictures for persons with
disabilities. The most used categories were: planning (68), documenta-
tion (65), situation of choice (65), and sequence (61). The other
categories were used by approximately 50% of the users. See Figure 2 for
more details.

The result of the statistical analysis showed a significant difference
between the discussion and what was actually done (χ2 (8)=97,9,
p<0.05). The answers in the e-mail survey were also analyzed to see if
they differentiated from random answers with the χ2 test “goodness of
fit”. There was a significant difference and the categories that made a
significant contribution to that were “documentation”,“situation of
choice”,“planning”and “wishes and dreams”(χ2 (8)=38,9, p<0.05).
    More about different types of digital photo usage can be found in
Bildpraktik (Picture Practice) [12].

7.Discussion and conclusions
Our experience of the effects of the introduction of personal digital
photographs as language stretches over an eight-year period. Our
reasoning has progressively moved in the direction that all higher
mental activity is language related (in its broadest sense).According to
this line of reasoning, a reduction of the language disability would most
likely result in just the developments that we have seen.
    But the models we use to explain these effects have never been
analyzed by linguists or other researchers in the language sciences. They
can be briefly stated, considerably abridged, as follows:

1. It is essentially the direct and indirect effects of the reduction in
   language disability that we see.
2. The most important effect of pictures-as-language is that it is a
   means for some people to gain their first user-worthy language and
   in that way drastically improve their ability to learn.
3. The prerequisite of all learning is some form of a relation between a
   person and her surroundings: the material, the natural and
   (particularly) the human. Language in its broadest sense can be said

27   •   pictures as language
     to constitute this relationship, be the mental means of conveyance.
     Many of those we are writing about have previously had weak and
     monotonous relationships to their surroundings.
4.   It seems as though the personal digital photographs are often more
     real for the users than reality itself, i.e., it is easier for them to esta-
     blish a relationship to the phenomenon through the picture than in
     direct contact with it. This can be due to a number of factors, among
     them, that the picture is a still life and immutable, that it is bright,
     that it has a frame, that it makes it possible for the viewer to look at
     the situation from the side, etc. This and other factors have already
     been discussed in 1996 in The Computer’s Power of Attraction. [22].
5.   If you examine what a relationship to another person can signify for
     learning, the most important function of the picture is that it allows
     an exchange in initiative taking between the differently abled person
     and her surroundings; that it enables conversation based on a
     common focus and—perhaps most of all—that it enables variation.
6.   Language is hardly a uniform “something”that arises and develops
     in a predetermined manner, neither on the individual level nor
     collectively.Verbal expression arises in specific contexts, is used for a
     specific purpose, differentiates successively and gradually develops
     into an enormous collection of concepts that are interwoven in all
     that we do.With an emphasis on “do”. It is crucial that action is
     coupled to language. This is most likely the same for people with
     cognitive disabilities.
7.   Rich action and rich language presuppose variation [18, 29]. It is
     variation rather than repetition that is the mother of all learning
     [25]. But the world around us tends to be more repetitive, the greater
     the advantage it has intellectually, power- or age-wise. In language
     there are, however, small shifts of already existing meaning, which
     gradually build up a whole world of internal conceptions of how
     things are related. The strength of digital photos in this context is
     that they can easily become so many that they provide a variation
     that offsets what is static and at times meager.
8.   Applied to the theme of this conference,“Language and Visual-
     ization”, one can say that for many people, pictures-as-language is
     their complete version of language and visualization; a special case,
     so to speak, because the use of pictures is both language and
     visualization, all in one. For others who through pictures are able to
     preserve or strengthen their spoken language, it is the variation in
     visualization that interacts with the variation in language in a
     dynamic developmental and learning process.

We can go no further than these eight points on our own. In spite of
eight years of intensive work on the project, we cannot carry out
sufficiently qualified and relevant analyses and evaluations of the

28   •   pictures as language
results.We have not been able to find more penetrating, applicable
explanatory models for the effects, nor relevant, basic theories that we
can rely on. Indeed, the digital picture concept is less than a decade old,
and on the basis of that it is not unreasonable that theory building
hasn’t yet caught up. But what we have discovered appears to be so
powerful, perhaps even primitive, that it has in all likelihood been
studied in the language field.We are asking, in other words, for help.
    If it is possible for us together to identify relevant existing theories
or to come up with new, useworthy theoretical foundations, the effects
would be at least twofold: To better understand and utilize the results
achieved so far; and, with a good theoretical basis, be able to define
outstanding sources of inspiration for an even more effective
implementation than what we have hitherto accomplished.

29   •   pictures as language
[1] Andersson, G., Knall, G. (1994). Gun Andersson och hennes väg framåt – om
rehabilitering efter en hjärnskada (Gun Anderson and Her Road Forward).
Report, Certec, LTH, 1: 1994.
[2] Andersson, G. (1997). Min dagbok med digitala bilder (My Diary with
Digital Photos).
[3] Andersson, G. (1998). Ord som kommer tillbaka men också en del nya ord
(The Return of Words).
[4] Andersson, G. (1999). Det tar aldrig slut (It Never Ends). Report, Certec,
[5] Bauth, R., Jönsson, B., Svensk,A. (1995). Just Give Us The tools. Natur och
[6] Bray, N., Fletcher, K., & Turner, L. (1997). Cognitive competencies and
strategy use in individuals with mental retardation.
[7] Brown,A.L., & Campione, J. (1972). Recognition memory for perceptually
similar pictures in preschool children: J. Exp. Psychol., v. 95, p. 55-62.
[8] Bryant, P. (1965). The effects of verbal labelling on recall and recognition in
severely subnormal and normal children: Journal of Mental Deficiency
Research.1965; 9(4): 229-236, p. -2366.
[9] Dahlöf,A., m.fl. (1999). Ovanlig vänskap (Uncommon Friendship). Film.
Uppsala Publishing House.
[10] Dahlöf,A., Larsson, S. (1999). Växtbok för annorlunda människor (Growth
Book for Differently Abled People). Uppsala Publishing House.www.uppsala-
[11] Danielsson, H. (1998). Figuratively speaking. Report, Certec, LTH, 1:1998
[12] Danielsson, H. (2000). Bildpraktik – Om digitala bilders betydelse för
personer med kognitiva funktionshinder (Picture Practice—The Significance of
Digitial Photos for People with Cognitive Disabilities). Internal Report, Certec,
LTH, 2:2000.
[13] Danielsson, H., Svensk,A. (2001). Digital Pictures as Cognitive Assistance,
The 6th European Conference for the Advancement of Assistive Technology in
Europe,AAATE 01, p. 148 – 152.
[14] Dobson, E., Rust, J. (1994). Memory for objects and faces by the mentally
retarded and nonretarded: Journal-of-Psychology.1994 May;Vol 128(3): 315-
322, p. -3222.
[15] Dulaney, C., Raz, N., Devine, C. (1996). Effortful and automatic processes
associated with Down syndrome and nonspecific mental retardation: American-
Journal on Mental Retardation.1996 Jan;Vol 100(4): 418-423, p. -4233.
[16] Elliott, D. (1981). Visual and kinesthetic memory and integration of
mentally retarded and nonretarded adults: Am. J. Ment. Defic., v. 86, p. 194-200.
[17] Ellis, N.,Woodley-Zanthos, P., Dulaney, C. (1989). Memory for spatial
location in children, adults, and mentally retarded persons: Am. J. Ment. Retard.,
v. 93, p. 521-526.
[18] Emanuelsson, J. (2001). A question about questions.Acta Universitatis
Gothoburgensis. ISBN 91-7346-415-5

30   •   pictures as language
[19] Ferretti, R., Cavalier,A. (1991). Constraints on the problem solving of
persons with mental retardation.
[20] Jolliffe, T., Baron-Cohen, S. (1999). A test of central coherence theory:
linguistic processing in high-functioning adults with autism or Asperger
syndrome: Is local coherence impaired?: Cognition, v. 71, p. 149-185.
[21] Jönsson, B., Svensk,A. (1994). Technology and Differently Abled People.
Natur och Kultur.
[22] Jönsson, B. (1996). Datorns attraktionskraft (The Computer’s Power of
Attraction). Internal Report, Certec, 7:1996.
[23] Jönsson, B., Philipson, L., Svensk,A. (1998). What Isaac Taught Us. Certec.
ISSN 1101 9956
[24] Jönsson, B. (2001). Unwinding the Clock: Ten Thoughts on Our
Relationship to Time. Harcourt Inc.
[25] Jönsson, B., Rehman, K. (2000). Den obändiga söklusten (The Unyielding
Desire to Search). Brombergs.
[26] Jönsson, B. (2001). Tankekraft (Thinking Power). Brombergs.
[27] Kamio,Y., Toichi, M. (2000). Dual access to semantics in autism: Is pictorial
access superior to verbal access? J. Child Psychol. Psychiatry, v. 41, p. 859-867.
[28] Kamio,Y., Toichi, M. (1999). Semantic systems in high-functioning autism:
Pictorial or verbal?: Seishin Igaku Clinical Psychiatry. Mar;Vol 41(3): 255-262,
p. -2622.
[29] Marton, F., Booth, S. (1997). Learning and awareness. Lau´wrence
Erlbaum Associates.
[30] Mottron, L., Peretz, I., Menard, E. (2000). Local and global processing of
music in high-functioning persons with autism: Beyond central coherence?: J.
Child Psychol. Psychiatry, v. 41, p. 1057-1065.
[31] Mottron, L., Belleville, S. Menard, E. (1999). Local bias in autistic subjects
as evidenced by graphic tasks: Perceptual hierarchization or working memory
deficit?: J. Child Psychol. Psychiatry, v. 40, p. 743-755.
[32] Mottron, L., Belleville, S. (1993). A study of perceptual analysis in a high-
level autistic subject with exceptional graphic abilities: Brain Cogn., v. 23, p. 279-
[33] Plaisted, K., Swettenham, J., Rees, L. (1999). Children with autism show
local precedence in a divided attention task and global precedence in a selective
attention task: J. Child Psychol. Psychiatry, v. 40, p. 733-742.
[34] Plenkovic,V. (1988). Pamcenje auditivnog i vizualnog slijeda kod autista
(Auditory and visual sequential memory of autistic subjects): Revija-za-
Psihologiju.(1988) Dec;Vol 18(1-2): 43-51, p. -2.
[35] Plato, G., Jönsson, B. (2001). Art and Science. Internal Report, Certec, LTH,
[36] Rinehart, N., Bradshaw, J., Moss, S., Brereton,A., Tonge, B. (2000). Atypical
interference of local detail on global processing in high- functioning autism and
Asperger’s disorder: J. Child Psychol. Psychiatry, v. 41, p. 769-778.
[37] Sporre, M. (2000). Digitala bilders kommunikativa funktion för människor
med kommunikationshandikapp (The Communicative Function of Digital
Photos for People with Cognitive Disabilities). Department of Logopedics,

31   •   pictures as language
Phoniatrics and Audiology.
[38] Stratford, B., (1979). Discrimination of size, form and order in mongol and
other mentally handicapped children: J. Ment. Defic. Res., v. 23, p. 45-53.
[39] Svensk,A. (2001). Design for Cognitive Assistance. Licentiate thesis, Certec,
LTH, 1: 2001

32   •   pictures as language
Appendix 1: Pages 3-8 from What Isaac Taught Us [23].

Appendix 2: “Thomas leans to measure with a yardstick” from What Isaac
Taught Us [23].

33   •   pictures as language
34   •   pictures as language
35   •   pictures as language

Shared By: