Document Sample
ACCORD Powered By Docstoc
					                                                                    DELIVERABLES TABLE

Project Number:           IST-2000-26364

Project Acronym:          ACCORD
Title:                    Usage and Development Scenarios for the Tangible Toolbox

Del. No.     Revision                                    Title                                    Type1   Classifi-    Due        Issue
                                                                                                          cation2      Date       Date
   D1.3                     Usage and Development Scenarios for the Tangible                     Report     Pub       01-09-30   01-12-20

1 R:Report; D: Demonstrator; S: Software; W: Workshop; O: Other – Specify in footnote
2 Int.: Internal circulation within project (and Commission Project Officer + reviewers if requested)
 Rest.: Restricted circulation list (specify in footnote) and Commission SO + reviewers only
 IST: Circulation within IST Programme participants
 FP5: Circulation within Framework Programme participants
 Pub.: Public document
                                      Administering Connected Co–Operative Residential Domains

  D1.3 Usage and Development
   Scenarios for the Tangible

Abstract: This deliverable considers a number of common tasks that might be
supported around the domestic environment and the range of functionality required to
realise these tasks. This work has been informed by a collection of patterns describing
how domestic technologies are woven into the fabric of everyday life, and through a
consideration of how different technological possibilities might be plausibly used. The
main result presented by the deliverable is the definition of two scenarios that will be
used to focus the development of the toolbox and point towards the tangible
interactive environment (TIE) that the project will use to finally demonstrate its

                                   Document ID               ACCORD D1.3
                                   Status                    Final
                                   Type                      Deliverable
                                   Version                   1.0
                                   Date                      December 2001
                                   Tasks                     1.2/3.1
                                   Authors                   Karl-Petter Åkesson,
                                                             Jan Humble,
                                                             Andy Crabtree,
                                                             Adrian Bullock.
ACCORD IST-2000-26364

Project coordinator:

Adrian Bullock
Swedish Institute of Computer Science
Box 1263
164 29 Kista

Tel. +46 8 633 15 73
Fax. +46 8 751 72 30

The ACCORD project includes the following institutions:

The Swedish Institute of Computer Science
The University of Nottingham

Authors of this report:

Karl-Petter Åkesson (
Jan Humble (
Andy Crabtree (
Adrian Bullock (
ACCORD, IST–2000–26364                                                                                           Deliverable D1.3

Table of Contents

1. INTRODUCTION ................................................................................................... 2

2. PATTERNS OF HOME LIFE ................................................................................. 3

3. POTENTIAL SCENARIOS .................................................................................... 5
       3.1 HOUSEHOLD MANAGEMENT .......................................................................................... 5

       3.2 FAMILY MANAGEMENT .................................................................................................. 7
       3.3 COMMUNICATION ......................................................................................................... 8

       3.4 W ORK AND LEISURE..................................................................................................... 9

       3.5 W ELFARE .................................................................................................................... 9

4. TECHNICAL FUNCTIONALITY AND REALISATION......................................... 11
       4.1 AWARENESS .............................................................................................................. 11

       4.2 DIGITAL CONTENT...................................................................................................... 13

       4.3 ALERTS ..................................................................................................................... 15

       4.4 DISPLAYING ............................................................................................................... 15

       4.5 CONTROLLING ........................................................................................................... 17

       4.6 APPLICATIONS ........................................................................................................... 17

       4.7 GENERAL ISSUES ....................................................................................................... 17

5. DEMONSTRATION SCENARIOS ....................................................................... 19

ACCORD, IST–2000–26364                                                   Deliverable D1.3

1. Introduction
This deliverable is concerned with the identification of demonstration scenarios
through which, or some closely derived alternative(s), the final results of the project
will be showcased. More immediately the identification of these scenarios will
provide necessary focus as the development work on the tangible toolbox moves
Section two describes how the work involved in identifying scenarios has been
informed and motivated from ethnographic sources. A number of patterns of
technology use in modern homes have been identified and made available via a
website. These patterns show different activities and the use of different technologies,
with video footage available to back up the assertions. This resource has been used as
a starting point in the design process of identifying scenarios within the domestic
environment. A brief description of the resource is given, followed by a description of
the household kitchen that exemplifies just how the resource can be used. Other
influences in deriving the scenarios have been our own personal experiences and just
what is technically possible and available right now. This latter aspect is touched on
in part in section four.
Section three describes common activities that take place in domestic environments,
giving examples of a wide range of different activities including household
management, communication and welfare. These are the result of open brainstorming
sessions, where we have tried to identify a number of different things that families
and individuals do in homes.
Section four considers the functional requirements from a technological standpoint
that are necessary to realise the kinds of activities presented in section three. A
mapping between the activities in section three and the functionality in section four is
Finally, section five presents two demonstration scenarios, which draw from the kinds
of activities and functionality described in the previous two chapters. While those
discussions were wide-ranging and open, the aim of these scenarios is to provide the
project with a specific target to aim for at the end of the project. They offer concrete
examples in which to ground the ongoing development of the toolbox, as well as
signposting the way to a successful final demonstrator for the project.

ACCORD, IST–2000–26364                                                    Deliverable D1.3

2. Patterns of Home Life
In the previous deliverables 1.1 and 1.2 the industrialised domestic environment has
been studied and three main influences on its development identified; architecture,
technology and social interaction. We are now developing a pattern language
framework, allowing an appreciation of the socially organised ways in which
domestic technologies are woven into the fabric of everyday life to be an available
resource to designers. Just what are the everyday, mundane activities that take place
in the home? and how can we support these appropriately? Our work is to uncover
new patterns in the home and make these available as design guidelines through an
evolving website,
This website is a resource for the designers of computer-based technologies for the
domestic environment. It provides an evolving corpus of empirical patterns of
technology usage in the home, which may be used to think about and to ground design
in the practical day-to-day circumstances that make up domestic life.
You may access the patterns in four ways.
   1) You can search the patterns listing using the edit/find function on your
      browser if looking for particular topics or patterns.
   2) You can browse by using the patterns log, which groups patterns of
      technology usage around routine domestic activities in the home.
   3) You can browse by using the patterns index, which groups patterns around
      specific technologies used in particular sub-environments in the home.
   4) You can browse by following the links embedded in the patterns themselves.
      Links are located in two places in the patterns: at the beginning, where they
      connect to other patterns occurring in the same domestic routine, and at the
      end of the pattern, where they connect to other patterns using the same
      technologies. Furthermore, the title of each pattern is linked to the video
      sequence from which the pattern is derived. This allows you to inspect real
      world events in the home for yourself and extract your own findings
To date the patterns concentrate on one location in the house, the kitchen (as this was
the first part of the data to be analysed – the living room and bedroom remain to be
analysed). We have found the patterns to provide a rich collection of activities and
technology use, and they have helped us in the identification of the scenarios later on
in this deliverable.
Patterns of home life in the kitchen show us that the kitchen table is the central
technology in this location. The patterns show us that the table is an activity centre,
used in a wide variety of domestic events. Supporting existing patterns, functionality
would include providing drawing, colouring, and painting applications on the
tabletop. The range of functionality would be extended to provide diaries and note
applications. We could also include email applications, as the table is a prime site for
the placing and sorting of mail. We may also wish to consider the relationship of
paper mail to the digital, exploring bar code readers or some similar device.
The processing of mail extends our attention to technologies that are situated around
the kitchen table. Mail is initially placed on the table for person’s attention. On being

ACCORD, IST–2000–26364                                                   Deliverable D1.3

read it may then be placed on a nearby notice board. How do we support the
displaying of mail, or notes? Might users “tag” an email or note and in placing the tag
on the notice board thereby display email or note at the location where it is placed?
Not all items placed on notice boards are passive. Shopping lists, for example are
actively constructed over time. There is a need, then, to provide for the active
construction of displayed objects. This might be extended in new ways to provide
access to displayed objects from outside of the home, via a PDA for example. Thus,
the user might be able to retrieve the details of their shopping list whilst in the
supermarket, view emails from the office, or check notes left for them while travelling
home from work. Alternately, they may be alerted of appointments recorded on a
The character of list construction and its relation to everyday activities opens up
possibilities for sensing technologies. Patterns in the kitchen show us that things live
in places that are known in common and ready to hand. Food lives in this cupboard,
medicine in that cupboard, cleaning stuff over there. One pattern shows that things
are not always to be found where they should be; that certain items of food have been
used and are no longer there. This suggests that we might develop sensing
technologies for cupboards. These would include sensing the presence of foodstuffs,
and displaying their absence – which could be related to the construction of a
shopping list. Displays would not be located on cupboard doors or the fridge but at the
table or the notice board. The list would be split – one side indicating absent
foodstuffs, which may be dragged on to the other active side, which is the shopping,
list itself. Other sensors might be concerned with such things as medicine or
household cleaning products. These present a different set of sensing challenges, as
the user needs to know if such things are being tampered with here and now by
children. (N.B. There is a vast social and economic benefit to had here as many, many
accidents are caused by drinking bleach or medicines, etc., every year). More still,
what of security?
Considerations of the active construction of displayed objects draw our attention to
peripheral devices. Notice boards provide one example of these, PDAs another. This
class of devices might be extended in considering other patterns that occur around the
kitchen table. Children often use the kitchen table to read magazines at, or to do their
homework at. In both situations they are bringing a portable device (a book) to the
table. But both are not the same – magazines are read whereas schoolbooks are
annotated. So we need to provide some intermediate class of device – smaller than a
notice board but larger than a PDA – that allows users to bring things to the table,
annotate them if required, and transport them to other locations in the home.

Having shown how the patterns of home life have been used in the design process, we
now move on to consider a wide-ranging collection of scenarios that demonstrate how
technology can be used to support different domestic activities.

ACCORD, IST–2000–26364                                                    Deliverable D1.3

3. Potential Scenarios
The following compilation is an attempt to gather as much overall insight into the
issues of introducing technological solutions into the home. The categorization is not
strict and is intended for organization of the main topics each scenario addresses.
Despite the fact that most of these scenarios will not be part of an Accord
demonstrator, they should be feasible and easily implemented by the toolbox. We will
show how the underlying technology, despite volatile solutions, is common to the
whole set (see section 4).
We refer to the house system as the underlying technology supported by the toolbox,
mostly in the form of individual components or the combinations of such. It is not
intended to specify a centralized design.

3.1 Household Management
The following scenarios describe how technology can be used in sustaining the
integrity of a household as a physical entity, as well as external stimuli and the in and
out flow of materials.

3.1.1 Maintenance
There is a constant logistic flow of items in and out as well as within the house.
Whenever the family runs out of some ingredient the house system keeps track of the
missing ingredient and adds this on a provisional shopping list.
Before the family goes to the shop to by food a shopping list is composed from the
provisional list and transferred to a personal digital assistant (PDA). The PDA gives
access to other items that reside in the kitchen, such as the calendar. They discover
while in the shop that the cousins are coming over for this weekend, so they have to
buy extra supplies. The PDA provides information on the availability and quantity of
food items in the home.
By connecting to the house system while at work one can get suggestions for meals
one can prepare with the readily available supplies. One can also see which
ingredients are missing for a particular desirable dish. Missing ingredients could be
ordered via the Internet and picked up at the shop on the way home.
The house system does not only detect groceries but other items as well. If the family
runs out of diapers or clean clothes for the children there will be a notification,
optionally connected to the shopping list. By interconnecting the house system with
other houses or locales the system can also keep track of the children’s clothes. The
parents have difficulties in knowing if they should look for their children’s clothes at
home or if they are already are at the day-care centre. By tagging the children’s
clothes the house system can inform where the clothes are located.
Also the car is connected into the system and provides the home system with facts
regarding amount of gas, how far you can drive with the amount of gas, and other
maintenance required such as oil refill, water, adjusting tyre pressure, overhaul at the
garage, etc.

ACCORD, IST–2000–26364                                                  Deliverable D1.3

Christmas is the time to decorate the house and all the decorations have to be found,
usually a task for the father. He finds all the decorations for the Christmas tree but
cannot remember where he put the Christmas goat last year. Fortunately the system
helps him to locate it in the storage space under the stairs.

3.1.2 Visitors
The home has at times visitors when no one in the family is at home, such as
plumbers or delivery services. With the use of the house system the family can let
those people in by providing them with a time constrained entry code or by providing
the system with the maintenance worker’s identification. A remote messaging system
and a video surveillance unit at the entrance makes it possible to let in an incidental
The system lets also the family be aware of a visitor even though the family members
are not far from the home, such as when the family are at the playground. The system
also works in the opposite direction if the person approaching the front door is trusted
he or she gets a notification about where the family is at the moment.

3.1.3 Shelter Structure and Environment
The house is a complex construction of several systems, some independent and others
intricately connected. Installations and reparations of these systems, such as
electricity, water and heating, require a practical know-how as to where they are
internally located in the house. Introduction of more technology into the house makes
this harder to manage. To provide assistance the house system has different ways to
visualize the underlying infrastructures. When re-building the bathroom the family
can bring up a model of the bathroom with all existing pluming and electricity. They
can change this model into what fits their needs and send it to the construction firm
that is about to carry out the job.
The house system can also control ambient systems and adapt to different situations.
The home climate control system considers who is at home and in which rooms.
Cooling, heating, lighting and ventilation can be individually controlled. This can also
be used to let the house attain a certain status when the family is leaving the house,
depending on if they leave for work, shopping or vacation etc.
One can also provide the system with a vacation “script” that includes actions to be
taken for example; no newspaper, lamps to go on or off at certain times, automatically
lawn mowing etc.

3.1.4 Bookkeeping
Changes in the home take place over longer time periods, such as rebuilding of the
house and the family economy. The house system provides a budget organizer, where
one registers bills, signs them electronically to be paid and are informed by the system
when they are due. It also keeps track of all expenses.
With the bookkeeping it is also possible to track changes of the construction of the
house and keep a history of it. When thinking about changing a room the family can

ACCORD, IST–2000–26364                                                   Deliverable D1.3

see that, 20 years ago and before they moved in, that room was actually two separate
rooms but one of the walls has been torn down.

3.2 Family Management
The following scenarios concentrate on the personal and group activities of the family
members. Technology resources are for this case mostly information services that
help family members organize and adjust to those activities and changes in them.

3.2.1 Family activities
Calendars are important mnemonic objects, they remind people of things they must
do. An electronic version located on the kitchen wall allows the family to input
important dates and reminders: for example, a relative’s birthday.
The calendar synchronizes dates and reminders with other sources defined by the
family. In that way it helps the family to remember and keep track of notes received
from the child day-care centre regarding such things as when the centre closes and the
date for the photo session. Along with the reminder about the photo session the
parents are also asked if they want any pictures taken of their child. The house system
also communicates with the school and day-care centre to inform about things such as
when the parents are free from work over Christmas.
A common “To Do List” provides the family members with their responsibilities,
along with suggested individuals to take care of pertaining list items. All relevant
artefacts in the home allude to the list. Reminders both from the calendar and the To-
Do-List are provided via a range of devices, at work the fathers PDA reminds him
about the engagement anniversary.
In the kitchen the family have instead of a clock with hour figures one that has a face
with places, at home, at work, at school, in car, at friends. It has one hand for every
family member that indicates that person’s current activity. Among other things they
use it when the phone rings, as it is very convenient to know if the person asked for is
at home.

3.2.2 Family Travels
Not all members of the household use public transportation enough to learn the
timetables and the youngest child has not yet learned to read the clock. Therefore they
have a string hanging by the door, which is periodically wound up by a small electric
motor, the length thereby representing the time left until the next bus leaves. Their
neighbours have a similar device but it looks much more like a thermometer but gives
the same information.
In the hall they also have another decorative display with air bubbles rising up
through the water filled pipe indicates the amount of cars at a certain junction, which
the mother who takes the car to work has to drive through.

ACCORD, IST–2000–26364                                                    Deliverable D1.3

3.3 Communication
Conveying thoughts across physical spaces and generation gaps is an important part
of family harmony. The following scenarios demonstrate how the house system
enables a wider range of communication channels between relatives and

3.3.1 Inter-family Communication
One of the children leaves a note in the kitchen on the notice board to tell the parents
that she is visiting a friend after school. With the aid of the house system this note can
be written at the friends house and be delivered to both the home and the parents.
The notice board can also be used to leave reminders, such as Post-It notes. The
farther writes a note while at work and places it on the notice board in the kitchen. It
reminds him to remember to bring photographs to work the next day to give to a
colleague. He could have recorded a voice mail instead if the note was intended for
one of the younger children.
The family have most of their photos digitally converted and kept in the digital family
album. In this album resides also links to other memories of the family and some of
these memories are tagged and associated with physical artefacts, such as souvenirs.
The photographs that the father was supposed to show for his colleague have not been
scanned yet and thus have to bring them tomorrow.
The grandparents, as they live abroad, does not have much physical contact with the
children but use the family album and the picture phone to stay in touch. On the
notice board the family have created a certain area where all they put there are shared
directly with the grandparents. The children use this feature to let the grandparents see
their drawings. They make a drawing on the screen embedded in the kitchen table,
which they tag and then put in the special grandparents area on the notice board.
With the aid of the tags they can also put up notes, emails, lists and etc onto the notice
board if they want other family members to become aware of them. For closer
inspection the tag can be brought to the kitchen table where the content is displayed.

3.3.2 External Communication
Communication is not only internal within the family. The family has a wide and
changing network of relatives, friends, colleagues etc. In some situations one does not
want to be interrupted and it is then when the activity controlled answering machine
comes in. It knows when the family has dinner, put the children to bed etc. It can also
prioritise the incoming calls and give individual messages.
The family has a device connected to the phone line that displays who is calling. It
diverts this information to the ones not at home as well. The mother receives a short
text message on her mobile phone if someone called while she was at the gym.
They also have other tools that help them to know who has been trying to get in touch
with the family. A guest book keeps track of who has been at the front door when they
are not at home. Visitors can also leave small notes in the guest book. A lamp in the

ACCORD, IST–2000–26364                                                  Deliverable D1.3

living room briefly changes colour to indicate arriving emails for a certain household
member assigned to that colour.

3.4 Work and Leisure
In general, most predominant home activity tends to be leisure related. Despite
questionable value and interest in interactive entertainment media, we observe how
whatever the form of activity, for the most part passive, can be made more time and
space flexible.

3.4.1 Entertainment
One important part of modern life is to entertain oneself. This is often interleaved
with other tasks within the house. People often watch television while working in the
kitchen, orienting the TV set to their location. Instead of a dedicated device the
different displays located throughout the house act as intermediate displays and a TV
show can be moved around and placed in an appropriate location. These displays are
also used for other activities; the children draw on the kitchen table display and then
move on into the living room and continue.
The different available spare time activities such as watching movies, sport events, etc
are presented by the house system for each person in the household with care taken to
that persons “pleasure/spare time” profile. Quite often the son in the house uses the
system to record a scheduled program he has read about in the newspaper and that he
would want to see but somehow can’t. While not at home he suddenly realizes that a
particular show is on, he calls the home system and instructs it to record the specified

3.4.2 Homework
A child comes into the kitchen and starts doing his homework at the table. He decides
to go into the living room as to not be disturbed by the younger siblings. He associates
the document he is working on a tag and drops the tag onto an intermediate A4 size
display and takes the display into the living room where continues with his work.
From the portable display he can move the document to the larger display in the living

3.5 Welfare
We now describe specific scenarios that deal with the welfare of family members.
This includes individual health concerns (weight, diet, etc), well-being (suitable
clothing, home climate control, etc), security of the home, and accident prevention.

3.5.1 Health
The following scenarios demonstrate both support for personal health surveillance and
family well-being.
When the father walks into the bathroom some bodily fluid (urine) is tested and used
for automatic health analysis. The system detects a high level of stress hormones and

ACCORD, IST–2000–26364                                                   Deliverable D1.3

recommends him to take vacation. By measuring weight and length when someone
enters the bathroom, the person can be recognized. This information is also stored
along with the other medical data collected. Thus the system can use this information
to keep track of people within the household.
Before the children are sent off to school, the mother notice the temperature is +10 C,
but the system warns that the afternoon temperature is expected to be considerably
lower, around –5 C, and she tells the children to dress accordingly.

3.5.2 Accident Prevention
The youngest child has picked up a bottle of bleach, or medicine. Sensors located in
the kitchen cupboards where household cleaning materials and medicines are stored
detect this. Together with position information of the inhabitants, the system triggers
an alert. Since the oldest person at home is the fourteen-year-old daughter upstairs in
her bedroom, the alert is sent to a device in the girl’s bedroom.
The family has a hunting cabin in the northern part of Sweden, almost a day’s drive
by car. There they have a fridge where all frozen meat from the elk hunts in the
autumn, as well as other kinds of meat. Last spring they had a power failure and
somehow the fridge did not start again and 50 kilos of meat could have been
destroyed. The cabin alarm system alerted them, and they took the car up there and
saved it.

3.5.3 Security
Security is a mundane but important matter in the home, e.g. when the last family
member leaves the house an alert brings an open window to attention. Sensors detect
whether or not windows and doors are open and alert users via an appropriate device
(the kitchen table, notice board, PDA, etc.) if they are. Similarly, when appliances,
such as the stove, have been left on, alerts are posted.
When the parents are sitting in front of the TV a textual reminder pops up on the
screen to alert the family that the stove was left on. Since the TV is already an
established display it is used to display alerts about various things, show video from
other rooms/front door view, "programming" the home, etc. It, like other display
devices, can also help in indicating and assisting at different threats such as fire,
burglaries, water leaks etc.

In the next section we examine the functional requirements, from a technological
standpoint, that are necessary in order to realise the kinds of scenarios outlined above.

ACCORD, IST–2000–26364                                                     Deliverable D1.3

4. Technical functionality and realisation
We now proceed to determine the different types of components based on their
functionality. They cover the technical solutions needed to implement the potential
scenarios before mentioned. Each type of component is described in non-technical
level, how it is technically realised and what technical and functional issues we might
have identified. We provide a reference for each component of an example scenario it
might be used in.
The most fundamental functional components are to receive input, generate output,
and interconnect these flows of data. Input comes either in the form of digital content
or through the more simplistic awareness functions. Driven through some sort of alert
this input is either displayed or controls a device or mechanism.
The preliminary underlying platform works with the notion of shared data spaces, by
which some of the devices will be aware of others and are capable of informing
listening devices of applicable outputted data. The data space is generic in nature, and
by means of an underlying data type transformer a particular data type will be relayed
in the proper format to its listeners (other devices or controlling software).

4.1 Awareness
One of the most important features of an intelligent house is its ability to be aware of
its surroundings. These functions provide an awareness of activities within or outside
the home. Some form of status value, numerical or Boolean, represents their outputs.
In most cases these are digital representations of physical sensors.
To associate the reading from each sensor with its physical counterpart a unique
identifier is required for each sensor. One way of providing this would be to use the 1-
wire line of sensors from Dallas Semiconductors that provides each sensor with a
unique identifier.

4.1.1 Absolute position
By tracking the position of users within the house, the system can have an awareness
of their activities. The absolute position of an object or a user within a locale will give
the system a possibility to draw conclusions of activities. E.g. if a certain person is in
the kitchen standing in front of the oven for sometime, the system can draw the
conclusion that he or she is most likely cooking. Acquiring the absolute position of a
device can be achieved in a number of ways with varying accuracy. GPS provides
almost universal coverage with an accuracy of +-10 meters though GPS does not
work indoors which limits its use within ACCORD. Ultrasound based positioning
systems exist and work quite well. An accuracy ranging from a couple of centimetres
to millimetres can be achieved. However, the effect of ultrasound on house pets is a
matter of possible concern. Other means for positioning is through triangulation of
base-stations for different wireless communication systems such as GPS, WaveLan
and BlueTooth.

ACCORD, IST–2000–26364                                                   Deliverable D1.3

4.1.2 Relative positioning
This indicates the position of an object relative to another object, e.g. to give a
warning if the child and the bottle of bleach are near each other. Relative positioning
can of course be achieved through the absolute position of each device but one can
achieve simpler and cheaper solutions if only the relative position is needed. An
ultrasound microphone can be used to measure the time it takes for a pulse to travel
from its source. Measuring this time gives the relative distance between the source
and the microphone with accuracy about the same as for absolute positioning with
ultrasound. By measuring the signal strength from a radio transmitter the
corresponding receiver can get a measure of the distance between the two. This gives
an accuracy of about a meter or two. An RFID tag reader can also be said to read the
relative distance between the tag and the reader but this value is binary as there are
only two states, read or not read.

4.1.3 Absence/Presence
Detecting the absence of certain objects from a known position will indicate that they
have left their spot. If we assume that most consumables in a home are stored at more
or less the same location, one can therefore quite easily detect the amount of each
type. Examples of this include detecting food items missing in cupboards, or any
other location for the readers, for more than a couple of hours, which would indicate
these are missing, or indicate something is running out (such as being out of diapers).
RFID tags embedded into the package of the grocery could be used to identify and
detect if it is present in a cupboard by embedding the RFID tag receiver in the shelf. If
the grocery is placed at the same spot each time a small scale can be embedded
instead of an RFID tag system. Barcodes could also be used instead of RFID tags and
do already exist on most groceries today. The barcode reader would have to be placed
in a way so the package is read all the time it is placed on the shelf. Both the relative
and absolute position of the packages could also be used to determine if something is

4.1.4 Movement
Detecting movement in a room indicates some sort of activity and can be used to
communicate general activity in a non-intrusive way. E.g. the grandmother can see if
her grandchildren are at home because their activity indicator shows this, and thus she
can make a phone call to them. Motion detectors for normal burglary alarms sense
movement and can be used to technically implement this. A video camera with some
image processing software can also achieve the same though it is more sensitive to
lightning conditions. The change of either relative or absolute position will also
indicate movement.

4.1.5 Open/close
Detecting if certain doors or windows are opened or closed can give indication of
activities. E.g. the medicine cupboard is opened while a child alone is in the vicinity
(detecting the child’s absolute position) sets off an alarm. The burglary alarm is set
off if a door/window is open while no one at home. Technically to implement this one

ACCORD, IST–2000–26364                                                     Deliverable D1.3

can use the vast range of burglary detectors that exists on the market that easily detect
if a door or windows is closed or opened. A simple switch embedded into the frame
that is closed when the door is could also be used. Contact-free measurement tools
could also be used to detect an opened door.

4.1.6 On/Off
Detecting if an appliance is on or off indicates if it is being used or not. E.g. the coffee
machine is on indicates someone is about to have coffee. This could be implemented
technically in a number of different ways. A current measuring instrument in the
power outlet would indicate how much effect an appliance is currently consuming and
tell if it is being used or not. The power switch of the appliance could be modified and
connected to a closed circuit detector.

4.1.8 Switches
A switch is a very generic type of input source and the user gives the meaning to it. A
switch could be used to sense both explicit and implicit actions from a user, e.g. turn
on the light or step on the doormat to leave the house. Switches are among the easiest
kind of sensors to make robust and interface with a computer. They come in a wide
range of design ranging from micro-switches to switch mats used as boat alarms.
Switches on paper via a regular printer, part of the Acreo technology, have the
advantage of providing a way for users to print an interface of choice. A
disadvantage, however, is that such switches don’t provide tactile feedback.

4.1.9 Other sensors
There exist many kinds of sensors and the ones mention above are the ones that we
have found most likely to be used. To mention some that also could be useful but
have not yet been identified as such are temperature and humidity sensors,
accelerometers and light sensors.

4.2 Digital Content
A fundamental component is the digital content; there are a number of functions
associated with it. The content can be produced, either manually or automatically, or
it can be brought in from an external source. Once in the home it has to be supervised.
It can also be shared with others, either family members or outside the family.
This data is stored within the shared data space and can be accessed by all services
within the home. Of course there are security issues that has to be regarded, such as
who has access to what. See section 4.8 for further discussion.

4.2.1 Manually producing content
There are numerous ways content can be produced. Some examples from the
scenarios include writing an e-mail on a keyboard, taking still images with a digital
camera or drawing an image on a touch screen embedded in the kitchen table.

ACCORD, IST–2000–26364                                                     Deliverable D1.3 Automatic content production
Through combining several services within the home new content can be produced.
E.g. producing a provisional shopping list by detecting what groceries are missing in a
cupboard. Also external resources can be used to produce new content, suggestions of
Internet food providers that can deliver the needed groceries. This category also
includes stream of data origin from the home, e.g. the entrance video camera.

4.2.2 Retrieving content
Content can also be brought into the home from outside. This might come from other
family members, friends or commercial producers to mention some. To bring this
content it has to be retrieved from some kind of source outside the home. From the
scenarios we have the following example demonstrating this, the water tower in the
hallway that indicates the amount of traffic at a certain junction. This information is
brought from the city council of transportations.

4.2.2 Sharing content
Content can also be shared, explicit or implicit, i.e. one can specifically send it to a
receiver or just make it available for other resources. The sharing can also be
restricted, e.g. the family is sharing their digital photo album for others to see but this
is restricted only to the grandparents.

4.2.5 Content supervising
To achieve a well working home, and from what we have seen in the patterns work,
organizing content, either it is physical or digital, is a fundamental part of home work.
On the desktop we have a physical metaphor on how we organize the electronic
content and we have to spend quite sometime to keep it organised. In the home we
also spend time to organize the physical props that inhabit our homes. Providing ways
of automatic filtering would be beneficial since it would minimize the time one have
to spend on doing it manually. Also, different ways of visualizing the digital content
is needed to get an understanding of the amount and how it is organized.

4.2.4 Associate content with physical artefact
To have a link between some digital content and a physical artefact one needs to
associate the two. E.g. to have your morning news displayed on the wall display in the
kitchen when you put down your coffee cup on the table you need to associate that
cup with your personal news service.

4.2.3 Moving electronic content
Content can be created in one place and be brought to another, e.g. a child makes a
drawing at school and wants to bring it home. That would include a way of
automatically share and move data between the school and the home. Content can also
be moved within the home, something produced in the kitchen is moved it into the
living room to view it. Video and audio streams can also be seen as electronic content

ACCORD, IST–2000–26364                                                    Deliverable D1.3

and thus are treated as moveable content, moving a TV show from TV in the living
room into the kitchen to start preparing dinner but still watch it.

4.3 Alerts
Alerts are event-based actions triggered by a certain factor, either time-based or
sensor-based. These could be used to activate reminders or set off alarms. Alerts are a
fundamental component to achieve more complex behaviours as these introduce a
time aspect as well as the possibility to combine input and generate new. An alert
does not necessarily have to pay the users attention but to do so both sounds based or
graphical displays can be used.

4.3.1 Time based
A time based alert is triggered when a certain time criteria is fulfilled. This criteria
may either be an isolated event or a repetitive, e.g. Monday 14th September or every
2nd hour. This is very easily implemented in software but not necessarily ease to
visualize and create an interface for. A classical example is the VCR where this is the
only thing the user have to give along with which channel to record, still these are so
poorly implemented that hardly no-one can use a VCR without reading the manual.

4.3.2 Sensor based
A sensor-based alert is triggered by a sensor with a value equal to, less than or greater
than some prerequisite for this alert, e.g. the distance between the child and the bottle
of bleach is less than 0.5 meters.

4.4 Displaying
Information from the system has to be displayed in a number of ways. All of the
scenarios involve some way of displaying information, but how to do this within the
home is not obvious. In some situations the use of ambient displays or background
sounds are more suitable but some information cannot be communicated through that
way. Another specific issue is how to best display information depending on the
capability of the device. One could imagine the combination of ambient and direct
All displays listed below can also be categorized as either being stationary or mobile
as well as being either personal or public.

4.4.1 Direct Graphical
These kinds of displays give information in a very direct way and have to be
explicitly read by a user, i.e. it gives information in a precise and direct way. They can
either be a more general-purpose device that can display wide range of information, or
a specific display always giving the same information. E.g. a display next to the door
displaying the number of minutes until the next bus leaves cf. a thermometer outside
the window to indicate the outside temperature. More examples of these displays are

ACCORD, IST–2000–26364                                                    Deliverable D1.3

Personal Digital Assistants (PDAs), displays embedded in furniture (e.g. kitchen table
as a display)
The patterns of home life identify the kitchen table as the activity centre in the
kitchen. This would indicate the kitchen table as a suitable place for a generic
graphical display. This could be realized either as an active surface, i.e. a display
embedded into the table or a projected one, i.e. projection on top of the table. These
have two fundamental differences. From the patterns we see that the kitchen table
tend to be used as a temporary storing facility, i.e. mail, newspapers etc. are placed on
the table until taken care of. The table is cleaned for other activities, such as having a
meal. An embedded display will be covered with these items, where a projection will
stay on top. A touch surface might also generate problems, as items will be placed on
top and generate input.
Another generic display of interest is a projector that can be directed onto different
surfaces to match the dynamics of family life. The use of such display have been
demonstrated by the Everywhere Display1 project at IBM and a commercial solution
is available from High End Systems and their Catalyst2 projector.
Paper based displays are also of interest, and one of the technologies being developed
by Acreo. They have functioning displays for predefined images, but by switching
between images simple animations can be reproduced. Pixel based paper displays are
still on the prototype level and are expected on the market in 3-5 years. Textual
A simple text display that can only be used to display text is a sub-class of Direct
Graphical displays. These can be used for several applications, e.g. a simple reminder
display next to the entrance of the house, or a remote PostIt-note in the kitchen. Small
and cheap LCD displays that can only display text can be used to realize these.

4.4.2 Ambient Graphical
An ambient display resides in the background and gives hints about activities, not
necessarily in a readable way. A string hanging next to the door to indicate the time
left until the next bus leaves. A lamp indicating through its colour each time a family
member receives a new e-mail. Graphics projectors that can be directed anywhere in
the home can also be used for these applications. The difficulty with these displays is
not the technical implementation but the actual design of the display and the media
used to convey the information.

4.4.4 Direct Sound
Sound is the medium that is less used in traditional desktop interfaces, mainly due to
the fact that the user constantly has his or her attention directed on the screen. In the
home where ones attention is diverted to other tasks and constant change among these
sound can be used to call someone’s attention. Sound can also be used to give the user


ACCORD, IST–2000–26364                                                   Deliverable D1.3

explicit information about something without requiring him or her to move her visual
Sound is also easily implemented in the house since most households have one or
several sound systems in use. These systems can easily be made to become a part of
the background environment and draw attention from the user only when needed.

4.4.5 Ambient Sound
An ambient sound can be used to give awareness about certain activities. Ambient
sounds are already used as an source of information in the home, cf. the father in the
kitchen making dinner while the children are playing upstairs, he can hear them
playing but not exactly what they are doing, but if something goes wrong they will
become quite or start yelling, and he becomes aware of the situation.

4.5 Controlling
There are a variety of devices and appliances that can be controlled within a house. To
mention some, the electronic lock on the entrance door, the heating in the different
rooms as well as lightning and air condition. In most houses today the current status
of these are not that they can directly be controlled, but standards, such as X10, are
being established. To change an existing house to be controllable through say X10 is
not any mayor re-work but it has not reached a break-through since the advantages is
not much higher than the functionality achieved today.

4.6 Applications
It is not only data and representations of physical artefacts that will exist within the
data space, but also applications. These applications may use data in the data space as
their input.
By combining several functions through a simple scripting language much more
complex behaviours could be achieved. Preferably this scripting language should have
some kind of graphical representation. It is intended that family members, via perhaps
a touch screen, to be able to program complex behaviour in the system directly and
not just via the tangible components around the house. This will require some
experience and understanding of consumer-friendly flow diagrams, such as dragging
icons representing heating, climate settings, etc into a calendar window.
The household economy is important and a budget application that keeps tracks of the
household economy will help people with the task.
Messaging applications, drawing tools, and the rest of the necessary services integrate
into the house system in the same way as physical components. They deliver and
receive output to and from the data space. How they are subsequently handled
depends on the scenario.

4.7 General issues
We have identified a number of important issues all crucial if the goal is to develop a
commercial product but not necessarily for prototype development. Security is of

ACCORD, IST–2000–26364                                                  Deliverable D1.3

great importance since people are very anxious about intrusiveness of the family life.
They have to stay in control of what leaves and enters the house. Interaction
paradigms have not yet been established for tangible and physical interfaces. With
this in respect one have to be careful when evaluating these applications. Robustness
becomes an important issue specifically for services that will run over very long time
without any direct user supervision or interaction.

4.7.1 Security
What information leaves the house and what can be extracted from it are the main
concerns for security breaches in a home system. Sensor data could be used to make
quite sophisticated analyses of family life and thus misused. But also within the
family there are no clear rules who will have access right to certain things. A teenager
will not accept that the parent will have access to private documents such as a diary.
The parents may not want their children to access the family budget but an economy
advisor at the bank will.
The borders of a family are not that easily defined and are dynamic, e.g. grandparents
will be included when regarding the family photo album. There is a constant shifting
need to interconnect different physical locales, the actual home, friends home, day-
care centre, school and work. A calendar or shopping list would probably be the
typical thing you would to access from a remote terminal outside the home.

4.7.2 Interaction
Previous work within the KidStory project has shown that it is not trivial at all to
design working tangible interfaces. Associating a physical artefact with some digital
content exists in several of the scenarios described and is quite simple to understand
as a concept but actually how to perform the task in an understandable way is not
easily designed. This has not to do with the actual technical implementation but how
the actual interface is created; the procedure to carry out the task. Once the
association is made the problem is to tell what is being associated with what.

4.7.3 Robustness
Robustness does not only mean few errors in the software, but hardware and precision
errors as well. For instance it is not trivial to accurately detect and track both all
inhabitants and all devices in a house. Systems may interfere with each other and the
environment is not controlled, thus hard to predict. Building robust and durable
hardware demands specific skills that none of the ACCORD partners do have. For a
demonstration scenario though it would be feasible to create a system that is robust
enough to be evaluated.

Finally, we present the two main scenarios.

ACCORD, IST–2000–26364                                                   Deliverable D1.3

5. Demonstration scenarios
Based on the previous chapters we outline two scenarios that will be the focusing
point for the next step of work within ACCORD. These are selected to both highlight
the use of the toolbox and reflect upon the patterns of home life. These scenarios will
give the project a foundation to work on when creating the different devices and
components of the tangible toolbox. The actual technical implementations will be
based on the solutions described in section four.
Through the ethnographic studies we have achieved descriptions of activities as well
as patterns of activities. The possible scenarios outlined in section three also describe
activities taking place within the home and the applications described are static. The
main feature of the tangible toolbox is how it supports the re-configuration of services
and how each family can take existing services and adapt to their specific situation.
Firstly the toolbox is not mainly a set of components to build these new applications
upon, rather a set of tools to reconfigure a set of components in an understandable
way. Though to realize this there is a need of an underlying infrastructure that let us
perform these operations. Based on this infrastructure we will compose a number of
components among those outlined in section four. These will be our building blocks
when constructing the tools within the tangible toolbox.
Focusing on a scenario will help us to fine-tune the infrastructure and to realize those
components we need. Though to focus the work on the main objective of the tangible
toolbox we need to address re-configuration and management. This will be achieved
by addressing two scenarios, the first emphasizing on the component and their use,
the second emphasizing on the change and management of the system.
As the patterns of home life were derived from activities within the kitchen the first
scenario is concentrated around this area. The kitchen table was identified as the
activity centre within the kitchen and thus this scenario takes place around it.
     The mother of the house has just arrived home from picking up two of
     the three children from kindergarten. She is about to start preparing
     dinner and therefore brings up a drawing application on the kitchen table
     to let the children play with and keep them occupied. She also drops off
     the mail she picked up at the front door, and by doing so brings up all e-
     mails received during the day. She looks it through while the kids start
     to play but since there were no interesting mails she starts to look in the
     cupboards to see what ingredients are available. Seeing that they have a
     lot of pasta at home and upon receiving a calendar notification of today
     being her mother’s birthday, she comes to think of a pasta dinner her
     mother used to prepare that she has not made for many years. She calls
     her mother to congratulate her and also to ask for the recipe, as she could
     not find it at home. Her mother is very happy to hear from her daughter
     and they talk for while. Before she hangs up she receives the recipe as
     well. For the sauce she uses the last tomatoes, but new ones will be
     delivered tomorrow.
     The younger daughter gets bored of drawing and brings her toy cars to
     play with on the table instead. A newspaper becomes the shopping mall
     and the sugar bowl is the house. She drives around on the tabletop

ACCORD, IST–2000–26364                                                    Deliverable D1.3

       buying food in the store and brings it home. The third child, who is
       attending school, comes home and needs to do his homework before
       dinner. He needs his mother’s help and therefore needs to sit by the
       kitchen table as well.
       The two younger children are shown into the living room where they
       continue to play. They bring what they have drawn on the kitchen table
       onto the floor in the living room as well as the toys they used.
       The father comes home about 30 minutes before the dinner is ready so
       he starts helping the son with the schoolwork to be ready by dinnertime.
       The son finishes just before dinner and sends it off to school to have it
       available by tomorrow morning. When contacting the school he gets a
       reminder to tell the parents about the meeting next week with the
       teacher. The father looks up in the family calendar and writes a note to
       the teacher suggesting two days. He also writes a note to remind the
       family about this event next week. As it is time for dinner he collects the
       girls in the living room and stores their drawings in their personal
       After the dinner it turns out that a bit too much food was prepared so the
       father decides he can take some of it tomorrow to work, puts it in the
       refrigerator, and sets an alarm to remind him tomorrow morning.
The scenario above includes several of the applications from section three and
components from section four. Even though this scenario’s main objective is to
identify a number of devices and components to realize, it also addresses the more
important features of the ACCORD toolbox: how the family configures and
administers these applications. From the above scenario the following items for
configuration can be identified:
       What application should be easily at hand at the kitchen table?
       How to associate physical objects to particular content, and create a context on
        which these objects communicate with each other. This in the case of
        transferring the drawing content in the kitchen to another medium in the living
       How to give the grandmother access to part of their data space, in this case the
       How is the shopping list to arrive at the right supermarket and for them to
        receive the right address to deliver to?
       How to access the school data space.
       How to let the school receive partial data from the calendar application.
       Personal storage within the data space.
       Personal identification.

The next scenario deals more with how to change the behaviour of the system.
Intelligent power management and burglar alarm systems are frequently used as the
selling arguments of today’s intelligent houses. We believe there is a need for the

ACCORD, IST–2000–26364                                                      Deliverable D1.3

habitants of a house to get an understanding of how their house alarm system work
and provide them with tools to change it to fit their needs. Most of the technology
already exists, and we just need to interconnect it and make it visible to the users.
       The family is about to go on summer vacation for four weeks. They have
       rented a house on the beach and are looking forward for their vacation.
       They want to set up their home system to divert all calls and electronic
       communication, except those that are work related, to the summerhouse.
       To prevent burglars from attempting to break in they want to configure
       the house to behave as if they still lived there, to turn on the lights in the
       evening and turn them off when they have usually go to bed, and to have
       the TV on when they usually watch TV.
By focusing on the movement of the household from the permanent location of the
house to the temporary summerhouse a number of re-configuration issues will be
highlighted. The following changes can be identified directly:
        Divert incoming telephone calls and other means of communication.
        Divert reminders.
        Turn off the automatic shopping system.
        Have personal data to follow into the new household.
        Change automatic systems to follow some form of scripts.
It also highlights visualization of the underlying infrastructure and the data within the
system. To be able to judge from a user perspective if the system will replicate the
inhabitant’s patterns in a believable way these must be visualized in an
understandable way. There will also be a need to visualize these extracted patterns,
extracted from user behaviours through application interaction and sensors. A visual
representation of the scripts has to be achieved as well.
The two scenarios address the different interests of the partners within the project.
Acreo are working with other projects on how to embed tagging and sensor
technology into grocery packages to both identify specific packages and the status of
their content. Nottingham provides the Equip platform that will be the underlying
infrastructure for shared data spaces. SICS is interested in achieving a better
understanding of interaction with tangible interface, and on different ways of
visualizing the underlying infrastructure and data space.


Shared By: