SENSECAM VISUAL DIARIES GENERATING MEMORIES FOR LIFE
Georgina Gaughan Hyowon Lee Cathal Gurrin Alan F. Smeaton Noel E. O’Connor Gareth J. Jones
SENSECAM: WHAT IS IT? AUTOMATIC STRUCTURING OF SENSECAM IMAGES PRESENTING SENSECAM IMAGES ?
SenseCam is a wearable digital Twenty events with highest Novelty Score are initially chosen for the interactive
browser. Each event bears:
camera you hang around your SenseCam Images of a day An “event” is a period of a day when something
neck, with various sensors: (about 3,000) specific happened, e.g. a meeting in an office, a • Landmark image
short chat with a colleague in the corridor, having • Novelty Score
• Light sensor lunch, drive a car, are all examples of an event. • Time and duration
• Passive infra-red sensor
A SenseCam: its fish-eye lens By comparing neighbouring Each landmark image is re-sized based on the Novelty Score, and displayed in
• Accelerometer (X-Y-Z axes) maximises the field-of-view Event Detection images (adjacent and n-ary temporal order, resulting in a visual summary of the most important events of
• Ambient thermometer distance) in terms MPEG-7 the day.
features (colour, texture, shape, etc.) and
It automatically takes images along with stroring data spatiograms, event boundaries are
from the above sensors as you go about your daily detected. The Interactive Browser is an automatically composed SenseCam browser,
business, passively capturing to chronicle your day into providing an efficient review of thousands of images from a given day. In
composing the browser, we use schemes for the following factors:
a visual archive of images and associated sensor data.
Comparison • Number of events to be presented
• Size of each photo (# different sizes)
To determine the importance (or uniqueness) of each of
the events of the day, we use past one week’s event • Layout (where each photo is to be placed)
WHAT DOES IT GENERATE? Event database containing
last 7 days’ Events database.
Top 19 important events have
been chosen from this day
Passive capture usually results in a large number of images. Timeline indicates when SenseCam
Mon By calculating the average feature vectors within each
SenseCam generates about 3,000 images on an average day event and comparing them, event-event similarity
was turned on, the period of each
event whose landmark image is
(640 x 480 resolution), although the exact number depends on Tue
among all events is established. An event that has presented below
what kind of activity the wearer did that day. Wed many highly similar events are routine events that
happen regularly throughout the week. Events that Landmark images are
Thr are not similar to any other events are the unique chronologically ordered (left to
right, top to bottom)
events that are deemed novel.
Chatting with a friend Walking on the corridor
Mouse-Over will start fast replay of
Similar events: John waiting for bus all images within that event (user
Sun Low Novelty has control over the pace of slide
Similar events: John at the office corridor Score
Similar events: John working at the desk
Working at the desk Walking on the At home Unique events ...
street High Novelty Score Meeting a friend in a hotel lobby
was the most novel event of the
Landmark Image day, thus the largest size
Selection The system adaptively re-ranks the
THE PROBLEM: ACCESS Novelty Score of each event within that day
as the day’s events come into the event
database using a 7-day window.
It’s good to have visual archive of a
day... BUT the large number of The events with smallest sizes
images means it’s difficult to access mean they appear frequently and
least important on the day for
them, for example: reviewing
• I want to quickly review what
0.1 0.7 0.1 0.1 0.3 0.4 0.8 0.1 0.9
happened yesterday... How can I flip The average feature vectors for each event calculated
through all 3,000 images without above are then compared to each of the images within the
event, and most similar photo is selected as a landmark
spending too much time?
• I want to find that particular person I
image, a image that visually represents the event.
met a few days ago... How can I find • We are working on the scaling issue by building up the collection
it from the archive of thousands of of images to test our techniques on months of images
images? • We are testing higher-level semantic features such as face
detection to filter out non-person events from the events with
CENTRE FOR DIGITAL VIDEO PROCESSING ADAPTIVE INFORMATION CLUSTER DUBLIN CITY UNIVERSITY IRELAND