poster - Centre for Digital Video Processing_ Dublin City University

Document Sample
poster - Centre  for Digital Video Processing_ Dublin City University Powered By Docstoc
					SENSECAM VISUAL DIARIES GENERATING MEMORIES FOR                                                                                                                                                                                                         LIFE
                                                                                                                                          Georgina Gaughan  Hyowon Lee  Cathal Gurrin  Alan F. Smeaton  Noel E. O’Connor  Gareth J. Jones

 SENSECAM: WHAT IS IT?                                                            AUTOMATIC STRUCTURING OF SENSECAM                                                                       IMAGES                            PRESENTING SENSECAM IMAGES ?
  SenseCam is a wearable digital                                                                                                                                                                                                 Twenty events with highest Novelty Score are initially chosen for the interactive
                                                                                                                                                                                                                                 browser. Each event bears:
  camera you hang around your                                                       SenseCam Images of a day                                     An “event” is a period of a day when something
  neck, with various sensors:                                                       (about 3,000)                                                specific happened, e.g. a meeting in an office, a                                   •   Landmark image
                                                                                                                                                  short chat with a colleague in the corridor, having                                •   Novelty Score
  • Light sensor                                                                                                                                   lunch, drive a car, are all examples of an event.                                 •   Time and duration
  • Passive infra-red sensor
                                            A SenseCam: its fish-eye lens                                                                         By comparing neighbouring                                                        Each landmark image is re-sized based on the Novelty Score, and displayed in
  • Accelerometer (X-Y-Z axes)              maximises the field-of-view                                   Event Detection                         images (adjacent and n-ary                                                       temporal order, resulting in a visual summary of the most important events of
  • Ambient thermometer                                                                                                                            distance) in terms MPEG-7                                                       the day.
                                                                                                                                                   features (colour, texture, shape, etc.) and
  It automatically takes images along with stroring data                                                                                           spatiograms, event boundaries are
  from the above sensors as you go about your daily                                                                                                 detected.                                                                     The Interactive Browser is an automatically composed SenseCam browser,
  business, passively capturing to chronicle your day into                                                                                                                                                                        providing an efficient review of thousands of images from a given day. In
                                                                                                                                                                                                                                  composing the browser, we use schemes for the following factors:
  a visual archive of images and associated sensor data.
                                                                                                             Comparison                                                                                                               • Number of events to be presented
                                                                                                                                                                                                                                      • Size of each photo (# different sizes)
                                                                                                                                                       To determine the importance (or uniqueness) of each of
                                                                                                                                                       the events of the day, we use past one week’s event                            • Layout (where each photo is to be placed)
  WHAT DOES IT GENERATE?                                                                       Event database containing
                                                                                               last 7 days’ Events                                      database.
                                                                                                                                                                                                                                                                                              Top 19 important events have
                                                                                                                                                                                                                                                                                              been chosen from this day

  Passive capture usually results in a large number of images.                                                                                                                                                                                                                                Timeline indicates when SenseCam
                                                                                         Mon                                                           By calculating the average feature vectors within each
  SenseCam generates about 3,000 images on an average day                                                                                              event and comparing them, event-event similarity
                                                                                                                                                                                                                                                                                              was turned on, the period of each
                                                                                                                                                                                                                                                                                              event whose landmark image is
  (640 x 480 resolution), although the exact number depends on                           Tue
                                                                                                                                                        among all events is established. An event that has                                                                                    presented below

  what kind of activity the wearer did that day.                                         Wed                                                             many highly similar events are routine events that
                                                                                                                                                         happen regularly throughout the week. Events that                                                                                    Landmark images are
                                                                                         Thr                                                              are not similar to any other events are the unique                                                                                  chronologically ordered (left to
                                                                                                                                                                                                                                                                                              right, top to bottom)
                                                                                                                                                          events that are deemed novel.
       Chatting with a friend      Walking on the corridor
                                                                                                                                                                                                                                                                                              Mouse-Over will start fast replay of
                                                                                                                                                                 Similar events: John waiting for bus                                                                                         all images within that event (user
                                                                                         Sun                                                                                                                       Low Novelty                                                                has control over the pace of slide
                                                                                                                                                                 Similar events: John at the office corridor       Score
                                                                                                                                                                 Similar events: John working at the desk
             Working at the desk            Walking on the            At home                                                                                                          Unique events ...
                                               street                                                                                                                                  High Novelty Score                                                                                     Meeting a friend in a hotel lobby
                                                                                                                                                                                                                                                                                              was the most novel event of the
                                                                                                                             Landmark Image                                                                                                                                                   day, thus the largest size
                                                                                                                             Selection                      The system adaptively re-ranks the
 THE PROBLEM: ACCESS                                                                                                                                        Novelty Score of each event within that day
                                                                                                                                                             as the day’s events come into the event
                                                                                                                                                             database using a 7-day window.
 It’s good to have visual archive of a
 day... BUT the large number of                                                                                                                                                                                                                                                               The events with smallest sizes

 images means it’s difficult to access                                                                                                                                                                                                                                                        mean they appear frequently and
                                                                                                                                                                                                                                                                                              least important on the day for
 them, for example:                                                                                                                                                                                                                                                                           reviewing

 • I want to quickly review what
                                                                                                 0.1   0.7 0.1   0.1   0.3    0.4   0.8    0.1   0.9
   happened yesterday... How can I flip                                                                                                                         The average feature vectors for each event calculated
   through all 3,000 images without                                                                                                                             above are then compared to each of the images within the
                                                                                                                                                                 event, and most similar photo is selected as a landmark
   spending too much time?
 • I want to find that particular person I
                                                                                                                                                                 image, a image that visually represents the event.
                                                                                                                                                                                                                                         FUTURE WORK
   met a few days ago... How can I find                                                                                                                                                                                                  • We are working on the scaling issue by building up the collection
   it from the archive of thousands of                                                                                                                                                                                                     of images to test our techniques on months of images
   images?                                                                                                                                                                                                                               • We are testing higher-level semantic features such as face
                                                                                                                                                                                                                                           detection to filter out non-person events from the events with
                                                                                                                                                                                                                                           people involved.

 CENTRE FOR DIGITAL VIDEO PROCESSING                                           ADAPTIVE INFORMATION CLUSTER                                                DUBLIN CITY UNIVERSITY                                  IRELAND

Shared By: