Learning Center
Plans & pricing Sign in
Sign Out



									Your Life as a Movie

        Sailesh Chutani
 Director, Worldwide University
      Microsoft Research
             What if..
Everything you ever say, hear, observe,
do, and sense is recorded
Those recordings are stored permanently
and can be accessed easily
You could analyze or modify copies of
these recordings and share them
         Some Questions
Is this scenario technically feasible?
What would such a future look like?
What new opportunities and risks will we
be confronted with?

Elements of such a future are already here
    but just not uniformly distributed (to
        paraphrase William Gibson)
       Microsoft Research
Founded in 1991
Staff of over 700 in over 55 areas
Internationally recognized research teams
200+ WW university relationships
Research lab locations :
– Redmond, Washington
– San Francisco, California
– Cambridge, United Kingdom
– Beijing, People’s Republic of China
– Mountain View, California
 A University CS Department
       inside Microsoft
University organizational model
– Flat structure, critical mass groups
Open research environment
– Aggressive publication of results in peer-
  reviewed literature
– Frequent visitors, daily seminars
Strong ties to University Research
– Nearly 15% of basic research budget directly
  invested in Universities
    Lab grants, research grants, fellowships, etc.
– Hundreds of interns and visitors
            MSR Mission
Expand the state of the art in each of the
areas in which we do research
Rapidly transfer innovative technologies
into Microsoft products
Ensure that Microsoft products have a
 Expanding the State of the Art
Thousands of peer-reviewed publications
    13% of papers at 2001 CHI (UI)
    20% of papers at 1996 SIGGRAPH (graphics)
    25% of papers at 1999 PLDI (prog languages)
    30% of papers at 2001 PLDI
    More papers at SIGGRAPH 2003, SIGMOD 2004 , SIGIR
    2004 and other key conferences than any other organization
Community leadership
    Professional societies
   Segmenting the Problem
Information Capture
Query and Retrieval
Analysis and Processing
          A Digital Memory for Everyone

Lyndsay Williams & Ken Wood @ MSRC and the MyLifeBits team @ BARC
Essentially a “Black Box” data and image recorder
for the human body
Current wearable prototype can be worn for a day
and captures up to 2000 images in that time

Image capture triggered by sensors, e.g. motion,
light, temperature, people in field of view, ...
Sensor data is also recorded for later presentation,
analysis, and correlation
    Ultra Wide-Angle Lens
Captures everything regardless of exact camera orientation
         Image Stabilisation
Accelerometer measures motion of the device
When any sensor indicates image capture, we wait a few
tens of milliseconds to capture image when movement is
less than 10 degrees/second if possible
Greatly reduces the number of blurred images

        Before                       After
A Short Walk Through Cambridge
   How does SenseCam work?
Accelerometers for motion detection
Passive Infrared (PIR) sensors
triggered by people or other living
beings in front of camera
Digital sensors for light level and
Images captured automatically on
sensor events caused by, e.g.
• turning around, sitting down,
  standing up, running, falling over, ...
• moving between locations (rooms,
  car, bus, indoor/outdoor, etc.)
• meeting a friend in the street
Simple hand gesture allows
intentional image capture
Accelerometers also allow image
    SenseCam Applications
Personal memory enhancement
– Where did I leave my umbrella?
– What was the name of that wine we had last week?
– What was written on the whiteboard?
– Automatic photo journals or blogs of specific trips,
  e.g. to the Edinburgh Festival
Clinical support for the memory-impaired
– We are working with a Cambridge neuropsychologist
  on a trial for Alzheimer’s patients
Now building next-gen SenseCams
– smaller, better power management, audio
  trigger, wireless comms
           Storage Needs
Number of seconds in 80 years
Data recording rate for MPEG-4 is 4Mb/s
Storage required for full motion video of 80
years of someone’s life is 10k tera bits or 10
peta bits (10^15)
Cost of such a storage in today’s terms is
approximately $10m
Storage capacity is doubling every year, in 10
years 10 peta bits may cost <$10k
Storage Trends

           Source: Ed Grochowski, IBM Research Almaden
                               Moore’s Law
                 1998          2001          2004         2007          2010

“A threshold event will take place early in the 21st century: the emergence of machines
more intelligent than their creators. By 2019, a $1,000 computer will match the processing
power of the human brain - about 20 million billion calculations per second. Organizing
these resources - the “software” of intelligence - will take us to 2029, by which time your
average personal computer will be equivalent to 1,000 human brains.” -- Ray Kurzweil
     Human Scale Storage
Terabyte ==
– $1600 today
– $400 in 2-3 years
Terabyte can hold
– A lifetime of all your conversations
– A year of what you can see
When did I meet the Dalai Lama?
How long did I live in Lausanne?
How many people did I encounter in my
life with whom I spend more than 5
minutes in conversation?
Where was I on my wife’s birthday during
the last 10 years?
What were the scariest moments in my
mountaineering trips?
       Stuff I’ve Seen (SIS)
Support information re-use
– Many locations
– Many interfaces

With SIS:
– Unified index
– Fast, flexible
– Immediate update
– Rich UI
  possibilities for
  your content
     SIS Usage Observations
Very short queries (1.6
words), quick filtering/sorting                                                               August, 1986!
Few searches for “best
matching” item                                      3.5

                                                                                  All Items
– Know and use other attributes                      3

Age of items opened                                 2.5

– 50% more than one month old,                       2

  some up to 8 years                                1.5

Importance of people                                 1

– 25% of queries involve people                     0.5
                                                              f(x) = -1.111862*x + 3.809612
                                                              R^2 = 8.017632E-1

Importance of time
                                                          0       0.5       1        1.5      2       2.5     3   3.5   4
                                                                                log(Days Since Item Creation)

– Date by far the most common
  sort attribute
– “Useful” date varies by source
     Contextualizing Search
Search is not the end goal …
Goal is information management in the context
of ongoing work activities
– Search always available

– Search from within apps

– Show results in context of apps
Harness models of human memory
  to assist with navigating large
    stores of personal content
 Search UI

• Calendar events
• Images from store
• Key news stories
• Holidays

      Hits over time
Modeling the interface after the
   way people think & feel
– Memory
    associations vs. facts/details
– Deep History
    objects recede vs. disappear
– Dynamic organization
    multiple simultaneous categories vs. fixed
Personality & Emotion
– Know what people care about and use to help organize
– Watch users habits & behavior to customize
– Many people, places and events

Adjustable threshold
  for memorability
Re-live the past, explore the events as
they happened at your own pace
Analogous to “action replay” but at a large
 WWMX ( Geography
                for Photography

The World-Wide Media eXchange is

 a centralized index of
 the world’s image media
 tagged by location (and other standard metadata).
 Why Location for Photos?
Provides immediate context, hints at semantics
– E.g., photos taken at Disneyland, video shot in Hawaii…

Repeatedly appears in user studies regarding
– Family trips strongly associated with location
– Event = time + location
– Subject as person doing something somewhere

Universal index (like date/time)
– Independent of ontology/taxonomy, language, culture, etc.
– Familiar visual browsing UI via maps
– Scales well

Technology to acquire location becoming common
– GPS, e911 services, radio-tower-based, 802.11-based…
Real estate tours
Travel and tourism
Shared travelogues
Driving directions with photos of
Centralized media depository for
community events
           Creating Art
Recordings can serve as the raw material
for creating art
         Meta Questions
Who would regulate the recording of this
Who will own this information?
Who would be able to access it?
Should copies and modifications be
What would be the legitimate uses of this
             Meta Questions
What would it mean for us as human beings?
– Will human behavior change in light of the “permanent record”?
– How will we substitute for “forgetting” if memory becomes
– Would we become more empathetic if we can literally see other
  points of view?
– Would resolving conflicts be just a matter of comparing
– Would we create a class of people who live vicariously?
– Would life be safer?
– Is video/experience editing the next big growth industry?
        Clues for Answers
Current usage in forms of
– video surveillance,
– credit histories, public records
– policy on picture phones,
– popularity of reality TV
– use of web search to get background
Exploration in Art, movies that have
touched the themes
– Being John Malkovich, After Life, Slacker
Scientists and engineers create the
intellectual and physical building blocks;
Artists imagine the possibilities, but
It is the interplay of Society and Culture
that shape the actual usage and norms

To top