The Vision and Reality of Ubiquitous Computing by jianghongl


									The Vision and Reality of Ubiquitous

                    Prof. Henning Schulzrinne
                    Dept. of Computer Science
                       Columbia University
          (with Arezu Moghadam, Ron Shacham, Suman Srinivasan,
           Xiaotao Wu and other IRT members as well as the IETF
                        GEOPRIV and ECRIT WGs)

January 2008                      UCSB                            1

•   The original vision of ubiquitous computing
•   User challenges
•   Beyond terminal mobility
•   Location as new core service
•   Beyond connectivity: 7DS
•   CS technology into reality

January 2008           UCSB                   2
Ubiquitous computing  ubiquitous

• “It is invisible, everywhere computing that does not live on a
  personal device of any sort, but is in the woodwork everywhere.”
• Weiser’s original vision (“Nomadic Issues in Ubiquitous Computing”,
     –   “one person, many computers”
     –   many computers embedded in environment
     –   dynamic ownership
     –   PC phonebooth
     –   “IR use will grow rapidly”
• Updated version, 2007
     –   not physically invisible, but transparent
     –   emphasis on communications, not computing
     –   most devices are mobile (or nomadic)
     –   cheap electronics personal devices
     –   radio (channelized and UWB)
January 2008                        UCSB                            3

• The original vision of ubiquitous
• User challenges
• Beyond terminal mobility
• Location as new core service
• Universality: 7DS
• CS technology into reality
January 2008       UCSB               4
User challenges vs. research challenges

• Are we addressing real user needs?
     – Engineering vs. sports scoring
• My guesses
                                ease of use             no manual
        reliability                          no re-entry
                                            no duplication

                                                             data loss
                                             limited risk

January 2008                     UCSB                                    5
Example: Email configuration

• Application configuration
  for (mobile) devices painful
• SMTP port 25 vs. 587
• IMAP vs. POP
• TLS vs. SSL vs. “secure
• Worse for SIP...

January 2008                UCSB   6
Example: SIP configuration
                                                        partially explains
•   highly technical parameters, with differing names
•   inconsistent conventions for user and realm
•   made worse by limited end systems (configure by multi-tap)
•   usually fails with some cryptic error message and no
    indication which parameter
•   out-of-box experience not good

January 2008                         UCSB                                    7
Mobile why’s

•   Not research, but examples of real annoyances
•   Why does each mobile device need its own power supply?
•   Why do I have to adjust the clock on my camera each time I travel?
•   Why do I have to know what my IMAP server is and whether it uses
    TLS or SSL?
•   Why do I have to type in my address book?
•   Why do I have to “synchronize” my PDA?
•   Why do I have to manually update software?
•   Why is connecting a laptop to a projector a gamble?
•   Why do we use USB memory sticks when all laptops have 802.11b?

January 2008                     UCSB                                8
Consumer wireless & mobile devices

     Prius key           Garage door opener          TAN display

                                                                   Water leak alarm

SPOT watch

                                                                    wireless door bell

    MSN Direct weather

January 2008                                  UCSB                                9
Mobile systems - reality
•   idea: special purpose (phone) --> universal
    communicator                                                                                          QuickTime™ an d a
                                                                                                 TIFF (Uncompressed) decompressor
                                                                                                    are need ed to see this p icture.

     –   idea is easy...                                                                                                                                       QuickTime™ an d a
                                                                                                                                                     TIFF (Uncompressed) decompressor
                                                                                                                                                        are need ed to see this p icture .

•   mobile equipment: laptop + phone
     –   sufficiently different UI and capabilities                                     location data
•   we all know the ideal (converged) cell phone
•   difficulty is not technology, but integration
    and programmability
     –   (almost) each phone has a different flavor of
                                                                          QuickTime™ and a

         OS                                                     TIFF (Un compressed) decompressor
                                                                   are neede d to see this picture.                                               QuickTime™ and a
                                                                                                                                        TIFF (Uncompressed) decompre ssor
                                                                                                                                           are neede d to see this picture.

     –   doesn’t implement all functionality in Java
     –   no dominant vendor (see UNIX/Linux vs.
     –   external interfaces crippled or unavailable
          •    e.g., phone book access                                                                                                                                       QuickTime™ and a
                                                                                                                                                                   TIFF (Un compressed) decompressor
                                                                                                                                                                      are neede d to se e this picture.

                                                                                                                       Quic kTime™ and a
                                                                                                             TIFF (Uncompres sed) dec ompress or
                                                                                                                are needed to s ee this pic ture.

January 2008                                             UCSB                                                                                                                      10
Example: displays and speakers

January 2008     UCSB            11
The mobile ubiquitous challenge

                   Mobile phone

               Mobile Internet access

               Interconnected devices
                                        of things”

January 2008             UCSB                        13
What do we need?

• Standards, not new technology
• Radio connectivity
     – 802.11a/b/g/n, 802.15.4 
     – better discovery of networks
• Location information everywhere
• Discovery: devices & services
     – network-local discovery via Bonjour (mDNS) 
     – missing: location-based discovery
• Advanced mobility: session, personal, service
• Event notification
• Data formats
     – location, sensor events, ...

January 2008                          UCSB            14
Examples of “invisible” behavior

• MP3 player in car automatically picks up new files in home server
• A new email with vcard attachment automatically updates my cell
  phone address book
• The display of my laptop appears on the local projector
     – without cable or configuration
• I can call people I just met at UCSB
     – without exchanging business cards
•   My car key opens my front door
•   My cell phone serves as a TAN (one-time password) generator
•   My cell phone automatically turns itself off during a lecture
•   My camera knows where the picture was taken

January 2008                            UCSB                          15
 An interconnected system
    opens doors

                           generates TAN

incoming call                                      updates location

                                  time, location

                                           address book

                                                                      alert, events

     any weather service     acoustic alerts
     school closings

 January 2008                              UCSB                                16
Thinking beyond 802.11 and UMTS

• Many interesting networks beyond those covered in conferences
     – ease of access by researchers vs. importance
     – 90% of papers on 802.11b and maybe GPRS, BlueTooth
• New wireless networks
     –   broadcast instead of unicast -- useful for many ubiquitous applications
     –   S5 for low-rate sensors (city scale)
     –   Zigbee (802.15.4) for local sensors (20 - 250 kb/s)                   QuickTime™ and a
                                                                     TIFF (Uncompressed) decompresso
     –   FM subcarrier (not really new) -- MSN Direct                   are needed to see this picture.

     –   FM Radio Data System -- TMC
     –   Sirius / XM
     –   HD radio
     –   paging

January 2008                                 UCSB                                             17

•   The original vision of ubiquitous computing
•   User challenges
•   Beyond terminal mobility
•   Location as new core service
•   Universality: 7DS
•   CS technology into reality

January 2008           UCSB                   18
Application-layer mobility

• terminal mobility
     – one terminal, multiple network addresses
• Personal mobility
     – one person, multiple terminals
     – e.g., Grandcentral
• session mobility
     – one user, multiple terminals in sequence or in parallel
• service mobility
     – services move with user

January 2008                       UCSB                          19
Session mobility

•   Walk into office, switch from cell
    phone to desk phone
     – call transfer problem  SIP
•   related problem: split session
    across end devices
     – e.g., wall display + desk phone +
       PC for collaborative application
     – assume devices (or stand-ins) are
     – third-party call control

January 2008                             UCSB   20
How to find services?

               •   Two complementary developments:
                    –   smaller devices carried on user instead of stationary
                    –   devices that can be time-shared
                          •   large plasma displays
                          •   projector
                          •   hi-res cameras
                          •   echo-canceling speaker systems
                          •   wide-area network access
               •   Need to discover services in local environment
                    –   SLP (Service Location Protocol) allows querying for
                          •   “find all color displays with at least XGA resolution”
                          •   slp://
                    –   SLP in multicast mode
                    –   SLP in DA mode
                    –   Apple Bonjour
               •   Need to discover services before getting to
                    –   “is there a camera in the meeting room?”
                    –   SLP extension: find remote DA via DNS SRV
                    –   LoST to find services by geographic location

January 2008                                            UCSB                           21
   Session mobility
                                                                    Local Devices


                                                SLP DA

                                                          SLP SA   SLP UA

                                                          SIP SM   SIP UA

          SIP UA

Correspondent                                                          SLP
                              SIP SM            SIP UA                 SIP
  Node (CN)
                                       SLP UA
   January 2008                                    UCSB                             22

                              Mobile Node (MN)

• The original vision of ubiquitous
• User challenges
• Beyond terminal mobility
• Location as new core service
• Universality: 7DS
• CS technology into reality
January 2008       UCSB               23
Context-aware communication
               •   context = “the interrelated conditions in which something
                   exists or occurs”
               •   anything known about the participants in the (potential)
                   communication relationship
               •   both at caller and callee
     time                              chronology, uniqueness
     capabilities                      caller preferences

     location                          location-based call routing
                                       location events (emergency alerts)
                                       security (access control)
                                       service discovery
     activity/availability             presence
     sensor data (mood, bio)           privacy issues similar to location data

January 2008                                UCSB                                 24
GEOPRIV and SIMPLE architectures


               publication   location       notification    location
  target       interface                    interface                   GEOPRIV
                              server                        recipient

                             presence                                   SIP
presentity                                                  watcher
               PUBLISH        agent                                     presence

  caller                                                     callee     SIP
                INVITE                      INVITE                      call
January 2008                            UCSB                                   25
Presence data architecture

           presence sources


                                              presence                                privacy
                    (compose)                 document                                filtering

                            XCAP                                 depends on watcher        XCAP
                                       select best source
                                       resolve contradictions

                    composition                                                       privacy
                      policy                                                           policy

                   (not defined yet)
January 2008                             UCSB                                                           26
Presence data architecture

               candidate                                raw
               presence                 watcher      presence              post-processing
               document                  filter      document                (merging)

                   remove data not of
                   interest                         difference
                                                    to previous notification

                        watcher                                            document

January 2008                                 UCSB                                            27
    Presence data model

 person         “calendar”              “cell”             “manual”


services               audio, video, text               video


 January 2008                               UCSB                      28
RPID = rich presence

•   Provide watchers with better information about the what, where, how of
•   facilitate appropriate communications:
    –   “wait until end of meeting”
    –   “use text messaging instead of phone call”
    –   “make quick call before flight takes off”
•   designed to be derivable from calendar information
    –   or provided by sensors in the environment
•   allow filtering by “sphere” – the parts of our life
    –   don’t show recreation details to colleagues

January 2008                              UCSB                               29
Rich presence

• More information: for (authorized) people and applications
• automatically derived from
    – sensors: physical presence, movement
    – electronic activity: calendars
• Rich information:
    –   multiple contacts per presentity
        •      device (cell, PDA, phone, …)
        •      service (“audio”)
    –   activities, current and planned
    –   sphere (home vs. work)
    –   current user mood
    –   surroundings (noise, privacy, vehicle, …)
    –   contact information
    –   composing (typing, recording audio/video IM, …)

January 2008                                  UCSB             30
Presence and privacy

•   All presence data, particularly
    location, is highly sensitive       <tuple id="sg89ae">
•   Basic location object (PIDF-            <gp:geopriv>
    LO) describes                              <gp:location-info>
     – distribution (binary)                      <gml:location>
                                                    <gml:Point gml:id="point1“
     – retention duration                                         srsName="epsg:4326">
•   Policy rules for more detailed                     <gml:coordinates>37:46:30N 122:25:10W
    access control                                       </gml:coordinates>
     – who can subscribe to my                    </gml:location>
       presence                                </gp:location-info>
     – who can see what when                  <gp:usage-rules>

January 2008                          UCSB                                                31
Events as missing Internet capability

• aka PUB/SUB
• Used across applications, e.g.,
     –   email and voicemail notification
     –   presence
     –   replace RSS (= poll!)
     –   web service completion
     –   emergency alerts (“reverse 9-1-1”)
     –   network management
     –   home control
     –   data synchronization
• Rich research history
     – but too complex, optimize the wrong thing
• XMPP and SIP as likely transport candidates
January 2008                          UCSB         32
                                    Local Switch


                                                      Collaboration between
                                                    local phone providers and
                                                   local public safety agencies

    January 2008                    UCSB                                 33
911 technology failures

•   NY Times (“An S O S for 911 Systems in Age of High-Tech”), 4/6/07:
     – “40% of … counties, most of them rural or small-town …, cannot yet pinpoint the
       location of the cellphone callers, though the technology to do so has been
       available for at least five years.”
       “In … Okmulgee, Okla., last November, 4-year-old Graciella Mathews-Tiger died
       in a house fire after a 911 operator who lacked the technology to pinpoint the call
       misheard the address.”
           • Phase II wireless; billions of dollars spent
           • In Mississippi, only 1 of out 5 counties
     – “As it ages, it is cracking, with problems like system overload, understaffing,
       misrouted calls and bug-ridden databases leading to unanswered calls and
       dangerous errors.”
           • operator (CAMA) trunks, with 8-digit number delivery
           • MSAG and ALI databases

January 2008                                     UCSB                                    34
Location delivery



January 2008                UCSB                     35
Location, location, location, ...

                 Voice Service Provider (VSP)
                     sees emergency call
               but does not know caller location

                 ISP/IAP knows user location
                   but does not handle call

January 2008                 UCSB                  36
Options for location delivery

•   Wireless
     –   GPS
     –   S5 wireless (active sensors + triangulation)
     –   Skyhook (802.11 in urban areas)
     –   HDTV
•   L2: LLDP-MED (standardized version of CDP + location data)
     – periodic per-port broadcast of configuration information
•   L3: DHCP for
     – geospatial (RFC 3825)
     – civic (RFC 4676)
•   L7: proposals for retrievals: HELD, SIP, …
     –   for own IP address or by third party (e.g., ISP to infrastructure provider)
     –   by IP address
     –   by MAC address
     –   by identifier (conveyed by DHCP or PPP)
     –   HTTP-based

January 2008                                UCSB                                       37
Locating Caller using LLDP-MED
         LLDP-MED stands for: *                                 * From Wikipedia
         Link Layer Discovery Protocol
           “a vendor-neutral Layer 2 protocol that allows a network device
         to advertise its identity and capabilities on the local network.”
         Media Endpoint Discovery
          “an enhancement to the LLDP that allows discovery of other
         things including location “
     “I am LLDP-MED Capable.
I can process location information.”

                CALLER                                           LLDP-MED
               EQUIPMENT                                          SWITCH
January 2008                            UCSB           “Your location is:     38
                                               500 W 120TH st. New York NY 10027”
Location determination options
Method          CDP or LLDP-          DHCP             HELD            GPS              manual entry
Layer           L2                    L3               L7 (HTTP)       -                user
advantages      • simple to           • simple to      • traverses     • accurate       • no
                  implement             implement        NATs          • mobile           infrastructure
                • built into switch   • network        • can be          devices          changes
                • direct port/room      locality         operated by   • no carrier     • no carrier
                  mapping                                L2 provider     cooperation      cooperation
problems        may be hard to        mapping MAC      mapping IP      • indoor         • fails for
                automate for large    address to       address to        coverage         mobile
                enterprises           location?        switch port?    • acquisition      devices
                                                                         time           • unreliable for
Use             Ethernet LANs         Enterprise       DSL, cable      mobile devices   fall back
                                      Some ISPs

 January 2008                                       UCSB                                            39
SIP message for Location Info.
  INVITE urn:service:sos SIP/2.0
                                              request line           ------- =_ZGY1NTFlZDJkMDkxY2FkMTIxMWI2MzIzNjE1M2U0OTY=
                                                                     MIME-Version: 1.0
                                                                     Content-Type: application/pidf+xml
  To: urn:service:sos                                                Content-Transfer-Encoding: 8bit
  Call-ID: 763782461@
  Via: SIP/2.0/TCP;rport                          <?xml version="1.0" encoding="ISO-8859-1"?>
  Content-Type: multipart/mixed; boundary                            <presence xmlns="urn:ietf:params:xml:ns:pidf"
  From:                                  xmlns:gp="urn:ietf:params:xml:ns:pidf:geopriv10"
  Contact: <sip:eddie@>                                xmlns:cl=" urn:ietf:params:xml:ns:pidf:geopriv10:civilLoc"
  CSeq: 1 INVITE                                                        xmlns:gml="urn:opengis:specification:gml:schema-xsd:feature:v3.0"
  Content-Length: 1379                        header fields             entity="">
                                                                       <tuple id="28185">
   ------ =_ZGY1NTFlZDJkMDkxY2FkMTIxMWI2MzIzNjE1M2U0OTY=
   MIME-Version: 1.0
   content-Type: application/sdp
                                                                            <cl:A2>new york</cl:A2>
   Content-Transfer-Encoding: 8bit
                                                                            <cl:A3>new york</cl:A3>
   o=eddie 1127764654 1127764654 IN IP4
   s=SIPC Call
   c=IN IP4
   t=0 0
   m=audio 10000 RTP/AVP 0 3
   m=video 20000 RTP 31
                                                                       <contact priority="0.8">sip:eddie@</contact>
                                                    SDP                </tuple>
                                                                     ------- =_ZGY1NTFlZDJkMDkxY2FkMTIxMWI2MzIzNjE1M2U0OTY=--

January 2008                                                  UCSB                                                      PIDF-LO
January 2008   UCSB   41

January 2008   UCSB   42
Data formats: location

• Civic (street)
     – jurisdictional & postal
• Geo (longitude & latitude)
     – point, polygon, circle, …
• see GeoRSS for simple example

January 2008                       UCSB   43
ECRIT: LoST Functionality

•   Civic as well as geospatial queries            •   Indicates errors in civic location data 
     –   civic address validation                      debugging
                                                        –   but provides best-effort resolution
•   Recursive and iterative resolution             •   Can be used for non-emergency services:
•   Fully distributed and hierarchical                  –   directory and information services
    deployment                                          –   pizza delivery services, towing companies, …
     –   can be split by any geographic or civic
     –   same civic region can span multiple
         LoST servers

                     <findService xmlns="urn:…:lost1">
                     <location profile="basic-civic">
                           <A6>Neu Perlach</A6>
January 2008                             UCSB                                                         44
   LoST: Location-to-URL Mapping
            VSP1       cluster serving VSP1
                                                                              root information
123 Broad Ave
Leonia                                                     serves VSP2
Bergen County

                                                NJ                       NY
                                                US                       US            nodes
                                                     Bergen County
                                                         NJ US
                                                                              NJ US

   January 2008                               UCSB                                               45
    LoST Architecture
     tree guide                       G
                               G                       T1: .us                 broadcast (gossip)
                                                       T2: .de

313 Westview
Leonia, NJ US

                                                      (.de)                         T3
                              (.us)   Leonia, NJ 

    January 2008                              UCSB                                                  46
Left to do: event notification

•   notify (small) group of users when something of interest happens
     –   presence = change of communications state
     –   email, voicemail alerts
     –   environmental conditions
     –   vehicle status
     –   emergency alerts
•   kludges
     – HTTP with pending response
     – inverse HTTP --> doesn’t work with NATs
•   Lots of research (e.g., SIENA)
•   IETF efforts starting
     – SIP-based
     – XMPP

January 2008                            UCSB                           47

• The original vision of ubiquitous
• User challenges
• Beyond terminal mobility
• Location as new core service
• Universality: 7DS
• CS technology into reality
January 2008       UCSB               48
Problems with Wide Area Wireless

          • 802.11 currently hard to deploy across city or large area
          • Problem: How can mobile devices / gadgets get information
            while on the move?
          • Use local peer-to-peer wireless networks to exchange
               – Peers can get information they do not have from another peer

                                Solution: 7DS!
January 2008                          UCSB                                      49
How 7DS Works

           1. When devices are in the same BSS (Basic service set) of
              802.11 ad-hoc network, they discover each other using
              service discovery of Zeroconf


January 2008                        UCSB                                50
How 7DS Works

          2. If there is no Internet connection, the devices can communicate
             with each other to exchange information

January 2008                        UCSB                                  51
Web Delivery Model

• 7DS core functionality: Emulation of web content access
  and e-mail delivery

January 2008               UCSB                         52

           • Peer-to-peer network set up using zeroconf
                – Protocol enables devices to communicate with each other
                  without a DHCP server, a DNS server and a Directory server
           •   Proxy server serves content
           •   Search engine searches for local data
           •   MTA store and forward
           •   In progress: File synchronization, BBS

January 2008                          UCSB                                     53
Store and Forward

• Forwarding e-mail in the ad-hoc network
• Acts as an MTA

January 2008              UCSB              54
Search Engine

• Provides ability to query self
  for results
• Searches the cache index
  using Swish-e library
• Presents results in any of three
  formats: HTML, XML and plain
• Similar in concept to Google

January 2008                     UCSB   55
Query Multicast Engine

•   Used to actually exchange
    information among peers
•   Requesting peer broadcasts a
    query to the network
•   Responding peers reply if they
    have information
      – Send encoded string with list
        of matching items
•   Requesting peer retrieves suitable

January 2008                         UCSB   56
File Synchronization
                                      SRV : 7ds-fs1.filesync._7ds._udp.local.
                             “I want
                             Word.doc and presentation.ppt”
                                      TXT : file1.xml
                                      TXT : file2.xml

                                 “I want: 7ds-fs2.filesync._7ds._udp.local.
                                 File1.xml and file2.xml”
                                     TXT : word.doc                             File1.xml
          File1.xml                                                             File2.xml
                                     TXT : presentation.ppt


January 2008                                     UCSB                                       57

• The original vision of ubiquitous
• User challenges
• Beyond terminal mobility
• Location as new core service
• Universality: 7DS
• CS technology into reality
January 2008       UCSB               58
CS research to reality

               CS as science
               CS as engineering

January 2008                       UCSB   59
CS tech transfer, mode 1

January 2008      UCSB     60
CS tech transfer, mode 2

January 2008      UCSB     61
The standards route

January 2008     UCSB   62
Role of standards

•   Widespread use of technology requires standards
     – railroad gauges, nuts & bolts, Morse code, phone system
           • industrial age = interchangeable parts
     –   CD ROM, DVD, HD-DVD/BlueRay
     –   JPEG, MPEG, ...
     –   SQL
     –   C, Java
•   Particularly for network technology
     – many mid-size actors
     – “network effect” = utility increases with number of users
           • e.g., cars have inverse network effects
           • chess game vs. email and word processor
     – DECnet, SNA, ARCnet  Ethernet, IP
     – “hypermedia”  HTML, HTTP

January 2008                                  UCSB                 63

• Tanenbaum: “The nice thing about standards is that you
  have so many to choose from.”

                           VHS + Beta
  component technologies
                           Ethernet + Tokenring
                           ATM + IP


January 2008                   UCSB                    64
Standards: technology translator

• Similar in some ways to text books
• “accepted technology”
     – lower/known risks (“vetted”)
     – infrastructure (“eco system”)
• requires expertise and broader training
     – many CS standards don’t have either
           • example: HTTP/1.0, HTML 1.0
• thus, way to provide technology leadership

January 2008                       UCSB        65
Examples of standards needed

• each with multi-B$ impact:
     – medical records
     – energy control
     – transportation
           • inter-vehicle safety systems, traffic alerts, ...
     – academic records
     – business transactions
           • e.g., your travel expense report
• requires both CS and domain expertise

January 2008                              UCSB                   66

• Motivate mobile ubiquitous research by user problems
• From stovepipe mobile & wireless systems to personal
  and shareable wireless networks
• Thinking beyond single applications
     – presence event notification
     – 9-1-1 location  time & location as infrastructure
• Need new models of creating services
     – domain-specific languages, not Java APIs

January 2008                       UCSB                     67

To top