Telepresence:
An Umbrella Research Topic
Jim Gray
Microsoft Research
Gray@Microsoft.com
http://research.Microsoft.com/~Gray/
1
NSF: Nerve Center of Science
If it’s not broke, don’t fix it.
But….
US Science is the engine of progress
BUT…..
Best and brightest are spending
increasing time fundraising
Seems excessive to me.
Venture capital community is
richer and
more generous
than NSF 2
Outline (ambitious!)
Microsoft Research (census)
Tele-Presentations (Gordon Bell, Jim Gemmell)
Microsoft Research initiative on Telepresence
What if you could record everything you see & hear?
The architecture revolution:
processing moves to transducers
3
Microsoft Research -- 1991
Founded in 1991
Goal:
pursue strategic technologies
for Microsoft
Original research groups:
– Natural Language Processing
– Operating Systems
– Programming Languages
Overall size 600 in FY00,
primarily in Redmond
Major impact on MS products
– Virtually all MS products shipped today use
technology from MS Research
Critical role in MS growth
– Pioneering research in software that allows
computers to see, hear, speak and understand
7
Microsoft Research
Philosophy
University organizational model
– Flat structure, critical mass groups
Open research environment
– Aggressive publication of research results
in literature and on world wide web
– Frequent visitors, daily seminars
– Over 70 visiting professors and interns in
1997
– Over 110 visiting researchers in 1998 8
Some Key Senior Researchers
Systems
– Rick Rashid, Butler Lampson, Gordon Bell
– Anoop Gupta, Roger Needham, Chuck Thacker
Databases & Data Mining
– David Lomet, Jim Gray, Usama Fayyad
Graphics
– Jim Kajiya, Jim Blinn, Alvy Ray Smith, Michael Cohen
Speech & Language
– Karen Jensen, George Heidorn, X.D. Huang, Alex
Acero, Hsiao-Wuen Hon, Scott Meredith
9
Some Key
Senior Researchers
UI Design, Intelligent Systems, IR
– George Robertson, Linda Stone, Susan Dumais, David
Heckerman, Eric Horvitz, Jack Breese
Computer Vision & Signal Processing
– Steve Shafer, Rick Szeliski, P. Anandan, Rico Malvar
Cryptography & Theory
– Yacov Yacobi, Jennifer Chayes, Christian Borg,
Michael Freedman
Languages & Compilers
– Daniel Weise, Chris Fraser, Amitabh Srivastava, Luca
Cardelli, David Hanson, Charles Simonyi, Todd
Proebsting 10
Microsoft Research
1997 BusinessWeek Poll of Academia:
– Voted #7 lab (overall) in Computer Science
– Voted #3 industrial research lab
(after Bell Labs and IBM Research)
– Voted #2 most desirable lab to work
(after Stanford)
11
Outline (ambitious!)
Microsoft Research (census)
Tele-Presentations (Gordon Bell, Jim Gemmell)
Microsoft Research initiative on Telepresence
What if you could record everything you see & hear?
The architecture revolution:
processing moves to transducers
12
Gordon Bell on
Tele Presentations
13
http://research.microsoft.com/barc/GBell/
Motivation:
Telepresentations
• Presenter and/or
audience
telepresent
NOT: meeting or collaboration
settings
Forget the nasty social
issues!
Mostly one-way
14
Telepresentation
Elements
Slides
Audio
Video
Script,
text
comments,
hyperlinks,
etc.
15
Telepresentations:
The Essentials
Slide and audio a must
Add some video
(low quality)
to make us feel good
Storage and
transmission costs low
16
Telepresentations:
The Killer App
Increased attendance & lower
travel costs
Practical and low-cost NOW
e.g. ACM97 - 2,000 visitors in real
space, 20,000 visitors on Internet
http://research.microsoft.com/acm97
17
Today’s
Experiment
Would you like to pause, rewind, browse?
Do you wish you could have seen this
– At home?
– At another time?
How much does a present speaker add?
How much would you pay for real
presence?
18
Outline (ambitious!)
Microsoft Research (census)
Tele-Presentations (Gordon Bell, Jim Gemmell)
Microsoft Research initiative on Telepresence
What if you could record everything you see &
hear?
The architecture revolution:
processing moves to transducers
19
Changing role of computation
Past: Computers for:
– computing (Cray)
– business data processing (IBM)
– “document” creation (PC)
Future: Computers for:
– understanding & learning
– communicating
– consuming & entertaining
Requires new User Interface to machines
20
Flows
Making “Flows” a Reality
Computer Graphics
– Creating realistic looking environments,
people
Computer Vision
– Analyzing posture, gaze, gestures
Speech input/output
Natural Language
– Analysis, IR
Implicit requests for information 22
Building life-like human
characters
Recognizing gestures
Area of
Live video motion H flow V flow
Generating life-like speech
from textual data
Data-driven stochastic speech
– Natural sounding
– Rapid, automatic customizability
Examples
– Synthetic voice w/ transplanted speech
contours
25
Artificial singing
AT&T Voder, 1962, by Homer Dudley
– Daisy (Inspiration for HAL‟s voice in 2001)
Microsoft Research Whistler, 1997
– Scarborough Fair
26
Analyzing language
Language recognition shipped in Word 97
General purpose text-critiquing,
summarization, Japanese word-breaking
27
Inside The Office
Grammar Checker
28
Understanding language:
MindNet
A huge language
knowledge base
Automatically created
from dictionaries
Words (nodes) linked
by relationships
Millions of links
Recently added
(Encarta)
encyclopedia
knowledge 29
MindNet -- “Going to the birds”
chicken
Is_a poultry Purpose supply Typ_obj
clean Is_a Quesp
smooth Typ_0bj_of keep
Is_a hen duck
Is_a
Typ_obj meat
Purpose
preen Typ_subj Cause
Is_a egg
Means quack
Not_is_a plant
chatter Typ_subj animal
Is_a Is_a
Is_a Is_a creature
make bird Is_a
Typ_obj sound
Part feather
Is_a
gaggle Is_a goose wing Is_a limb
peck Is_a
Means Typ_subj_of Is_a
claw
Is_a Is_a
beak Part_of Part
hawk Is_a
Typ_obj
strike Typ_subj_of
fly
leg
turtle catch
Is_a Typ_subj Is_a
bill arm
face Locn_of mouth Is_a opening
Changing balance between
user & software systems
Yesterday:
– Applications were single programs running in
isolation
– Users used to (more or less) understand systems
that they used
Today:
– Componentized applications operate in concert
– Sophisticated users understand only small
percentage of systems they use
31
Tomorrow‟s Systems and
Applications
Users will not be able to predict
– where computations will be performed,
– when they will be performed or
– by what software components
Gap between system capabilities and
user understanding will grow to the
point that the only way user will be able
to use system is through assisting
agents
32
Examples of user agents &
implicit actions
Lumiere (Office 97)
– Monitoring user and program events to
provide user help and assistance
Implicit queries
– Inferring information needs from browsing
Lookout/SpamKiller
– Monitoring mail activity to auto-categorize it
33
User Modeling
Models of a user‟s informational goals
– User‟s query (when available…)
– User‟s background
– Acute and long-term search activity
– Acute actions with objects and documents
– Program data structures
Explicit and implicit information access
and display
Outline (ambitious!)
Microsoft Research (census)
Tele-Presentations (Gordon Bell, Jim Gemmell)
Microsoft Research initiative on Telepresence
What if you could record everything you see &
hear?
The architecture revolution:
processing moves to transducers
35
Some Tera-Byte Databases Kilo
Mega
The Web: 1 TB of HTML
TerraServer 1 TB of images
Giga
Several other 1 TB (file) servers
Hotmail: 7 TB of email Tera
Sloan Digital Sky Survey:
40 TB raw, 2 TB cooked Peta
EOS/DIS (picture of planet each week)
– 15 PB by 2007
Exa
Federal Clearing house: images of checks
– 15 PB by 2006 (7 year history)
Zetta
Nuclear Stockpile Stewardship Program
– 10 Exabytes (???!!) Yotta
36
A letter
Info Capture A novel
Kilo
You can record Mega
everything you see
or hear or read. A
Movie Giga
What would you do
with it? Library of Tera
Congress (text)
How would you
LoC (image) Peta
organize & analyze
it?
All Disks Exa
Video 8 PB per lifetime (10GBph)
Audio 30 TB (10KBps) All Tapes Zetta
Read or write: 8 GB (words)
See: http://www.lesk.com/mlesk/ksg97/ksg.html Yotta 37
Kilo A letter
A novel
Mega
A
Giga Movie
Library of
Congress (text)
Tera
LoC LoC
(sound +
(image)
cinima) Peta
All Disks Exa All
Photos
All Tapes Zetta
All Information!
Yotta 38
Michael Lesk’s Points
www.lesk.com/mlesk/ksg97/ksg.html
Soon everything can be recorded and kept
Most data will never be seen by humans
Precious Resource: Human attention
Auto-Summarization
Auto-Search
will be a key enabling technology.
39
Outline (ambitious!)
Microsoft Research (census)
Tele-Presentations (Gordon Bell, Jim Gemmell)
Microsoft Research initiative on Telepresence
What if you could record everything you see &
hear?
The architecture revolution:
processing moves to transducers
40
Put Everything
in Future (Disk) Controllers
(it‟s not “if”, it‟s “when?”)
Acknowledgements:
Dave Patterson explained this to me a year ago
Kim Keeton
Erik Riedel Helped me sharpen
these arguments
Catharine Van Ingen
41
Remember Your Roots
42
Kilo
Mega Technology Drivers: Disks
Disks on track
Giga
100x in 10 years
Tera
2 TB 3.5” drive
Peta
Shrink to 1” is 200GB
Exa
Disk replaces tape?
Zetta
Yotta
Disk is super
computer!
43
Data Gravity
Processing Moves to Transducers
Move Processing to data sources
Move to where the power (and sheet metal) is
Processor in
– Modem
– Display
– Microphones (speech recognition)
& cameras (vision)
– Storage: Data storage and analysis
44
It‟s Already True of Printers
Peripheral = CyberBrick
You buy a printer
You get a
– several network interfaces
– A Postscript engine
cpu,
memory,
software,
a spooler (soon)
– and… a print engine.
45
All Device Controllers will be Cray 1‟s
TODAY
– Disk controller is 10 mips risc engine
with 2MB DRAM
Central
– NIC is similar power
Processor &
SOON
Memory
– Will become 100 mips systems
with 100 MB DRAM.
They are nodes in a federation
(can run Oracle on NT in disk
controller).
Advantages
– Uniform programming model
– Great tools Tera Byte
– Security Backplane
– economics (CyberBricks)
– Move computation to data (minimize
traffic)
46
Basic Argument for x-Disks
Future disk controller is a super-computer.
– 1 bips processor
– 128 MB dram
– 100 GB disk plus one arm
Connects to SAN via high-level protocols
– RPC, HTTP, DCOM, Kerberos, Directory Services,….
– Commands are RPCs
– Management, security,….
– Services file/web/db/… requests
– Managed by general-purpose OS with good dev
environment
Apps in disk saves data movement
– need programming environment in controller 47
The Slippery Slope
Nothing =
Sector Server
If you add function to server
Then you
add more function to server
Function gravitates to
data.
Everything =
App Server
48
Why Not a Sector Server?
(let‟s get physical!)
Good idea, that‟s what we have today.
But
– cache added for performance
– Sector remap added for fault tolerance
– error reporting and diagnostics added
– SCSI commends (reserve,.. are growing)
– Sharing problematic (space mgmt, security,…)
Slipping down the slope to a 2-D block server
49
Why Not a 1-D Block Server?
Put A LITTLE on the Disk Server
Tried and true design
– HSC - VAX cluster
– EMC
– IBM Sysplex (3980?)
But look inside
– Has a cache
– Has space management
– Has error reporting & management
– Has RAID 0, 1, 2, 3, 4, 5, 10, 50,…
– Has locking
– Has remote replication
– Has an OS
– Security is problematic
– Low-level interface moves too many bytes 50
Why Not a 2-D Block Server?
Put A LITTLE on the Disk Server
Tried and true design
– Cedar -> NFS
– file server, cache, space,..
– Open file is many fewer msgs
Grows to have
– Directories + Naming
– Authentication + access control
– RAID 0, 1, 2, 3, 4, 5, 10, 50,…
– Locking
– Backup/restore/admin
– Cooperative caching with client
File Servers are a BIG hit: NetWare™
– SNAP! is my favorite today 51
Why Not a File Server?
Put a Little on the Disk Server
Tried and true design
– Auspex, NetApp, ...
– Netware
Yes, but look at NetWare
– File interface gives you app invocation interface
– Became an app server
Mail, DB, Web,….
– Netware had a primitive OS
Hard to program, so optimized wrong thing
52
Why Not Everything?
Allow Everything on Disk Server
(thin client‟s)
Tried and true design
– Mainframes, Minis, ...
– Web servers,…
– Encapsulates data
– Minimizes data moves
– Scaleable
It is where everyone ends up.
All the arguments against are short-term.
53
The Slippery Slope
Nothing =
Sector Server
If you add function to server
Then you
add more function to server
Function gravitates to
data.
Everything =
App Server
54
Disk = Node
has magnetic storage (100 GB?)
has processor & DRAM
has SAN attachment
has execution Applications
environment Services DBMS
RPC, ... File System
SAN driver Disk driver
OS Kernel
55
Technology Drivers:
System on a Chip
Integrate Processing with memory on one chip
– chip is 75% memory now
– 1MB cache >> 1960 supercomputers
– 256 Mb memory chip is 32 MB!
– IRAM, CRAM, PIM,… projects abound
Integrate Networking with processing on one
chip
– system bus is a kind of network
– ATM, FiberChannel, Ethernet,.. Logic on chip.
– Direct IO (no intermediate bus)
Functionally specialized cards shrink to a chip.
56
Technology Drivers: What if
Networking Was as Cheap As Disk IO?
TCP/IP Disk
– Unix/NT – Unix/NT
100% cpu @ 40MBps 8% cpu @ 40MBps
Why the Difference?
Host does Host Bus Adapter does
TCP/IP packetizing, SCSI packetizing,
checksum,… checksum,…
flow control flow control
57
small buffers DMA
Technology Drivers:
The Promise of SAN/VIA:10x in 2 years
http://www.ViArch.org/
Today:
– wires are 10 MBps (100 Mbps Ethernet)
– ~20 MBps tcp/ip saturates 2 cpus
– round-trip latency is ~300 us
In the lab
– Wires are 10x faster Myrinet, Gbps Ethernet,
ServerNet,…
– Fast user-level communication
tcp/ip ~ 100 MBps 10% of each processor
round-trip latency is 15 us 58
SAN:
Standard Interconnect
Gbps Ethernet: 110 MBps
LAN faster than
memory bus?
PCI: 70 MBps 1 GBps links in
lab.
100$ port cost
UW Scsi: 40 MBps soon
Port is computer
FW scsi: 20 MBps
scsi: 5 MBps 59
Technology Drivers:
100 GBps Ethernet replaces SCSI
Why I love SCSI
– Its fast (40MBps)
– The protocol uses little processor power
Why I hate SCSI
– Wires must be short
– Cables are pricey
– pins bend
60
Functionally Specialized Cards
P mips processor
Storage
ASIC Today:
P=50 mips
M= 2 MB
M MB DRAM
Network
In a few years
ASIC P= 200 mips
M= 64 MB
Display
ASIC
61
Technology Drivers
Plug & Play Software
RPC is standardizing: (DCOM, IIOP, HTTP)
– Gives huge TOOL LEVERAGE
– Solves the hard problems for you:
naming,
security,
directory service,
operations,...
Commoditized programming environments
– FreeBSD, Linix, Solaris,…+ tools
– NetWare + tools
– WinCE, WinNT,…+ tools
– JavaOS + tools
Apps gravitate to data.
General purpose OS on controller runs apps.
62
Basic Argument for x-Disks
Future disk controller is a super-computer.
– 1 bips processor
– 128 MB dram
– 100 GB disk plus one arm
Connects to SAN via high-level protocols
– RPC, HTTP, DCOM, Kerberos, Directory Services,….
– Commands are RPCs
– management, security,….
– Services file/web/db/… requests
– Managed by general-purpose OS with good dev
environment
Move apps to disk to save data movement
– need programming environment in controller 63
Summary
Microsoft Research (census)
Tele-Presentations (Gordon Bell, Jim Gemmell)
Microsoft Research initiative on Telepresence
What if you could record everything you see &
hear?
The architecture revolution:
processing moves to transducers
64