The Past, Present, and Future of Video Telephony A Systems

Document Sample
The Past, Present, and Future of Video Telephony A Systems Powered By Docstoc
					        The Past, Present, and Future of
Video Telephony: A Systems Perspective

                              Dave Lindbergh

                  Stanford Networking Seminar
                              27 January 2005




                                           1
 Thanks


           Thanks to Athina for inviting me
           I’m going to take advantage of the opportunity
           to present some opinions about video
           telephony
           I hope at least they’re educated opinions




Past, Present, and Future of Video Telephony – Lindbergh – January 2005, Stanford University   2
 Contents

          A little bit about my perspective
          Where we’ve come from
          Where we are now
            – What is succeeding
            – What is not succeeding (yet)
            – Current problems & challenges

          The mass-market barrier
            – Expectations vs. reality
            – What it will take to succeed

          Where we go from here
Past, Present, and Future of Video Telephony – Lindbergh – January 2005, Stanford University   3
 A little bit about my perspective


           Engineering background
            – Modems & data communications
            – Protocols, real-time systems, image processing

           1993: PictureTel, largest vendor of video
           conferencing gear
            – ISDN, H.320, H.261 128+ kbit/s minimum

           Soon got sucked into standardization work
            – Mid-90s: Chaired H.324 Systems Experts Group
                   • Edited ITU-T Rec. H.324
                   • Basis of today’s 3G-324M system
                   • Precursor to H.323 (yes, I take some of the blame)

Past, Present, and Future of Video Telephony – Lindbergh – January 2005, Stanford University   4
 What I’ve been doing lately


           H.264 video compression standardization
            – Profiles/Levels
            – Applications

           Editor, ITU-T Rec. H.239 – Role management
            – Live = People
            – Presentation = Content

           Editor, ITU-T Rec. H.241 – Video signaling
           Editor, H.324 (again)
           Rapporteur, ITU-T Q.23/16 (“Media Coding”)

Past, Present, and Future of Video Telephony – Lindbergh – January 2005, Stanford University   5
 Video telephony system


           18 frames/second
           Progressive scan
           Plasma display
           Pixel aspect ratio 3:2
           Image quality described as “excellent”
           End-to-end latency 1 millisecond (great!)



Past, Present, and Future of Video Telephony – Lindbergh – January 2005, Stanford University   6
 April 7, 1927 – Bell Labs




Past, Present, and Future of Video Telephony – Lindbergh – January 2005, Stanford University   7
 New York – Washington DC




   Walter Gifford                                                     Herbert Hoover
   President, AT&T                                                    US Sec’y of Commerce
   New York                                                           Washington DC
Past, Present, and Future of Video Telephony – Lindbergh – January 2005, Stanford University   8
 “Television” = Telephone + Vision


           50x50 pixel display, neon bulbs
           Camera: Scanning arc lamp beam
           Optional projection to 2x3 feet
            – But “results were not so good”



                  Edna Mae Horner
                                          Operator
       Chesapeake and Potomac
            Telephone Company
Past, Present, and Future of Video Telephony – Lindbergh – January 2005, Stanford University   9
   AT&T Picturephone



1957 “Experimental Model”




  Past, Present, and Future of Video Telephony – Lindbergh – January 2005, Stanford University   10
 Early 1960s

                                                                                                    Mirror




Past, Present, and Future of Video Telephony – Lindbergh – January 2005, Stanford University   11
 AT&T was very serious

           Plenty of smart business people!




                                1964


Past, Present, and Future of Video Telephony – Lindbergh – January 2005, Stanford University   12
 Framing




Past, Present, and Future of Video Telephony – Lindbergh – January 2005, Stanford University   13
 Did it “cost too much”?




           AT&T finally gave up in the early 1970s
Past, Present, and Future of Video Telephony – Lindbergh – January 2005, Stanford University   14
 1980s – Still image picture phones


           Mid-1980s: Japanese consumer electronic
           firms introduced still-image picture phone
            –    Used existing regular analog phone line
            –    POTS modem
            –    ~ 5 seconds to send 1 black & white frame
            –    No audio during picture transmission
            –    ~$200 each

           Very few takers




Past, Present, and Future of Video Telephony – Lindbergh – January 2005, Stanford University   15
 1992 – AT&T Videophone 2500


“Predicting that 10 years from now
video phones will be as popular as
cordless phones and fax machines,
last week AT&T introduced the first
full-color motion video phone that
operates over regular phone
lines…AT&T officials say the picture
quality was acceptable to test-market
consumers”
  – Newsweek, January 20, 1992
10 frames/second, $1500
Marconi, others, had similar products
Past, Present, and Future of Video Telephony – Lindbergh – January 2005, Stanford University   16
 Many more videophones since then


  Mostly based
  on ITU-T
  standards

  H.320 (ISDN)
  H.323 (IP)
  H.324 (POTS)
  and SIP

  They all worked
                                      Siemens T-View H.320 (ISDN) Phone ~ 1997
Past, Present, and Future of Video Telephony – Lindbergh – January 2005, Stanford University   17
 More videophones




Past, Present, and Future of Video Telephony – Lindbergh – January 2005, Stanford University   18
 And more…




Past, Present, and Future of Video Telephony – Lindbergh – January 2005, Stanford University   19
 And more




Past, Present, and Future of Video Telephony – Lindbergh – January 2005, Stanford University   20
 FOMA experience in Japan

           FOMA = H.324/Mobile, 64 kbit/s channels
           Video calls cost ~2x price of voice calls
           3 million phones deployed (as of Sept. 2004)
           Average monthly video usage = 2 minutes
           Top 20% of users do 20 minutes/month
           Most users young
            – Show where they are, who they’re with
            – Don’t point camera at themselves

           DoCoMo is hopeful that usage will increase
           when penetration > 1 phone/family
Past, Present, and Future of Video Telephony – Lindbergh – January 2005, Stanford University   21
 Still-image camera phones


           2nd generation – Camera is on back of phone




Past, Present, and Future of Video Telephony – Lindbergh – January 2005, Stanford University   22
 Did they all cost too much?

           Many had good video quality
           Most were reliable & easy to use
           Many < $50 PC cameras with videophone apps
           MS NetMeeting & Messenger are free
           Clearly, people do want video phones
            – Witness all the attempts, user excitement

           But they don’t buy or use them when offered
            – For some reason people are disappointed
            – We need to understand why before we can fix this
Past, Present, and Future of Video Telephony – Lindbergh – January 2005, Stanford University   23
 What is succeeding?

           The real killer app: TELEVISION




           But TV is doing fine without help from me…
Past, Present, and Future of Video Telephony – Lindbergh – January 2005, Stanford University   24
 What else is succeeding?


           Video conferencing
            – $2B/year industry, profitable
            – Top vendors: Polycom, Tandberg, Sony




Past, Present, and Future of Video Telephony – Lindbergh – January 2005, Stanford University   25
 Video conferencing today


           Most use is in large organizations
            – Industry
            – Government
            – Education

           Most use is internal
            – Between sites of the same organization

           Most use is scheduled
            – Planned meetings, not spontaneous

           Only a few meeting rooms have VC equipment
            – Much talk about ubiquitous access, but not real yet

Past, Present, and Future of Video Telephony – Lindbergh – January 2005, Stanford University   26
 Situations where VC works well

           With people you already know
            – Already introduced, not strangers

           Not too many people on screen at once
            – Need to see facial expressions clearly

           Good lighting
           Good room & furniture layout
           People & Content at same time
           How I use it every week
            – Offices in Boston, California, Texas, Atlanta, Israel…
Past, Present, and Future of Video Telephony – Lindbergh – January 2005, Stanford University   27
 Why is VC successful?


           Relatively big picture size, high resolution
            – Less restriction on where people are in the frame
           Good lighting
           High-value application
           Work environment, pre-scheduled meetings
            – People come dressed & prepared to meet others
            – Reduces discomfort with “being on camera”
           Yet, VC is still in < 2% of conference rooms
            – Lots of room for growth 
            – Similar problems as stopped video telephony 
            – It works, but not nearly as well as we want it to!
Past, Present, and Future of Video Telephony – Lindbergh – January 2005, Stanford University   28
 How are we doing?


           We’re doing an excellent job on the classical
           technical challenges
            –    Video and audio coding
            –    Cost: $250K (1989) to $2000 (2004), less for PCs
            –    Bandwidth is getting cheaper all the time
            –    Simplicity, reliability have improved greatly

           Some immediate challenges
            – Standards and network issues – being worked

           Longer-term challenges for video telephony
            – Expectations vs. reality
            – Human factors
Past, Present, and Future of Video Telephony – Lindbergh – January 2005, Stanford University   29
 Standards

           Wonderful thing: So many to choose from!
           Religion: H.323, SIP, MGCP, proprietary…
            – No real differences from user perspective
            – Some want to start over…again

           Every standard is unnecessarily complex
            – Over-reaction to past mistakes, too little experience
            – The POTS network was also incredibly complex
            – Limits of human complexity management ability

           Directory services
            – ENUM/DNS, H.350/LDAP, UMMAP…
            – This will settle out with time
Past, Present, and Future of Video Telephony – Lindbergh – January 2005, Stanford University   30
 Latency


           Lots of denial – this is not helpful
           ITU-T G.114 gives 150 ms as an upper limit
            – For total end-to-end latency
                   • Including propagation over distance
                   • This is about right, but difficult to achieve

           IP networks inherently have latency issues
            – Usually make ARQ, backchannel schemes
              impractical
           Low frame rates make things worse

Past, Present, and Future of Video Telephony – Lindbergh – January 2005, Stanford University   31
 Quality of Service on IP


           Lots of solutions in theory
            – DiffServ, MPLS, IP Precedence, etc…

           Zero penetration on public Internet
            – There is no pricing model

           Most private networks provide QoS with
           massive over-provisioning
            – This is often cheaper than “clever” schemes

           QoS will remain a problem on the public
           Internet until there is a way to charge for it

Past, Present, and Future of Video Telephony – Lindbergh – January 2005, Stanford University   32
 More network issues


           NATs and Firewalls
            – IP is effectively unusable between organizations
            – Virtually all inter-organization calls are still ISDN

           Network fragmentation
            – IP, ISDN, POTS, 3G, 4G…
            – Lack of public/automatic gateways

           These are all being worked; will get solved
            – Some things will take time to shake out

           A “killer app” could force more rapid change
            – But this hasn’t happened yet

Past, Present, and Future of Video Telephony – Lindbergh – January 2005, Stanford University   33
 Video compression coding

           Ideal lossy video compression system
            – Every possible bit sequence decodes as something
              meaningful to human perception
            – On various time scales
            – Might this be the way the brain works?
                   • Markov chain text generators sound a lot like dreams…

           Lots of room for improved coding
           Past: Biggest challenge was reducing bitrate
            – Bandwidth and storage were expensive

           Today: Computational efficiency a challenge
            – Bits are getting cheaper faster than computes are
Past, Present, and Future of Video Telephony – Lindbergh – January 2005, Stanford University   34
 Beyond video coding


           Most video research focused on coding
            – Without compression video was unmanageable

           Between improved compression and cheaper
           bandwidth and storage, things can now change
            – Computation has gotten much cheaper

           Fast, cheap video DSP means we can do more
            – Stitching, warping, perspective correction…
            – Searching, indexing, processing, recognizing content

           Analogous to audio & still-image DSP
            – There will be new apps unique to video
Past, Present, and Future of Video Telephony – Lindbergh – January 2005, Stanford University   35
 The mass-market barrier


           Video conferencing is a successful niche
           Video telephony hasn’t succeeded yet
            – Yet, clearly there is a market desire!

           Current issues don’t explain past failures
            – Standards, directories were solved for videophones
            – Latency was not a problem in the analog world
            – QoS, NAT/FW issues didn’t exist for switched circuits

           Then what will it take for success?
            – Why have users not yet embraced video telephony?


Past, Present, and Future of Video Telephony – Lindbergh – January 2005, Stanford University   36
 What will it take to succeed?


           Cost?
            – PC apps are nearly free, yet very little used

           Reliability?
            – Current systems work well on private networks
            – Video telephones were quite reliable

           Complexity?
            – Video phones are easy to use, as are modern VCS

           Video quality?
            – VC systems provide TV-like quality, yet haven’t
              broken into the mass-market

Past, Present, and Future of Video Telephony – Lindbergh – January 2005, Stanford University   37
 Expectations – Fiction




                           Metropolis (Fritz Lang, 1926)
Past, Present, and Future of Video Telephony – Lindbergh – January 2005, Stanford University   38
 From Jetsons to Star Trek

           Countless science-fiction films & TV shows
           – Perfect framing, perfect lighting
           – People look straight into the camera
           – Nobody is nervous “being on camera”




      2001: A Space Odyssey (Stanley Kubrick, 1968)
Past, Present, and Future of Video Telephony – Lindbergh – January 2005, Stanford University   39
 Expectations – Film & TV experience


           We’ve all grown up with film & TV
            –    Professional cinematography / videography
            –    Studios with proper lighting, layout
            –    Multiple camera angles
            –    Directors to choose the best shots

           With video telephony we get
            –    Single camera viewpoint
            –    Bad lighting
            –    Bad or no framing
            –    Often, poor resolution and video quality
            –    Sometimes, tiny pictures

Past, Present, and Future of Video Telephony – Lindbergh – January 2005, Stanford University   40
 What it will take: Human factors

          Framing
            – Keeping people in the picture

          Camera viewpoint & perspective
            – Psychological factors, geometrical distortion

          Eye contact & gaze direction
            – And “camera shyness”

          Peripheral vision
            – Sense of space – close-up vs. wide views

          Attention requirements
            – Tradeoffs of different media
Past, Present, and Future of Video Telephony – Lindbergh – January 2005, Stanford University   41
 Framing

           Keeping people in the picture
            – Close enough to see faces clearly
            – Far enough for freedom of movement
            – Consciousness of framing, control can be distracting
            – Scenes in movie Mother (Albert Brooks, 1996) –
              Rob Morrow and Debbie Reynolds on videophone

           Automatic speaker-following not ideal
            –    Often used when multiple people are in the room
            –    Want to see listener reactions (not just talker)
            –    Want to see VIPs (even if they’re listening)
            –    Close-ups can lose sense of relative position
            –    Still, often better than doing nothing
Past, Present, and Future of Video Telephony – Lindbergh – January 2005, Stanford University   42
 Camera viewpoint & perspective


           If the camera is too close
            – Geometric distortion – big noses, etc.

           If your camera is above eye-level
            – They’re looking down on you – you look submissive

           If your camera is below eye-level
            – They’re looking up at you – you look dominant

           This is why royal thrones are tall
           There is no single “right” position
            – People can either stand or sit

Past, Present, and Future of Video Telephony – Lindbergh – January 2005, Stanford University   43
 Eye contact


           What happens when you stare at someone?
           Does it happen if you stare at video?
           What happens when someone stares at you?
            – Do you feel comfortable?

           Eye contact is a form of innate, highly evolved
           non-verbal communication
            – A deep part of human nature
            – Lots of emotional charge
            – Not present in video telephony - unnatural


Past, Present, and Future of Video Telephony – Lindbergh – January 2005, Stanford University   44
 Eye contact & gaze direction


       People can detect eye contact at great distances
         – They can tell when they’re being observed
         – They may respond with a glance, or return contact

       Cooperation or liking = more direct gaze
       Disagreement or dislike = less direct gaze
       Gaze and emotional signals
         – Unwavering gaze - dominance or threat
         – Gaze avoidance – submission or fear
         – Gaze can signal sincerity, discomfort, challenge…



Past, Present, and Future of Video Telephony – Lindbergh – January 2005, Stanford University   45
 Eye contact and video


           We need to solve eye contact on video
            – I think this will reduce “camera shyness”

           Need to know who is looking at you
           Need to know if/when you’re stared at
           Need to allow natural feedback response




Past, Present, and Future of Video Telephony – Lindbergh – January 2005, Stanford University   46
 Other “naturalness” issues


           Peripheral vision
            – Noticing what other people are doing

           Many people prefer to see but not be seen
            – At least with current video systems

           Attention demand & media
            –    Text: Least – can carry on several IMs at once
            –    Audio: More – one at a time, can do other things
            –    Video: Most – can’t do other things
            –    Not a flaw, just something to take into account



Past, Present, and Future of Video Telephony – Lindbergh – January 2005, Stanford University   47
 Why is this all so complicated?


           Voice telephony doesn’t have these problems
           Because people are evolved to talk in the dark
            – This is why telephones “work”
           Because video is not “just another channel”
            – But that’s how engineers usually think about it
            – It’s something very different than audio
           The video telephony experience needs to feel
           more natural and intuitive
            – I think this is the real reason it hasn’t succeeded yet
            – This is where research needs to focus

Past, Present, and Future of Video Telephony – Lindbergh – January 2005, Stanford University   48
 We are still in the “mainframe” era


           Video telephony is not unique in facing this
           challenge
            – Automobiles: Benz Motorwagen to Ford Model T
                   • Mass production, simplicity
            – Aviation: Wright Flyer to Douglas DC-3
                   • Efficiency, safety
            – Computing: Mainframes to PCs
                   • VLSI microprocessors

           From possibilities in theory to useful practice
            – High-value niche applications come first
            – These teach us about what is missing
            – When technology matures, the mass-market arrives
Past, Present, and Future of Video Telephony – Lindbergh – January 2005, Stanford University   49
 Where we go from here


           “Show-me” 3G video phones can succeed now
            –    See where I am
            –    See who I’m with
            –    Human factors issues not a problem in this model
            –    Limited usage compared to voice minutes

           Video conferencing, other high-value apps will
           continue to mature & expand
           Human factors improvements needed
            – For “talking heads” video telephony to succeed

           A fertile field for research – please work on it!
Past, Present, and Future of Video Telephony – Lindbergh – January 2005, Stanford University   50
 It may take a visionary individual




                                                                                           ?
Past, Present, and Future of Video Telephony – Lindbergh – January 2005, Stanford University   51
                                   Thank you!

           Polycom has an opening for a video DSP
           researcher to work on these topics
           Send CVs to lindbergh@92F1.com


Past, Present, and Future of Video Telephony – Lindbergh – January 2005, Stanford University   52