Docstoc

Schubin_AES_Lip_Sync

Document Sample
Schubin_AES_Lip_Sync Powered By Docstoc
					                                              Lip Sync
                           What It Is, How It Works




Mark Schubin, AES Lip Sync, 2009 October 11              1
                                     Lip Sync Stink
                           What It Is, How It Works,
                               Why It Doesn’t



Mark Schubin, AES Lip Sync, 2009 October 11            2
                       Definition of Lip Sync
    NOT a perfect temporal match
between picture and sound

    a perceptually appropriate temporal match
     between picture and sound
           some examples of sounds that shouldn’t
            exactly temporally match pictures:
                distant thunder
                antiphonal trumpets

                intentionally obvious dubbing
                         auto-correction disastrous


Mark Schubin, AES Lip Sync, 2009 October 11            3
               Why Is It An Issue Now?
    1973: Field Synchronizer & DVE
    1976: Live Stereo Simulcast Network
    1982: LCD TV
    1984: Solid-State Broadcast Camera
    1994: Digital Emission
    1994: Virtual Sets
    1997: U.S. HDTV and Up/Downconversion
    2008: Election Night “Hologram”
    2009: U.S. Full-Power Analog-TV Cutoff
Mark Schubin, AES Lip Sync, 2009 October 11   4
             Factors Affecting Lip Sync
    1. Physics: Nothing is instantaneous
           speed of sound: 37 feet ≈ 1 30-fps frame
            (varies with temperature & humidity)
           speed of circuitry
                simple processing is cumulative
                look-ahead processing, requires a buffer:
                 imager/display, format conversion, digital effects,
                 frame synchronization
           speed of light (and electronic signals)
    2. Psychophysics
           Visual distance allows delayed sound
    3. Action
Mark Schubin, AES Lip Sync, 2009 October 11                            5
    Display Experiment: HPA 2006

                                              DLP :28
                                              LCD :00
          Latency, itself, is                 CRT :01
          not lip-sync error add more for
                                              multiviewers



                                                              Bill Hogan &
                                                             Michael Smith
Mark Schubin, AES Lip Sync, 2009 October 11                              6
                                              Action
    Improper Setting
           Not matching delay (sometimes audio)
                not bothering
                                                       Squarehead
                not knowing                           Audioscope
    Intentional Choice
           Dubbing
           Use of low-data-rate wireless camera
           Use of virtual set, “hologram,” sports marker
    Improper Design
           Not compensating
                for uses (common and possible)
                for an imperfect world
Mark Schubin, AES Lip Sync, 2009 October 11                   7
                     BRR Decoder Problem
                                                   again, latency, itself,
                                                   is not lip-sync error




                                              presentation
                                                (decoding)
                                               time stamp
                                              OPTIONAL!!!




Mark Schubin, AES Lip Sync, 2009 October 11                                  8
           Common Impossibilities - I
    Live control room
           Video delays: imaging chips, LBR wireless,
            frame syncs, format conversion, digital
            effects, multi-viewers, display processors
           Compensation problems:
            instructions, return bleed, conversation




Mark Schubin, AES Lip Sync, 2009 October 11              9
          Common Impossibilities - II
    Simulcast to TV and cinemas
           Video delay: bit-rate reduction
           Compensation problem: Viewers at different
            distances from screens




Mark Schubin, AES Lip Sync, 2009 October 11              10
                           Metropolitan Opera House
        about three frames of                 more than four frames of
           acoustic delay                          acoustic delay

Mark Schubin, AES Lip Sync, 2009 October 11                              11
                 Visual-Aural Perspective

                                                                      distant
                                                                      sound
                                                                      ok




                      the
                    killer
                                                                      distant
                    close                                             sound
                      -up                                             not ok


                                        speed of sound ≈ 1100’/sec
                                        37 feet ≈ 1 30-fps TV frame
Mark Schubin, AES Lip Sync, 2009 October 11                                     12
Today, viewing distances
 can be measured in city
  blocks – or even miles




 Mark Schubin, AES Lip Sync, 2009 October 11   13
                                          Conclusions
    Lip-sync is a “new” problem due to recent
     developments
    Big-distance acoustic delay might be
     insurmountable on the viewer end;
     control-room error might also be
    Lock video & audio signals           3 cheers for
     to prevent cumulative error          audio-signal
                                          embedding!
    Auto-correction might harm intent
    MPEG Police should be
     organized to enforce
     frequent PTS use
           meantime, change channels
Mark Schubin, AES Lip Sync, 2009 October 11             14

				
DOCUMENT INFO
Shared By:
Categories:
Stats:
views:4
posted:3/5/2010
language:English
pages:14