The Concord Algorithm for Synchronization of Networked Multimedia

Document Sample
The Concord Algorithm for Synchronization of Networked Multimedia Powered By Docstoc
					       The Concord Algorithm for Synchronization of Networked
                        Multimedia Streams
                 N. Shivakumar           C. J. Sreenan       B. Narendran          P. Agrawal
                                           AT&T Bell Laboratories
                                           Murray Hill, NJ 07974

                      Abstract                             di erence which is undesirable in situations where, for
                                                           example, lip-sync is required. We describe these two
   Synchronizing di erent data streams from multiple       problems as single and multiple stream synchroniza-
sources simultaneously at a receiver is one of the basic   tion respectively.
problems involved in multimedia distributed systems.           To solve these synchronization problems, one must
This requirement stems from the nature of packet based     introduce additional delay by bu ering packets at or
networks which can introduce end-to-end delays that        near the point of presentation. In the single stream
vary both within and across streams. In this paper, we     case this is used to smooth out jitter prior to passing
present a new algorithm called Concord, which pro-         data samples to an output device. For multiple related
vides an integrated solution for these single and mul-     streams it is used to compensate for inter-stream delay
tiple stream synchronization problems. It is notable       di erences. There are two basic approaches to oper-
because it de nes a single framework to deal with both     ating this synchronization bu er (sometimes called an
problems, and operates under the in uence of parame-       elastic bu er) 5]:
ters which can be supplied by the application involved.
In particular, these parameters are used to allow a              Packet-Preserving: In this scheme, the receiver
trade-o between the packet loss rates, total end-to-end          waits until a packet p arrives to play back the
delay and skew for each of the streams. For applica-             packet.
tions like conferencing this is used to reduce delay by          Time-Preserving: If a packet p does not arrive
determining the minimum bu er delay/size required.               within a certain time bound, the packet is as-
                                                                 sumed to be lost, and subsequent packets are
                                                                 played back. If p does arrive later, it is thrown
1 Introduction                                                   away.
   The use of packet based communication networks          Our focus is on time-preserving applications and the
to transport digitized streams of audio and video is be-   additional requirements associated with the total end-
coming increasingly common. A characteristic of these      to-end delay: the cumulative delay su ered by pack-
networks is that the total delay experienced by each       ets in the network and bu er. Clearly, if every packet
packet is a function of variable delays due to physical    is delayed in the bu er such that it su ers cumula-
media access and relay queueing, in addition to xed        tively (in the network and bu er) a delay equal to the
propagation delays. The result is that the time dif-       maximum network delay, the receiver can reproduce a
ference between transmitting any two packets at the        jitter-free playback. Problems with this approach are
source is unlikely to be the same as that observed be-     that it may be di cult to obtain an accurate estimate
tween their arrival at the destination. This is a prob-    of maximum network delay over a stream lifetime, and
lem for a stream of multimedia packets, because the        such a value may be quite large in relation to the aver-
presence of delay variations (known as jitter) can have    age delay. This is important for real-time applications
an impact on the audio-visual quality as perceived by a    like conferencing and telephony, where large delays are
human user. Delay is also a problem when using mul-        detrimental to interactivity. For example, in packe-
tiple related streams, because it introduces a temporal    tized voice applications, gures in the 150 - 250 msec
   Now at the Department of Computer Science, Stanford     range are often quoted as being the maximum accept-
University.                                                able total end-to-end delay. It is possible, however, to
take advantage of the built in redundancy of a multi-       for minimum, maximum and/or mean delays and de-
media stream to reduce the bu er delay at the expense       lay distributions. The PDD constructed from this in-
of throwing away packets which arrive too late. This        formation is approximate, but because its long-term
may not be the case with other data streams such as         accuracy is important, we assume that this can be im-
  le transfers.                                             proved dynamically as described in Section 2.2.5. The
    In this paper, we present the Concord algorithm,        basic problem is then to nd the minimum bu er size
which is notable because it de nes a single framework       at the receiver that will smooth out network jitter so
to deal with both forms of synchronization, and oper-       that the stream's requirements of PLR and MAD are
ates under the in uence of parameters which can be          satis ed, if possible. That is, to nd B the minimum
supplied by the application involved. We perform a          bu er size satisfying the following:
trade-o of packet loss against the total end-to-end de-
lay su ered by packets, by applying these parameters             For every packet p: ndp + bdp is a constant TED,
to probabilistic data describing packet delay distribu-          where TED is the total end-to-end delay, ndp is
tions. Section 2 gives the details and also describes            the network delay su ered by packet p, and bdp is
extensions to deal with the e ects of sender/receiver            the induced bu er delay for p, a function of the
clock drift. In Section 3 the solution for multiple              bu er size.
stream synchronization extends this scheme by tak-              The chosen TED is lesser than the MAD of the
ing user input describing upper bounds on the accept-           stream.
able di erence in end-to-end delay (known as skew) for
the related streams. This information is used, if nec-          The chosen TED does not lead to more than
essary, to increase the selected bu er sizes to achieve         PLR packets/sec being thrown away.
synchronization between the streams. The scheme op-
erates dynamically by updating its probability delay
distributions and actual bu er sizes over time in re-
sponse to the observed behavior of the network. Some                         Network
early simulation work is presented in Section 4; fol-
lowed by a review of related work and our conclusions        Sender                                          Receiver
in Sections 5 & 6 respectively.                                                Jitter

                                                                          Figure 1: System elements
2 Single Stream Solution
   This section addresses the problem of single stream      2.2 Algorithm Details
synchronization. After giving a formal de nition of
the problem, our algorithm is described in detail. The      2.2.1 Computing Total End-to-End Delay
initial discussion ignores issues of imperfect clocks, an
assumption which is removed and dealt with in Sec-          In this section we describe how to calculate the end-to-
tion 2.3.                                                   end delay, using the type of PDD shown in Figure 2
                                                            to illustrate the procedure. This PDD shows a nor-
2.1 Problem Formulation                                     mal distribution of packets delays, which is typical of
                                                            packet switched networks 2]. From this plot it is clear
   As shown in Figure 1, we have a network N , over         that all packets that su er more than TED in net-
which sender S wishes to send stream M to receiver          work delay will be declared lost in the time-preserving
R, with the packets in M being produced every Tr            scheme. Hence (1 ? Cdf (TED)) packets will be lost
seconds. All packets have sequence numbers. Stream          every second, where Cdf is the cumulative distribution
M is characterized by the maximum acceptable delay          function on the PDD. There are a few possible direc-
(MAD) it can su er, and maximum packet loss rate            tions at this point to choose the TED of the packets.
(PLR) it can tolerate. For the network N we con-
struct a packet delay distribution (PDD): an estimate         1. Minimize PLR: Choose TED so that PLR is
of the probable delays su ered by packets in the net-             minimized while satisfying the MAD. Cdf (t) is
work over a time window. This PDD may draw on                     a monotonically increasing function, hence (1 ?
existing tra c conditions, history information or any             Cdf (t)) is monotonically decreasing. That is, the
negotiated service characteristics to derive estimates            packet loss rate decreases with an increase in t.
    Hence, the choice of TED = MAD, will have                    R replays packets (and at which the S produces pack-
    the minimum packet loss rate, since MAD is the               ets), and B is the bu er size. The packet in the zeroth
    maximum permissible TED.                                     slot, henceforth termed the EXIT slot will be given to
                                                                 the receiver also every Tr seconds (see Figure 3). The
 2. Minimize TED: Minimize TED such that the                     bu er controller uses two variables - PLAYED to in-
    PLR is satis ed. This can be achieved by choos-              dicate which packet is to be played next, and TIMER
    ing the minimum TED such that (1?Cdf (TED))                  which can be initialized and reset to keep track of Tr .
    is PLR. Choosing any t greater than this TED                 In this, we allow a network to resequence and dupli-
    will reduce the packet loss rate of the stream as            cate packets.
    seen above. Conversely, any t lesser than TED
    will lead to a greater packet loss rate than PLR.
    Hence the given choice of TED will be the mini-                                              EXIT
    mum t satisfying the given PLR.                                Before shift     4    3   2    1
 3. Minimizing Hybrid-Cost: Find a TED such that
    some cost metric cost f (TED; plr)] is minimized,
    where f (TED; plr) ! < with the constraints that
    TED MAD, and plr PLR, where plr is                              After shift          4   3    2      1     Receiver
    (1 ? Cdf (TED)).                                                                             EXIT

                                                                                  Figure 3: Bu er mechanism

                                                                  1. Given the initial PDD of the network, and the
                                                                     PLR, MAD of the stream and the cost function
                                                                     to minimize, BC will decide on the TED.
        % of packets

                                                                  2. When the rst packet arrives (with sequence num-
                                                                     ber i), the ndi su ered by the packet is calcu-
                                                                     lated, and the packet is placed into the appro-
                                                                     priate bu er slot as de ned by bi = b(TED ?
                                                                     ndi )=Tr c. Set PLAYED = (i ? bi), and TIMER =
                                                                     Tr ? ((TED ? ndi) mod Tr ). Note that i could be
                          Network Delay                              any packet, and not necessarily the rst packet.
 Figure 2: A network delay distribution for packets               3. When TIMER = Tr seconds, BC will pass the
                                                                     packet in the EXIT slot to the receiver, will incre-
                                                                     ment PLAYED by one, and will shift all enqueued
   When we choose the TED for the packets, we are                    packets by one. This is a discretely timed event.
in e ect choosing the bu er delay to be encountered                  Reset TIMER to zero.
by the packets since network delay is a function of
the network. It is clear that TED can be chosen                   4. When a new packet arrives with sequence num-
satisfying both the MAD and PLR constraints iff                      ber j , the BC throws away the packet if j is less
(1 ? Cdf (MAD)) PLR.                                                 than the current value of PLAYED. Otherwise,
                                                                     the packet is placed into the (j ?PLAY ED)th slot
2.2.2 Operation of the Bu er Controller (BC)                         in the bu er. Observe that by the shifting mech-
                                                                     anism, packet j will continue to remain in the
Conceptually, the bu er for delaying packets is viewed               (j ?PLAY ED)th slot even as PLAY ED changes.
as a simple shift-register mechanism1. Every Tr sec-                 And since all distinct packets have unique se-
onds, packets in the ith slot, are moved to the (i ? 1)th            quence numbers, no two such packets will collide
slot for all i = 1, 2 : : : B , where Tr is the rate at which        in the bu er. Duplicated packets do not create
   1 A practical implementation avoids copying packet contents       any problems, since the slot will be occupied by
by manipulating pointers to packets or using a circular bu er.       exactly one instance of the packet.
Lemma 1 The rst packet with sequence number i              2.2.3 Computing Bu er Size
that arrives at the receiver with ndi network delay,
ndi TED, would su er exactly TED from being                Finding the minimum bu er size required can be al-
produced to being serviced at the receiver.                ternatively posed as nding the maximum number of
                                                           bu er slots that could be occupied at a given time.
Proof of Lemma 1 : By construction i will be               Let us say that the rst packet received had sequence
placed into slot bi = b(TED ? ndi )=Tr c. After            number 1 at time 0. The worst case occurs when
((TED ? ndi ) mod Tr ), the TIMER would go o ,             the kth packet arrives at the receiver's bu er after
and the packet will be shifted to the right by one.        having su ered minimum network delay min, at time
The packet will take exactly Tr bi from then to be         ((k ? 1) tr + min). The rst packet would have been
serviced. Hence, the packet would su er a cumulative       serviced at time TED by the receiver. The maximum
delay of                                                   number of packets in the bu er is

ndi +((TED ? ndi) mod Tr )+ Tr (b(TED ? ndi)=Tr c)         number produced ? number serviced =

= ndi + (TED ? ndi ); since b(p=q)c q + p mod q = p.       k ? b (((k?1) T +min)?TED) c = ?b (min?TED?T ) c:
                                                                           T  r                   T     r

                                                              The above relation can be proved with a little more
Lemma 2 Subsequent packets that arrive with higher         e ort to any packet reaching the bu er rst, if one
or lower sequence numbers than the rst packet i re-        should consider the relation between PLAYED and
ceived, will be played back if their nd TED, or else       the rst packet inserted, and consider the bu er to be
will be thrown away.                                       a circular bu er for analytical reasons. An alternative
                                                           proof is from the fact that all packets have to su er
Proof of Lemma 2 : Let i, the rst packet received,         the maximum delay in the bu er if they su ered the
be produced at time ti . Hence, it would be played at      minimum delay in the network so that the cumulative
time ti + TED, by Lemma 1. At ti + TED, PLAYED             delay is TED.
would be set to i + 1. Consider a packet (i + j ) that
arrives at the bu er, where j could be positive or neg-    2.2.4 Computing Network Delays
ative. It must have been produced at time (ti + j Tr ).
(i + j ) would lose its chance of being played if it ar-   In some networks, link speed between two nodes in
rived when PLAYED > (i + j ). This would happen            bytes/second is typically known. This de nes the min-
when time is greater than (ti + TED + j Tr ), by           imum amount of propagation time that will be needed
construction. Hence if (i + j ) su ers nd <= TED,          for a packet of sizep bytes to be transmitted. That
(i + j ) would reach the bu er before the deadline of      is min = sizep=link secs independent of switching/
(ti + TED + j Tr ) and will be played, else it will        bu ering etc. Note that this information is not crit-
be thrown away. Note that we are not using the ex-         ical. If the link speed cannot be computed, min can
plicit value of the network delay su ered by subse-        be set to zero for the computation of bu er size. How-
quent packets, instead relying on sequence numbers         ever, if min is known in the network, the bu er size
to derive timing information.                              can be optimized with this added knowledge.
                                                               The algorithm proposed depends on knowing the
Theorem 1 Packets that su er nd          TED will be       network delay su ered by the rst arriving packet. A
played such that they su er exactly a cumulative delay     review of possible approaches to computing the net-
of TED.                                                    work delay of packets is given in 7]. One common
                                                           approach is to keep the source and the receiver clocks
Proof of Theorem 1 : Let i be the rst packet that          synchronized2. A time-stamp on the packets when
arrived. If it was produced at ti , from Lemma 1, we       they are produced, will allow us to compute the net-
see that the rst packet arrived at ti + TED. Any           work delay su ered by the packet. Another approach
packet (i + j ) will be produced at ti + Tr j , and        is to have the packet switches record propagation and
would lose its chance of being played back Tr j after      bu er delays in the packets. Note that none of the
i was played. Hence every packet that arrives within       lemmas/ theorem depend on knowledge of network de-
TED, will su er a cumulative delay of (ti + TED +          lays for packets received after the rst packet, because
j Tr ) ? (ti + j Tr ), the di erence between the time        2 Distributed clock synchronization schemes such as 6] can
of service and production, which is TED.                   maintain accuracy to within milliseconds.
of the use of sequence numbers to derive the timing.          the rst packet is serviced at t1 , at the local clock of
Hence certain optimizations can be made in case of            the receiver.
more knowledge of the network, and packets after the            1. When a packet p is inserted into the bu er, time-
  rst received packet need not carry the extra timing              stamp the packet with the clock timearrive at the
information, or have time stamps.                                  receiver. When it is being removed for service
    If one of the schemes in 7], cannot be implemented,            at timesvc, compute the bu er delay bdp to be
and clock synchronization algorithms are not feasible              timesvc ? timearrive. Since by Theorem 1, packet
we cannot obtain the network delay su ered by the                  p should su er a cumulative delay of TED, we
  rst packet. Hence, we cannot guarantee the optimal-              have the ndp su ered by the packet to be TED ?
ity of our algorithm since the rst slot of insertion can-          (timesvc ? timearrive ).
not be obtained implicitly. One alternative approach
considered is to use a Special Control Packet . If              2. If the ith packet arrives after TED, by Lemma 2,
the network can guarantee that the packet is delivered             the packet will not be inserted into the bu er
between Min; Min + ], then                                         since the packet's service time would have elapsed
                                                                   at time t1 +(i ? 1) Tr . By computing timeextra to
    if Tr insert the rst packet into the bu er slot,               be timearrive ? t1 + (i ? 1) Tr for the ith packet,
    denoted by END that will delay the packet to the               we see that the ith packet has su ered exactly
    maximum extent. Hence the TED su ered by the                   TED + timeextra in the network.
    packets is actually bounded by TEDoptimal + or
                                                              It should be noted that all the above times are by the
    if the stream is sequenced, and > Tr , insert             local clock of the receiver. The above scheme helps us
    the rst packet into END, and whenever a new               maintain the PDD of the network over any required
    packet i arrives, insert it into the appropriate slot     time window without any additional packet informa-
    o set from the PLAYED packet. If the slot com-            tion or undue computation. The user application can
    puted is higher than the size of the bu er by k,          be given access to this data, and can respond to in-
    shift all the packets to their right by k slots so that   crease/ decrease bu er size based on the application's
    the new packet is accommodated. Some packets              requirements.
    may drop o the left edge. This is because they
    have su ered excessive delay relative to the new          2.3 Dealing with Imperfect Clocks
    packet. In the beginning, the actual total end-to-
    end delay TEDbeg is bounded by TEDoptimal + .             2.3.1 Clock O set and Drift
    Over time, this can be reduced by the throwing            Note that we do not rely on the absolute value of the
    away of packets based on the arrival pattern of           clocks at the source and the receiver in our bu er con-
    packets as in 2].                                         troller mechanism. However, if we do choose to use
                                                              time-stamps to compute the network delay of the rst
2.2.5 Dynamic Operation                                       packet as opposed to other methods in 7], we will need
                                                              to worry about clock o sets. Let us say that the clock
As mentioned in Section 2.1, an initial estimate of           synchronization algorithms executed guarantee that
the network PDD is constructed based on informa-              the source and receiver's clocks are never o by more
tion such as history and/or the prevalent tra c con-          than . Hence the real network delay ndtrue of the
ditions at the time of stream setup. But over time,             rst packet is within ndcomputed ? ; ndcomputed + ],
the conditions of the network and the delays su ered          where ndcomputed is the network delay computed from
by the packets can change. If the PDD of the net-             the time-stamp. Choose TEDoperation, the intended
work changes, the assertions on the computed TED              total end-to-end delay in the range TEDoptimal ?
and the bu er sizes will no longer be valid. Hence             ; TEDoptimal + ] depending on the PLR/TED trade-
we need some mechanism to update the PDD of the               o metric of the user application, where TEDoptimal
system.                                                       is the optimal TED that would have been chosen
   It is trivial to compute the PDD of the network if         in absence of the -uncertainty. For instance, if one
the network delays of the packets are known. Since we         still wants to ensure the PLR is maintained, choose
do not assume time stamps in our packets, we do not           TEDoperation to be TEDoptimal + and when the rst
have an explicit value for the network delays. In this        packet arrives, assume the packet su ered the best
section, we will introduce a scheme to compute the            case delay of (ndcomputed ? ), and perform the ap-
network delays at the receiver end. Let us say that           propriate bu er initializations. Better bounds on
will of course ensure increased accuracy of the com-                      Consider only the 2factor that depends on i, which
puted network delay of the rst packet. Since none of                      is i T p?TT 2 pp?T T .
                                                                                  2     2
                                                                                     r         s       s   r

the lemmas/ theorem need the network delay informa-                                            r

tion for subsequent packets, even if the time-stamping                    The easiest way to make the bu er size to be in-
clocks do drift over time, the bu er mechanism is not                     dependent of i, is to set the above term to zero,
a ected.                                                                  and solve p to be (Ts2 Tr )=(Tr2 ? Ts2). Hence
                                                                          throwing one packet away every p secs, ensures
2.3.2 Drifting Production and Service Rates                               that the bu er size is bounded.
Typically, there are clocks associated with the device                    Source Slower than Receiver: In this case,
generating the packets and the receiver that processes                    the receiver services the packets faster than the
the packets. The bu er controller algorithm presented                     source produces. The receiver will be ready to ser-
thus far depends on the device clocks of the source                       vice the ith packet at time ti = TED +(i ? 1) Tr
and receiver not drifting. That is, rate of production                    after the rst packet was produced. The number
and service is assumed to be the same and to remain                       of packets that would have been produced by time
constant. In practice, physical clocks may operate at                     ti would be (ti Tr =Ts2). A packet should have left
slightly di erent rates, which over time can lead to                      the source TED before its due time at receiver.
bu er overrun or over ow at the receiver and cause a                      Hence the number of packets that would reach
loss of synchronization. In this section, we make a re-                   the EXIT slot in the bu er by ti is packetarrive
laxation to allow for device clock drifts, and describe                   = b ((t ?TED) T ) c.
                                                                                    T2     s

a technique which makes infrequent adjustments to                         We need to have (packetarrive ? i) zero so that
stream delay at the receiver in order to compensate                       the receiver can always process a packet. Again
for drift e ects. Observe that a clock drift of sender or                 we see that the di erence depends on i, and we
receiver can be considered a drift of the sender relative                 do not have bounds on the length of a multimedia
to the receiver. Hence we can work with the assump-                       conversation. One possibility is to add a dummy
tion that the receiver does not drift. Let Ts and Tr                      packet, or make a copy of another packet every p
be the device period of the source and receiver, that                     secs so that the source- receiver relation is main-
is the rate of production and service respectively.                       tained such that
     Source faster than Receiver: If this sce-
     nario is left unchecked, the source will ood the
                                                                          b (ti ? TED) Tr c ? i + b (ti ? TED) Tr c 0:
                                                                                         Ts2                   Ts p
     receiver and the bu er will over ow. Consider
     the ith packet. The bu er size can be exceeded                        We can make the above equation independent of
     i the bu er is already full, and the ith packet                       i, when p is (Tr2 Ts )=(Ts2 ? Tr2). So every p secs,
     arrives before any packet is serviced. The worst                      we insert an extra packet into the bu er.
     case is when the ith packet reaches the bu er at                    In devices and applications, where the exact drifts
     time i Ts + min. Due to the drift, if the source's               are known, we can compute the exact rate at which
     device is at time t, the receiver's clock will be at             packets can be thrown or dummy packets be inserted.
     time (t Ts )=Tr . Similar to the section on com-                 If the exact drift is not known, but the drift can be
     puting the bu er size, we see that the bu er size                bounded, we can resort to choosing the earliest time
     is actually                                                      obtained by computing the times associated with the
     maxbufsize = i ? b (i T +min?TED) T c:
                                                                      various combinations of direction of drift and maxi-
                                                                      mum drift in the two cases. The bu er controller can

    Clearly, the above depends on i, the number of                    then have an estimate on when to check the bu er,
    packets sent. However, for real-time applications,                and depending on how full the bu er is, the user ap-
    an advance bound cannot usually be obtained on                    plication will have the exibility to insert or throw
    the length of the conversation. One possible ap-                  away a packet.
    proach is to throw away a packet every p secs so
    as to maintain the bound on the number of bu er
    slots required. Therefore, the new maxbufsize                     3 Multiple Stream Extension
    = i?b (i T +min?TED) T c?b (i T +min?p ) T c
                                                                        In the previous section, we considered how to com-
       i ? (i T +min?TED) T ? (i T +min?pTED) T +2.
                   T2 r
                                                                      pute the total end-to-end delay and the bu er size for
a single stream case. In this section, we consider the          Otherwise, choose a t 2 (max ti ? ; min MADi )
multi-stream case where N streams of data are being             and TEDi 2 (t; t + ) \ (ti ; MADi ), so as to min-
sent over di erent links from multiple sources Si , i =         imize the desired cost metric.
1, 2 : : : N, to a common receiver R. Computing the
TED for a set of streams is similar, but more involved          Compute bu er size Bi of stream i to be (TEDi ?
due to the PLRi and MADi requirements for each of               mini )=Tr .
the N streams. We will rst consider a simple case of
synchronizing the N streams such that:                      3.3 Semantics of Streams
    TED is the same for all the N streams.                     The TED should be chosen depending on the re-
                                                            lation between streams since this will give us more
    TED MADi for i = 1, 2 : : : N.                          information about what choices are allowable to im-
    The packet loss rate for stream i is bounded by         prove the synchronization mechanism.
    PLRi , for i = 1, 2, : : : N.                                Master-Slave Streams: In some cases, one
                                                                 stream is designated the master stream, and the
    TED is the minimum t satisfying the above con-               rest of the streams, termed the slaves, are main-
    ditions. (Other metrics de ned in section 3.1 are            tained in synchronization with the master. In this
    also clearly possible.)                                      scheme, all received packets of slaves are thrown
                                                                 away if the master's packet is not received. If the
3.1 Computing Operational Parameters                             master packet is received, then all the available
                                                                 slave packets are played with the master packet.
   We now determine the system operation metrics ac-             The PLRsystem of the system is typically speci-
cording to the following steps:                                    ed. This is merely a special case of the system
    As before, calculate each stream delay ti =                  described in section 4 with the PLRmaster being
    Cdfi?1 (1 ? PLRi ) for each i = 1, 2, : : :, N.              the PLRsystem , and the PLRslave being set to
    Choose TED to be the maximum of the com-                     Inter-Dependent Streams: Sometimes, the
    puted ti , i = 1, 2 : : : N, hence satisfying all PLR        streams are related such that even if a packet on
    requirements.                                                one stream is late, the corresponding packets in
    If TED > MADi for any i = 1, 2, : : : N, there is            other streams are to be thrown away in an all-or-
    no feasible solution for these stream parameters.            nothing format. The PLRsystem is then speci ed
                                                                 to be satis ed. The joint cumulative distribution
    Choose Bi , the bu er sizes for the N streams to             function of the PDD0 s of the streams will then
    be (TED ?mini )=Tr , where mini is the minimum               be useful. For instance, if there are n streams X1 ,
    network delay for sources Si , i = 1, 2, : : : N.            X2 , : : :, Xn that need to be synchronized in this
3.2 Relaxing the Delay Constraints                                  { Compute Cdf (t) to be JX1 t;X2 t;:::X t    n

   In some applications, streams have less strict con-                 which speci es the probability that the pack-
straints regarding their synchronization. For example,                 ets of all streams arrives within some time t.
audio streams may be required to be within 100 msec                    Hence (1 ? Cdf (t)) speci es the probability
of related video streams. Using this extra degree of                   that packets are to be thrown away.
freedom, some otherwise infeasible requirements can                 { Find TED to be minimum t that satis es
be satis ed. We consider the case where the TEDs                       (1 ? Cdf (t)) PLRsystem .
between any two streams are permitted to be at most              Redundant Streams: Critical transmissions
   apart:                                                        are sometimes replicated and transmitted so that
    Compute ti = Cdfi?1 (1?PLRi ) for the ith stream             the receiver can mix the streams by some sort of
    as in the previous section.                                  a majority scheme. In this case the BC needs
                                                                 to choose TED to be the minimum t satisfying
    The constraints are infeasible if max ti ?         >         JX1 >t;X2 >t;:::X >t PLRsystem since the com-
    min MADi .                                                   puted joint probability de nes the probability
     that none of the packets are received in time to           The rate of packet production, and packet service
     be used for playing back.                                  were assumed to be one packet every 33 msecs.
     Hybrid Streams: Several streams having hy-                 Each of the streams were assumed to have a max-
     brid relations can also be speci ed in this scheme.        imum PLR to be 5%, and their maximum accept-
     For example, if there are n sets of streams, and           able delays were 250 msecs.
     there are mi streams in the ith set that follow
     a master-slave relation, while the sets themselves       We show below the most interesting plots for the
     are inter-dependent or redundant, we can nd the       cases when the variance, packet loss rates and the rate
     TED as follows.                                       of production were varied.
        { For the ith set, set the PDDi to be that of           In Figure 4, we varied the rate of service and pro-
           the master stream. i = 1, 2, : : : n.                duction for the three packets from 1 msec through
        { Compute the TED with these new PDDi as                100 msecs, while xing the rest of the parameters.
           de ned in the inter-dependent or redundant           We plot the bu er sizes required to service the
           scheme.                                              packets at the given rate. As expected, the bu er
                                                                sizes are maximum when the rate of production
     Other combinations can be addressed similarly.             is minimum.
   In multimedia synchronization, related streams can           In Figure 5, we varied the max PLR of the third
have di erent rates of production. For instance, in             stream from 1% through 15%. We plot the change
video-conferencing four audio packets may be gener-             in TED for the given PLR's. The TED is maxi-
ated for every video frame produced. This relation              mum for the low PLR3 , and beyond the 5% mark,
can be modeled at the receiver's end that four audio            stays constant since the other two streams have
packets are to be serviced for every video frame ser-           maximum acceptable PLRs of 5%.
   Let us consider the lip-syncing example where                In Figure 6, we change the variance of the third
audio packets are to be serviced for every video packet.        stream from 1 msec through 25 msecs. As ex-
Modify the bu er size computing algorithm as follows.           pected the TED increases beyond the variance of
                                                                5 msec. This is because until that point, the vari-
     Compute the TED based on the semantic relation             ance of the third stream is dominated by the rst
     between audio-video.                                       and second streams. Beyond the 5 msecs point,
     Compute the size of the audio stream's bu er to            the third stream is the dominant factor.
     be (TED ? Minaudio)=Taudio .
     Compute the size of the video stream's bu er to
     be (TED ? Minvideo )= Taudio .                        5 Related Work
                                                              Early work on packetized voice studied the use of
4 Simulation Results                                       a receiver bu er to smooth jitter 3, 7]. A number
                                                           of experimental systems such as 1, 10], used a time-
   This section presents some early simulation results     preserving scheme. More recent work has produced
of Concord based on a system with the following char-      a scheme for calculating accurate delay estimates 2]
acteristics:                                               in the absence of global clocks, and compared poli-
                                                           cies for changing the bu er size in packet-preserving
     Three streams are to be synchronized perfectly.       operation 8].
     That is, they share the same TED.                        For the most part, work on multiple stream syn-
     The three streams are sent over di erent links        chronization has been looked at separately from the
     from di erent sources.                                single stream requirements. Some proposals, such
                                                           as 4] assume that reasonable upper bounds on the de-
     The packet delay distributions for each of the        lay across streams are available, while other schemes
     links followed a normal distribution with means of    set a xed upper delay bound for a stream, discard-
     100, 90, 100 msecs. The respective variances were     ing data which arrives late and would cause a loss
     10, 5, and 5 msecs. The minimum link speeds           of synchronization 11]. Other e orts have looked at
     were assumed to be 5, 10, and 5 msecs.                describing synchronization using some form of data
                                                                             frames, relying on the underlying transport protocol
                          100                                                to constrain jitter and delay 5, 9]. The added compli-
                           90                                                cations of dealing with streams from di erent senders
                                                                             was discussed recently in 12] as the inter-participant
                                                            Stream 1
                                                            Stream 2
                           80                               Stream 3
                                                                             synchronization problem.
 Buffer Size (packets)

                                                                             6 Conclusion
                                                                                 This paper presented the Concord algorithm for
                                                                             synchronizing networked multimedia streams. It is no-
                                                                             table because it de nes a common framework to deal
                                 0   10 20 30 40 50 60 70 80 90 100          with both single and multiple stream synchronization,
                                           Service Period (msecs)
                                                                             and operates under the in uence of parameters which
                                                                             can be supplied by the application involved. In par-
Figure 4: E ect of changing the rate of service                              ticular, these parameters are used to allow a trade-o
                                                                             between the packet loss rates, total end-to-end delay
                                                                             and skew. Thus, an application can directly indicate
                                                                             an acceptable lost packet rate, rather than having the
                         105.2                                               synchronization mechanism operate by always trying
                                                            Stream 3         to minimize losses.
                                                                                 We have described the details of the basic algorithm
                                                                             and its extension for dealing with multiple stream syn-
                                                                             chronization. Issues of calculating the receiver bu er
 TED (msecs)


                         104.4                                               size and estimating network delays were discussed. In
                                                                             addition, features for dealing with imperfect clocks
                                                                             and dynamic network behavior were described. Some
                         104.0                                               early simulation results were presented; future work
                         103.8                                               will include a more extensive evaluation with real traf-
                         103.6                                                 c traces.
                             0.00 0.02 0.04 0.06 0.08 0.10 0.12 0.14 0.16
                                                PLR (%)

                         Figure 5: E ect of changing the PLR                 References
                                                                              1] S. Ades et al. Protocols for Real Time Voice Com-
                                                                                 munication on a Packet Local Network. In Pro-
                                                                                 ceedings of IEEE ICC '86, pages 525{530, June
                         108.5                                                   1986.
                         108.0                              Stream 3
                                                                              2] F. Alvarez-Cuevas et al. Voice Synchronization
                                                                                 in Packet Switching Networks. IEEE Network,
                                                                                 pages 20{25, September 1993.
 TED (msecs)

                         106.0                                                3] G. Barberis and D. Pazzaglia. Analysis and Op-
                         105.5                                                   timal Design of a Packet-Voice Receiver. IEEE
                         105.0                                                   Transactions on Communications, 28(2):217{227,
                         104.5                                                   February 1980.
                                                                              4] J. Escobar et al. Flow Synchronization Protocol.
                                 0      5        10       15       20   25       In Proceedings GlobeCom '92, pages 1381{1387,
                                            Delay Variance (msecs)               1992.
Figure 6: E ect of changing the delay variance                                5] L. Li et al. Synchronization in Real Time Mul-
                                                                                 timedia Data delivery. In Proceedings of IEEE
                                                                                 ICC '92, pages 587{591, June 1992.
 6] D. Mills. Internet Time Synchronization: The
    Network Protocol. IEEE Transactions on Com-
    munications, COM-39(10):1482{1493, October
 7] W. A. Montgomery. Techniques for Packet Voice
    Synchronisation. IEEE Journal on Selected Ar-
    eas in Communications, 1(6):1022{1028, Decem-
    ber 1983.
 8] R. Ramjee et al. Adaptive Playout Mechanisms
    for Packetized Audio Applications in Wide-Area
    Networks. In Proceedings of IEEE INFOCOM
    '94, pages 680{688, June 1994.
 9] K. Ravindran and V. Bansal. Delay Compen-
    sation Protocols for Synchronization of Multime-
    dia Data Streams. IEEE Transactions on Knowl-
    edge and Data Engineering, 5(4):574{589, August
10] D. C. Swinehart et al. Adding Voice to an Of-
      ce Computer Network. In Proceedings of IEEE
    GlobeCom '83, November 1983. Also available as
    Xerox PARC technical report CSL-83-8, Febru-
    ary 1983.
11] R. Yavatkar. MCP: A Protocol for Coordina-
    tion and Temporal Synchronization in Multime-
    dia Collaborative Applications. In Proceedings
    of IEEE International Conference on Distributed
    Computing Systems, pages 606{613, June 1992.
12] P. N. Zarros. Statistical Synchronization Among
    Participants in Real-Time Multimedia Confer-
    ence. In Proceedings of IEEE INFOCOM '94,
    pages 912{919, June 1994.

Shared By: