Lee, et al.: Design and Implementation of a Predictable High-Throughput Video Conference Recorder 63
DESIGN AND IMPLEMENTATIONOF A PREDICTABLE
HIGH-THROUGHPUT VIDEO CONFERENCE RECORDER*
Meng-Huang Lee,’S2 Chia-Hsiang Chang,’ Meng Chang Chen,’
Jan-Ming Ho,’ Ming-Tat KO,‘
Yen-Jen Oyang,* Kuo-Hui Tsai,’ and Shie-Yuan Wang’
Abstract tems. The finding motivates us to design a new file
system for video conference recording.
In this paper, the design and implementation of a
video conference recorder on an inexpensive PC is
described. The recorder can simultaneously record Although recording has many similarities with
several video and audio (A/V) streams in Unix file movieplayback [l,6 , 7 , 9 , 10, 11, 12, 14, 15, 161, such
format. The objectives of the file system design are as requirement of high throughput and predictable
high throughput and low loss rate. A hard disk de- response from file systems, there are some dissimilar-
vice driver and a file system are modified and tuned ities. In the movie playback, the d a t a fed from disk
for A/V recording. The new file system can have is more stable than the d a t a fed from video confer-
up to 60% performance improvement over the tra- encing via network. Movie playback can read ahead
ditional file system. Performance results from our and cache video/audio (A/V) streams in buffer to
experiments are presented and discussed in the pa- avoid jitter, while video conference recording can-
per. not assume having sufficient buffer for incoming data
because of the unpredictability of network commu-
nication. File layout on disk is another discrepancy.
1 Introduction Movie playback requires A/V frames well arranged
on the disks in order to be read efficiently. For video
As the advances in networking and computers, it conference recording, the file layout is not a con-
becomes feasible for people to collaborate remotely cern as long as data can be efficiently written into
using multimedia computing. Among them, video disk. Just for the dissimilarities discussed above, the
conference application is one of the most promising file system for movie playback cannot be suitable for
mechanisms. Video conference recording is an in- recording.
dispensable function of video conferencing that al-
low users to review the conference instantly or later. An application of the video conference recording
There have been much research on video conferenc- technology is video caching of a multi-level storage
ing [3, 5, 81. However, to the best of our knowledge, hierarchy of video on demand systems. In [4, 131, a
there is no publication specifically discussing video multi-level storage hierarchy is proposed for video on
conference recording. demand applications, in which hard disks are used to
Video conference recording, at the first glance, cache video d a t a downloaded from a video jukebox.
seems a simple disk 1/0 problem. After a careful The key technology shared by video cache and video
study and experiments, we find out lower-end work- conference recording is how t o write A/V frames to
stations, such as PC’s, cannot simultaneously dis- disks without packet loss. Apparently, the results of
play and record a video conference without jitter or this project can be applied to video caching.
data loss. The problem is due to low throughput
and unpredictable response of traditional file sys-
In this project, we focus on the implementation
*This work has been supported in part by the National of video conference recorders on inexpensive PC’s.
Science Council under Grant NSC84-2221-E-001-005. In Section 2, the design and implementation of the
Institute of Information Science, Academia Sinica, recorder are described. Experiments and their re-
Department of Information Engineering and Computer sults are presented in Section 3. Finally, the conclu-
Science, National Taiwan University, Taiwan sion is in Section 4.
Manuscript received December 12, 1995 0098 3063/96 $04.00 1996 IEEE
64 EEE Transactions on Consumer Electronics, Vol. 42, No. 1, FEBRUARY 1996
2 System Design and Imple- 5003
ReadWnle Throughputs v s Dillerenl Inleileaving Facolr
The goal of the design of t h e video conference
recorder is to provide a recording facility with high
throughput and low loss rate. In the context of a f -
multi-user , multi-session video conference, there are 5
many audio/video (A/V) streams periodically flow- a
ing into a user’s workstation. T h e amount of d a t a f m
is large. For instance, a JPEG compressed video 150
stream with 25 frames/second requires a bandwidth 1wO
of 1 2 MB/second. Because o f the unpredictability
of network communication, A/V frames may arrive
a t workstation in a burst that will occupy a large 0
amount of buffer. The recorder has t o promptly InleileavingFactor
store the A/V frames into disks from buffer. Oth-
erwise, the buffer will be used up and, as a result, Figure 1: Throughput vs. Interleaving Factor (on
frames will be lost. Hence, a file system with high disk zone=1).
throughput and large buffer can reduce the chance of Read Ihroughputs Y s dillerenl block size
frame loss. Another reason for frame loss is the un-
predictable response of file systems. T h e traditional
file systems have been optimized for relatively smail-
sized text files in a multi-user environment which are
not efficient for handling large amounts of periodical
data In some adverse situations, writing a frame t o
disk requires several disk 1/0 operations in conven-
tional file systems t h a t greatly increases the chance
of frame loss.
In the design of the recorder, we aim t o increase
the throughput and reduce the unpredictability t o
optimize system performance. Since the target work-
stations are inexpensive PC’s, customized hardware
solutions for recorders are not considered. Rather,
0 8 16 24 32 40 48 56 64 72 80 88 96 104 112 120 128 136
we examine the operating system (Mach 2.6), and Blocksire (KO)
propose our enhancements. T h e major work is on
the hard disk device driver and file system. Figure 2: Throughput vs. Block Size (interleaving
2.1 Performance Tuning on Hard
Disk Driver without filling, the hard disk performs the best in
all contexts. T h e reason is when d a t a are sequen-
AHA-2842VL VL-bus to SCSI/II host adapter is se- tially stored in disk, the overhead of disk seeks can
lected in the research. As Mach 2.6 Operating Sys- be reduced. Figure 2 shows the throughput of disk
tem does not support the adapter, we have t o write 1/0 is saturated when d a t a block size (i e. the size
a driver for the adapter t,hat give us an opportunity of 1/0 request) reaches 64KB which is the track size
to fine tune the performance. Several parameters of Zone 1 of the disk. If the size of d a t a block is less
of the device driver might affect hard disk perfor- t h a n 64KB, some disk 1/0 capacity is wasted; while
mance, including interleaving factor, block size and the size exceeds 64KB, the d a t a other than the first
disk zone (i.e. physical position on hard disk). Be- 64KB is written t o the next track that does not out-
fore setting values of the parameters in t h e device perform sequential write of 64KB blocks. Besides,
driver, we run extensive experiments t o understand larger d a t a block implies a larger buffer. Thus, we
the effect of each parameter. T h e performance d a t a set the block size as 64KB.
from experiments are list in Fig 1, Fig 2, and Fig As the outer zone has higher capacity than the
3 respectively. inner zones, it is natural that the throughput of
From Figure 1, when the interleaving factor is set outer zones is higher, as shown in Figure 3 . How-
as 1, i.e. blocks are contiguously laid on the disk ever, many conventional file systems (such as UNIX
Lee, et al.: Design and Implementation of a Predictable High-ThroughputVideo Conference Recorder
OndDuai disk thmughpuls at differentzones
mw blocks are generated and written t o disk only when
5 all their linked d a t a blocks are written to disk.
5ow A large chunk of contiguous blocks in disk are re-
served for recording. A/V streams are written se-
quentially t o the next available block of the chunk
that it becomes unnecessary t o search for free blocks.
A buffer pool is shared between the recorder and the
I / O thread, depicted in Figure 4, that the synchro-
nization between them can be done by lock/unlock
operations on the buffers. T h e recorder uses asyn-
chronous 1/0 operations t o eliminate the waiting
time for 1/0 completion which can increase 1/0
2.3 Video Conference Recorder
Figure 3: Throughput vs. Different Disk Zone (in- The recorder consists of several recording threads,
terleaving factor=l). a file system server thread , an 1/0 thread and a
buffer pool (see fig 4). Each thread is responsible
for recording an A/V stream. Contrary t o the fact
SVR4.0) format the disks t o have equal number of that BSD file system of Mach 2.6 resides in the ker-
sectors in each track and reserve the extra sectors nel, the recorder runs as a user process. When a
in the outer zones as spares. As a result, through- recording request is admitted, a recording thread is
put of the conventional file systems is indifferent created and a number of buffers are assigned. The
from the zones. In our design, we relax the con- admission criteria include system load, 1/0 capacity,
straint of having equal number of sectors per tracks disk capacity and available buffers. The number of
to have a higher throughput. Another experiment, buffers assigned a t the creation is the minimum re-
as in Figure 3, shows that two disks attached t o same quirement for the recording, which can be increased
bus easily overload the bus. Consequently, the total as more buffers become available.
throughput of the two disks maintains a t the peak Frames are attached with the arrival time a t the
transfer rate of the bus. workstation for synchronization of A/V streams in
playback. The frames stored in the buffer pool are
2.2 Design of File System organized as a time-ordered queue. When all the
buffers are occupied, a frame in the buffer pool will
In this project, we modify a UNIX SVR4.0 file sys- be dropped when a new frame arrives. The frame t o
tem for A/V stream recording, which has features be dropped is selected in a leapfrog fashion. T h a t
inefficient for recording, such as excessive read/write is, a t first, the oldest frame is dropped. Then the
metadata(i.e. superblock and inodes) from/to disk, third oldest, and so on. T h e apparent advantage of
access of indirect blocks, search for free blocks and this arrangement is t o avoid jitter by not dropping
synchronous 1/0 operations. adjacent frames. Note that audio and video frames
For maintaining consistency between d a t a blocks are treated differently that audio frames have higher
and metadata blocks, conventional file systems read priority not t o be dropped.
and then write metadata blocks t o disk after updat- The file system server thread serves all the record-
ing the metadata, when the contents of d a t a blocks ing threads and writes frames t o buffer pool. In ad-
are changed. These read and write operations can be dition, it schedules 1 / 0 requests and manages the
delayed or reduced for A/V streams recording, since buffer pool in a first come first serve order. 1/0
the files are single-writer write-only during record- thread reads/writes frames from/to disk using raw
ing that race condition is impossible. Crash recov- device 1/0 in a scanning order t o reduce waiting
ery is the other reason for maintaining consistency. latency. The frames are written t o the reserved con-
However, crash recovery is not a problem in this tiguous partition of disks. T h e optimization of file
application which is explained in the next section. layout for playback is not considered in the recorder
As d a t a blocks in UNIX SVR4.0 file system may be design since off-line processes can easily optimize the
linked via indirect blocks of various length, writing recorded A/V streams layout for playback. As the
a data block requires various number of 1/0 oper- A/V streams are stored as a time-ordered sequential
ations for updating indirect blocks that makes the file, it is straight forward t o rebuild metadata blocks
response time unpredictable. In the design, indirect from the stored A/V frames. Hence, crash recovery
66 IEEE Transactions on Consumer Electronics, Vol. 42, No. 1,FEBRUARY 1996
recording requests fro
Figure 4: T h e Recorder Architecture
is not a concern. 200
Accumulated Packets Loss at time = 8 Sec.
3 Experiments 'mach-bufLZM8
150 ,x ,
Accumulated Packets Inss at tine = 6 Sec. 1
10 12 14 16 20
number af streams
Figure 6: Accumulated Packet Loss a t time = 8 sec.
10 12 14 16 18 20
number of streams buffer pool of various sizes (ZME, 2 h B , and 4MB)
and runs on both Mach/BSD and our modified file
Figure 5: Accumulated Packet Loss at time = 6 sec. system. We measure the packet loss rate every 2
seconds. At the 6th second, it can be observed from
Figure 5 t h a t when with less than 10 input streams,
In the section, we will present experiment results recorder does not lose any packet i n all contexts.
showing the performance of the video conference When with 11input streams, the recorder with 1MB
recorder, and compare the results with Mach/BSD buffer running on Mach/BSD begins t o drop packets,
file system. The experiments run on a PC with 486 while with 2MB buffer running on Mach/BSD can
DX2-66, 16MB memory, HP C3323A SCSI disks, keep excessive packets in the buffer without drop-
AHA2842VL VL-to-SCSI/II adapter a n d Mach 2.6 ping them. With more t h a n 19 streams, packet loss
operating system. appears on all contexts.
In the experiments, a packet generator is designed At the 8th and 10th seconds, we can see from Fig-
to generate streams, of which the rate of each stream ure 6 and Figure 7 respectively t h a t the numbers of
is 200 KB/s, t o simulate the incoming data. T h e packet loss were accumulated. T h e recorder running
packet size of the modified file system is 64 KB, on the modified file system performs much better
while Mach/BSD file system only allows a maximum than on MachlBSD. Coincident with intuition, the
packet size of 8 KB. The recorder is assigned with a recorder with larger buffer has less packet loss. Af-
Lee, et al.: Design and Implementation of a Predictable High-Throughput Video Conference Recorder
Accumulated Packets Loss at tine i 10 sec 3.2 MB/s is sufficient t o support up t o 17 1.5 Mb/s
'OUr-bUf=lm' + MPEG streams or 16 recordings of 200 KB/s A/V
250 'our-buf=4W - 0 . .
streams of video conferencing.
~ 200 Write Response Time Analysis
10 12 14 16 18 20 a
number of streams
Figure 7: Accumulated Packet Loss a t time = 10
Picket Loss per Second after 10 Seconds
Figure 9: Response Time Distributions
'our-buf-4MB' - 0 . -
'mach-buf=2MB' * -
'mach-buf=4mk -X- - In examining the predictability of the modified
# , I
system, the response times of each 1/0 requests in
the previous experiments are saved. From the graph
depicted in Figure 9 , it shows the response time of
the modified file system is very predictable. T h a t
is, 92.5% of responses are within 10 ms, and the
other 6.5% within 20ms. Mach/BSD does not per-
form so predictable t h a t only 5% within 10ms, and
60% within 20ms.
Figure 8: Packet Loss per Second after 10 Seconds. In this paper, we describe the design and implemen-
tation of a video conference recording facility on in-
expensive workstations. We first study the problems
ter 10 seconds, the file system reaches a n equilibrium of tradition file systems in supporting A/V stream
state that packet loss rate maintains a t a constant recording and then propose our enhancements. Our
level for each context. Figure 8 shows the statistics proposal includes tuning hard disk device driver and
after 10 seconds that the packet lost per second is modifying UNIX SVR4.0 file system. T h e goal of our
roughly equivalent t o the number of packet exceed- modification is to increase throughput and reduce
ing the processing capability of the file system, inde- unpredictability, which are verified from the experi-
pendent from the size of buffer. The result verifies ments. T h e modified file system has a n improvement
the theory that the buffer is only for accommodating of M 60% over the traditional file system.
the burst incoming A/V frames, but not to increase We observe that the size of buffer pool does not
throughput or reduce the loss rate after the system affect the packet loss rate when system reaches an
reaches an equilibrium state. equilibrium state. In other words, system overload
From Figure 8, it can be observed that Mach/BSD is the major reason for packet loss. However, the
file system can only handle 10 streams without larger the size of buffer is, the better the recorder
packet loss, i.e. a throughput of 2 MB/s. T h e modi- perform against anomaly, such as burst input. In
fied file system can run as many as 16 streams with- order to prevent system from overloaded, a strict
out packet loss, which is equivalent t o a throughput admission control is needed, of which the criteria
of 3.2 MB/s. The improvement of the modified file can be obtained via experiments similar t o the one
system is 60%. A recorder with the throughput of described in this paper.
68 IEEE Transactions on Consumer Electronics, Vol. 42, No. 1, FEBRUARY 1996
The video conference recorder is part of the ASIS [I31 W.D. Sincoskie. System architecture for a large
conferencing system [a] developed in this lab. Dur- scale video on demand service. Computer Networks
ing a video conferencing with 4 sessions, experi- and I S D N Systems, 22:155-162, 1991.
ments show that the recorder can record the 4 ses-  F.A. Tobagi and J. Pang. StarWorksTM - a video
sions without overloading the conferencing system applications server. In Proceedings Thirty-Eighth
and causing any packet loss on the workstation. I E E E Computer Society Int. Conf., COMPCON’93,
Currently we look into the problem of reading February 1993.
A/V frames from and writing to file system at the 1151 James Yee and Pravin -Varaiya. An analytical model
same time, that is useful for users reviewing ear- for real-time multimedia disk scheduling. In Proc.
lier session of the conference during the conferenc- Third Symp. o n Operating S y s t e m Supports f o r A u -
ing. The problem is more difficult since the perfor- dio and Video, June 1993.
mance balance between reading and writing has t o  Philip S. Yu, Mon-Song Chen, , and Dilip D.Kand-
be achieved. lur. Design and analysis of a grouped sweeping
scheme for multimedia storage management. In
Proc. Third S y m p . o n Operating S y s t e m Supports
eferences for A u d i o and Video, pages 44-55, June 1993.
[I] D.Anderson, Y. Osawa, and R. Govindan. Real-
time disk storage and retrieval of digital audio
and video. Technical report, U.C. Berkeley, 1991.
UCB/ERL Tech. Rep. M91/646.
 C.-H. Chang, M.C. Chen, J.-M. Ho, M.-T. KO,
K.-H. Tsai, C.-F. Wang, and D. Wang. ASIS -
Academia Sinica Multimedia Interactive System.
Technical report, Inst. of Info. Sci., Academia
Sinica, December 1994.
 S.E.Deering. Host extensions for IP multicasting.
Technical report, August 1989. RFC-1112.
 Craig Federighi and Lawrence Rowe. A distributed
hierarchical storage manager for a video-on-demand
system. In Proc. S P I E , 1994.
 R. Frederick. nv - UNIX manual page. Technical
report, Xerox Parc, 1992.
 Jim Gemmell and Stavros Christodoulakis. Princi-
ples of delay-sensitive multimedia d a t a storage and
retrieval. A CM Transactions o n Information Sys-
tems, January 1992.
 Shahram Ghandeharizadeh and Luis Ramos. An
overview of techniques t o support continuous re-
trieval of multimedia objects. A C M , SIGOPS, 1993.
 H.Schulzrinne. A transport protocol for audio and
video conferences and other multiparticipant real-
time applications. Technical report, 1992. Internet
 P. Lougher and D. Shepherd. T h e design of a storage
server for continuous media. T h e Computer Journal,
[IO] P. Venkat Rangan and Harrick M. Vin. Efficient
storage techniques for digital continuous multime-
dia. IEEE Transactions o n Knowledge and Data
Engineering, 5(4), August 1993.
[I11 P. Venkat Rangan and Harrick M. Vin. Designing
an on-demand multimedia service. I E E E Commzs-
nications Magazine, July 1992.
[I21 Lawrence Rowe and Brian Smith. A continous me-
dia player. In Proc. Third Workshop o n Network
and OS Supports for Digital A u d i o and Video, 1992.
Lee, et al.: Design and Implementation of a Predictable High-ThroughputVideo Conference Recorder
Meng-Huang Lee is currently a Ph.D. student in the Department of Com-
puter Science and Information Engineering of National Taiwan Univer-
sity. He received the B.S. degree in Electrical Engineering from National
Cheng Kung University in 1987, and the M.S. degree in 1989. His major
research interests include expert systems, multimedia storage systems,
and real-time file systems.
Chia-Hsiang Chang received a B.S. in computer science from National
Chia-Tung University in 1980. He received a M.S. and a Ph. D. degree in
computer science from New York University in 1986 and 1992, respec-
tively. From 1982 to 1984, he joined Institute of Information Science
as a research assistant. From 1985 to 1988, he worked for department
of physics, New York University. Since 1992, he joined Institute of In-
formation Science as an assistant research fellow. Dr. Chang's research
interest includes operating system, communication, geographical infor-
mation system, programming language, program transformation and the
design and analysis of algorithm.
Meng Chang Chen received the B.S. and M.S. degrees in computer sci-
ence from National Chiao-Tung University, Taiwan, in 1979 and 1981,
respectively, and the Ph.D. degree in computer science from the Univer-
sity of California, Los Angles, in 1989.
He joined AT&T Bell Labs in 1989 as Member of Technical Staff and
led several projects in the area of data quality problems of databases. He
was an Associate professor at Department of Information Management,
National Sun Yat-Sen University, Taiwan, from 1992 to 1993 when he
joined Institute of Information Science, Academic Sinica, Taiwan. He is
an Assistant Research Fellow now.
His current research interests include multimedia systems, database sys-
tems and query processing, operating systems, distributed processing,
groupware, inexact reasoning, knowledge discovery and knowledge rep-
Jan-Ming Ho received his Ph.D. degree in electrical engineering and
computer science from Northwestern University in 1989. He received
his B.S. in electrical engineering from National Cheng Kung Univer-
sity in 1978 and his M.S. at Instititute of electronics of National Chiao
Tung University in 1980. He joined Institute of Information Science,
Academia Sinica, Taiwan, R.0.C as a associate research fellow in 1989,
and is promoted to research fellow in 1994. He visited IBM T. J. Watson
Research Center in summer 1987 and 1988, Leonard0 Fibonacci Institute
for the Foundations of Computer Science, Italy, in Summer 1992, and
Dagstuhl-Seminar on "Combinatorial Methods for Integrated Circuit De-
sign", IBFI-Geschuftsstelle, SchloP Dagstuhl, Fachbereich Informatik,
Bau 36, Universitut des Saarlandes, Germany, in October 1993.
70 LEEE Transactions on Consumer Electronics, Vol. 42, No. 1, FEBRUARY 1996
His research interests include realtime operating systems, realtime multi-
media applications, e.g., video conference and video on demand, compu-
tational geometry, combinatorial optimization, VLSI design algorithms,
and implementation and testing of VLSI algorithms on real designs.
Ming-Tat KOreceived a B.S. and an M.S. in mathematics from National
Taiwan University in 1979 and 1982, respectively. He received a Ph.D.
in computer science from National Tsing Hua University in 1988. Since
then he joined the Institute of Infromation Science as an associate research
Dr. KO’s major research interest includes the design and analysis of
algorithm, computational geometry, graph algorithm, real-time systems
and computer graphics.
Yen-Jen Oyang received the B .S. degree in Information Engineering from
National Taiwan University in 1982, the M.S. degree in Computer Science
from the California Institute of Technology in 1984, and the Ph.D. degree
in Electrical Engineering from Stanford University in 1988. He is cur-
rently an Associate Professor in the Department of Computer Science and
Information Engineering, National Taiwan University. His research inter-
ests include mobile computing systems, multimedia storage systems, and
multiprocessor systems. He can be reached at email@example.com.
Kuo-Hui Tsai received a B.S. in electrical engineering from National
Tsing Hua University in 1982, an M.S. in computer engineering from
National Chiao Tung University in 1984, and a Ph.D. in computer science
from Northwestem University in 1991. From 1986 to 1988, he worked
as an assistant researcher at the Institute of Information Science. In 1991,
he was a post-doctoral research fellow at the Department of Electrical
Engineering and Computer Science, Northwestern University. Since
August 1991, he joined the institute as an assistant research fellow.
Dr. Tsai’s research interest includes combinatorial optimization, design
and analysis of algorithms, graph theory, computational geometry, com-
puter networking and communication.
Shie-Yuan Wang is a Ph.D. student in computer science at Harvard Uni-
versity, Cambridge, Massachusetts. Before studying at Haravrd Univer-
sity, he worked at Institute of Information Science, Academia Sinica, Tai-
wan participating in the video conference and video on-demand project.
He received an master degree in computer sceicne from National Taiwan
University in 1992 and a bachelor degree from National Taiwan Normal
University in 1990.