Scene Change Detection Algorithms & Techniques: A Survey by ijcsiseditor


More Info
									                                                              (IJCSIS) International Journal of Computer Science and Information Security,
                                                                                                            Vol. 9, No. 12, December 2011

                     Scene Change Detection Algorithms
                          & Techniques: A Survey

                                                                                              Manisha Sharma2
                      Dolley Shukla1                                             dept. of Electronics & Telecommunacation
             dept. of information Technology                                       Bhilai Institute of Technology, Durg
       Shri Shankaracharya College of Engg., Tech.                                               Durg, India
                        Bhilai, India                                      

Abstract— A video scene change detection method is necessary              DCT coefficient comparison. Gradual scene changes
for managing the growing amount of video information                      result from chromatic edits, spatial edits and combined
efficiently. For recognizing the video content, many advanced             edits. Gradual scene changes include special effects like
video applications such as video on demand (VOD), digital                 zoom, camera pan, dissolve and fade in/out etc.
library and digital watermarking, requires the scene change
detection. Different techniques on scene change detection are
used for compressed & uncompressed videos. Scene change
detection has been studied and researched over the last three               II     Systematic Survey on Scene change detection
decades. As a result, many scene change detection techniques
have been proposed and published in the literature. This paper                     Scene change detection has received a great
gives a brief description of different algorithms and comparative         interest in the research community. In this section a
analysis of different scene change detection techniques. The              survey on significant scene change detection techniques
classification of algorithms into a relatively small number of
categories will provide useful guidance to the algorithm designer.
                                                                          of videos has been presented. Scene change detection
(Abstract)                                                                algorithms are based on the pixel differences[1],
                                                                          compressed(MPEG-2)domains, temporal segmentation
   Keywords-Scene    change        detection;compressed    &              luminance histograms based framework for temporal
uncompressed video; histogram;pixel difference ; abrupt scene             segmentation, sudden scene change detection for MPEG-
change (key words)                                                        2 compressed video, algorithm using direct edge
                                                                          information extraction from MPEG video data is used.
                                                                                   Lock Ye0 and Bede Liu et al proposed rapid
                                                                          scene analysis algorithms for detecting scene changes and
                I.   INTRODUCTION                                         flashlight scenes directly on compressed video[2].These
           Video is the most effective media for capturing                algorithms operate on the DC sequence which can be
   the world around us. Video scene change detection is a                 readily extracted from video compressed using Motion
   fundamental operation used in many multimedia                          JPEG or MPEG without full-frame decompression. The
   applications such as digital libraries, video on demand                DC images occupy only a small fraction of the original
   and digital watermarking. Scene change detection is the                data size while retaining most of the essential “global”
   procedure for identifying changes in the scene content of              information. Operating on these images offers a
   a video sequence. Video data can be divided into                       significant computation saving. Experimental results
   different shots. A shot is a video sequence that consists of           show that the proposed algorithms are fast and effective
   continuous video frames for one action. Scene change                   in detecting abrupt scene changes, gradual transitions
   detection is an operation that divides video data into                 including fade-ins and fade-outs, flashlight scenes and in
   physical shots.                                                        deriving intrashot variations. The temporal segmentation
              Generally scene changes are divided into two                is done only on compressed video, not for uncompressed
   types: Abrupt scene change and Gradual scene change.                   video.
   Abrupt scene changes result from editing “cuts” and                               K. Tse et al presents scene change detection
   detecting them is called cut detection either by colour                algorithms which is based on the pixel differences and
   histogram comparison on the uncompressed video or by                   compressed(MPEG-2) domains[3]. The main problem of
                                                                          this method is that it suffers from the variations incurred

                                                                                                   ISSN 1947-5500
                                                         (IJCSIS) International Journal of Computer Science and Information Security,
                                                                                                       Vol. 9, No. 12, December 2011

by camera motion. This algorithm has the potential to                   detection, but also obtain object level information of the
detect gradual scene changes.                                           video frames, which is very useful for video content
                                                                        indexing and analysis. The algorithm compares the
                                                                        segmentation mask maps between two successive video
        Haitao Jiang et al presents a scene change                      frames & cannot be used for compressed video.
detection techniques for video database system[4] which
                                                                                Anastasios Dimou et al proposed a scene change
provide automatic segmentation, annotation, and indexing                detection for H.264[10]. It describes the correlation
of video data. It provide a taxonomy that classifies the                between local statistical characteristics, scene duration
existing algorithms into three categories: full video image
                                                                        and scene change. It uses only previous frames for the
based, compressed-video-based and model-based                           detection.
algorithms.                                                                    Priyadarshinee Adhikari et al proposed a new
         P.Bouthemy et al gives approach to scene
                                                                        segmentation method based on colour difference
change detection & characterization[5]. The method                      histogram[11]. In these developed a new scene change
relies on statistical technique robustness and efficiency               detection method by scaling the histogram difference
for compressed video. The image is represented in 2D
                                                                        between the two frames. It provide the scaled frame
affine model.                                                           difference that is dynamically compressed by log formula
                                                                        and it is more convenient to decide the threshold. Its
         Xinying Wang' et al suggested a twice difference
                                                                        approach is on edge detection which is based on detecting
of luminance histograms based framework for temporal
                                                                        edges in two neighbouring images and comparing these
segmentation[6].The proposed method utilizes the
luminance histogram twice difference in order to
                                                                               Purnima. S. Mittalkod et al present a paper in
determine the dynamic threshold needed to evaluate the
                                                                        which it gives the classification of shot boundary
break. The adaptive determination of the threshold,
                                                                        detection algorithms,including those that deal with
minimizes the number of incorrect decisions leading to a
                                                                        gradual shot transitions[12].The paper compares different
robust and accurate determination of the actual scene
                                                                        shot boundary detection algorithms and techniques &
break. The method is simple, computationally attractive
                                                                        concluded that these techniques can be further refined
and capable of detecting changes in a variety of visual
                                                                        and implemented for automatic shot detection. However,
inputs. Experimental results indicate that the new method
                                                                        video analysis remains a challenging task with respect to
constantly outperforms existing techniques that are based
                                                                        unstructured home videos.
on static thresholds However, this method need to be
further explored to detect complex transitions between
                                                                         III. Different Techniques on scene change detection
scene change as fades and dissolve.
                                                                               There are a number of methods for video scene
        Seong-Whan Lee has presented a method for                       change detection in the literature. Many of them use the
scene change detection algorithm using direct edge                      low-level global features such as the luminance pixel-
information extraction from MPEG video data[7].The                      wise difference[13],luminance or color histogram
paper proposed a fast scene change detection algorithm                  difference to compare two consecutive frames. However,
using direct feature extraction from MPEG compressed                    since luminance or color is sensitive to small change,
videos, and evaluate this technique using sample video                  these low-level features cannot give a satisfactory answer
data. This process was made possible by a new                           to the problem of scene change detection.
mathematical formulation for deriving the edge                                 The major techniques that have been used for scene
information directly from the discrete cosine transform                 change detection are pixel differences, sum of absolute
coefficients.                                                           differences, block differences, statistical differences,
       W. A. C. Fernando proposed a novel algorithm for                 histogram differences, edge tracking and compression
sudden scene change detection for MPEG-2 compressed                     differences. This section gives a brief description of
video[8].This uses the number of interpolated                           different techniques used for scene change detection.
macroblocks in B-frames to identify the sudden scene
changes. A gradual scene change detection algorithm                A.    Pixel differences
based on statistical features is also presented.
                                                                       This method proves to be the easiest, As its basis consists
                                                                   of counting the number of pixels that have changed
      Shu-Ching Chen1 et al proposed a technique for               considerably between two consecutive images, deciding if
uncompressed video data[9].It presents an effective scene          the difference is higher than a predefined threshold. After
change detection method using an unsupervised                      counting has been accomplished, it will check if the amount
segmentation algorithm and the technique of object                 is enough to be considered a scene change comparing it with
tracking based on the result of the segmentation. This             a second threshold. But the main problem of this method is
method can perform not only accurate scene change

                                                                                               ISSN 1947-5500
                                                             (IJCSIS) International Journal of Computer Science and Information Security,
                                                                                                           Vol. 9, No. 12, December 2011

that it suffers from the variations incurred by camera
motion[3].                                                             F. Edge tracking
B.   Sum of absolute differences (SAD)
                                                                         To fight against intensity variations, such as illumination
    In this method, the absolute mean value of intensity               changes due to flashlight, edge information can be used. The
difference between consecutive frames is computed and a large          percentage of edges that enter and exit between two frames
difference is assumed to be a scene change. The difference is          was computed and scene changes were recognized by
calculated from the intensity mean matrix. The main problem            looking for large edge change percentages. Dissolves and
of this method is that it is quite susceptible to noise.
                                                                       fades were identified by looking at their relative values of the
C. Statistical difference                                              entering and existing percentages.
   This technique is based on the assumption that the frame
                                                                       G.     Scene change Detection Using Macroblocks
intensity variances in a shot do not demonstrate large
fluctuations while that of the neighbouring shot will be
                                                                          One of the approaches to handle different shot boundaries
different. Because it is possible for the variance to remain
                                                                       is using Macroblock. Depending on the types of the
the same as the mean values differ, the use of both intensity
                                                                       macroblock the MPEG pictures have different attributes
mean and variance may be more robust. This method
                                                                       corresponding to the Macroblock. The Macroblock types can
proves to work better when camera and object motion is
                                                                       be divided into forward prediction, backward prediction or
present but is sensitive to object appearance            and
                                                                       no prediction at all[8].
disappearance and fast pans and zooms[5].
                                                                              IV Comparison among different techniques
D. Block differences
   Unlike other previous methods, which may be susceptible
                                                                           Table 1 compares different scene change detection
to local changes, block differences methods have been
                                                                       techniques with advantages & limitations of the proposed
proposed to handle local changes better. These start with
block division for each frame and calculate the differences
between the collocated or matching (in the sense of motion
compensation) blocks in two frames.

E. Histograms differences

    A colour histogram is a representation of the distribution
of colours in an image. The difference between the
histograms of two consecutive frames is evaluated resulting
in the metrics [6]. Histogram comparisons are usually based
on one of the following three distance metrics: bin-to-bin
difference, chi-square and histogram intersection. Among
these, histogram intersection is the best of them all.
Histograms are robust to object and camera changes. On the
other hand, they are sensitive to intensity variations, such as
flashlight and shadow effects. Furthermore, as it is a global
measure, two frames with considerably different content may
have the same histogram, which results in higher amount of
missed detections. The colour histogram of an image can be
computed by dividing a colour space, e.g., RGB, into discrete
image colours called bins and counting the number of pixels
falling into each bin.

     Identify applicable sponsor/s here. (sponsors)

                                                                                                   ISSN 1947-5500
                                                                                      (IJCSIS) International Journal of Computer Science and Information Security,
                                                                                                                                    Vol. 9, No. 12, December 2011

                                                                                                            scene change detection is a fundamental operation used
                   Table1 : Comparison of Different Techniques                                              in many multimedia applications video on demand
                                                                                                            (VOD), and it must be performed prior to all other
S.No     Method           Description of              Advantage              Limitation                      processes.
                                                                                                       •     Digital Library - Over the years that industry has
                                                                                                             developed detailed and complete procedures and
 1.        Pixel        Its basis consists of    • This method is the    • It suffers from                   techniques to index, store, edit, retrieve, sequence and
                        counting the number        easiest.                                                  present video material. Conceptually the video
       Differences      of pixels that have      • Threshold to the        the variations                    retrieval system should act like a library system for
                        changed.                   input sequence         incurred        by                 the users. Video materials should be modeled and
            [3]                                    provided good          camera motion.                     stored in a similar way for effective retrieval[14].
                                                                                                             Shot change detection is the procedure for identifying
 2.       Sum of        Mean value of            • Shows the              • It is quite
         absolute       intensity difference       difference in results      susceptible to
                                                                                                             changes in the scene content of a video sequence so
        differences     between consecutive        between a fixed or a       noise.                         that alternate representation may be derived for the
                        frames is computed.        dynamic threshold                                         purposes of browsing and retrieval. e.g. key frames
                                                                                                             may be extracted from a distinct shot to represent it.
 3.     Statistical      It is based on the      • It proves to be quite   • Slow
        difference         frame intensity         tolerant to noise.        technique                 •     Digital Watermarking - The block analysis is done
                         variances in a shot.                                due the                         only for the first picture of the scene change. This can
          Es [5]                                                             complexity of                   reduce the computation of watermark embedding as
                                                                             the statistical                 compared to the conventional method, which
 4.       Block              Calculate the       • Block difference        • Computation
                                                                                                             computes edge information or block energy for every
                         differences between       remove the                of block                        block for block analysis. This increases a
        differences        the collocated or       sensitivity to object     statistics is                   computational complexity of watermark embedding.
                         matching blocks in        and camera.               costly.                         To reflect a characteristic of each block, motion
                             two frames.                                                                     vector information is used in the block analysis[15].
 5.    Histograms             Histogram          • They are sensitive to   • Different
                           comparisons are         intensity variations,     content may
       Differences[      based on one of the       such as flashlight        have the same                                   VI. Conclusion
       6]                  following three         and shadow effects.       histogram,
                        distance metrics: bin-                               which results                The scene change detection is a potential approach to
                          to-bin difference ,                                in higher              recognize video contents.       In this paper, comparison and
                            chi-square and                                   amount of              analysis of different scene change detection techniques in
                              histogram                                      missed
                                                                                                    compressed & uncompressed domain have been discussed.
                                                                                                    The successful performance of an algorithm depends entirely
 6.        Edge         Compared the number      • To fight against        • Main                   on the context of its application. While in this comparison no
         tracking       and position of edges      intensity variations      drawback is            specific application context was envisaged, the analysis
                           in the images.                                    the                    would hopefully guide a designer to select the algorithms
                                                                             l cost.
                                                                                                    according to specific characteristics, as dictated by the
 7.      Using          Handle different shot    • Macroblock types                                 application context.
       Macroblocks          boundaries             can be       divided
                                                   into        forward
            [8]                                    prediction,
                                                   backward                                                                    References
                                                   prediction or no
                                                   prediction                                         [1]    H. Zhang, A. Kankanhalli, and S. W. Smoliar, “Automatic
                                                  at all.                                                    partitioning of full-Motion video,” Multimedia System, vol. 1,
                                                                                                             No.1, pp. 10-28, 1993.
                                                                                                      [2]    Boon-Lock Yeo, and Bede Liu, “Rapid Scene Analysis on
                                                                                                             Compressed Video,” IEEE Transactions on Circuits & Systems
                                                                                                             for Video Technology, vol.5, No. 6, pp. 533-544, 1995.
                                                                                                      [3]     K. Tse, J. Wei and S. Panchanathan, “A Scene Change
                         V. Applications of Scene Change Detection                                            Detection Algorithm for MPEG Compressed Video
                                                                                                              Sequences,” Electrical & computer engineering, Canadian
                                                                                                              conference on, 2 , 827-830, 1995 .
                   •   Video on Demand - Recently, multimedia information                             [4]     Haitao Jiang, Abdelsalam Helai, Ahmed K. Almagarmid,
                       has been made overwhelmingly accessible with the                                       And Anupam Joshi, “Scene change detection Techniques for
                       rapid advances in communication and multimedia                                         video database systems,” Multimedia Systems @ Springer
                       computing technologies. The requirements for                                           Verilog , vol. 6, pp. 186-195, 1998.
                                                                                                      [5]     P.Bouthemy, M. Gelgon, and F.Ganansia, EIRISA, Rennes,
                       efficiently accessing mass amounts of multimedia data                                  “A unified approach to shot change detection and camera
                       are becoming more and more important[9]. Video                                         motion characterization,” Circuits & systems for Video

                                                                                                                                ISSN 1947-5500
                                                                     (IJCSIS) International Journal of Computer Science and Information Security,
                                                                                                                   Vol. 9, No. 12, December 2011

           Technology,IEEE Transactions on, vol. 9, No. 7, pp. 1051-            [11]    B.G. Hogade ,Jyothi Digge, Neeta Gargote, and
           8251, 1999.                                                                  Priyadarshinee Adhikari, “Abrupt Scene Change Detection,
                                                                                        World Acadamy of Science,” Engineering and Technology
   [6]     Xinying Wang, and Zhengke V C Teng, “Scene Abrupt                            540543(2008), pp. 711-716, 2008.
           Change Detection,” Electrical & computer Engineering 2000,           [12]   Purnima.S.Mittalkod, and Dr. G.N.Srinivasan, “Shot Boundary
           Canadian conference on, Halifax, NS, Canada, vol.2, pp. 880-                 Detection Algorithms and Techniques: A Review,” Research
           883, 2000.                                                                   Journal of Computer Systems Engineering- An International
   [7]     Seong-Whan Lee, Young-Min Kim, and, Sung Woo Choi.,                          Journal, vol. 02, No. 02, pp. 115-121, 2011.
           Fast Scene Change Detection using Direct Feature, Extraction
           from MPEG Compressed Videos, IEEE Transactions on                    [13]     Chong-Wah Ngo, Ting-Chuen Pong, and Hong-Jiang
           multimedia, vol. 2, No. 4, pp. 240-254, 2000.                                 Zhang,R.T.Chin, “Motion-Based Video Representation for
   [8]     W.A.C.Fernando, C.N.Canagarajah, and D.R.Bull, “Scene                         Scene Change Detection,” Patttern Recognition,2000
           change detection algorithms           for content based video                 proceedings 15th International Conference on, 2000, vol. 1,
           indexing and retrieval,” Electronics & Communication                          Barcelona, Spain, pp. 827-830, 2000.
           Engineering Journal, vol. 13, No. 3, pp. 117-126, 2001.              [14]     U.     Gargi,R.    Kasturi,     ,S.H.    Strayer,Performance
   [9]     Shu-Ching Chen, Mei-Ling Shyu, Cheng-Cui Zhang, and R.                        Characterization of Video-Shot-Change Detection Methods,
           L. Kashyap, “Video Scene change Detection method using                        IEEE transaction on circuits and systems for video
           Unsupervised Segmentation and object Tracking,” Multimedia                    technology,IEEE Transactions on vol. 1, No. 01 ,pp. 1-13,
           and Expo,2001(ICME 2001), IEEE International Conference                       2000.
           on, florida, pp. 56-59, 2001.                                        [15]     T.-S. Choi, Y.-H.Choi, and Y.K. Seong, “Scene-Based
   [10]    Anastasios Dimou, and Olivia Nemethova, “Scene Change                         Watermarking method for Copy-Protection using Image
           Detection for H.264 Using Dynamic Threshold Techniques,”                      complexity and motion vector amplitude,” In: IEEE
           Proceedings of 5th EURASIP Conference on Speech and                           International Conference on Acoustics, Speech and Signal
           Image Processing, Multimedia Communications and Service,                      Processing(ICASSP ‘04), vol. 3, pp. 409-412, 2004.
           Smolenice, Slovak Republic, pp. 80-227, 2005.


Dolley Shukla was born in chhattisgarh, India in 1975.She received the B.E. degree

from Pt. Ravishankar University, M.Tech. degree from M.A.N.I.T., Bhopal in
Electronics & Telecommunication engineering in 1999 and in 2006 respectively. She is
currently pursuing the Ph.D. degree at the CSVTU, Bhilai , India.
Dolley Shukla is currently Associate Professor at SSCET, Bhilai, India.
Her research interests include Image Processing, Video Processing & Watermarking.

Dr. Manisha Sharma was born in 1970. She received the B.E. from Barkhattullah
 University, Bhopal in 1992 , M.E. from Government Engineering College,
Jabalpur Rani Durgavati University, Jabalpur in 1995 and Ph.D. from C.S.V.T.U.,
 Bhilai, India in 2010. Presently she is working as a professor & Head of the department
 at, Bhilai Institute of Technology , Durg, CHHATTISGARH, India. Her research
Interest includes Image Processing , Image Segmentation , Video Processing ,                                                        t
 watermarking and Authentication

                                                                                                           ISSN 1947-5500

To top