Color Image Demosaicking with Adaptation to Varying Spectral Correlation using linear minimum mean square estimation

Document Sample
Color Image Demosaicking with Adaptation to Varying Spectral Correlation using linear minimum mean square estimation Powered By Docstoc
					National Conference on Role of Cloud Computing Environment in Green Communication 2012

                  Color Image Demosaicking with Adaptation to Varying Spectral
                    Correlation using linear minimum mean square estimation
                                              C.Jegatheesh, 2A.Jayachandran
              PG Student ,2 Assistant professor,CSE Department PSN college of Engineering and technology ,Tirunelveli

          Abstract                                                            solutions    are   proposed: one is
                                                                     based on linear minimum mean-square
                     Almost         all        existing     color
                                                                     estimation and the other         based on the
          demosaicking         algorithms           for    digital
                                                                     support vector       regression Experimental
          cameras are designed on the assumption o
                                                                     results demonstrate that the new hybrid
          f high correlation between red, green, blue
                                                                     of color      artifacts of existing        color
          (or some other primary color) bands. they
                                                                     demosaicking .
          exploit spectral correlations between the
                                                                     Index Terms
          primary color bands to interpolate the
                                                                      Autoregressive     model,       color
          missing color samples, but in areas o f                    demosaicking, color saturation, digital
                                                                     cameras, linear minimum mean-square
          no or weak spectral correlations, these
                                                                     estimation (LMMSE), support vector
          algorithms          are          prone      to large       regression (SVR).
          interpolation errors. Such demosaicking
                                                                         I. INTRODUCTION
          errors are visually objectionable because
                                                                               In quest of low cost, compact size,
          they tend to            correlate        with    object
                                                                     and long battery life, most digital cameras
          boundaries        and      edges.        This    paper
                                                                     use a single sensor array to capture color
          proposes a remedy to the above problem
                                                                     images. At each pixel position only one
          that has long been over looked in the
                                                                     instead of three or more primary colors
          literature. The main             contribution of this
                                                                     (e.g., red, green and blue) is captured with
          work is a hybrid demosaicking approach
                                                                     a color filter array (CFA). the most
          that     supplements            an    existing    color
                                                                     commonly used CFA is that of bayer
          demosaicking algorithm by combining its
                                                                     pattern that consists of a quincunx lattice
          results with those of adaptive intraband
                                                                     of green samples interleaved with one
          interpolation. This is formulated as an
                                                                     square lattice of red samples and another
          optimal data fusion problem, and two
                                                                     square lattice of blue samples, as depicted
                                                                     by the bayer pattern. the full color image

          Department of CSE, Sun College of Engineering and Technology
National Conference on Role of Cloud Computing Environment in Green Communication 2012

          is      reconstructed         by interpolating the       estimates the color difference signals.
          missing           color     samples,    a      process   The estimates of missing red and blue
          commonly known as color demosaicing.                     samples are anchored on the green band
          Bayer pattern of color mosaic for digital                that has the least chance of aliasing in
          cameras.                                                 Bayer CFA. Moreover, PCSD imposes
                    A basic premise of CFA mosaic                  the same interpolation direction in all
          under sampling schemes for color                         three color bands. However, this work
          reproduction is the following. Most                      demonstrates that aggressive uses of
          scenes in nature comprise of pastoral                    spectral       correlations       in       color
          colors, and highly saturated colors are                  demosaicking can sometimes backfire.
          rare,     i.e.,     strong     correlations     exist    When the assumption of high spectral
          between different spectral bands. Indeed,                correlations does not hold, for instances,
          most       existing         color   demosaicking         in areas of highly saturated colors, and if
          algorithms,          particularly      those      of     large      sensor    noises   are       present,
          competitive performance assume and                       demosaicking            methods            over
          exploit the spectral correlations when                   emphasizing spectral correlations can
          interpolating the missing color samples.                 produce highly visible, objectionable
          In       general,         those     demosaicking         color artifacts. The problem of overusing
          techniques that process different color                  spectral       correlation        in       color
          bands in isolation are inferior to the                   demosaicking was overlooked by many
          interband approach. Among previously                     researchers     in    academia,        ourselves
          published             color         demosaicking         included. The reason seemed to be that
          techniques, the primary-consistent soft                  they were misled by a peculiarity of the
          decision          demosaicking      (PCSD)        [5]    popular Kodak set of test images that
          performs the best over a diverse set of                  were commonly used to simulate CFA
          color images The PCSD algorithm                          data    and    benchmark      demosaicking
          exemplifies an explicit and thorough use                 performance.
          of spectral correlations in demosaicking.
          It   assumes          the     difference      signals
          between green and red and between
          green and blue to be low-pass, and

          Department of CSE, Sun College of Engineering and Technology
National Conference on Role of Cloud Computing Environment in Green Communication 2012

                                                        Kodak set seems to include a quite
                                                        narrow range of color images. The goal
                                                        of this research is to lend a degree of
                                                        universality      to       the     existing      color
                                                        demosaicking           technology        for      still
                                                        cameras. Our main contribution is a new
                                                        mechanism of adapting to varying
                                                        spectral correlations in demosaicking.
                                                        The idea is to judiciously combine
                                                        spatial-spectral demosaicking with an
                   Fig 1 bayer CFA
                                                        adaptive intraband interpolation process.
                 The spectral correlations of these     Pure spatial demosaicking is carried out
          test images are considerably higher           separately in each of the color bands to
          (much smoother hue) than typical color        produce supplementary estimates of
          images, probably due to some post             missing    samples.              These       intraband
          processing.    Let   us    compare     the    estimates are fused with the interband
          correlation coefficients    PGR (between      estimates produced by an existing
          the green and red bands), (between the        demosaicking method to mitigate the
          green and blue), PGB and (between the         bad   artifacts        f   the     latter.     Spatial
          red and blue) PRB. for the Kodak set and      demosaicking is a problem of image
          another set of 30 images randomly             interpolation in quincunx or square sub
          chosen from the JPEG, MPEG, SMPTE             lattice of the original image, depending
          test sets and various internet sites. The     on whether the green or blue/red band is
          averages of , PGR, PGB, PRB , and for the     in question. The proposed adaptation
          two test sets are tabulated in Table I. The   mechanism is general, and it can work
          Kodak set has substantially higher            with any image interpolation algorithm.
          spectral correlations than normal. The        But the performance of the chosen
          table also reveals that the Kodak set has     interpolation      algorithm          affects      the
          a significantly smaller standard deviation    quality of spatial demosaicking and in
          than normal (0.146 versus 0.316) in           turn the final fused result. For this
          spectral correlation. In other words, the     reason, we choose the recent technique

          Department of CSE, Sun College of Engineering and Technology
National Conference on Role of Cloud Computing Environment in Green Communication 2012

          of soft-decision adaptive interpolation                which is extensively studied in the
          for its superior performance [10]. The                 machine       learning      literature,   can
          key to the success of the proposed                     incorporate preknowledge of a training
          hybrid approach is how well the                        set in the weighting of inter and
          underlying data fusion problem can be                  intraband estimates. Therefore, the latter
          solved. We propose two solutions of                    is more powerful and robust than the
          different       complexity-performance                 former. Simulation results verify the
          tradeoffs: one is based on classic linear              superior performance of the proposed
          minimum      mean-square         estimation            techniques to existing methods, in both
          (LMMSE); the other computes the                        PSNR measure and perceptual image
          optimal fusion weight by support vector                quality.
          regression (SVR). The SVR technique,

                        Fig 2 Robust color demosaicking with adaptation to varying spectral correlations

          Department of CSE, Sun College of Engineering and Technology
National Conference on Role of Cloud Computing Environment in Green Communication 2012

       II. Intraband Demosaicking via                        First, consider the interpolation of
       Piecewise Auto Regression                             the blue/red band that constitutes a

       When there is no spectral correlation to              square sublattice of the original

       exploit, the better strategy is to perform            image in the Bayer CFA. Let IHh be

       spatial interpolation rather than spectral            the blue/red image to be estimated,

       demosaicking         based       on       erroneous   and be the Il down-sampled blue/red

       assumption. The intraband demosaicking of             image. Let x(i) in Il and y(n) in Ih

       three down-sampled bands in the Bayer                 bethe pixels of Il and Ih respectively.

       CFA is a problem of image interpolation in            Interpolation of the missing pixels is

       quincunx or square sub lattice of the original        carried out in two passes. In the first

       image, depending on whether the green or              pass y(i) in Ih, those missing pixels ,

       blue/red band is in question. We solve this           whose four 8-connected neighbors

       problem by a new image interpolation                  are known observed pixels ,x(I,) in

       method based on piecewise autoregressive              Il.,   are    interpolated   Upon    the

       modeling (PARM) [10]. In this section, we             completion of the first pass, the

       sketch the main idea of the PARM technique            blue/red band becomes a quincunx

       and how it can be applied to intraband                lattice, being the same as the green

       demosaicking.                                         band of Bayer CFA. The remaining
                                                             missing blue/red pixels are to be
                                                             interpolated in the second pass. The
                                                             interpolation problem in the second
                                                             pass is essentially the same as the
                                                             previous one. The only difference is
                                                             that we interpolate the missing pixels
                                                             y(i) in Ih using their four 4-
                                                             connected      neighbors,    which   are
                                                             either known in il or estimated in the
                                                             first pass.
                                                                              x1(n)= x1’(n)+€1(n
       Fig-3 Spatial configuration in second   pass of
       PARM.                                                                  x2(n)=x2’(n)+ €2(n)

       Department of CSE, Sun College of Engineering and Technology
National Conference on Role of Cloud Computing Environment in Green Communication 2012

                                                                     SVR operates in a feature space to
                                                                     approximate unknown functions in To learn
                                                                     this model, we choose an appropriate
                                                                     training set consisting of input-output pairs.
                                                                     To this end, we postulate that an estimate of
                                                                     , denoted by ,can be expanded in terms of a
                                                                     set of nonlinear basis functions. Now the
                                                                     estimation problem can be stated as one of
                                                                     minimizing the empirical risk. The most
                                                                     critical factor in designing a good learning
       Fig 4 Row and a column of mosaic data that intersect
                                                                     machine for color demosaicking is the
       at a blue sampling position.
                                                                     selection of suitable features that an output
                                                                     space, aiming to linearly estimate an
       III. Intra- And Interband Estimation By
                                                                     unknown      regression     with    nonlinear
       Support Vector Regression
                                                                     functions. Suppose that the dependence of a
       In the proceeding section, the fusion of the
                                                                     scalar on a vector can be described by a
       intra band and interband estimates is treated
                                                                     nonlinear regressive model d=f(z) can
       and solved as a problem of linear minimum
                                                                     indicate which demosaicking method is
       mean-square estimation. But this method
                                                                     better for the current pixel. Besides and, we
       can fail when the image signal exceeds the
                                                                     need to furnish SVR with more statistically
       Nyquist     frequency.         In        this   case,   the
                                                                     significant features. The sample variance is
       estimates of the missing color values are
                                                                     a useful feature. The other features are the
       wrong but they still match the autoregressive
                                                                     interband linear correlation coefficients
       model, and we grossly underestimate the
                                                                     IV. Experimental Results And Remarks
       error variance. To overcome this weakness,
                                                                     The proposed hybrid color demosaicking
       we take a machine learning approach to
                                                                     algorithm is implementedAdams [6], the
       solve     the    problem            of     data    fusion.
                                                                     wavelet-based method of Gunturk adaptive
       Specifically,     we      use            support    vector
                                                                     homogeneity method of Hirakawa and Parks
       regression (SVR) to find the optimal fusion
                                                                     [3], the primary-consistent soft-decision
       weights and Support vector regression
                                                                     demosaicking (PCSD) method[5], and the
       (SVR) is a SVM-based regression technique.
                                                                     EUSSC method of Chang and Tan [8]. To

       Department of CSE, Sun College of Engineering and Technology
National Conference on Role of Cloud Computing Environment in Green Communication 2012

       evaluate the impact of different intraband            CFA images we can compare the different
       interpolation algorithms in the proposed              demosaicking methods in PSNR since the
       hybrid demosaicking framework, we also                underlying fully sampled color images are
       include in the comparison group two                   known. The second test set consists of three
       different     combinations:         a)     bicubic    raw CFA images taken by Canon digital
       interpolator and Hamilton-Adam’s method,              cameras, listed in which are downloaded.
       and b) the NEDI interpolator and Hamilton-            This website provides these raw CFA
       Adam’s method.                                        images for the very purpose of evaluating
                                                             the performance of different digital cameras.
                                                             The second test image set allows us to
                                                             compare the visual quality of different
                                                             demosaicking      methods    in   real    action,
                                                             although we cannot measure PSNR in this
                                                             case. For the SVR variant of the new
                                                             method, we include the training set and use
                                                             the other images for testing. In both
                                                             LMMSE and SVR variants. To demonstrate
                                                             the adaptability of the proposed method to
          Fig 5 Test images used in this paper               varying spectral correlations,         present a
                                                             example, the intensity map of the weight in
               We use two sets of color images in            in comparison with the interband correlation
       our comparison study. The first set consists          map for image .To quantify this adaptability
       of 20 test color images that have a wide              we plot in the curve of ideal weight as a
       range of spectral correlations. These 20              function of interband correlation. Also, to
       images, listed are chosen from the test sets          demonstrate      the   effectiveness     of   the
       of JPEG, MPEG, SMPTE and as well as                   proposed fusion method, the curve of the
       from the Kodak test set They are originally           ideal weight versus the estimated value .
       fully   sampled     RGB      images       and   the
                                                             V. CONCLUSION
       corresponding mosaic images are simulated
       by down-sampling the test images with the                        We proposed a new hybrid
       Bayer CFA pattern. For the said simulated             approach    of    color   demosaicking        that

       Department of CSE, Sun College of Engineering and Technology
National Conference on Role of Cloud Computing Environment in Green Communication 2012

       combines inter and intraband estimates of
                                                              [8]    L. Chang and Y.-P. Tan, “Effective use of
       missing samples. The new approach cures a                        spatial and spectral correlations for color
                                                                        filter array demosaicking,” IEEE Trans.
       common flaw of existing demosaicking                             Consum. Electron., vol. 50, no. 1, pp. 355–
                                                                        365, Jan. 2004.
       techniques: susceptibility to color artifacts in
       areas of weak spectral correlation or sensor           [9]      L. Chang and Y.-P. Tan, “Hybrid color filter
                                                                       array demosaicking for effective artifact
       noises. Two embodiments of the new                              suppression,” J. Electron. Imag., vol. 15,
       approach, an LMMSE-based demosaicking                  [10]      X. Zhang and X. Wu, “Structure preserving
                                                                       image interpolation via adaptive 2d
       technique and a SVR-based technique were                        autoregressive modeling,” presented at the
                                                                       IEEE Int. Conf. Image Process., San
       developed and evaluated. Both of them                           Antonio, TX, 2007.
       eliminate    visually     objectionable      color     [11]      V. Vapnik, Statistical Learning Theory.
                                                                       New York: Wiley, 1998.
       artifacts of existing color demosaicking               [12]     C. Kwan and X. Wu, “A classification
                                                                       approach to color demosaicking,” in Proc.
       methods, and achieve significantly higher                       IEEE Int. Conf. Image Process., San
                                                                       Antonio, TX,Oct. 2004, vol. 4, pp. 2415–
       PSNR and superior image quality.                                2418.
                                                              [13]    Fan Zhang, Xiaolin Wu, Xiaokang Yang,
                                                                       Wenjun Zhang, Lei Zhang, “Robust Color
       REFERENCES                                                      Demosaicking With Adaptation to Varying
                                                                       Spectral Correlations”, IEEE Trans., Image
       [1]     B. E. Bayer, Eastman Kodak         Company,             Process.,
               “Color Imaging Array,” U.S.Patent 3 971                  Dec. 2009
               065, Jul. 20, 1976.
       [2]     B. K. Gunturk, Y. Altunbasak, and R. M.
               Mersereau, “Color plane interpolation using
               alternating projections,” IEEE Trans. Image
               Process., vol. 11, no. 9, pp. 997–1013, Sep.
       [3]     K. Hirakawa and T. W. Parks, “Adaptive
               homogeneity-directed             demosaicing
               algorithm,” IEEE Trans. Image Process.,
               vol. 14, no. 3, pp.360–369, Mar. 2005.
       [4]     X. Li, B. Gunturk, and L. Zhang, “Image
               demosaicing: A systematic survey,” SPIE,
               vol. 6822, pp. 68 221J–68 221J-15, 2008.
       [5]      X. Wu and N. Zhang, “Primary-consistent
               soft-decision color demosaicking for digital
               cameras,” IEEE Trans. Image Process., vol.
               13, no.9, pp. 1263–1274, Sep. 2004.
       [6]     J. F. Hamilton and J. E. A. , “Adaptive
               Color Plane Interpolation in Single Sensor
               Color Electronic Camera,” U.S. Patent 5 629
               734, May 13, 1997.

       [7]      X. Wu and L. Zhang, “Improvement of
               color video demosaicking in
               temporal domain,” IEEE Trans. Image
               Process., vol. 15, no. 10, pp. 3138–3151,
               Oct. 2006.

       Department of CSE, Sun College of Engineering and Technology

Shared By: