Docstoc

EE 7730 Lecture 1

Document Sample
EE 7730 Lecture 1 Powered By Docstoc
					EE 7700



    Demosaicking Problem in Digital Cameras
Multi-Chip Digital Camera
        To produce a color image, at least three spectral
         components are needed at each pixel.
        One approach is to use beam-splitters and multiple chips.




                             Lens




                     Scene




                                     Beam-
                                    splitters   Spectral   Sensors
                                                 filters

Bahadir K. Gunturk                                                   2
Single-Chip Digital Camera
        Multi-chip approach is expensive. Precise chip alignment is
         required.
        The alternative is to use a color filter array.




                            Lens
                                   Color filter   Sensors
                                     array



               Scene




Bahadir K. Gunturk                                                     3
Single-Chip Digital Camera
        The missing color samples must be estimated to produce
         the full color image.
        Since a mosaic of samples are available, this estimation
         (interpolation) process is called demosaicking.




Bahadir K. Gunturk                                                  4
Single-Chip Digital Camera
        Images suffer from color artifacts when the samples are not
         estimated correctly.




                     Original image        Bilinearly interpolated
                                         from CFA-filtered samples
Bahadir K. Gunturk                                                     5
Demosaicking Approaches
        Non-Adaptive Single-Channel Interpolation: Interpolate
         each color channel separately using a standard technique,
         such as nearest-neighbor interpolation, bilinear
         interpolation, etc.
        Edge-Directed Interpolation: Estimate potential edges,
         avoid interpolating across the edges.

                                 Edge-directed interpolation

                                 1.Calculate horizontal gradient ΔH = |G1 – G2|
                         3       2.Calculate vertical gradient ΔV = |G3 – G4|
                     1   x   2   3.If ΔH > ΔV,
                                 Gx = (G3 + G4)/2
                         4
                                     Else if ΔH < ΔV,
                                 Gx = (G1 + G2)/2
                                     Else
                                 Gx = (G1 + G2 + G3 + G4)/4




Bahadir K. Gunturk                                                                6
Demosaicking Approaches
        Edge-Directed Interpolation: Based on the assumption that
         color channels have similar texture, various edge detectors
         can be used.



                     1           Edge-directed interpolation
                     2
                                 1.   Calculate horizontal gradient ΔH = | (R3 + R7)/2 – R5 |
            3   4    5   6   7
                                 2.   Calculate vertical gradient ΔV = | (R1 + R9)/2 – R5 |
                     8           3.   If ΔH > ΔV,
                     9                   G5 = (G2 + G8)/2
                                           Else if ΔH < ΔV,
                                         G5 = (G4 + G6)/2
                                           Else
                                         G5 = (G2 + G8 + G4 + G6)/4




Bahadir K. Gunturk                                                                              7
Demosaicking Approaches
        Constant-Hue-Based Interpolation: Hue does not change
         abruptly within a small neighborhood.
                    Interpolate green channel first.
                    Interpolate hue (defined as either color differences or color
                     ratios).
                    Estimate the missing (red/blue) from the interpolated hue.




               Red                              Interpolate            Interpolate
                                                                          d Red




               Green       Interpolate




Bahadir K. Gunturk                                                                   8
Demosaicking Approaches
        Edge-Directed Interpolation of Hue: It is a combination of
         edge-directed interpolation and constant-hue-based
         interpolation. Hue is interpolated as in constant-hue-based
         interpolation approach, but this time, hue is interpolated
         based on the edge directions (as in the edge-directed
         interpolation algorithm).




Bahadir K. Gunturk                                                     9
Demosaicking Approaches
        Using Laplacian For Enhancement: Use the second-order
         gradients of red/blue channels to enhance green channel.

                             1.   Calculate horizontal gradient ΔH = |G4 – G6| + |R5 – R3 + R5 – R7|
                1            2.   Calculate vertical gradient ΔV = |G2 – G8| + |R5 – R1 + R5 – R9|
                2            3.   If ΔH > ΔV,
                                           G5 = (G2 + G8)/2 + (R5 – R1 + R5 – R9)/4
        3   4   5    6   7
                                  Else if ΔH < ΔV,
                8
                                           G5 = (G4 + G6)/2 + (R5 – R3 + R5 – R7)/4
                9                 Else
                                           G5 = (G2 + G8 + G4 + G6)/4 + (R5 – R1 + R5 – R9 + R5 – R3 + R5 – R7)/8




Bahadir K. Gunturk                                                                                           10
Aliasing                                     f2


      Frequency spectrum of an image:
                                                  fm     f1




       After CFA sampling:
                                                         f2
                       f2




                                        f1                                f1




                     Green channel                     Red/Blue channel

Bahadir K. Gunturk                                                             11
Demosaicking Approach
        Alias Cancelling: Based on the assumption that red, green,
         and blue channels have similar frequency components, the
         high-frequency components of red and blue channels are
         replaced by the high-frequency components of green
         channel.

                                f2




                                            f1



                         Red/Blue channel




Bahadir K. Gunturk                                                    12
Experiment

                                                 HL    HL    HL
               Full                         LL
                               Subband            LL    LL
          Red/Green/Blue
             channels
                            decomposition                    HH
                                            LH    LH    LH

                 CFA
               Sampling




              Interpolate                        HL    HL    HL
                               Subband      LL    LL    LL
                            decomposition                    HH
                                            LH    LH    LH

Bahadir K. Gunturk                                                13
Constraint Sets

      Detail Constraint Set: Detail subbands of the red and blue
     channels must be similar to the detail subbands of the green
     channel.

                           GHL
                                            HL    HL            RHL
                                            HH    HH
                                       LH    LH




                      R(n1 , n2 ) : Rk (n1 , n2 )  Gk (n1 , n2 )  T (n1 , n2 ),
                                                                                 
                Cd                                                              
                      for k  HL, LH , HH
                                                                                 
                                                                                  



Bahadir K. Gunturk                                                                    14
Constraint Sets

      Observation Constraint Set: Interpolated channels must be
     consistent with the observed data.
                                                    Sensors    CFA




                                                 O(n1 , n2 )
                      R


               Co  R(n1, n2 ) : R(n1, n2 )  O(n1, n2 ),  (n1, n2 ) R 


Bahadir K. Gunturk                                                             15
Projection Operations

      Projection onto the Detail Constraint Set:
             Decompose the color channels.
             Update the detail subbands of red and blue channels.

                     GHL (n1 , n2 )

                                           HL



                                           HH                         RHL (n1 , n2 )
                                      LH
                                                GHL (n1 , n2 )  T (n1 , n2 )



             Apply synthesis filters to reconstruct back the channels.


Bahadir K. Gunturk                                                                     16
Projection Operations

      Projection onto the Observation Constraint Set:
             Insert the observed data to their corresponding positions.

                                               Sensors    CFA




                                            O(n1 , n2 )




Bahadir K. Gunturk                                                         17
Alternating Projections Algorithm

   Samples of                         Initial
  color channels                  interpolation




                          Projection onto the              Projection onto the
                          detail constraint set         observation constraint set

                     h0                      g0
                                                               Insert the
                                                             observed data
                     h1        Update        g1




                                            Iteration



Bahadir K. Gunturk                                                                   18
Results




            Original      Hibbard 1995   Laroche and Prescott 1994




Hamilton and Adams 1997   Kimmel 1999         Gunturk 2002


Bahadir K. Gunturk                                             19
   Results




                                  Laroche
                                    and
                        Hibbard   Prescott
Original                 1995      1994




Hamilton
   and
 Adams                  Kimmel    Gunturk
  1997                   1999      2002
   Bahadir K. Gunturk                        20
Previous Methods



                                                                                [Gunturk02]




    Gunturk et al, “Demosaicking: Color Filter Array Interpolation in Single-Chip Digital
    Cameras,” to appear in IEEE Signal Processing Magazine.



Bahadir K. Gunturk                                                                          21
References
      [Gunturk02] Gunturk et al, “Color Plane Interpolation Using Alternating
     Projections,” IEEE Trans. Image Processing, 2002.
     [Hibbard 1995] R. H. Hibbard, “Apparatus and method for
     adaptively interpolating a full color image utilizing luminance
     gradients,” U.S. Patent 5,382,976, January, 1995.
      [Laroche and Prescott 1994] C. A. Laroche and M. A. Prescott,
     “Apparatus and method for adaptively interpolating a full color
     image utilizing chrominance gradients,” U.S. Patent 5,373,322,
     December, 1994.
      [Hamilton and Adams 1997] J. F. Hamilton Jr. and J. E. Adams,
     “Adaptive color plane interpolation in single sensor color electronic
     camera,” U.S. Patent 5,629,734, May, 1997.
      [Kimmel 1999] R. Kimmel, “Demosaicing: Image reconstruction
     from CCD samples,” IEEE Trans. Image Processing, vol. 8, pp.
     1221-1228, 1999.


Bahadir K. Gunturk                                                               22

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:6
posted:8/25/2011
language:English
pages:22