Docstoc

EE 7730 Lecture 1

Document Sample
EE 7730 Lecture 1 Powered By Docstoc
					EE 7700



    High Dynamic Range Imaging
References
        Slides and papers by Debevec, Ward, Pattaniak, Nayar, Durand, et al…
        http://people.csail.mit.edu/fredo/PUBLI/Siggraph2002/




Bahadir K. Gunturk                                                              2
High Dynamic Range (HDR) Imaging
        The range of luminances is more than 10^14 candela/m2




  star light         moon light         office light        day light            search light

10-6                     10-2         100 101 102             104                       108

Range of human eye at an instant is around 10^4:1 (4log units)
Human eye can adapt to see much wider range.

Candela (cd) is the unit of luminous intensity (power emitted by a light source in a
particular direction, with wavelengths weighted by the sensitivity of the human eye.)
A common candle emits roughly 1 cd.
A 100 W incandescent lightbulb emits about 120 cd.
Bahadir K. Gunturk                                                                       3
Spectral Sensitivity of Human Visual
System: Luminosity Function

                                                          One candela is defined as the luminous intensity of a
                                                          monochromatic 540 THz light source that has a radiant intensity
                                                          of 1/683 watts per steradian, or about 1.464 mW/sr. The 540 THz
                                                          frequency corresponds to a wavelength of about 555 nm, which is
                                                          green light near the peak of the eye's response. A typical candle
                                                          produces very roughly one candela of luminous intensity.

                                                          Quantity           Derived SI Unit           Symbol
                                                          Luminance          candela per square meter cd/m2
                                                          Luminous flux     lumen                     cd * sr = lm
                                                          Illuminance        lux                      lm/m2 = lx




Photopic (black) and scotopic [1] (green) luminosity functions. The photopic includes the CIE 1931
standard [2] (solid), the Judd-Vos 1978 modified data [3] (dashed), and the Sharpe, Stockman,
Jagla & Jägle 2005 data [4] (dotted). The horizontal axis is wavelength in nm. (from Wikipedia)


Bahadir K. Gunturk                                                                                                   4
HDR
        The range of radiances is more than 10^14 candela/m2




  star light         moon light      office light    day light   search light

10-6                     10-2      100 101 102         104            108



    Range of Typical Displays:
      from ~1 to ~100 cd/m2
                                    0    255

Bahadir K. Gunturk                                                      5
           Sensitivity of Eye

                                                                        Cone dominated


             rod                              Gain
log Gain




                     cone




           -6       -4          -2     0      2   4   6   1000 cd/m^2
                                     log La
           Bahadir K. Gunturk                                                   6
           Sensitivity of Eye

                                                                        Rod dominated


             rod                              Gain
log Gain




                     cone




                                                          0.04 cd/m^2
           -6       -4          -2     0      2   4   6
                                     log La
           Bahadir K. Gunturk                                                  7
Sensitivity of Eye




Bahadir K. Gunturk   8
HDR
        The range of image capture devices is also low




Bahadir K. Gunturk                                        9
HDR
        The range of image capture devices is also low




Bahadir K. Gunturk                                        10
HDR
        HDR image rendered to be displayed on a LDR display.




Bahadir K. Gunturk                                              11
HDR Problems:
        • How to capture an HDR image with LDR cameras?
        • How to display an HDR image on LDR displays?




Bahadir K. Gunturk                                        12
                     • Capture multiple images with varying exposure.
                     • Combine them to produce an HDR image.




Bahadir K. Gunturk                                              13
Creating HDR from Multiple Pictures
                     Measured intensity, z



                                  t1         t2




                                                   Irradiance, E (=total power per unit area)




                                 t1           t2



Bahadir K. Gunturk                                                                        14
Creating HDR from Multiple Pictures
                     Measured intensity, z


                                   t1
                            z1               t2
                                                                    t1            t2
                            z2


                                    E                      Irradiance, E
  z1 = t1 * E
  z2 = t2 * E


Estimates:
                                                  Take a weighted sum of E1 and E2:
       E1=z1/t1
                                  w1    w2
       E2=z2/t2                                         E=( w1*E1 + w2*E2 ) / (w1+w2)



                                                                E

Bahadir K. Gunturk                                                                      15
Creating HDR from Multiple Pictures
                     Measured intensity, z


                                   t1
                            z1               t2
                                                                      t1             t2
                            z2


                                    E                       Irradiance, E
  z1 = t1 * E
  z2 = t2 * E


Estimates:
                                                  Take a weighted sum of E1 and E2:
       E1=z1/t1
                                        w
       E2=z2/t2                                          E=( w(z1)*E1 + w(z2)*E2 ) / (w(z1)+w(z2))



                                                                 z
                                                   255
Bahadir K. Gunturk                                                                           16
Creating HDR from Multiple Pictures
                     Measured intensity, z


                                      t1
                              z1                   t2
                                                                            t1             t2
                              z2


                                       E                          Irradiance, E
  z1 = t1 * E
  z2 = t2 * E


Estimates:
                                                        Take a weighted sum of E1 and E2:
       E1=z1/t1
                                             w
       E2=z2/t2                                                E=( w(z1)*E1 + w(z2)*E2 ) / (w(z1)+w(z2))



                                                                       z
                                                         255
Bahadir K. Gunturk                                                                                 17
                     Question: If t1 and t2 are not given, how can we estimate them?
 Creating HDR from Multiple Pictures
  In general, the camera response is not linear.
                                                  f

            z1 = f ( t1 * E )
            z2 = f ( t2 * E )
                                                                         t1               t2

                                                 g
    E1= g (z1) / t1

    E2= g (z2) / t2
                                                            z


                                                      w
                                                                      w is sometimes chosen as the
E=( w(z1)*E1 + w(z2)*E2 ) / (w(z1)+w(z2))                             derivative of f. (Mann)

                                                            z



 Questions: How to estimate g and t?  One approach is based on polynomial model (Nayar).

 Bahadir K. Gunturk                                                                                  18
  Radiometric Self Calibration
                         Irradiance               Intensity


Polynomial model



Exposure ratios:


                           Pixel    Image number


Cost function




          Solve using




  If exposure ratios are not known, solve iteratively




  Bahadir K. Gunturk                                          19
Tone Mapping
   Given an HDR image, how are we going to display it in an LDR display?




Bahadir K. Gunturk                                                         20
Tone Mapping
   Given an HDR image, how are we going to display it in an LDR display?

                                                                 Linear




                                                                Nonlinear




Bahadir K. Gunturk                                                          21
Bahadir & Dorsey
DurandK. Gunturk   22
Bahadir & Dorsey
DurandK. Gunturk   23
Bahadir & Dorsey
DurandK. Gunturk   24
Bahadir & Dorsey
DurandK. Gunturk   25
Bahadir & Dorsey
DurandK. Gunturk   26
      Durand & Dorsey




Bahadir & Dorsey
DurandK. Gunturk                        27
                    Bilateral filter
Bahadir & Dorsey
DurandK. Gunturk   28
Fattal et al in 1D

                     log   derivative



        2500:1




                                         attenuate
        7.5:1




                     exp     integrate




Bahadir K. Gunturk                                   29
Reinhard et al.




        L_white is the smallest luminance that will be mapped to pure white (1).
        Set L_white = L_max to have no “burn-out”.




Bahadir K. Gunturk                                                                 30
Bahadir & Dorsey
DurandK. Gunturk   31
Bahadir & Dorsey
DurandK. Gunturk   32
   Informal comparison




Gradient domain                Bilateral        Photographic
 Gradient domain                Bilateral        Photographic
  [Fattal et al.]
Bahadir K. Gunturket al.]
                            [Durand et al.]
                             [Durand et al.]   [Reinhard et al.]
                                                [Reinhard et33al.]
    [Fattal
Spatially Varying Exposures

    Instead of capturing multiple pictures, allow different
     amounts of light pass for different pixel positions.
    Estimate the missing pixels.
    Combine to obtain an HDR image.

                                  100%            75%

                                   50%            25%




                         Nayar



Bahadir K. Gunturk                                             34
Image Reconstruction: Interpolation




Bahadir K. Gunturk                    35
Image Reconstruction: Aggregation




Bahadir K. Gunturk                  36
HDR image examples




Bahadir K. Gunturk   37
HDR image examples




Bahadir K. Gunturk   38
HDR image examples




Bahadir K. Gunturk   39
The Bilateral Filter (BF)

    The SUSAN filter, which is essentially the bilateral filter, was
     used for corner/edge detection and denoising in [Smith &
     Brady 97].
    The BF was presented in [Tomasi & Manduchi 98].
    [Elad 02] and [Barash 02] show that the BF is related to the
     weighted least squares estimation and anisotropic diffusion.
    Fast implementations/approximations have been proposed,
     e.g., in [Paris & Durand 06].
    In addition to image denoising, the BF is used in tone
     mapping of HDR images, contrast enhancement, 3D mesh
     smoothing, blocking artifact reduction, etc.




Bahadir K. Gunturk                                                      40
    Bilateral Filtering




             Intensity (range)
             proximity
                                                Spatial (domain)
                                                proximity



ˆ( x )  1   e      I ( y )  I ( x )  / 2 r        y  x / 2 d
                                       2        2              2     2
I                                                   e                    I ( y)
         K     y


    Bahadir K. Gunturk                                                            41
Bilateral Filtering

               Input




           Gaussian
               d  10




            Bilateral
                d  10
                r  1.3

Bahadir K. Gunturk         42
What are the optimal values of the
parameters of the Bilateral Filter?
                                  d  2    d  4



                      r  10




                                MSE=49.8   MSE=50.9


      MSE=100.0
        n  10
                      r  30




                                MSE=30.3   MSE=43.4




                      r  50



Bahadir K. Gunturk                                    43
                                MSE=42.5   MSE=71.5

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:6
posted:2/9/2012
language:
pages:43