Docstoc

Blind_Deconvolution

Document Sample
Blind_Deconvolution Powered By Docstoc
					Single Image Blind Deconvolution

            Presented By:
            Tomer Peled
                  &
         Eitan Shterenbaum
                                   Agenda
    1. Problem Statement
    2. Introduction to Non-Blind Deconvolution
    3. Solutions & Approaches
      A. Image Deblurring
         PSF Estimation using Sharp Edge Prediction / Neel Joshi et. Al.

      B. MAPx,k Solution Analysis
         Understanding and evaluating blind deconvolution algorithms / Anat Levin et. Al.

      C. Variational Method MAPk
         Removing Camera Shake from a Single Photograph / Rob Fergus et. Al

    4. Summary
2
           Problem statement

• Blur = Degradation of sharpness and contrast
  of the image, causing loss of high frequencies.
• Technically - convolution with certain kernel
  during the imaging process.




3
    Camera Motion blur




4
    Defocus blur




5
    Defocus blur




6
    Defocus blur




7
              Blur – generative model


                                        =

Sharp image        Point Spread Function       Blured image




                                       =


fft(Image)         Optical Transfer Function   fft(Blured image)
    Object Motion blur




9
     Local Camera Motion




10
     Depth of field – Local defocus




11
     Evolution of algorithms

               ?               Volunteers ?



                               Shan              2008

       Camera motion blur      Fergus            2006


         Simple kernels        Joshi             2008


                               Lucy Richardson   1972
     Non blind deconvolution
                               Wiener            1949
12
     Introduction to Non-Blind
           Deconvolution

                         sharp
      blurred image               blur kernel   noise
                         image



             Deconvolution Evolution:
                      Simple no-Noise Case


              Noise Effect Over Simple Solution


                      Wiener Deconvolution
                         RL Deconvolution
13
      Simple no-Noise Case:


     Blurred
                   Recovered
     Image




14
                    Noisy case:



     Original (x)    Blured + noise (y)   Recovered x




15
     Noisy case, 1D Example:
          FT of original signal         Original signal




       Reconstructed FT of the signal
         FT of convolved signals   Convolved signals w/o noise
                                                            sd

                                      Noisy Signal
                                      Original Signal




16      High Frequency Noise Amplified
             Wiener Deconvolution



     Blurred noisy image      Recovered image




18
Non Blind Iterative Method :
            Richardson –Lucy Algorithm
Assumptions:
            Blurred image yi~P(yi), Sharp image xj~P(xj)
            i point in y, j point in x

                                        where


Target:
Recover P(x) given P(y) & P(y|x)
From Bayes theorem Object distribution can be expressed iteratively:




Richardson, W.H., “Bayesian-Based Iterative Method of Image Restoration”, J. Opt. Soc. Am., 62, 55, (1972).
Lucy, L.B., “An iterative technique for the rectification of observed distributions”, Astron. J., 79, 745, (1974).

19
Richardson-Lucy Application
Simulated Multiple Star




     measurement       PSF    Identification
     reconstruction           of 4th Element




20
                  Solution Approaches
     A. Image Deblurring
        PSF Estimation using Sharp Edge Prediction
        Neel Joshi Richard Szeliski David J. Kriegman

     B. MAPx,k Solution Analysis
        Understanding and evaluating blind deconvolution algorithms
        Anat Levin, Yair Weiss, Fredo Durand, William T. Freeman

     C. Variational Method MAPk
        Removing Camera Shake from a Single Photograph
        Rob Fergus, Barun Singh, Aaron Hertzmann, Sam T. Roweis, William T. Freeman




23
        PSF Estimation by Sharp Edge
                 Prediction
Given edge steps, debluring can be reduced to Kernel Optimization

Suggested in PSF Estimation by Sharp Edge Prediction \ Neel Joshi et. el. in


                          Select Edge Step (Masking)




                           Estimate Blurring Kernel




                            Recover Latent Image


  24
      PSF Estimation by Sharp Edge
           Prediction - Masking
     Original Image   Edge Prediction           Masking


                                          Max




                                Valid Region


                          Min




25
              Masking, Cont.
          Which is Best the Signals ?
            Original           Blurred


Edge




Impulse




   26
      PSF Estimation by Sharp Edge
       Prediction – PSF Estimation

     Blurr Model: y=x*k+n, n ~ N(0,σ2)

     Bayseian Framework: P(k|y) = P(y|k)P(k)/P(y)
     Map Model:
     argmaxk P(k|y) = argmink L(y|k) + L(k)



27
      PSF Estimation by Sharp Edge
          Prediction – Recovery
Recovery through Lucy-Richardson Iterations
given the PSF kernel

                     Blurred           Recovered




 28
      PSF Estimation by Sharp Edge,
       Improvements & Summary
1. Handle RGB Images – perform processing in
   parallel
2. Local Kernel Variations:
     Sub divide image into sub-image units

Limitations:
     – Highly depends on the quality of the edge detection
     – Requires Strong Edges in multiple orientations for
       proper kernel estimation
     – Assumes knowledge of noise error figure.

29
     MAPx,k , Blind Deconvolution Definition



        blurred image blur kernel sharp                       noise
                                  image

              Input             Unknown, need to
            (known)                 estimate




                                ?

                                ?
30                    Courtesy of Anat Levin CVPR 09 Slides
           MAPx,k Cont. - Natural Image Priors

                                                                               Gaussian:
                                                                                     -x2




                                             Log prob
                                                                              Laplacian: -
                                                                    -|x|0.5            |x|



                                                                -|x|0.25


               x
                                                                              x
     Derivative histogram                                           Parametric models
     from a natural image


           Derivative distributions in natural images are sparse:


31                          Courtesy of Anat Levin CVPR 09 Slides
              Naïve MAPx,k estimation

     Given blurred image y,
     Find a kernel k and latent image x minimizing:



                      Convolution                     Sparse prior
                       constraint


         Should favor sharper x explanations




32                  Courtesy of Anat Levin CVPR 09 Slides
                     The MAPx,k paradox



P(                   ,   kernel
                                   )>P(                                           ,   kernel
                                                                                               )
      Latent image                                                 Latent image



Let be an arbitrarily large image sampled from a sparse
prior , and
Then the delta explanation is favored




33                         Courtesy of Anat Levin CVPR 09 Slides
         The MAPx,k failure
     sharp                                           blurred




                          ?
34           Courtesy of Anat Levin CVPR 09 Slides
                         The MAPx,k failure
                15x15 windows     25x25 windows       45x45 windows


  simple
derivatives
[-1,1],[-1;1]




 FoE filters
(Roth&Black)




    35              Red windows = [ p(sharp x) >p(blurred x) ]
                      The MAPx,k failure - intuition
                      P(step edge)                   >                P
                                                                      (blurred step edge)
                                                                                         k=[0.5,0.5]




    sum of derivatives:                 cheaper




                       P(impulse)                    <                P(blurred impulse)

                                                                                           cheaper
sum of derivatives:

   36                         Courtesy of Anat Levin CVPR 09 Slides
 MAPx,k Cont. - Blur Reduces Derivative Contrast

     P(sharp real image)            <                   P(blurred real image)




                                                                                cheaper



           Noise and texture behave as impulses -
           total derivative contrast reduced by blur

37                    Courtesy of Anat Levin CVPR 09 Slides
         MAPx,k Reweighting Solution
High Quality Motion Debluring From Single Image / Shan et al.

 Alternating Optimization Between x & k

 Minimization term:




                  MAPx,k
     MAPx,k Reweighting
         - Blurred




39
                          39
     MAPx,k Reweighting
       - Recovered




40
                          40
                  Solution Approaches
     A. Image Deblurring
        PSF Estimation using Sharp Edge Prediction
        Neel Joshi Richard Szeliski David J. Kriegman

     B. MAPx,k Solution Analysis
        Understanding and evaluating blind deconvolution algorithms
        Anat Levin, Yair Weiss, Fredo Durand, William T. Freeman

     C. Variational Method MAPk
        Removing Camera Shake from a Single Photograph
        Rob Fergus, Barun Singh, Aaron Hertzmann, Sam T. Roweis, William T. Freeman




47
                   MAPk estimation

     Given blurred image y, Find a kernel minimizing:




                    Convolution Sparse prior   Kernel prior
                     constraint

      Again, Should favor sharper x explanations
48
                        Superiority of MAPk over MAPk,x


                             Toy Problem : y=kx+n




                                                             uncertainty of p(k|y) reduces
The joint distribution p(x, k|y).   p(k|y) produce optimum
                                                             given multiple observations
Maximum for x → 0, k → ∞.           closer to true k∗.
                                                             yj =kxj + nj .


     49
           Evaluation on 1D signals




        Exact MAPk                     Favors delta           MAPx,k
                                         solution
         Favor correct
        solution despite
          wrong prior!




     MAPk Gaussian prior                 MAPk variational approximation
50                                              (Fergus et al.)
                      Courtesy of Anat Levin CVPR 09 Slides
             Intuition: dimensionality asymmetry




                                                                   kernel k
  blurred image y                  sharp image x
~105 measurements          Large, ~105 unknowns Small, ~102 unknowns



        MAPx,k– Estimation unreliable.
               Number of measurements always lower than number
               of unknowns: #y<#x+#k


        MAPk – Estimation reliable.
               Many measurements for large images: #y>>#k

   51                      Courtesy of Anat Levin CVPR 09 Slides
52   Courtesy of Rob Fergus Slides
     Three sources of information




53             Courtesy of Rob Fergus Slides
     Image prior p(x)




55       Courtesy of Rob Fergus Slides
     Blur prior p(b)




56     Courtesy of Rob Fergus Slides
     The obvious thing to do




57          Courtesy of Rob Fergus Slides
     Variational Bayesian approach




58             Courtesy of Rob Fergus Slides
     Variational Bayesian methods
• Variational Bayesian = ensemble learning,
• A family of techniques for approximating intractable
  integrals arising in Bayesian inference and machine learning.
• Lower bound the marginal likelihood (i.e. "evidence") of
  several models with a view to performing model selection.




59
Setup of Variational Approach
       Ensemble Learning for Blind
    Source Separation / J.W. Miskin , D.J.C. MacKay
                                                                   Gradients of
                                            Cartoon
                                                                     natural
                                            images
                                                                     images



                                             Small
                                                                    large real
                                           synthetic
                                                                   world blurs
                                             blurs




An introduction to variational methods for graphical models \ JORDAN M. et al.
     61
Independent Factor Analysis \ H. Attias
63
64
65
66   Courtesy of Rob Fergus Slides
     Example 1




67
     Output 1




68
     Example 2




69
     Output 2




70
     Example 3




71
     Output 3




72
               Achievements
• Work on real world images
• Deals with large camera motions
  (up to 60 pixels)
• Getting close to practical generic solution
  of an old problem .




73
                         Limitations
• Targeted at camera motion blur
    – No in plane rotation
    – No motion in picture
    – Out of focus blur
• Manual input
    – Region of Interest
    – Kernel size & orientation
    – Other parameters e.g. scale offset, kernel TH & 9 other
      semi-fixed parameters
• Sensitive to image compression, noise(dark images)
   & saturation
• Still contains artifacts (solvable by upgrading from Lucy Richardson)
74
                                              Evaluation

                                                                          Fergus, variational MAPk
                                                                          Shan et al. SIGGRAPH08
                                                                          MAPx,k sparse prior
                                                                          MAPk, Gaussian prior

                    100
     Successes percent




                         80

                         60

                         40

                         20




                              Cumulative histogram of deconvolution successes :

75                                        bin r = #{ deconv error < r }
                                   Summary
Method               Quasi-MAPK             Reweighted MAPKX        Variational MAPk
                     Joshi                  Shan                    Fergus
Distortion model     Defocus blur           Camera motion blur      Camera motion blur
                     simple PSF             Complex sparse PSF      Complex sparse PSF



Region of interest   Edge region            Edge region             User selected


Optimization model   Quasi-MAPK             MAPKX                   Variational Bayes for
                                                                    K estimation (MAPk
                                                                    equivalent)
Degrees of freedom   O(K)                   O(K+X)                  O(K+Xprior+PRIOR)

Scheme               Gradient based least   Alternating iterative   Multiscale iterative
                     squares                                        (internal altering)
    78
    Debluring is underconstrained
                                       under constrained
Debluring single image
                                           problem




                                          Recovered
                                            kernel

                     Recovered image
Blured image
                                                       ?
               Priors do the trick




                        Image    Recovered
                         prior     kernel

Blured image
                                             ?
               Kernel marginalization




                          Image    Recovered
                           prior     kernel

Blured image
                                               ?
    Back to non-blind deconvolution




                                     Recovered
                                       kernel

    Recovered image   Blured image

?
          Existing challenges
        and potential research
• Robustness to user’s parameters & initial
  priors
• Solutions to spatially varying kernels




84
The End
  Thank You
 Eitan & Tomer

				
DOCUMENT INFO