# Blind_Deconvolution

Document Sample

```					Single Image Blind Deconvolution

Presented By:
Tomer Peled
&
Eitan Shterenbaum
Agenda
1. Problem Statement
2. Introduction to Non-Blind Deconvolution
3. Solutions & Approaches
A. Image Deblurring
PSF Estimation using Sharp Edge Prediction / Neel Joshi et. Al.

B. MAPx,k Solution Analysis
Understanding and evaluating blind deconvolution algorithms / Anat Levin et. Al.

C. Variational Method MAPk
Removing Camera Shake from a Single Photograph / Rob Fergus et. Al

4. Summary
2
Problem statement

• Blur = Degradation of sharpness and contrast
of the image, causing loss of high frequencies.
• Technically - convolution with certain kernel
during the imaging process.

3
Camera Motion blur

4
Defocus blur

5
Defocus blur

6
Defocus blur

7
Blur – generative model

=

Sharp image        Point Spread Function       Blured image

                        =

fft(Image)         Optical Transfer Function   fft(Blured image)
Object Motion blur

9
Local Camera Motion

10
Depth of field – Local defocus

11
Evolution of algorithms

?               Volunteers ?

Shan              2008

Camera motion blur      Fergus            2006

Simple kernels        Joshi             2008

Lucy Richardson   1972
Non blind deconvolution
Wiener            1949
12
Introduction to Non-Blind
Deconvolution

sharp
blurred image               blur kernel   noise
image

Deconvolution Evolution:
Simple no-Noise Case

Noise Effect Over Simple Solution

Wiener Deconvolution
RL Deconvolution
13
Simple no-Noise Case:

Blurred
Recovered
Image

14
Noisy case:

Original (x)    Blured + noise (y)   Recovered x

15
Noisy case, 1D Example:
FT of original signal         Original signal

Reconstructed FT of the signal
FT of convolved signals   Convolved signals w/o noise
sd

Noisy Signal
Original Signal

16      High Frequency Noise Amplified
Wiener Deconvolution

Blurred noisy image      Recovered image

18
Non Blind Iterative Method :
Richardson –Lucy Algorithm
Assumptions:
Blurred image yi~P(yi), Sharp image xj~P(xj)
i point in y, j point in x

where

Target:
Recover P(x) given P(y) & P(y|x)
From Bayes theorem Object distribution can be expressed iteratively:

Richardson, W.H., “Bayesian-Based Iterative Method of Image Restoration”, J. Opt. Soc. Am., 62, 55, (1972).
Lucy, L.B., “An iterative technique for the rectification of observed distributions”, Astron. J., 79, 745, (1974).

19
Richardson-Lucy Application
Simulated Multiple Star

measurement       PSF    Identification
reconstruction           of 4th Element

20
Solution Approaches
A. Image Deblurring
PSF Estimation using Sharp Edge Prediction
Neel Joshi Richard Szeliski David J. Kriegman

B. MAPx,k Solution Analysis
Understanding and evaluating blind deconvolution algorithms
Anat Levin, Yair Weiss, Fredo Durand, William T. Freeman

C. Variational Method MAPk
Removing Camera Shake from a Single Photograph
Rob Fergus, Barun Singh, Aaron Hertzmann, Sam T. Roweis, William T. Freeman

23
PSF Estimation by Sharp Edge
Prediction
Given edge steps, debluring can be reduced to Kernel Optimization

Suggested in PSF Estimation by Sharp Edge Prediction \ Neel Joshi et. el. in

Estimate Blurring Kernel

Recover Latent Image

24
PSF Estimation by Sharp Edge

Max

Valid Region

Min

25
Which is Best the Signals ?
Original           Blurred

Edge

Impulse

26
PSF Estimation by Sharp Edge
Prediction – PSF Estimation

Blurr Model: y=x*k+n, n ~ N(0,σ2)

Bayseian Framework: P(k|y) = P(y|k)P(k)/P(y)
Map Model:
argmaxk P(k|y) = argmink L(y|k) + L(k)

27
PSF Estimation by Sharp Edge
Prediction – Recovery
Recovery through Lucy-Richardson Iterations
given the PSF kernel

Blurred           Recovered

28
PSF Estimation by Sharp Edge,
Improvements & Summary
1. Handle RGB Images – perform processing in
parallel
2. Local Kernel Variations:
Sub divide image into sub-image units

Limitations:
– Highly depends on the quality of the edge detection
– Requires Strong Edges in multiple orientations for
proper kernel estimation
– Assumes knowledge of noise error figure.

29
MAPx,k , Blind Deconvolution Definition

blurred image blur kernel sharp                       noise
image

Input             Unknown, need to
(known)                 estimate

?

?
30                    Courtesy of Anat Levin CVPR 09 Slides
MAPx,k Cont. - Natural Image Priors

Gaussian:
-x2

Log prob
Laplacian: -
-|x|0.5            |x|

-|x|0.25

x
x
Derivative histogram                                           Parametric models
from a natural image

Derivative distributions in natural images are sparse:

31                          Courtesy of Anat Levin CVPR 09 Slides
Naïve MAPx,k estimation

Given blurred image y,
Find a kernel k and latent image x minimizing:

Convolution                     Sparse prior
constraint

Should favor sharper x explanations

32                  Courtesy of Anat Levin CVPR 09 Slides

P(                   ,   kernel
)>P(                                           ,   kernel
)
Latent image                                                 Latent image

Let be an arbitrarily large image sampled from a sparse
prior , and
Then the delta explanation is favored

33                         Courtesy of Anat Levin CVPR 09 Slides
The MAPx,k failure
sharp                                           blurred

?
34           Courtesy of Anat Levin CVPR 09 Slides
The MAPx,k failure
15x15 windows     25x25 windows       45x45 windows

simple
derivatives
[-1,1],[-1;1]

FoE filters
(Roth&Black)

35              Red windows = [ p(sharp x) >p(blurred x) ]
The MAPx,k failure - intuition
P(step edge)                   >                P
(blurred step edge)
k=[0.5,0.5]

sum of derivatives:                 cheaper

P(impulse)                    <                P(blurred impulse)

cheaper
sum of derivatives:

36                         Courtesy of Anat Levin CVPR 09 Slides
MAPx,k Cont. - Blur Reduces Derivative Contrast

P(sharp real image)            <                   P(blurred real image)

cheaper

Noise and texture behave as impulses -
total derivative contrast reduced by blur

37                    Courtesy of Anat Levin CVPR 09 Slides
MAPx,k Reweighting Solution
High Quality Motion Debluring From Single Image / Shan et al.

Alternating Optimization Between x & k

Minimization term:

MAPx,k
MAPx,k Reweighting
- Blurred

39
39
MAPx,k Reweighting
- Recovered

40
40
Solution Approaches
A. Image Deblurring
PSF Estimation using Sharp Edge Prediction
Neel Joshi Richard Szeliski David J. Kriegman

B. MAPx,k Solution Analysis
Understanding and evaluating blind deconvolution algorithms
Anat Levin, Yair Weiss, Fredo Durand, William T. Freeman

C. Variational Method MAPk
Removing Camera Shake from a Single Photograph
Rob Fergus, Barun Singh, Aaron Hertzmann, Sam T. Roweis, William T. Freeman

47
MAPk estimation

Given blurred image y, Find a kernel minimizing:

Convolution Sparse prior   Kernel prior
constraint

Again, Should favor sharper x explanations
48
Superiority of MAPk over MAPk,x

Toy Problem : y=kx+n

uncertainty of p(k|y) reduces
The joint distribution p(x, k|y).   p(k|y) produce optimum
given multiple observations
Maximum for x → 0, k → ∞.           closer to true k∗.
yj =kxj + nj .

49
Evaluation on 1D signals

Exact MAPk                     Favors delta           MAPx,k
solution
Favor correct
solution despite
wrong prior!

MAPk Gaussian prior                 MAPk variational approximation
50                                              (Fergus et al.)
Courtesy of Anat Levin CVPR 09 Slides
Intuition: dimensionality asymmetry

kernel k
blurred image y                  sharp image x
~105 measurements          Large, ~105 unknowns Small, ~102 unknowns

MAPx,k– Estimation unreliable.
Number of measurements always lower than number
of unknowns: #y<#x+#k

MAPk – Estimation reliable.
Many measurements for large images: #y>>#k

51                      Courtesy of Anat Levin CVPR 09 Slides
52   Courtesy of Rob Fergus Slides
Three sources of information

53             Courtesy of Rob Fergus Slides
Image prior p(x)

55       Courtesy of Rob Fergus Slides
Blur prior p(b)

56     Courtesy of Rob Fergus Slides
The obvious thing to do

57          Courtesy of Rob Fergus Slides
Variational Bayesian approach

58             Courtesy of Rob Fergus Slides
Variational Bayesian methods
• Variational Bayesian = ensemble learning,
• A family of techniques for approximating intractable
integrals arising in Bayesian inference and machine learning.
• Lower bound the marginal likelihood (i.e. "evidence") of
several models with a view to performing model selection.

59
Setup of Variational Approach
Ensemble Learning for Blind
Source Separation / J.W. Miskin , D.J.C. MacKay
Cartoon
natural
images
images

Small
large real
synthetic
world blurs
blurs

An introduction to variational methods for graphical models \ JORDAN M. et al.
61
Independent Factor Analysis \ H. Attias
63
64
65
66   Courtesy of Rob Fergus Slides
Example 1

67
Output 1

68
Example 2

69
Output 2

70
Example 3

71
Output 3

72
Achievements
• Work on real world images
• Deals with large camera motions
(up to 60 pixels)
• Getting close to practical generic solution
of an old problem .

73
Limitations
• Targeted at camera motion blur
– No in plane rotation
– No motion in picture
– Out of focus blur
• Manual input
– Region of Interest
– Kernel size & orientation
– Other parameters e.g. scale offset, kernel TH & 9 other
semi-fixed parameters
• Sensitive to image compression, noise(dark images)
& saturation
• Still contains artifacts (solvable by upgrading from Lucy Richardson)
74
Evaluation

Fergus, variational MAPk
Shan et al. SIGGRAPH08
MAPx,k sparse prior
MAPk, Gaussian prior

100
Successes percent

80

60

40

20

Cumulative histogram of deconvolution successes :

75                                        bin r = #{ deconv error < r }
Summary
Method               Quasi-MAPK             Reweighted MAPKX        Variational MAPk
Joshi                  Shan                    Fergus
Distortion model     Defocus blur           Camera motion blur      Camera motion blur
simple PSF             Complex sparse PSF      Complex sparse PSF

Region of interest   Edge region            Edge region             User selected

Optimization model   Quasi-MAPK             MAPKX                   Variational Bayes for
K estimation (MAPk
equivalent)
Degrees of freedom   O(K)                   O(K+X)                  O(K+Xprior+PRIOR)

Scheme               Gradient based least   Alternating iterative   Multiscale iterative
squares                                        (internal altering)
78
Debluring is underconstrained
under constrained
Debluring single image
problem

Recovered
kernel

Recovered image
Blured image
?
Priors do the trick

Image    Recovered
prior     kernel

Blured image
?
Kernel marginalization

Image    Recovered
prior     kernel

Blured image
?
Back to non-blind deconvolution

Recovered
kernel

Recovered image   Blured image

?
Existing challenges
and potential research
• Robustness to user’s parameters & initial
priors
• Solutions to spatially varying kernels

84
The End
Thank You
Eitan & Tomer

```
DOCUMENT INFO
Shared By:
Categories:
Stats:
 views: 844 posted: 1/8/2011 language: English pages: 72