NEW METHOD OF SUPER-RESOLUTION IMAGE RESTORATION FOR ELECTRONIC
IMAGING ISR SYSTEM
Chen H.X., 2Zhang D.M.
Southwest China Institute of Electronic Technology, Chengdu, 610036, China;
Southwest China Jiaotong University, Chengdu, 610031, China;
In electronic imaging ISR systems, degraded images are caused by the hardware limited
resolution or blurry imaging. In this paper, we propose a new super-resolution image restoration approach
from multiple low resolution images. The locally image edge preserving prior for the super-resolution
problem is realized by a scale-alterable Gauss-Makov adaptive prior model and the sample-blur kernel.
The experimental results show that the proposed method have high reconstructed image quality with good
edges-preserving and low spatial noise.
In remote electronic imaging ISR systems (ISR: Intelligence, Surveillance, Reconnaissance), high
resolution and clear images are often required for target identification or situation awareness of
battlefield. In real situation, the captured images are often of low-resolution (LR), blurry and noisy
observations, which limit related military and civil application extremely. Super-resolution restoration
(SRR), which to recover a high resolution image from multiple low resolution observations degraded by
warping, blurring et .al. using the signal processing way, is an effective solution to meet this problem, and
become one of the most important and hot topic of image research nowadays due to its wide usefulness.
In spatial domain related SRR approach [1~5], MAP SRR is one of the most popular and effective
approach, which is robust and flexible in modeling noise characteristics and a spatial priori knowledge
about the solution. In this paper, we propose a new MAP SRR approach for remote electronic imaging
reconnaissance. Specifically our approach is of key ideas as following: (i) scale-alterable Gauss-Makov
adaptive prior: edge-preserving prior with different noisy removal capability; (ii) sample-blur kernel:
model the sub-sampling and blur distortion jointly.
Observation Model and MAP Frame
We utilize the observation model for sub-sampling, de-blurring of image procedure as following:
y p DBM p x k n p kN pKN (1)
while x (k) is the original SR image vector of size q2WH×1 to be reconstructed, q is an integer-valued sub-
sampling factor, W and H is the width and height of LR images respectively. y (p) (k-N≤p≤k+N) is one of
2N+1 LR image vectors of size WH×1. n(p) (k-N≤p≤k+N) is one of 2N noising vectors of size WH×1.D is
an averaging sub-sampling matrix of size WH×q2WH,and M(p) (k-N≤p≤k+N) is one of 2N+1 motion
warping matrix between the p-th SR image and x (k) .
Super-resolution reconstruction based on MAP (MAP-SRR) frame is to maximize the posterior
probability P(x (k)|y (k-N),..., y (k) ,...,y (k+N)), and the related vector x (k) at the maximum is regarded as the
reconstructed SR image. Based on Bayesian theorem, the MAP-SRR frame can be represented as:
p k N
x(k ) arg min log P xk log P y p xk
p k N
In (2), the former logarithmic term represents the prior image model of SR image x (k), and the latter one
represents the conditional density between LR image y (p) and x (k) .
SRR method with image edge-preserving Scale-alterable Gauss-Makov Adaptive prior
In order to overcome the disadvantage of over-smoothing edges of conventional Gauss-Makov prior
model, we propose a edge-preserving adaptive Gauss-Makov prior model as follow:
1 1 qW 1 s qH 1 s s s
P(x) = exp i , j ( xi , j H P [n, m]xi n, j m ) 2 (3)
Z i s 1 j s 1 n s m s
where Z is a normalizing constant, λ is the “temperature” parameter of the density, αi,j is the adaptive
coefficient of pixel xi,j in location (i,j), HP is a (2s+1)×(2s+1) kernel, s is the scale factor. HP can usually
be chosen as averaging or Gaussian kernel, whose scale s can be adjust manually.
Sample-Blur Kernel and Probability Density
We define sample-blur kernel by combination the sub-sampling and blur distortion:
q 1 q 1
H B S [ n1 , n2 ]
j 0 i 0
B [n1 j S , n2 i S ] (4)
where, HB is a (2S+1)×(2S+1) blur kernel, S is the scale factor.
For np (k-N≤ p ≤ k+N, p≠k), modify probability of the model as fellow:
P(y p |x k )=
1 1 W H S q S q 2 (5)
2 ( y pi, j H B-S [ n, m] x k M _ TAB p (i*q n , j*q m ) ) )
(2 (p,k) ) n S m S
2 i=1 j=1
Where, M_TAB p(x,y) represents Mp which means the related matching pixels’ coordinates of xk from
xp(x,y)( p≠k),where xp is the top level up-sample image of yp during hierarchical block-matching
algorithm to obtain sub-pixel precision motion vector.
The peak signal-to-noise ratio (PSNR) is employed as holistic quality measurement for reconstructed
SR image, and the edge-signal-to-noise ratio (EPSNR) is used as measure the edge-preserving ability,
which formulation is as fellows:
2552 * q 2WH 2552 * q 2WH
PSNR 10 log10 dB EPSNR 10 log10 (6)
ˆ Ostu Sobel ( x) Ostu Sobel ( x)
where x and x are original and reconstructed SR images respectively, Sobel(.) and Ostu(.) are operators
which imply edge extraction by Sobel operator and threshold, respectively.
Fig.1(a) is a original high resolution image of electronic imaging system, downloaded in the
Internet; Fig.1(b) is one of synthesis low resolution degraded images, which generated from Fig.1(a) by
shift warping, 5×5 Gaussian blur, 2×2 sub-sampling.
Fig.1(c),(d) is the SRR results of a classic SRR algorithm proposed in. and our approach proposed
in this paper, respectively. Obviously, our method achieves the better visual result and good index of
PSNR and EPSNR of reconstruction image, which demonstrates the effectiveness of the method.
Fig.2(a),(b) show the curves of PSNR and EPSNR versus SNR corresponding to Fig.1, which
demonstrate the relative robustness of noise of our SRR algorithm proposed in this paper.
(a) Original high-resolution image (b) Synthesis low-resolution image,
downloaded in Internet PSNR=20.331dB, EPSNR=8.829dB.
(c) Result in , PSNR=20.795dB, (d) Result of this paper,
EPSNR=9.732dB. PSNR=22.340dB, EPSNR=10.326dB.
Fig.1 Experiment results of SRR image
(a) PSNR versus SNR (b) EPSNR versus SNR
Fig.2 PSNR and EPSNR of SRR image versus SNR
As for application of electronic imaging ISR systems, we proposed scale-alterable Gauss-Makov
adaptive prior model and the sample-blur kernel, improve the visual effect of SRR image with edge-
preserving prior and high spatial resolution. Furthermore, our method is of robust and flexible under
various noise levels, which offer a effective approach for SRR problem in practical applications.
1 Park S.C., Park M.K., Kang MG. Super-resolution image reconstruction: A technical overview. IEEE
Signal Processing Magazine, 2003 ,20 (3):21-36
2 Tom B.C., Katsaggelos A.K., Reconstruction of a high resolution image from multiple degraded
misregistered low resolution images. Proceedings of SPIE VCIP , Chicago , 1994 ,2308:971-981
3 Schultz R.R., Stevenson R.L., A Bayesian approach to image expansion for improved definition. IEEE
Transactions on Image Processing , 1994 , 3 (2):233-242
4 Schultz R.R., Stevenson R.L., Extraction of high resolution frames from video sequences. IEEE
Transactions on Image Processing , 1996 , 5 (6):996-1001
5 Elad M., Feuer A., Restoration of a single super resolution image from several blurred, noisy, and
undersampled measured images. IEEE Transactions on Image Processing,1997, 6(12): 1646-1658