Document Sample

2010 3rd International Congress on Image and Signal Processing (CISP2010) A PDE Approach to Super-resolution with Contrast Enhancement Weili Zeng Xiaobo Lu School of Transportation School of Automation Southeast University Southeast University Nanjing, Jiangsu 210096, P. R. China Nanjing, Jiangsu 210096, P. R. China zengwlj@yahoo.com.cn Corresponding author: xblu2008@yahoo.cn Abstract—Multi frame super resolution (SR) reconstruction developed. Hardie et al.[11] proposed a joint MAP registration algorithms make use of complimentary information among low and restoration algorithm using a Gibbs image prior. Schultz resolution (LR) images to yield a high resolution (HR) image. In and Stevenson [2] proposed a Bayesian MAP estimator using a this paper, we first present a fast partial differential equation Huber-Markov random field model as an edge preserving prior. (PDE) model for multi-frame image super resolution Farsiu et al [6] proposed an alternative data fidelity, or reconstruction. We then combine our proposed super resolution regularization term based on the L1 norm which has been model with the local histogram equalization (LHE), which shown to be robust to data outliers. More recently, X. L. Xue et perform super resolution and enhance image contrast al. [12] proposed two new regularization items, termed as simultaneously. It overcomes the shortcomings of recent locally adaptive bilateral total variation and consistency of promising super resolution methods dealt with super resolution and contrast enhancement separately. Our technique not only gradients, to keep edges and flat regions, which are implicitly reconstructs a high-resolution image from several overlapping described in LR images, sharp and smooth, respectively. noisy low resolution images, but also enhances edges and image In this paper, we first propose a fast partial differential contrast while suppressing image noise during the reconstruction equation (PDE) model for image super resolution. Moreover, it process. Experiments show the effectiveness and practicability of is well known that the classical super-resolution methods dealt the proposed method and demonstrating its superiority to the with super resolution and contrast enhancement separately. For existing SR method. example, If contrast enhancement is applied firstly, the noise Keywords-Super-resolution; contrast enhancement; Partial may be enhanced significantly. On the other hand, if super- differential equation (PDE) ; low resolution resolution is performed firstly, the weak edges may be destroyed. In both cases, the resulted images are hardly acceptable. Based on this reason, we combine our proposed I. INTRODUCTION super resolution model with the local histogram equalization, Super-resolution (SR) techniques aim at estimating a high- which perform super-resolution and enhance image contrast resolution (HR) image by utilizing complimentary information simultaneously. It overcomes the shortcomings of recent among low-resolution (LR) images. In many civilian and promising super resolution methods dealt with super resolution military applications, such as medical imaging, traffic video and contrast enhancement separately. Experiments show the sequences using low cost sensors, HR images are always effectiveness and practicability of the proposed method. required. Therefore, SR reconstruction is currently an active The structure of this paper is organized as follows. In research topic in image processing [1-7]. section II, we first describe the image degradation model, and SR reconstruction method can not only increase the image we then derive a PDE method for super resolution that pixels but also get more details and information by consider the corresponds to the solution of the described image degradation degradation process. There are many reasons leading to the model. The numerical scheme for the PDE is followed in degradation of the required video quality, such as atmospheric section III. Experimental results are presented in Section 4 and disturbances, motion, focus, down-sampling and the noise. Section 5 gives some conclusions. From a restoration point of way, the idea is to increase the information content in the reference image by using the II. PROPBLEM ALGORITHM additional spatial-temporal information that is available in each of the LR images. A variety of SR algorithms have been A. Image Degradation Model proposed since the multi-frame SR problem was first addressed in [1]. Irani and Peleg [8] proposed an iterative back-projection Given N consecutive low resolution images, denoted method to address the super-resolution problem. Mann and as Yk , k = 1, 2,..., n , which were cropped from a video，we Picard [9] extended this to the projective case, and Zomet and use the notation in [5] to formulate the general SR model. The Peleg[10] rendered the original implementation more efficient, relationship between the LR observation and the original HR and applied it to image mosaics. To preserve edge information image is expressed by a set of linear equations while removing image noise, many algorithms have been 978-1-4244-6516-3/10/$26.00 ©2010 IEEE 600 Yk = Dk Bk Fk X + Vk , 1 ≤ k ≤ n (1) Using the gradient descent method formulated as the time evolution procedure to solve the Euler-Lagrange equation (6), we have corresponding partial differential equation, where X represents the HR image of size [ N1 N 2 × 1] which is represented by rearranged in lexicographic order; Yk and Vk are ∂f n ∇f M 1M 2 elements column representing the kth observed LR = −∑ Tk [Η k ( f ) − yk ] + λ div( ) (7) ∂t k =1 | ∇f | images and the additive noise, respectively; Bk denotes the with homogeneous Neumann boundary conditions and camera lens blur matrix, of size [ N1 N 2 × N1 N 2 ] ; Fk is a implementing a simple bilinear interpolation of the reference [ N1 N 2 × N1 N 2 ] matrix that models the motion corresponding image as initialization. to the reference frame; Dk is the decimation matrix , of size C. Accelerating Evolution Procedure [ M1M 2 × N1 N 2 ] . The model (7) often converges very slowly to its steady state since the parabolic term is quite singular for small B. A PDE Method for Super-resolution gradients [14]. In order to accelerate the evolution procedure As we know, image SR problem is an ill-posed. In order to we multiply the Euler-Lagrange equation (6) by the magnitude obtain a stable solution, a regularization term is always of the gradient and our new time depend fast super resolution introduced to the observation model. The following expression (FSR) model reads as follows formulates a generalized minimization cost function in SR ∂f n ∇f reconstruction process [10] = − | ∇f | ∑ Tk [Η k ( f ) − yk ]+ | ∇f | λ div( ) (8) ∂t k =1 | ∇f | ⎧n ⎫ F ( X ) = arg min ⎨∑ ρ (Yk , Dk Bk Fk X ) + λΓ( X ) ⎬ . (2) with homogeneous Neumann boundary conditions and X ⎩ k =1 ⎭ implementing a simple bilinear interpolation of the reference image as initialization as above. where λ is the regularization parameter which provide a tradeoff between the constrain regularized term Γ(⋅) and the D. Simultaneous Super-resolution and Contrast residual data term ρ (⋅) . Enhancement We rewrite the minimization cost function (2) by using an We present a flow for simultaneous super resolution and image representation instead of a matrix-vector representation. contrast enhancement. This is just an example of the possibility It can be written, in a continuous form, as of combining different algorithms in the same PDE. Caselles et al. [15] proposed a PDE based approach to ⎧ n ⎫ F ( f ) = arg min ⎨∫∫ ∑ ρ ( yk ( x, y), Η k ( f )( x, y))dxdy + λΓ( f ( x, y)) ⎬ perform global histogram equalization. However, because of f ⎩ k =1 ⎭ the global modification not always produces good contrast, a (3) local histogram equalization algorithm which preserve the family of level-sets of the image was proposed in [16]. Most where Η k ( f ) is the transformed image of the super resolution recently, the authors in [17] generalized the PDE based image f , with the image operation corresponding to Dk Bk Fk , approach to contrast enhancement to adapt to any specified stretch function for global and local contrast enhancement, of and yk is the kth input image. The regularization term Γ(⋅) and the form the residual data term ρ (⋅) are given by ∂f ( x, y, t ) = ϕ ( f ( x, y, t )) − f ( x, y, t ) , (9) Γ( f ) = ∫∫ ∇( f ( x, y))dxdy (4) ∂t where ϕ (⋅) is any stretch function specified by user. A widely and used stretch function is the local histogram equalization (LHE) n 1 n [18], which will be used in our experiments. ∑ ρ(y ,Η k =1 k k ( f )) = ∑ || yk ( x, y) − Η k ( f )( x, y) ||22 . (5) 2 k =1 As we know, one of the advantages of the use of PDE for image processing is the possibility to combine algorithms, that where ∇ denotes the gradient [13]. is, if two procedures are given by The Euler-Lagrange equation associated to variational ∂f ∂f problem (3) is = F1 ( f ( x, y, t )) , = F2 ( f ( x, y, t )) , ∂t ∂t n ∇f −∑Τk [Ηk ( f )(x, y) − yk (x, y)] + λdiv( ) =0 (6) then they can be combined as k =1 | ∇f | ∂f = F1 ( f ( x, y, t )) + α F2 ( f ( x, y, t )) where Τk is the transpose operator to Η k , div denotes the ∂t divergence. 601 where F1 and F2 are two different operator, α ∈ R+ . Using the above property of PDE, the flows (8) and (9) can be combined to obtain a new flow which performs super resolution while improving the whole image contrast. The flow is given by ∂f ( x, y, t ) N = − | ∇f | ∑ Τk [Η k ( f ( x, y, t )) − yk ( x, y, t )] ∂t k =1 + λ1 R1 ( ∇ f ) + λ 2 R 2 ( f ) (10) where (a) (b) ∇f R1 (∇ f ) =| ∇ f | div ( ), | ∇f | R2 ( f ) = ϕ ( f ( x , y , t )) − f ( x , y , t ), with homogeneous Neumann boundary conditions and implementing a simple bilinear interpolation of the reference image as initialization. III. NUMERICAL SCHEME The PDE of (9) is not well defined at points where ∇f = 0 , (c) (d) due to the presence of the term 1/ ∇f . Then, it is common to Figure 1. (a) Original HR image. (b) One of six LR frames. (c) CSR. d) slightly perturb the R1 (∇f ) to be FSR ∇f + − ( Dy + Dy ) fi ,nj + + − ( Dx + Dx− )( Dy + Dy ) f i ,nj R1 (∇f ) =| ∇f | div[ ] n [ f ] := n , [ f xy ]ij := | ∇ f | +ε y ij 2h 4h 2 where ε is a small positive parameter. IV. EXPERIMENT RESULT Next we construct an explicit discrete scheme to In this section, we test our proposed method in both numerically solve the derived PDE in equation (9). Let f ijn be simulated and real acquired LR sequences. For the simulated the approximation to the value f ( xi , y j , tn ) , where xi = iΔx , experiment, the performance of the reconstruction algorithm was evaluated by measuring the peak signal to noise ratio y j = j Δy and tn = nΔt , where Δx , Δy and Δt are the (PSNR) [17]. we will use Gaussian blur kernel, defined as spatial step sizes and the time step size, respectively. Then our first order scheme of (9) reads as follows: 1 x2 + y2 h ( x, y ) = exp(− ) (13) n n −1 2πα 2 2α 2 f −f ij ij where α > 0 . In the following experiments, the criterion for Δt selecting the parameter in each experiment of each algorithm ⎡ N ⎤ is to choose parameters to produce visually most appealing = ( f xn ) 2 + ( f yn ) 2 ⎢ −∑ Tk [ yk − Η k ( f n )] + λ2 (ϕ ( f n ) − f n ) ⎥ results [19]. ⎣ k =1 ⎦ ij ⎡ f n (( f n ) 2 + ε ) − 2 f xy f xn f yn + f yy (( f xn ) 2 + ε ) ⎤ n n A. Evaluation of Simulation Experiment + λ1 ⎢ xx y ⎥ . (11) We used a good quality image of size 256 × 256 pixels for ⎢ ⎣ (( f xn ) 2 + ( f yn )2 + ε ) ⎥ ij ⎦ the simulated data [Fig. 1(a)]. Each pixel has a value in [0,255]. To compute the right hand side of (10), we denote by In the first simulation experiment, we design to evaluate the ± n n n fast convergence of our proposed method (denoted as “FSR”) D f := ±( f x i, j i ±1, j − f ) , D u := ±(u i, j ± n y i, j n i , j ±1 n −u ) i, j compared with the super resolution (denoted as “CSR”) proposed in [13]. Then we have the following formulas to approximate the term containing derivatives We first shifted this good quality image by two pixels in the + − vertical and horizontal direction. Then, this shifted image is Dx+ Dx−uin, j Dy Dy fi ,nj ( Dx+ + Dx− ) f i ,nj corrupted by blurring it with 5 × 5 Gaussian blur kernel with n [ f xx ]ij := , [ f yy ]ij := n [ f xn ]ij := h2 h2 2h standard deviation equal to 6. Independent white Gaussian noise of variance 10 is was then added. Finally, the resulting 602 TABLE I. COMPARISON PERFORMANCE FSR FSR+LHE LHE+FSR FSRLHE PSNR (dB) 22.8637 24.5508 24.3089 25.6703 Figure 2. PSNR values of interpolated images corresponding to 30 iterations in Fig. 1. (a) (b) image was down-sampled by a factor of 4 in each direction. One of these degraded low-resolution images is shown in Fig. 1(b), in which the visual quality of this image is poor. We use the method described in [20] to compute the motion vector. The PNSR versus iteration numbers are shown Fig. 2. From the plots of the corresponding PNSR values in Fig. 2 , we can get that the our proposed FSR method needs smaller iteration numbers but get better results than the CSR method. The second simulation experiment is presented to verify the good behavior of the simultaneous super resolution and enhance image contrast method. Fig. 3(a) is the original high (c) (d) resolution image. One of these degraded low-resolution images is shown in Fig. 3(b) by decreasing brightness and contrast of the six LR images generated in the previous experiment. Fig. 3(b) is the result using our proposed fast super resolution without contrast enhancement (denoted as, ‘FSR’). Figure 3 (c)gives the result when we firstly performed histogram equalization, and then followed by super resolution (denoted as ‘FSR+LHE’). If alternate the process order (denoted as, ‘LHE+FSR’), Fig. 4(d) is the result. The result of the simultaneous super resolution and local histogram equalization (denoted as, ‘FSR&LHE’) is given in Fig. 4(e). The PSNR corresponding to these results are listed in Table I. It is obvious that our proposed method outperforms the existing methods in (e) (f) terms of both the quantitative measurement and visual evaluation. Figure 3. (a) Original HR image. (b) One of six LR frames. (c) FSR. (d) FSR+LHE. (e) LHE+FSR. (f) FSR&LHE. B. Evaluation of Real Data Experiment Here we use real surveillance sequences to validate the effectiveness of the proposed super-resolution method. It is assumed that the motion of the images during the sequence is a globally translational motion. The first surveillance sequence experiment shows a SR reconstruction of a car sequence, which was obtained from a surveillance video camera. One frame of this sequence is shown in Fig. 4. Here the car is selected as our region of interest. We used six LR images with one frame shown in Fig. 4(a) to obtain a resolution enhancement factor of 2. Fig. 5(b-e) shows the result using FSR, SR+LHE, LHE+SR, and SR&LHE, respectively. Evidently, our proposed method has better visual quality. Figure 4. Fig. 4. One LR frame in the car video 603 Technologies R & D Program of China under grant 2009BAG13A06. REFERENCES [1] T. S. Huang, R. Y. Tsai, “Multi-frame image restoration and registration,” Adv. Comput. Vis. Image Process,Vol.1, 1984, 1: 317-339. [2] R. Schultz, R. Stevenson, “Extraction of high-resolution frames from video sequences,” IEEE Tran. Image Processing, Vol. 5, No. 6, 1996, pp. 996-1011. (a) (b) [3] M. Elad M , Y. Hel-Or, “A fast super-resolution reconstruction algorithm for pure translational motion and common space invariant blur.,” IEEE Tran. Image Processing, Vol. 10, 2001, pp.1187-1193. [4] S. Farsiu, D. Robinson, M. Elad, et al., “Fast and robust multi-frame super-resolution, ” IEEE Trans. Image Processing, Vol. 3. No. 10, 2004, pp. 1327-1344. [5] Takeda H, Milanfar P, Protter M and Elad M. Super-resolution without explicit subpixel motion estimation. IEEE Tran. Image Processing, Vol. 18 No. 9, 2009, pp. 1958-1975. [6] S. Farsiu, D. Robinson, M. Elad, et al., “Advances and challenges in super-resolution,” Int. Journ. Imaging Sys. Technol., Vol. 14, No. 2, 2004, pp. 47-57.. [7] S. C. Park, M. K. Park et al., “Super-resolution image reconstruction: a (c) (d) technical review,” IEEE Signal Processing Magazine,Vol. 5, 2003, pp. 21-36 [8] M. Irani, S. Peleg, “Motion analysis for image enhancement: Resolution, occlusion, and transparency,” J. Visual Commum. Image Represent, Vol. 4, 1993, pp.324-335. [9] S. Mann, R. W. Picard, “Virtual bellows: Constructing high quality stills from video,” Proc. IEEE Int. Conf. Image Process, 1994. [10] A. Zomet, S. Peleg, “Efficient super-resolution and applications to mosaics,” Proc. Intl. Conf. Patt. Recog., 2000, 579-583. [11] R. Hardie, K. Barnard, E. Armstrong, “Joint MAP registration and high resolution image estimation using a sequence of under-sampled images,” IEEE Transactions on Image Processing, Vol. 6, No. 12, 1997, pp. 1621- 1633. (e) [12] X. L. Li, Y. T. Hu, et al., “A multi-frame image super-resolution method,” Signal Processing, Vol. 90, 2010, pp. 405-414. Figure 5. SR reconstruction results from real LR video sequence. (a) One [13] A. Marquina, S. J. Osher, “Image super-resolution by TV Regularization frame of the LR images. (b) FSR. (c) FSR+LHE. (d) LHE+FSR. (e) and bregman iteration,” Journal of Scientific and Computing, Vol. 37, FSR&LHE. 2008, pp. 367-382. [14] A. Marquina, S. Osher, “Explicit algorithms for a new time dependent model based on level set motion for nonlinear deblurring and noise V. CONCLUSION removal,” SIAM:J. SCi. Comput, Vol. 22, No. 1, 2000, pp. 387-405. [15] G. Sapiro, V. Caselles, “Contrast enhancement via image evolution In this paper, we propose a PDE approach to super flows,” Graphical Models and Image Processing, Vol. 59, No. 6, 1997, resolution with contrast enhancement. The two main features of pp. 407-416. this paper are the following: On the one hand, we propose a [16] V. Caselles, J. L. Lisani, J. M. Morel, et al., “Shape preserving local fast partial differential equation (PDE) model for multi-frame histogram modification,” IEEE Transactions on Image Processing, Vol. super resolution reconstruction. One the other hand, we 8, No. 2, 1999, pp. 220-230. combine our proposed super resolution model with the local [17] J. Ke, Y. Q. Hou, et al., “An improved algorithm for shape preserving histogram equalization, which perform super-resolution and contrast enhancement,” Acta Photonica Sinica, Vol. 38, No. 1, 2009, pp. 214-219. enhance image contrast and simultaneously. It overcomes the [18] Q. Huynh-Thu, M. Ghanhari, “Scope of validity of PSNR in shortcomings of recent promising super resolution methods image/video quality assessment,” Electronics Letters, Vol. 44, No. 13, dealt with super resolution and contrast enhancement 2008, pp. 800-801. separately. Experiment results show the good behavior of the [19] L. P. Zhang, H. Y. Zhang, et al., “A super-resolution reconstruction proposed method. algorithm for surveillance images, ” Signal Processing, Vol. 90, 2010, pp. 848-859. ACKNOWLEDGMENT [20] G. Rochefort, F. Champagnat, et al., “Animproved observation model for super-resolution under affine motion,” .IEEE Transactionson Image This work was supported by National Natural Science Processing, Vol. 15, No. 11, 2006, pp.3325-3337. Foundation of China under grant 60972001 and National Key 604

DOCUMENT INFO

Shared By:

Tags:
adaptive, enhancement, enhancement algorithm, adaptive algorithm, low illumination, A binarization, method with learning, built rules, for document images, Adaptive, document, image binarization, An Objective, Evaluation, Methodology for Document, Image Binarization, Computer vision theory, global Thresholding Techniques, GRAY-SCALE, IMAGE ENHANCEMENT, Image Binarization By Back Propagation, Algorithm, Image Enhancement in the Spatial, Image Enhancement Techniques Using Local, Image Processing

Stats:

views: | 20 |

posted: | 3/29/2012 |

language: | English |

pages: | 5 |

Description:
A MODIFIED UNSHARP-MASKING TECHNIQUE FOR IMAGE CONTRAST, A NEW APPROACH FOR VERY DARK VIDEO DENOISING AND ENHANCEMENT, A PDE Approach to Super-resolution with, A Unified Histogram and Laplacian Based for Image, An Adaptive Image Enhancement Technique, An Improved Retinex Image Enhancement Technique, Automatic Exact Histogram Specification for, Bayesian Foreground and Shadow Detection in, Color Image Enhancement and Denoising Using an Optimized Filternet, Content Based Image Retrieval Using, Contrast Enhancement for Ziehl-Neelsen Tissue, DETAIL WARPING BASED VIDEO SUPER-RESOLUTION USING IMAGE GUIDES, Gray-level Image Enhancement By Particle Swarm, Image Contrast Enhancement based on Histogram, Image Enhancement and Segmentation Using Dark, Image Enhancement Technique Based On Improved, Image Quality Improvement fo r Electrophoretic Displays by, Image Reconstruction Using Particle Filters and, IMPROVED IDENTIFICATION OF IRIS AND EYELASH FEATURES, Improving Colour Image Segmentation on Acute, K�R ANALİZ Y�NTEMLERİ İLE İMGE İYİLEŞTİRME, NATURAL RENDERING OF COLOR IMAGE BASED ON RETINEX, Power-Constrained Contrast Enhancement, Research on Road Image Fusion Enhancement, Shadow Detection and Compensation in High Resolution Satellite Image Based, Smoothing Cephalographs Using Modified, Three-Dimensional Computational Integral Imaging Reconstruction by Use of, Towards integrating level-3 Features with

OTHER DOCS BY jasonndcosta

How are you planning on using Docstoc?
BUSINESS
PERSONAL

By registering with docstoc.com you agree to our
privacy policy and
terms of service, and to receive content and offer notifications.

Docstoc is the premier online destination to start and grow small businesses. It hosts the best quality and widest selection of professional documents (over 20 million) and resources including expert videos, articles and productivity tools to make every small business better.

Search or Browse for any specific document or resource you need for your business. Or explore our curated resources for Starting a Business, Growing a Business or for Professional Development.

Feel free to Contact Us with any questions you might have.