; Inpainting Algorithms for Image Coding and Scaling Patrick Ndjiki Nya
Documents
Resources
Learning Center
Upload
Plans & pricing Sign in
Sign Out
Your Federal Quarterly Tax Payments are due April 15th Get Help Now >>

Inpainting Algorithms for Image Coding and Scaling Patrick Ndjiki Nya

VIEWS: 2 PAGES: 24

  • pg 1
									 Inpainting Algorithms for
Image Coding and Scaling
         Patrick Ndjiki-Nya
                               Outline
 Motivation
 Image Inpainting and Coding
   • Structure Interpolation
   • Post-Processing Methods
   • Results

 Image Inpainting and Spatial Scaling
   • Proposed Method
   • Results

 Conclusions and Future Work




                                                                                                                slide 2
                    P. Ndjiki-Nya – Inpainting Algorithms for Image Coding and Scaling @ ITG FA 3.2 12/7/2007
                           Motivation

High-frequency textures are difficult to code, e.g. flower bed,
crushed stones, water etc.




              Where is the original texture ?




                                                                                                                 slide 3
                     P. Ndjiki-Nya – Inpainting Algorithms for Image Coding and Scaling @ ITG FA 3.2 12/7/2007
                  Coding: Block Diagram


                                      MSE Data
Video In           Encoder                                   Decoder                               Video Out



                                 Synth. Metadata



                TAS                                         TS


           Texture Analysis-Synthesis (TAS) Video Coding




                                                                                                                    slide 4
                        P. Ndjiki-Nya – Inpainting Algorithms for Image Coding and Scaling @ ITG FA 3.2 12/7/2007
Coding: Synthesizer Input

             Current frame (t0)




                                                                                                 slide 5
     P. Ndjiki-Nya – Inpainting Algorithms for Image Coding and Scaling @ ITG FA 3.2 12/7/2007
     Coding: Rigid Texture Synthesizer (TSI)

  Left ref. frame (t0-x)            Current frame (t0)                              Right ref. frame (t0+x)




 Texture synthesizer for rigid (flower bed, sand, …) textures
 Parametric approach
 Features some similarity with global motion compensation
 Typical drawback: Limited by assumptions of motion model




                                                                                                                       slide 6
                           P. Ndjiki-Nya – Inpainting Algorithms for Image Coding and Scaling @ ITG FA 3.2 12/7/2007
  Coding: Non-Rigid Texture Synth. (TSII, 2D)

         Orig. texture                                                 Keep as is


                                                        Overlap




                    Patch #2   Patch #1                                          Synth. texture



                                        Stuffing procedure



 Generic approach: Good for rigid (flower bed, sand, …) as well as
non-rigid textures (water, smoke,…)
 Non-parametric approach




                                                                                                                            slide 7
 Typical drawback: Complexity of texture representation

                                P. Ndjiki-Nya – Inpainting Algorithms for Image Coding and Scaling @ ITG FA 3.2 12/7/2007
Coding: Non-Rigid Texture Synth. (TSII, 2D+t)




                                                                                                           slide 8
               P. Ndjiki-Nya – Inpainting Algorithms for Image Coding and Scaling @ ITG FA 3.2 12/7/2007
Coding: Intermediate Results




                                                                                                   slide 9
       P. Ndjiki-Nya – Inpainting Algorithms for Image Coding and Scaling @ ITG FA 3.2 12/7/2007
Coding: Intermediate Results (cont.)




                                                                                                      slide 10
          P. Ndjiki-Nya – Inpainting Algorithms for Image Coding and Scaling @ ITG FA 3.2 12/7/2007
Coding: Challenges




                                                                                              slide 11
  P. Ndjiki-Nya – Inpainting Algorithms for Image Coding and Scaling @ ITG FA 3.2 12/7/2007
Coding: Structure Interpolation




                                                                                                   slide 12
       P. Ndjiki-Nya – Inpainting Algorithms for Image Coding and Scaling @ ITG FA 3.2 12/7/2007
Coding: Structure Interpolation




                                                                                                   slide 13
       P. Ndjiki-Nya – Inpainting Algorithms for Image Coding and Scaling @ ITG FA 3.2 12/7/2007
Coding: Structure Interpolation Results




                                                                                                        slide 14
            P. Ndjiki-Nya – Inpainting Algorithms for Image Coding and Scaling @ ITG FA 3.2 12/7/2007
Coding: Structure Interpolation Results




                                                                                                        slide 15
            P. Ndjiki-Nya – Inpainting Algorithms for Image Coding and Scaling @ ITG FA 3.2 12/7/2007
Coding: Structure Interpolation Results




                                                                                                        slide 16
            P. Ndjiki-Nya – Inpainting Algorithms for Image Coding and Scaling @ ITG FA 3.2 12/7/2007
Post-Processing Methods: Poisson Cloning




                                                                                                                  slide 17
     Simple Cloning                                            Poisson Seamless Cloning

                      P. Ndjiki-Nya – Inpainting Algorithms for Image Coding and Scaling @ ITG FA 3.2 12/7/2007
Post-Processing Methods: Formalization

                                                       S                                  f*
                                                                                          f


                                                                    Ω

  v                                                                            ∂Ω
                  g



      • S: a closed subset of R2
      • Ω: a closed subset of S, with boundary ∂Ω
      • f*: known scalar function over S\Ω
      • f: unknown scalar function over Ω
      • ∂Ω: boundary pixels
      • g: source function




                                                                                                             slide 18
      • v: vector field of g

                 P. Ndjiki-Nya – Inpainting Algorithms for Image Coding and Scaling @ ITG FA 3.2 12/7/2007
Post-Processing Methods: Formalization

 Poisson equation with Dirichlet boundary condition

                                2
         min ∫ ∇ - v
                f                       with           f |∂Ω f * |∂Ω
                                                           =
          f   Ω


          Δ f = divv         over Ω with              f |∂Ω f * |∂Ω
                                                          =




                                                                                                              slide 19
                  P. Ndjiki-Nya – Inpainting Algorithms for Image Coding and Scaling @ ITG FA 3.2 12/7/2007
Coding: Structure Interpolation Results




                                                                                                        slide 20
            P. Ndjiki-Nya – Inpainting Algorithms for Image Coding and Scaling @ ITG FA 3.2 12/7/2007
Image Inpainting and Small Holes




                                                                                                     slide 21
         P. Ndjiki-Nya – Inpainting Algorithms for Image Coding and Scaling @ ITG FA 3.2 12/7/2007
                 Scaling Results




                                                                                                              slide 22
Bicubic Interpolation                           Structure-aware Interpolation

                  P. Ndjiki-Nya – Inpainting Algorithms for Image Coding and Scaling @ ITG FA 3.2 12/7/2007
          Conclusions and Future Work
 Efficient tools for image inpainting presented
 Performance optimization of cloning in inpainting
  framework
 Extension of structure-aware methods to video (rigid and
  non-rigid textures)




                                                                                                                slide 23
                    P. Ndjiki-Nya – Inpainting Algorithms for Image Coding and Scaling @ ITG FA 3.2 12/7/2007
Thank You




                                                                                               slide 24
   P. Ndjiki-Nya – Inpainting Algorithms for Image Coding and Scaling @ ITG FA 3.2 12/7/2007

								
To top