# Image Quilting for Texture Synthesis and Transfer

Document Sample

```					Image Quilting for Texture
Synthesis and Transfer
Alexei A. Efros
William T. Freeman
Texture synthesis
• take a sample of texture and
generate unlimited amount of image
data
Motivation from Language
• [Shannon,’48] proposed a way to
generate English-looking text using N-
grams:
– Assume a generalized Markov model
– Use a large text to compute probability
distributions of each letter given N-1 previous
letters
• precompute or sample randomly
– Starting from a seed repeatedly sample this
Markov chain to generate new letters
– One can use whole words instead of letters too:
WE NEED TO EAT CAKE
Mark V. Shaney (Bell Labs)
• Results (using alt.singles corpus):
– “As I've commented before, really relating to
someone involves standing next to impossible.”
– "One morning I shot an elephant in my arms and
kissed him.”
– "I spent an interesting evening recently with a
grain of salt"
• Notice how well local structure is preserved!
– Now let’s try this in 2D...
Efros & Leung ’99
non-parametric
sampling

p

Input image
Synthesizing a pixel

• Assuming Markov property, compute P(p|N(p))
– Building explicit probability tables infeasible
– Instead, let’s search the input image for all similar
neighborhoods — that’s our histogram for p
• To synthesize p, just pick one match at rando
m
Efros & Leung ’99
- Synthesis Results

Increasing window size
Efros & Leung ’99 extended
non-parametric
sampling

p
B
Input image
Synthesizing a block

• Observation: neighbor pixels are highly correlated
Idea: unit of synthesis = block
• Exactly the same but now we want P(B|N(B))
• Much faster: synthesize all pixels in a block at once
• Not the same as multi-scale!
Texture synthesis
• Use quilting as a very simple texture
synthesis algorithm
Quilting
• Develop patch-based
texture synthesis
– Define the unit of
synthesis Bi (square
block) from the set Sb
– The set Sb – the set of
all overlapping blocks
in the input texture
image
Quilting
1
Minimum Error Boundary Cut
• Dynamic programming
– The minimal cost path through the error
surface – two blocks overlap along
vertical edge
– Error surface is defined
– To find the minimal vertical cut
• We traverse e(I=2,…,N)
The image quilting Algorithm
• Algorithm
– Pick size of block and size of overlap
– Synthesize blocks in raster order

– Search input texture for block that satisfies
overlap constraints (above and left)
• Easy to optimize using NN search [Liang et.al., ’01]
– Paste new block into resulting texture
• use dynamic programming to compute minimal error
boundary cut
Failures
Texture Transfer
• extend the algorithm to
perform texture transfer
Texture Transfer
Correspondence map
• 앞장 – luminance of the man’s face
– Bright patches of face and bright patches of
rice are defined to have a low correspondence
error.
• Modify error term of the image quilting
algorithm to be the weighted sum,
• a*(block overlap mapping error)
+ (1-a)*(squared error within
correspondence map)
Source      Target
texture      image
result

Correspondence map
Iteration
• Iterate over the synthesized image several
times
– Reducing the block size with each iteration
• Local texture constraint
– Their neighbor blocks on the overlap regions
– And. Whatever was synthesized at this block in
the previous iteration
• Works surprisingly well
– Starts out using large blocks to roughly assign
– Then used smaller blocks
– Our test
• N=3 to N=5 iterations , reducing the block size by a
third each time, and a=0.8*(i-1)/(N-1) + 0.1
+   =
conclusion
• Image quilting – a method of
synthesizing a new image by stitching
together small patches of existing
images
• Equal or better than Efros ’99 and
faster
• Extended our method to texture
transfer
Portilla & Simoncelli   Xu, Guo & Shum

input texture

Wei & Levoy         Image Quilting
Portilla & Simoncelli   Xu, Guo & Shum

input texture

Wei & Levoy         Image Quilting
Portilla & Simoncelli   Xu, Guo & Shum

input texture

Wei & Levoy         Image Quilting
ing

```
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
 views: 23 posted: 10/5/2012 language: English pages: 29
How are you planning on using Docstoc?