SAMPLING SCHEMES FOR 2-D SIGNALS WITH FINITE RATE OF INNOVATION

					A transfer talk on

SAMPLING SCHEMES FOR 2-D SIGNALS WITH FINITE RATE OF INNOVATION*
by

Pancham Shukla
supervisor

Dr P L Dragotti
Communications and Signal Processing Group Imperial College London

 This research is supported by EPSRC.

1/3/2005

SAMPLING SCHEMES FOR 2-D SIGNALS WITH FINITE RATE OF INNOVATION

OUTLINE
1. INTRODUCTION
• Sampling: Problem, Background, and Scope

2. SIGNALS WITH FINITE RATE OF INNOVATION (FRI) (non-bandlimited)
• • Definition, Extension in 2-D 2-D Sampling setup, Sampling kernels and their properties

3. SAMPLING OF FRI SIGNALS
• • • SETS OF 2-D DIRACS Local reconstruction (amplitude and position) BILEVEL POLYGONS & DIRACS using COMPLEX MOMENTS Global reconstruction (corner points) PLANAR POLYGONS using DIRECTIONAL DERIVATIVES Local reconstruction (corner points)

4. CONCLUSION AND FUTURE WORK

3

SAMPLING SCHEMES FOR 2-D SIGNALS WITH FINITE RATE OF INNOVATION

1. INTRODUCTION
• Why sampling? Many natural phenomena
are continuous (e.g. Speech, Remote sensing) and required to be observed and processed by sampling.
Many times we need reconstruction (perfect !) of the original phenomena.

ˆ Continuous x(t )  Discrete (samples) s[n]  Continuous x (t )
• Sampling theory by Shannon (Kotel’nikov, Whittaker)
‘bandlimited-sinc’ scenario with the assumption of Perfect Reconstruction !

x(t )
 fm
fm

f   fm

f  fm

fs  2 fm , T 

1 fs

 (t )  SINCT (t )

 (t )  SINCT (t )

ˆ x(t )  x(t )

n

  (t  nT )
(Although powerful and widely used since 5 decades)

•

Why not always ‘bandlimited-sinc’?

1. Real world signals are non-bandlimited. 2. Ideal low pass (anti-aliasing, reconstruction) filter does not exist. (Acquisition devices) 3. Shannon’s reconstruction formula is rarely used in practice with finite length signals (esp. images) due to infinite support and slow decay of ‘sinc’ kernel. (do we achieve PR in practice?)
PR ?

Motivation:

*

4

SAMPLING SCHEMES FOR 2-D SIGNALS WITH FINITE RATE OF INNOVATION

1. INTRODUCTION contd.
• • Extensions of Shannon’s theory
So… many papers but for comprehensive account, we refer to [Jerry 1977, Unser 2000].

Shift-invariant subspaces

[Unser et al.]

The classes of non-bandlimited signals (e.g. uniform splines) residing in the shift-invariant subspaces can be perfectly
reconstructed. The other non-bandlimited signals are approximated through their projections.

We look into:
Non-bandlimited signal

Non-bandlimited signals that do not reside in shift-invariant subspace but have a parametric representation. Non-traditional ways of perfect reconstruction… ….from the projections of such signals in the shiftinvariant subspace. Is it possible to perfectly reconstruct such signals from their samples?

projection

e.g. uniform spline

Shift-invariant subspace

Any examples of such signals ?
What type of kernels ? Sampling and reconstruction schemes? 5

SAMPLING SCHEMES FOR 2-D SIGNALS WITH FINITE RATE OF INNOVATION

2. FRI SIGNALS
Very recently such signals are identified and termed as

•

Signals with Finite Rate of Innovation (or FRI signals) [ Vetterli et al. 2002]
Model: Non-bandlimited signals that do not reside in shift-invariant subspace. Examples: Streams of Diracs, non-uniform splines, and piecewise polynomials.

Unique feature: A finite number of degrees of freedom per time (rate of innovation ) e.g. a Dirac in 1-D has a rate of innovation = 2 (i.e. amplitude and position). • • The sampling schemes for such signals in 1-D are given by [Vetterli, Marziliano and Blu 2002]. Extensions of these schemes in 2-D are given by [Maravic and Vetterli 2004], however, focusing on Sampling kernels: as sinc and Gaussian. Algorithms: Little more involved reconstruction algorithms (solution of linear systems, root finding) based on Annihilating filter method [from Spectral estimation, Error correction coding]. Reconstruction: Only a finite number of samples ( ) guarantees perfect reconstruction.

6

SAMPLING SCHEMES FOR 2-D SIGNALS WITH FINITE RATE OF INNOVATION

2. FRI SIGNALS contd.
• Assortments of kernels
[Dragotti, Vetterli and Blu, ICASSP-2005] For 1-D FRI signals, one can use varieties of kernels such as 1. 2. 3. That reproduce polynomials (satisfy Strang and Fix conditions) Exponential Splines (E-Splines) [Unser] Functions with rational Fourier transforms

•

Our Focus
Sampling extensions in 2-D using above mentioned kernels, in particular, for

Sets of 2-D Diracs  Local & Global schemes: Local kernels & Complex moments + (AFM)
Bilevel polygons  Global scheme: Complex moments + Annihilating filter method (AFM) Planar polygons  Local scheme: Directional derivatives + Directional kernels

7

g ( x, y )

g ( x, y )

g ( x, y )

SAMPLING SCHEMES FOR 2-D SIGNALS WITH FINITE RATE OF INNOVATION

2. SAMPLING FRI SIGNALS in 2-D
• Sampling setup
Set of samples in 2-D

Input signal Sampling kernel

S j , k  g ( x, y), xy ( x / Tx  j, y / T y  k )

•

Properties of sampling kernels
In current context, any kernel that reproduce polynomials 

of degrees  =0,1,2…-1 such that
Partition of unity:
jZ k Z

   xy ( x  j, y  k ) 1
  C , j xy ( x  j, y  k )  x


 0
C , j

Reproduction of polynomials along x-axis:
k Z j  Z

 1

Reproduction of polynomials along y-axis:
k Z j  Z

  C , k  xy ( x  j, y  k )  y



C , k

8 e.g., B-Splines (biorthogonal) and Daubechies scaling functions (orthogonal) are valid kernels

SAMPLING SCHEMES FOR 2-D SIGNALS WITH FINITE RATE OF INNOVATION

3. SETS OF 2-D DIRACS
• Sets of 2-D Diracs: Local reconstruction
j Z k  Z

Consider g ( x, y) 

  a j, k  xy ( x  x j , y  yk ) and  xy ( x, y) with support Lx  Ly such that

there is at most one Dirac a p, q xy ( x  x p , y  yq ) in an area of size LxTx  L yT y . Assume Tx , T y  1 . From the partition of unity (reproducing of polynomial of degree 0),it follows that

The amplitude

a p, q 

  S j, k
j 1k 1

Lx L y

This is derived as follows,

  S j, k
j 1 k 1

Lx L y

 a p, q  xy ( x  x p , y  y q ), 
 

2.5 5  a p, q  10  S j, k    1.2 1.3 L L
x



   a p, q  xy ( x  x p , y  y q )    xy ( x  j , y  k )  dx dy    j 1 k 1  x   y    
Lx L y j 1 k 1

j 1 k 1  Lx L y

  xy ( x  j, y  k )

y

Only Lx  L y inner products overlap the unique Dirac

 a p, q   a p, q

  xy ( x p  j, y q  k )

(property: partition of unity)

B-Splines of order one

9

SAMPLING SCHEMES FOR 2-D SIGNALS WITH FINITE RATE OF INNOVATION

2. SAMPLING FRI SIGNALS in 2-D

•

Properties of sampling kernels
In current context, any kernel that reproduce polynomials 

of degrees  =0,1,2…-1 such that
Partition of unity:
jZ k Z

   xy ( x  j, y  k ) 1
  C , j xy ( x  j, y  k )  x


 0
C , j

Reproduction of polynomials along x-axis:
k Z j  Z

 1

Reproduction of polynomials along y-axis:
k Z j  Z

  C , k  xy ( x  j, y  k )  y



C , k

10 e.g., B-Splines (biorthogonal) and Daubechies scaling functions (orthogonal) are valid kernels

SAMPLING SCHEMES FOR 2-D SIGNALS WITH FINITE RATE OF INNOVATION

3. SETS OF 2-D DIRACS contd.
• Sets of 2-D Diracs: Local reconstruction contd.
… and using polynomial reproduction properties along x and y directions, the coordinate positions are given by
Lx Ly

  C1, j S j , k
xp 
j 1 k 1

a p, q

Above relations are derived as,

Similarly, it is easy to follow that
As long as any two Diracs are sufficiently apart, we can accurately reconstruct a set of Diracs, considering one Dirac per time.

11

SAMPLING SCHEMES FOR 2-D SIGNALS WITH FINITE RATE OF INNOVATION

4. BILEVEL POLYGONS & DIRACS: Complex-moments
• Complex-moments for polygonal shapes: earlier works
Since decades, moments are used to characterize unspecified objects. [Shohat and Tamarkin 1943, Elad et al. 2004]. Here, we present a sampling perspective to the results of [Davis 1964, Milanfar et al. 1995, Elad et al. 2004] on reconstruction of polygonal shapes using complex-moments.

Definition: The nth

simple and weighted complex-moments of a given function g ( x, y ) over a

complex Cartesian plane z  x   1 y in the closure  are given by

s  n    g ( x, y ) z n dx dy O
Simple moment

w  n  n n  1)    g ( x, y ) z n  2 dx dy O
Weighted moment

Result of Davis (1964): For any non-degenerate, simply connected polygon g ( x, y)
points (z i ) in closure  of any analytic function h(z ) ,following holds

with N corner

  g ( x, y) h( z) dx dy   i h( zi )
where  i are complex weights that depend on the ordered connection of corner points z i Milanfar et al. (1995 ) extended the above work using h( z)  z n as follows 12

N

i 1

SAMPLING SCHEMES FOR 2-D SIGNALS WITH FINITE RATE OF INNOVATION

4. BILEVEL POLYGONS & DIRACS: Complex-moments
• Complex-moments for polygonal shapes: a modern connection
Results of Milanfar et al. (1995): Milanfar et al. considered a bilevel polygon that is non-degenerate, simply connected and convex. They showed that when h( z)  z n and g ( x, y ) is ‘1’ in the closure  and
‘0’ out side. It follows that for n  0,1,2....2 N  1

n2

i 1



N

 i z in 

 g ( x, y) h( z ) dx dy

n "

y

n 1 n0
x

  g ( x, y ) ( z ) dx dy


 n (n  1)  g ( x, y ) z ( n  2) dx dy
s  n(n  1)  n  2 (simple moment) w  n (weighted moment)

n0

n 1 n  2

Theorem [Milanfar et al.]: For a given non-degenerate, simply connected, and convex polygon in the
complex Cartesian plane, all its N corner points are uniquely determined by its weighted complexmoments  w up to order 2N-1.

n

Now, we will briefly review the annihilating filter method due to its relevance in finding weights and positions of the corner points zi from the observed complex moments.

13

SAMPLING SCHEMES FOR 2-D SIGNALS WITH FINITE RATE OF INNOVATION

4. BILEVEL POLYGONS & DIRACS: Complex-moments
• Annihilating filter method
(we refer to [Vetterli, Stoica and Moses] for more details) This method is well known in the field of Error-correcting codes and Spectral estimation. Especially, in second application, it is employed to determine the weights  i and locations u i of the spectral components, generally observed in form of

 [ n] 

i 1

  i uin

N

where  i  R, ui  C , n  N

(1)

The annihilating filter method consists of the following steps: 1. Design the annihilating filter A(z): such that for filter A[l ], l  0,1,...N with its z-transform A( z ) 
l 0 N N 1 i 0

 A[l ] z  l 

 1  ui z 1 .

the condition

A[n] [n]  0 holds.

2. Locations: The convolution condition is solved by the following Yule-Walker system
  [ N  1]  [ N  2]  [N ]  [ N  1]       [2 N  2]  [2 N  3]   

    [ N  1] 

 [0]   A[1]   [1]   A[2]   

 [N ]    [ N  1]               A[ N ]  [2 N  1]

The roots of the filter are the locations u i

3. Weights: Once the locations u i are known, eq. (1) is solved for the weights  i by the following Vandermonde system
1  1  u u1  0     N 1 N 1 u1 u 0  1    0    [0]   u N 1   1    [1]                 N 1    u N 1    N 1   [ N  1]  

Gives the weights  i 14

SAMPLING SCHEMES FOR 2-D SIGNALS WITH FINITE RATE OF INNOVATION

4. BILEVEL POLYGONS & DIRACS: Complex-moments
• A sampling perspective (using Complex-moments + Annihilating filter method)
Consider g(x,y) as a non-degenerate, simply Consider g(x,y) as a set of N 2-D Diracs connected, and convex bilevel polygon with N corner points S j , k  g ( x, y), xy ( x / Tx  j, y / T y  k ) ,

 xy ( x, y) can reproduce polynomials up to degree 2N-1 ( = 0,1...2N-1)
Then from the complex-moments formulation of Milanfar et al.

i 1

  i zin  O g ( x, y) h( z ) dx dy
 n(n  1)  g ( x, y ) z ( n  2) dx dy
w  n

N

i 1

  i zin  O g ( x, y) h( z ) dx dy

s  n

N

n 2

O

O g ( x, y) z

n

dx dy

n

Because of the polynomial reproduction property of the kernel, we derive that
i 1



N

 i z in  n(n  1)
w  n

 
j k

x C1, j 

 1 C1yk ( n  2) S j , k ,



i 1

 ai zin

N



 C1x, j 

 1 C1yk n S j , k ,



j k s  n

where  i are complex weights and z i are corner points of the bilevel polygon

where a i denotes amplitudes of the Diracs and z i are complex position of the Diracs

Now using annihilating filter method, it is straightforward to see that

w n

A(z )

zi

s n

A(z )

zi ai
15

SAMPLING SCHEMES FOR 2-D SIGNALS WITH FINITE RATE OF INNOVATION

4. BILEVEL POLYGONS & DIRACS: Complex-moments
• Simulation results
Bilevel polygon with N=3 corner points
A set of N=3 Diracs

g ( x, y) *  xy ( x, y)

g ( x, y )

g ( x, y )

g ( x, y) *  xy ( x, y)

5  xy ( x, y)   xy ( x, y)

w  n  A(z )

s  n  A(z )

S j, k

S j, k

16

SAMPLING SCHEMES FOR 2-D SIGNALS WITH FINITE RATE OF INNOVATION

4. BILEVEL POLYGONS & DIRACS: Complex-moments
•
1. 2.

Summary
A polygon with N corner points is uniquely determined from its samples using a kernel that reproduce polynomials up to degree 2N-1 along both x and y directions. A set of Diracs is uniquely determined from its samples using a kernel that reproduces polynomials up to degree 2N-1 along both x and y directions and that there are at most N Diracs in any distinct area of size NLxTx  NL yTy. Global reconstruction algorithm Complexity  Complexity of the signal Numerical instabilities in algorithmic implementations for very close corner points

3.

17

SAMPLING SCHEMES FOR 2-D SIGNALS WITH FINITE RATE OF INNOVATION

5. PLANAR POLYGONS: Directional-derivatives
• Problem formulation Intuitively, for a planar polygon, two successive directional derivatives along
two adjacent sides of the polygon result into a 2-D Dirac at the corner point formed by the respective sides.
Planar polygon with N corner points

Continuous model:

N pairs of orientations N pairs of directional derivatives

In present context, we have access to samples S j , k only.

N Diracs

Discrete challenge:
Local reconstruction scheme of 2-D Diracs

Amplitudes and positions of the corner points

Lattice theory: Directional derivatives  Discrete differences • Subsampling over integer lattices and • Local directional kernels in the framework of 2-D Dirac sampling (local reconstruction) 18

SAMPLING SCHEMES FOR 2-D SIGNALS WITH FINITE RATE OF INNOVATION

5. PLANAR POLYGONS: Directional-derivatives
• Lattice theory We refer to [Cassels, Convey and Sloan] for more detail.
Base lattice: is a subset of points of Z2 (integer lattice)

Each pattern of subsampling (or

 ) over the integer lattice is characterized by a non-unique

  v1   v1,1 v1,2  Sampling matrix V        v2  v2,1 v2,2  
For example, by taking V    , the base lattice  is illustrated as 2  1
1 2

 1 , v1
A

1  tan

1  v1,2   

 v1,1   

 tan(1 )  2

 2 , v2



 v2,2    tan( 2 )  1 / 2  2  tan 1   v2,1   

19

SAMPLING SCHEMES FOR 2-D SIGNALS WITH FINITE RATE OF INNOVATION

5. PLANAR POLYGONS: Directional-derivatives
• Proposed sampling scheme
Consider a planar polygon g ( x, y ) with N corner points and a sampling kernel  xy ( x, y) that satisfies partition of unity (reproduces polynomial of degree zero). The observed samples are given by S j , k  g ( x, y), xy ( x, y) Therefore, using lattice theory, apply a pair of directional differences D and D along 1 2  1 and  2 over the samples S j , k identified by the base lattice  and its sampling matrix V It then follows,

D 2 D1 S j , k  S (v 21  v1,1 ), (v 2, 2  v1, 2 )  S (v 2,1 ), (v 2, 2 )  S (v1,1 ), (v1, 2 )  S (0), (0) g ( x, y ), 

   





 xy ( x  (v21  v1,1 ), y  (v2,2  v1,2 ))   xy ( x  (v2,1 ), y  (v2,2 ))  xy ( x  (v1,1 ), y  (v1,2 ))   xy ( x  (0), y  (0))
     g ( x, y),  1 2 ( x, y)   2  1  
Dirac at A Modified kernel

By using Parseval’s identities and after certain manipulations, we have

D 2 D1 S j , k det (V )

  

where  1 2 ( x, y) 

0   ( x, y) *  xy ( x, y)
1 2

sin( 2  1 )

20

SAMPLING SCHEMES FOR 2-D SIGNALS WITH FINITE RATE OF INNOVATION

5. PLANAR POLYGONS: Directional-derivatives
• Directional kernels
D1 D 2 S j , k det(V )

 



     g ( x, y) ,  1 2 ( x, y)   2  1  
Dirac at A Modified kernel

where  1 2 ( x, y) 

0   ( x, y) *  xy ( x, y)
1 2

sin( 2  1 )

Modified kernel is a ‘directional kernel’. For each corner point  independent directional kernel.

For example,

 xy ( x, y)

0   ( x, y )
1 2

   ( x, y )
1 2


support: Lx  L y

support: v1,1  v2,1  Lx  v1,2  v2,2  L y 
21

SAMPLING SCHEMES FOR 2-D SIGNALS WITH FINITE RATE OF INNOVATION

5. PLANAR POLYGONS: Directional-derivatives
• Local reconstruction of the corner point
The directional kernel 1 2 ( x, y) can reproduce polynomials of degrees 0 and 1 in both x and y directions.

Assume that there is only one corner point  support of its associated directional kernel. Then from the local reconstruction scheme of Diracs, we can reconstruct the amplitude and the position of an equivalent Dirac at a given corner point (e.g. at point A ) as:

 D 2 D1 S j, k 
a p, q 
j k

1 , v1



 v1,2  1  tan 1    tan(1 )  2  v1,1     v2,2    tan( 2 )  1 / 2  2  tan 1   v2,1   

A

det (V )

 2 , v2



 C1x, j
xp 
j k

D 2 D1 S j , k

  
yq 

 C1y, k
j k

D 2 D1 S j , k

  
22

a p, q det (V )

a p, q det (V )

SAMPLING SCHEMES FOR 2-D SIGNALS WITH FINITE RATE OF INNOVATION

5. PLANAR POLYGONS: Directional-derivatives
• Simulation result
Original polygon with 3 corner points

Samples of the polygon using Haar kernel

Local reconstruction Pair of directional differences

After first pair of directional difference on samples

After second pair of directional difference on samples

After third pair of 23 directional difference on samples

SAMPLING SCHEMES FOR 2-D SIGNALS WITH FINITE RATE OF INNOVATION

5. PLANAR POLYGONS: Directional-derivatives
• Summary: reconstruction algorithm
Initial intuition:
Planar polygon with N corner points

Final realization:
Planar polygon with N corner points Only one corner point in the support of its directional kernel

v1,1  v2,1  Lx  v1,2  v2,2  L y 
tan( )  Q
Enough number of samples S j , k

N pairs of orientations N pairs of directional derivatives

 R

N pairs of orientations N pairs of directional differences

N Diracs

N Diracs

Advantage:
Local reconstruction scheme of 2-D Diracs Local reconstruction scheme of 2-D Diracs 1. 2. Local reconstruction. Only local reconstruction complexity, irrespective of the number of corner points in a polygon.

Amplitudes and positions of the corner points

Amplitudes and positions of the corner points

24

SAMPLING SCHEMES FOR 2-D SIGNALS WITH FINITE RATE OF INNOVATION

6. CONCLUSION & FUTURE WORK
• Conclusion
• • • We have proposed several sampling schemes for the classes of 2-D non-bandlimited signals. In particular, sets of Diracs and (bilevel and planar) polygons can be reconstructed from their samples by using kernels that reproduce polynomials. Combining the tools like annihilating filter method, complex-moments, and directional derivatives, we provide local and global sampling choices with varying degrees of complexity.

25

SAMPLING SCHEMES FOR 2-D SIGNALS WITH FINITE RATE OF INNOVATION

6. CONCLUSION & FUTURE WORK
• Future work

From March 2005 to October 2005: 1. 2. 3. Exploring a different class of kernels, namely, exponential splines (E-Splines). Extending the sampling schemes in higher dimension. For instance, using the notion of complex numbers in 4-D (quaternion). Considering more intricate cases such as piecewise polynomials inside the polygons, and planar shapes with piecewise polynomial boundaries.

We plan to submit a paper for IEEE Transactions on Image Processing by summer 2005.
From November 2005 to June 2006: 1. 2. Studying the wavelet footprints [Dragotti 2003] and then extending them in 2-D Integrating the proposed sampling schemes with the footprints in 2-D

3.
4.

Investigating the sampling situations when the signals are perturbed with the noise
Developing resolution enhancement algorithms for satellite images.

26

Questions?

27


				
DOCUMENT INFO