# Introduction Bregman divergences Bregman Voronoi diagrams Applications Voronoi diagrams

### Pages to are hidden for

"Introduction Bregman divergences Bregman Voronoi diagrams Applications Voronoi diagrams"

```					               Introduction
Bregman divergences
Bregman Voronoi diagrams
Applications

Voronoi diagrams of statistical
distributions

J-D. Boissonnat               F. Nielsen         R. Nock

´
Journee GeoTopAl
september 26, 2006

´
journee GeoTopAl      Voronoi diagrams of statistical distributions
Introduction
Bregman divergences
Bregman Voronoi diagrams
Applications

Tulipe
Introduction
Bregman divergences
Deﬁnition
Bisectors
Bregman balls
Orthogonality and geodesics
Centroid and Bregman information
Bregman Voronoi diagrams
Bregman Voronoi diagrams from polytopes
Bregman triangulations
Bregman Voronoi diagrams from power diagramms
Weighted Bregman diagrams
Applications
´
journee GeoTopAl    Voronoi diagrams of statistical distributions
Deﬁnition
Introduction
Bisectors
Bregman divergences
Bregman balls
Bregman Voronoi diagrams
Orthogonality and geodesics
Applications
Centroid and Bregman information

Deﬁnition of Bregman divergences
F a strictly convex and differentiable function deﬁned over a
convex set X

DF (p, q) = F (p) − F (q) − p − q,                 F (q)

F

ˆ
x
DF (x, y)
ˆ
y

Hy
X
y    x
´
journee GeoTopAl       Voronoi diagrams of statistical distributions
Deﬁnition
Introduction
Bisectors
Bregman divergences
Bregman balls
Bregman Voronoi diagrams
Orthogonality and geodesics
Applications
Centroid and Bregman information

Examples
F (x) = x 2 : Squared Euclidean distance

DF (p, q) = F (p) − F (q) − p − q,                             F (q)
2         2                                           2
= p − q − p − q, 2q = p − q

F (p) = p(x) log2 p(x)                                      (Shannon entropy)
p(x)
DF (p, q) = x p(x) log2 q(x)                                  (K-L divergence)

F (p) = − x log p(x)                                                (Burg entropy)
p(x)     p(x)
DF (p(x), q(x)) = x ( q(x) log p(y) − 1)                            (Itakura-Saito)

´
journee GeoTopAl        Voronoi diagrams of statistical distributions
Deﬁnition
Introduction
Bisectors
Bregman divergences
Bregman balls
Bregman Voronoi diagrams
Orthogonality and geodesics
Applications
Centroid and Bregman information

Basic properties of Bregman divergences

Non-negativity : DF (x, y) ≥ 0, ∀x, y ∈ X
Convexity : DF is convex in the ﬁrst argument
Linearity : DF +G = DF + DG , DλF = λDF
Pythagoras theorem :
DF (x, z) = DF (x, y) + DF (y, z) − x − y, z − y
For X a convex subset of X and y = arg mint∈X DF (t, z)
DF (x, y) + DF (y, z) ≤ DF (x, z)
with equality when X is an afﬁne set

´
journee GeoTopAl    Voronoi diagrams of statistical distributions
Deﬁnition
Introduction
Bisectors
Bregman divergences
Bregman balls
Bregman Voronoi diagrams
Orthogonality and geodesics
Applications
Centroid and Bregman information

Bisectors

DF (p, q) = F (p) − F (q) − p − q,                   F (q)

Several types of bisectors

Hpq : DF (x, p) = DF (x, q) (hyperplane)
∗
Hpq : DF (p, x) = DF (q, x) (hypersurface)

´
journee GeoTopAl    Voronoi diagrams of statistical distributions
Deﬁnition
Introduction
Bisectors
Bregman divergences
Bregman balls
Bregman Voronoi diagrams
Orthogonality and geodesics
Applications
Centroid and Bregman information

Legendre duality

F ∗ (y ) = sup      y , x − F (x) = y ,              F
−1
(y ) − F (         F
−1
(y ))
x

Properties
(F ∗ )∗ = F
F ∗ is strictly convex and differentiable
By taking y =            F (y)

F ∗ (y ) = −F (y) + y, y
DF (x, y) = F (x) + F ∗ (y ) − x, y = DF ∗ (y , x )

´
journee GeoTopAl    Voronoi diagrams of statistical distributions
Deﬁnition
Introduction
Bisectors
Bregman divergences
Bregman balls
Bregman Voronoi diagrams
Orthogonality and geodesics
Applications
Centroid and Bregman information

Theorem
∗
The curved second-type Bregman bisector Hpq is the image by
−1
F of the ﬂat ﬁrst-type Bregman bisector Hp q associated to
the dual divergence DF ∗

´
journee GeoTopAl    Voronoi diagrams of statistical distributions
Deﬁnition
Introduction
Bisectors
Bregman divergences
Bregman balls
Bregman Voronoi diagrams
Orthogonality and geodesics
Applications
Centroid and Bregman information

Bregman spheres                       σ(c, r ) = {x ∈ X | DF (x, c) = r }

Lemma                                                                                         F

The lifted image σ onto F of a Bregman
ˆ
sphere σ is contained in a hyperplane Hσ
ˆ
x
DF (x, y)
ˆ
y
Conversely, the intersection of any                   Hy
hyperplane H with F projects vertically
X
onto a Bregman sphere                                                        y      x

´
journee GeoTopAl    Voronoi diagrams of statistical distributions
Deﬁnition
Introduction
Bisectors
Bregman divergences
Bregman balls
Bregman Voronoi diagrams
Orthogonality and geodesics
Applications
Centroid and Bregman information

Polarity (for symmetric divergences)

F

The pole of Hσ is the point
σ + = (c, F (c) − r )
ˆ
x
DF (x, y)                 common to all the tangent
ˆ
y
Hσ                                                        hyperplanes at Hσ ∩ F
Hy
σ+

X
y        x

´
journee GeoTopAl   Voronoi diagrams of statistical distributions
Deﬁnition
Introduction
Bisectors
Bregman divergences
Bregman balls
Bregman Voronoi diagrams
Orthogonality and geodesics
Applications
Centroid and Bregman information

Orthogonality and geodesics
l(p, q) = {x : x = λp + (1 − λ)q}
c(p, q) = {x : x = λp + (1 − λ)q }
Orthogonality
X is Bregman orthogonal to Y if
∀x ∈ X , ∀y ∈ Y , ∀t ∈ X ∩ Y
DF (x, t) + DF (t, y) = DF (x, y)                 ⇔         x − t, t − y = 0

Lemma
c(p, q) is Bregman orthogonal to Hpq
∗
l(p, q) is Bregman orthogonal to Hpq

Lemma
−1
Geodesic (p, q) = {x : x =             F        ((1 − λ)p + λq ), λ ∈ [0, 1]}.

´
journee GeoTopAl         Voronoi diagrams of statistical distributions
Deﬁnition
Introduction
Bisectors
Bregman divergences
Bregman balls
Bregman Voronoi diagrams
Orthogonality and geodesics
Applications
Centroid and Bregman information

Lemma
The Bregman centroid of a domain D coincides with the (L2 )
centroid of D.

∂                                 ∂
DF (x, c) dx     =                     (F (x) − F (c) − x − c,                        F (c)   ) dx
∂c   x∈D                          ∂c        x∈D
2
= −                       F       (c)(x − c)dx
x∈D
2
= −         F       (c)                 xdx − c            dx
x∈D               x∈D
R
x∈D xdx
vanishes for   c∗   =    R       .
x∈D dx

´
journee GeoTopAl        Voronoi diagrams of statistical distributions
Deﬁnition
Introduction
Bisectors
Bregman divergences
Bregman balls
Bregman Voronoi diagrams
Orthogonality and geodesics
Applications
Centroid and Bregman information

Bregman information

x random variable proba p(x)
Distortion rate : DF (s) =           x∈X   p(x) DF (x, s) dx
Bregman information : infs∈X DF (s)
Bregman representative : s =                x∈X   p(x) dx = E(x)

´
journee GeoTopAl      Voronoi diagrams of statistical distributions
Introduction   Bregman Voronoi diagrams from polytopes
Bregman divergences     Bregman triangulations
Bregman Voronoi diagrams    Bregman Voronoi diagrams from power diagramms
Applications    Weighted Bregman diagrams

Theorem
The ﬁrst-type Bregman Voronoi diagram vorF (S) is obtained by
projecting by Proj⊥ the faces of the (d + 1)-dimensional
↑
polytope H = ∩i Hpi of X + onto X .

The Bregman Voronoi diagrams of type 1 or 2 of a set of n
d+1
d-dimensional points have complexity Θ(n 2 ) and can be
d+1
computed in optimal time Θ(n log n + n 2 ).

´
journee GeoTopAl    Voronoi diagrams of statistical distributions
Introduction   Bregman Voronoi diagrams from polytopes
Bregman divergences     Bregman triangulations
Bregman Voronoi diagrams    Bregman Voronoi diagrams from power diagramms
Applications    Weighted Bregman diagrams

´
journee GeoTopAl    Voronoi diagrams of statistical distributions
Introduction   Bregman Voronoi diagrams from polytopes
Bregman divergences     Bregman triangulations
Bregman Voronoi diagrams    Bregman Voronoi diagrams from power diagramms
Applications    Weighted Bregman diagrams

Deﬁnition
ˆ
S : the lifted image of S
T the lower convex hull of Sˆ
The vertical projection of T is called the Bregman triangulation
BT (S) = Proj⊥ (T ) of S

Properties
BT (S) is a triangulation embedded in Rd
For symetric divergences, BT (S) is dual to vorF (S)
Characteristic property : The Bregman sphere
circumscribing any simplex of BT (S) is empty
Optimality : BT (S) = minT ∈T (S) maxτ ∈T r (τ )
(r (τ ) = radius of the smallest Bregman ball containing τ )

´
journee GeoTopAl    Voronoi diagrams of statistical distributions
Introduction   Bregman Voronoi diagrams from polytopes
Bregman divergences     Bregman triangulations
Bregman Voronoi diagrams    Bregman Voronoi diagrams from power diagramms
Applications    Weighted Bregman diagrams

´
journee GeoTopAl    Voronoi diagrams of statistical distributions
Introduction   Bregman Voronoi diagrams from polytopes
Bregman divergences     Bregman triangulations
Bregman Voronoi diagrams    Bregman Voronoi diagrams from power diagramms
Applications    Weighted Bregman diagrams

Theorem
The ﬁrst-type Bregman Voronoi diagram of n sites of X is
identical to the power diagram of n Euclidean hyperspheres
centered at F (S) = {p | p ∈ S}

Bregman geodesic triangulation
The regular triangulation dual to the power diagram above
is a triangulation of the points pi
The image of this triangulation by −1 is a curved
F
triangulation whose vertices are the pi
The edges of this curved triangulation are geodesic arcs
joining two sites

´
journee GeoTopAl    Voronoi diagrams of statistical distributions
Introduction   Bregman Voronoi diagrams from polytopes
Bregman divergences     Bregman triangulations
Bregman Voronoi diagrams    Bregman Voronoi diagrams from power diagramms
Applications    Weighted Bregman diagrams

Weighted Bregman diagrams

WDF (pi , pj ) = DF (pi , pj ) + wi − wj

The weighted Bregman Voronoi diagrams of type 1 or 2 of a set
d+1
of n d-dimensional points have complexity Θ(n 2 ) and can
d+1
be computed in optimal time Θ(n log n + n 2 )

The k -order Bregman Voronoi diagram of n d-dimensional
points is a weighted Bregman Voronoi diagram

´
journee GeoTopAl    Voronoi diagrams of statistical distributions
Introduction
Bregman divergences
Bregman Voronoi diagrams
Applications

Aplications

Union of Bregman balls
d+1
The union of n Bregman balls of X has complexity Θ(n                            2        )
d+1
and can be computed in time Θ(n log n + n 2 )

Centroidal Bregman Voronoi diagrams

´
journee GeoTopAl    Voronoi diagrams of statistical distributions
Introduction
Bregman divergences
Bregman Voronoi diagrams
Applications

Sampling
ε-sample
error(P) = maxx∈D minpi ∈P DF (x, pi )

A ﬁnite set of points P of D is an ε-sample of D iff error(P) ≤ ε

Local maxima
error(P) = maxv∈V minpi ∈P DF (x, pi ).
where V consists of the vertices of BVD(P) and intersection
points between the edges of BVD(P) and the boundary of D

Ruppert’s algorithm
Insert a vertex of the current Bregman Voronoi diagram if its
Bregman radius is greater than ε
´
journee GeoTopAl    Voronoi diagrams of statistical distributions
Introduction
Bregman divergences
Bregman Voronoi diagrams
Applications

Termination of the sampling algorithm

Implied by the following lemma and a packing argument
Fatness of Bregman balls
If F is of class C 2 , there exists two constants γ and γ such
that                      √                        √
EB(c, γ r ) ⊂ B(c, r ) ⊂ EB(c, γ r )

´
journee GeoTopAl    Voronoi diagrams of statistical distributions
Introduction
Bregman divergences
Bregman Voronoi diagrams
Applications

Further applications ?

References
Benerjee, Merugu, Dhillon, Ghosh
Clustering with Bregman divergences
Journal of Machine Learning Research, 6: 1705-1749,
2005
Nielsen, Boissonnat, Nock
On Bregman Voronoi diagrams
SODA 2007

´
journee GeoTopAl    Voronoi diagrams of statistical distributions

```
DOCUMENT INFO
Shared By:
Categories:
Stats:
 views: 27 posted: 4/23/2009 language: English pages: 23