# Lecture 2 Convex sets

Document Sample

```					Lecture 2: Convex sets

August 28, 2008
Lecture 2

Outline

• Review basic topology in Rn
• Open Set and Interior
• Closed Set and Closure
• Dual Cone
• Convex set
• Cones
• Aﬃne sets
• Half-Spaces, Hyperplanes, Polyhedra
• Ellipsoids and Norm Cones
• Convex, Conical, and Aﬃne Hulls
• Simplex
• Verifying Convexity

Convex Optimization                                1
Lecture 2

Topology Review

Let {xk } be a sequence of vectors in Rn
Def. The sequence {xk } ⊆ Rn converges to a vector x ∈ Rn when
ˆ
xk − x tends to 0 as k → ∞
ˆ

• Notation: When {xk } converges to a vector x, we write xk → x
ˆ                ˆ

• The sequence {xk } converges x ∈ Rn if and only if for each component
ˆ
i: the i-th components of xk converge to the i-th component of x
ˆ
|xi − xi| tends to 0 as k → ∞
k   ˆ

Convex Optimization                                                        2
Lecture 2

Open Set and Interior
Let X ⊆ Rn be a nonempty set
Def. The set X is open if for every x ∈ X there is an open ball B(x, r) that
entirely lies in the set X , i.e.,
for each x ∈ X there is r > 0 s.th. for all z with z − x < r, we have z ∈ X
Def. A vector x0 is an interior point of the set X , if there is a ball B(x0, r)
contained entirely in the set X
Def. The interior of the set X is the set of all interior points of X , denoted
by (X)
• How is (X) related to X ?
• Example X = {x ∈ R2 | x1 ≥ 0, x2 > 0}
(X) = {x ∈ R2 | x1 > 0, x2 > 0}
(S) of a probability simplex S = {x ∈ Rn | x 0, e x = 1}
Th. For a convex set X , the interior (X) is also convex

Convex Optimization                                                           3
Lecture 2

Closed Set

Def. The complement of a given set X ⊆ Rn is the set of all vectors that do
not belong to X :
the complement of X = {x ∈ Rn | x ∈ X} = Rn \ X
/
Def. The set X is closed if its complement Rn \ X is open
• Examples: Rn and ∅ (both are open and closed)
{x ∈ R2 | x1 ≥ 0, x2 > 0} is open or closed?
hyperplane, half-space, simplex, polyhedral sets?
• The intersection of any family of closed set is closed
• The union of a ﬁnite family of closed set is closed
• The sum of two closed sets is not necessarily closed
• Example: C1 = {(x1, x2) | x1 = 0, x2 ∈ R}
C2 = {(x1, x2) | x1x2 ≥ 1, x1 ≥ 0}
C1 + C2 is not closed!
• Fact: The sum of a compact set and a closed set is closed

Convex Optimization                                                        4
Lecture 2

Closure

Let X ⊆ Rn be a nonempty set
Def. A vector x is a closure point of a set X if there exists a sequence
ˆ
{xk } ⊆ X such that xk → x
ˆ
ˆ
Closure points of X = {(−1)n/n | n = 1, 2, . . .}, X = {1 − x | x ∈ X}?
• Notation: The set of closure points of X is denoted by cl(X)

• What is relation between X and cl(X)?

Th. A set is closed if and only if it contains its closure points, i.e.,
X is closed iﬀ cl(X) ⊂ X

Th. For a convex set, the closure cl(X) is convex

Convex Optimization                                                       5
Lecture 2

Boundary

Let X ⊆ Rn be a nonempty set
Def. The boundary of the set X is the set of closure points for both the set
X and its complement Rn \ X , i.e.,

bd(X) = cl(X) ∩ cl(Rn \ X)

• Example X = {x ∈ Rn | g1(x) ≤ 0, . . . , gm(x) ≤ 0}. Is X closed?
What constitutes the boundary of X ?

Convex Optimization                                                         6
Lecture 2

Dual Cone

Let K be a nonempty cone in Rn
Def. The dual cone of K is the set K ∗ deﬁned by

K ∗ = {z | z x ≥ 0 for all x ∈ K}

• The dual cone K ∗ is a closed convex cone even when K is neither closed
nor convex
• Let S be a subspace. Then, S ∗ = S ⊥.
• Let C be a closed convex cone. Then, (C ∗)∗ = C .
• For an arbitrary cone K , we have (K ∗)∗ = cl(conv(K)).

Convex Optimization                                                         7
Lecture 2

Convex set

• A line segment deﬁned by vectors x and y is
the set of points of the form αx + (1 − α)y    for α ∈ [0, 1]

• A set C ⊂ Rn is convex when, with any two vectors x and y that belong
to the set C , the line segment connecting x and y also belongs to C

Convex Optimization                                                        8
Lecture 2

Examples

Which of the following sets are convex?
• The space Rn
• A line through two given vectors x and y
l(x, y) = {z | z = x + t(y − x), t ∈ R}
• A ray deﬁned by a vector x
{z | z = λx, λ ≥ 0}
• The positive orthant {x ∈ Rn | x 0} (    componentwise inequality)
• The set {x ∈ R2 | x1 > 0, x2 ≥ 0}
• The set {x ∈ R2 | x1x2 = 0}

Convex Optimization                                                    9
Lecture 2

Cone

A set C ⊂ Rn is a cone when, with every vector x ∈ C , the ray {λx | λ ≥ 0}
belongs to the set C

• A cone may or may not be convex
• Examples:      {x ∈ Rn | x 0} {x ∈ R2 | x1x2 ≥ 0}
For a two sets C and S , the sum C + S is deﬁned by
C + S = {z | z = x + y, x ∈ C, y ∈ S} (the order does nor matter)
Convex Cone Lemma: A cone C is convex if and only if C + C ⊆ C
Proof: Pick any x and y in C , and any α ∈ [0, 1]. Then, αx and (1 − α)y
belong to C because... . Using C + C ⊆ C , it follows that ... Reverse: Let
C be convex cone, and pick any x, y ∈ C . Consider 1/2(x + y)...

Convex Optimization                                                        10
Lecture 2

Aﬃne Set

A set C ⊂ Rn is a aﬃne when, with every two distinct vectors x, y ∈ C ,
the line {x + t(y − x) | t ∈ R} belongs to the set C
• An aﬃne set is always convex
• A subspace is an aﬃne set

A set C is aﬃne if and only if C is a translated subspace, i.e.,
C = S + x0 for some subspace S and some x0 ∈ C

Dimension of an aﬃne set C is the dimension of the subspace S

Convex Optimization                                                      11
Lecture 2

Hyperplanes and Half-spaces

Hyperplane is a set of the form {x | a x = b} for a nonzero vector a

Half-space is a set of the form {x | a x ≤ b} with a nonzero vector a
The vector a is referred to as the normal vector
• A hyperplane in Rn divides the space into two half-spaces
{x | a x ≤ b} and {x | a x ≥ b}
• Half-spaces are convex
• Hyperplanes are convex and aﬃne

Convex Optimization                                                           12
Lecture 2

Polyhedral Sets
A polyhedral set is given by ﬁnitely many linear inequalities
C = {x | Ax b} where A is an m × n matrix

• Every polyhedral set is convex
• Linear Problem
minimize   cx
subject to Bx ≤ b, Dx = d

The constraint set {x | Bx ≤ b, Dx = d} is polyhedral.

Convex Optimization                                                   13
Lecture 2

Ellipsoids

Let A be a square (n × n) matrix.
• A is positive semideﬁnite when x Ax ≥ 0 for all x ∈ Rn
• A is positive deﬁnite when x Ax > 0 for all x ∈ Rn, x = 0
An ellipsoid is a set of the form
E = {x | (x − x0) P −1(x − x0) ≤ 1}
where P is symmetric and positive deﬁnite

• x0 is the center of the ellipsoid E
• A ball {x | x − x0 ≤ r} is a special case of the ellipsoid (P = r2I )
• Ellipsoids are convex

Convex Optimization                                                         14
Lecture 2

Norm Cones
A norm cone is the set of the form
C = {(x, t) ∈ Rn × R | x ≤ t}

• The norm          ·   can be any norm in Rn
• The norm cone for Euclidean norm is also known as ice-cream cone
• Any norm cone is convex

Convex Optimization                                                     15
Lecture 2

Convex and Conical Hulls
A convex combination of vectors x1, . . . , xm is a vector of the form
m
α 1 x1 + . . . + α m xm    αi ≥ 0 for all i and   i=1 αi   =1
The convex hull of a set X is the set of all convex combinations of the
vectors in X , denoted conv(X)

A conical combination of vectors x1, . . . , xm is a vector of the form
λ1x1 + . . . + λmxm    with λi ≥ 0 for all i
The conical hull of a set X is the set of all conical combinations of the
vectors in X , denoted by cone(X)

Convex Optimization                                                               16
Lecture 2

Aﬃne Hull

An aﬃne combination of vectors x1, . . . , xm is a vector of the form
m
t1x1 + . . . + tmxm    with   i=1 ti   = 1, ti ∈ R for all i
The aﬃne hull of a set X is the set of all aﬃne combinations of the vectors
in X , denoted aﬀ (X)

The dimension of a set X is the dimension of the aﬃne hull of X
dim(X) = dim(aﬀ (X))

Convex Optimization                                                               17
Lecture 2

Simplex
A simplex is a set given as a convex combination of a ﬁnite collection of
vectors v0, v1, . . . , vm:
C = conv{v0, v1 . . . , vm}
The dimension of the simplex C is equal to the maximum number of linearly
independent vectors among v1 − v0, . . . , vm − v0.
Examples
• Unit simplex {x ∈ Rn | x 0, e x ≤ 1}, e = (1, . . . , 1), dim -?
• Probability simplex {x ∈ Rn | x 0, e x = 1}, dim -?

Convex Optimization                                                      18
Lecture 2

Practical Methods for Establishing Convexity of a Set

Establish the convexity of a given set X
• The set is one of the “recognizable” (simple) convex sets such as
polyhedral, simplex, norm cone, etc

• Prove the convexity by directly applying the deﬁnition
For every x, y ∈ X and α ∈ (0, 1), show that αx + (1 − α)y is also in X

• Show that the set is obtained from one of the simple convex sets through
an operation that preserves convexity

Convex Optimization                                                         19
Lecture 2

Operations Preserving Convexity

Let C ⊆ Rn, C1 ⊆ Rn, C2 ⊆ Rn, and K ⊆ Rm be convex sets. Then, the
following sets are also convex:
• The intersection C1 ∩ C2 = {x | x ∈ C1 and x ∈ C2}
• The sum C1 + C2 of two convex sets
• The translated set C + a
• The scaled set tC = {tx | x ∈ C} for any t ∈ R
• The Cartesian product C1 × C2 = {(x1, x2) | x1 ∈ C1, x2 ∈ C2}
• The coordinate projection {x1 | (x1, x2) ∈ C for some x2}
• The image AC under a linear transformation A : Rn → Rm:
AC = {y ∈ Rm | y = Ax for some x ∈ C}
• The inverse image A−1K under a linear transformation A : Rn → Rm:
A−1K = {x ∈ Rn | Ax ∈ K}

Convex Optimization                                                   20

```
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
 views: 9 posted: 8/14/2011 language: English pages: 21