Lecture 2 Convex sets

Document Sample
Lecture 2 Convex sets Powered By Docstoc
					Lecture 2: Convex sets



     August 28, 2008
                                            Lecture 2


                                  Outline

  • Review basic topology in Rn
  • Open Set and Interior
  • Closed Set and Closure
  • Dual Cone
  • Convex set
  • Cones
  • Affine sets
  • Half-Spaces, Hyperplanes, Polyhedra
  • Ellipsoids and Norm Cones
  • Convex, Conical, and Affine Hulls
  • Simplex
  • Verifying Convexity

Convex Optimization                                1
                                                                     Lecture 2


                           Topology Review

 Let {xk } be a sequence of vectors in Rn
Def. The sequence {xk } ⊆ Rn converges to a vector x ∈ Rn when
                                                   ˆ
                          xk − x tends to 0 as k → ∞
                               ˆ

   • Notation: When {xk } converges to a vector x, we write xk → x
                                                ˆ                ˆ

   • The sequence {xk } converges x ∈ Rn if and only if for each component
                                   ˆ
     i: the i-th components of xk converge to the i-th component of x
                                                                    ˆ
                          |xi − xi| tends to 0 as k → ∞
                            k   ˆ




 Convex Optimization                                                        2
                                                                          Lecture 2


                          Open Set and Interior
   Let X ⊆ Rn be a nonempty set
 Def. The set X is open if for every x ∈ X there is an open ball B(x, r) that
      entirely lies in the set X , i.e.,
   for each x ∈ X there is r > 0 s.th. for all z with z − x < r, we have z ∈ X
Def. A vector x0 is an interior point of the set X , if there is a ball B(x0, r)
     contained entirely in the set X
Def. The interior of the set X is the set of all interior points of X , denoted
     by (X)
     • How is (X) related to X ?
     • Example X = {x ∈ R2 | x1 ≥ 0, x2 > 0}
                         (X) = {x ∈ R2 | x1 > 0, x2 > 0}
             (S) of a probability simplex S = {x ∈ Rn | x 0, e x = 1}
  Th. For a convex set X , the interior (X) is also convex

   Convex Optimization                                                           3
                                                                     Lecture 2


                               Closed Set

Def. The complement of a given set X ⊆ Rn is the set of all vectors that do
     not belong to X :
               the complement of X = {x ∈ Rn | x ∈ X} = Rn \ X
                                                       /
Def. The set X is closed if its complement Rn \ X is open
  • Examples: Rn and ∅ (both are open and closed)
     {x ∈ R2 | x1 ≥ 0, x2 > 0} is open or closed?
     hyperplane, half-space, simplex, polyhedral sets?
  • The intersection of any family of closed set is closed
  • The union of a finite family of closed set is closed
  • The sum of two closed sets is not necessarily closed
      • Example: C1 = {(x1, x2) | x1 = 0, x2 ∈ R}
                           C2 = {(x1, x2) | x1x2 ≥ 1, x1 ≥ 0}
                                   C1 + C2 is not closed!
      • Fact: The sum of a compact set and a closed set is closed

 Convex Optimization                                                        4
                                                                     Lecture 2


                                  Closure

  Let X ⊆ Rn be a nonempty set
Def. A vector x is a closure point of a set X if there exists a sequence
              ˆ
     {xk } ⊆ X such that xk → x
                              ˆ
                                                     ˆ
  Closure points of X = {(−1)n/n | n = 1, 2, . . .}, X = {1 − x | x ∈ X}?
    • Notation: The set of closure points of X is denoted by cl(X)

    • What is relation between X and cl(X)?

Th. A set is closed if and only if it contains its closure points, i.e.,
                          X is closed iff cl(X) ⊂ X

Th. For a convex set, the closure cl(X) is convex


  Convex Optimization                                                       5
                                                                      Lecture 2


                                Boundary

 Let X ⊆ Rn be a nonempty set
Def. The boundary of the set X is the set of closure points for both the set
     X and its complement Rn \ X , i.e.,

                         bd(X) = cl(X) ∩ cl(Rn \ X)

   • Example X = {x ∈ Rn | g1(x) ≤ 0, . . . , gm(x) ≤ 0}. Is X closed?
     What constitutes the boundary of X ?




 Convex Optimization                                                         6
                                                                       Lecture 2


                                Dual Cone

  Let K be a nonempty cone in Rn
Def. The dual cone of K is the set K ∗ defined by

                        K ∗ = {z | z x ≥ 0 for all x ∈ K}




    • The dual cone K ∗ is a closed convex cone even when K is neither closed
      nor convex
    • Let S be a subspace. Then, S ∗ = S ⊥.
    • Let C be a closed convex cone. Then, (C ∗)∗ = C .
    • For an arbitrary cone K , we have (K ∗)∗ = cl(conv(K)).


  Convex Optimization                                                         7
                                                                    Lecture 2


                             Convex set

  • A line segment defined by vectors x and y is
    the set of points of the form αx + (1 − α)y    for α ∈ [0, 1]




  • A set C ⊂ Rn is convex when, with any two vectors x and y that belong
    to the set C , the line segment connecting x and y also belongs to C




Convex Optimization                                                        8
                                                                Lecture 2


                             Examples




Which of the following sets are convex?
 • The space Rn
 • A line through two given vectors x and y
                   l(x, y) = {z | z = x + t(y − x), t ∈ R}
 • A ray defined by a vector x
                              {z | z = λx, λ ≥ 0}
  • The positive orthant {x ∈ Rn | x 0} (    componentwise inequality)
  • The set {x ∈ R2 | x1 > 0, x2 ≥ 0}
  • The set {x ∈ R2 | x1x2 = 0}

Convex Optimization                                                    9
                                                                     Lecture 2


                                  Cone

A set C ⊂ Rn is a cone when, with every vector x ∈ C , the ray {λx | λ ≥ 0}
belongs to the set C




 • A cone may or may not be convex
 • Examples:      {x ∈ Rn | x 0} {x ∈ R2 | x1x2 ≥ 0}
For a two sets C and S , the sum C + S is defined by
C + S = {z | z = x + y, x ∈ C, y ∈ S} (the order does nor matter)
Convex Cone Lemma: A cone C is convex if and only if C + C ⊆ C
Proof: Pick any x and y in C , and any α ∈ [0, 1]. Then, αx and (1 − α)y
belong to C because... . Using C + C ⊆ C , it follows that ... Reverse: Let
C be convex cone, and pick any x, y ∈ C . Consider 1/2(x + y)...


Convex Optimization                                                        10
                                                                   Lecture 2


                                Affine Set

A set C ⊂ Rn is a affine when, with every two distinct vectors x, y ∈ C ,
the line {x + t(y − x) | t ∈ R} belongs to the set C
  • An affine set is always convex
  • A subspace is an affine set

A set C is affine if and only if C is a translated subspace, i.e.,
          C = S + x0 for some subspace S and some x0 ∈ C

Dimension of an affine set C is the dimension of the subspace S




Convex Optimization                                                      11
                                                                        Lecture 2


                      Hyperplanes and Half-spaces

Hyperplane is a set of the form {x | a x = b} for a nonzero vector a




Half-space is a set of the form {x | a x ≤ b} with a nonzero vector a
The vector a is referred to as the normal vector
  • A hyperplane in Rn divides the space into two half-spaces
                       {x | a x ≤ b} and {x | a x ≥ b}
  • Half-spaces are convex
  • Hyperplanes are convex and affine

Convex Optimization                                                           12
                                                                Lecture 2


                           Polyhedral Sets
A polyhedral set is given by finitely many linear inequalities
           C = {x | Ax b} where A is an m × n matrix




  • Every polyhedral set is convex
  • Linear Problem
                          minimize   cx
                         subject to Bx ≤ b, Dx = d

     The constraint set {x | Bx ≤ b, Dx = d} is polyhedral.

Convex Optimization                                                   13
                                                                     Lecture 2


                                 Ellipsoids

Let A be a square (n × n) matrix.
  • A is positive semidefinite when x Ax ≥ 0 for all x ∈ Rn
  • A is positive definite when x Ax > 0 for all x ∈ Rn, x = 0
An ellipsoid is a set of the form
                   E = {x | (x − x0) P −1(x − x0) ≤ 1}
where P is symmetric and positive definite




  • x0 is the center of the ellipsoid E
  • A ball {x | x − x0 ≤ r} is a special case of the ellipsoid (P = r2I )
  • Ellipsoids are convex

Convex Optimization                                                         14
                                                                  Lecture 2


                                   Norm Cones
A norm cone is the set of the form
C = {(x, t) ∈ Rn × R | x ≤ t}




  • The norm          ·   can be any norm in Rn
  • The norm cone for Euclidean norm is also known as ice-cream cone
  • Any norm cone is convex




Convex Optimization                                                     15
                                                                            Lecture 2


                           Convex and Conical Hulls
A convex combination of vectors x1, . . . , xm is a vector of the form
                                                              m
            α 1 x1 + . . . + α m xm    αi ≥ 0 for all i and   i=1 αi   =1
The convex hull of a set X is the set of all convex combinations of the
vectors in X , denoted conv(X)


A conical combination of vectors x1, . . . , xm is a vector of the form
                      λ1x1 + . . . + λmxm    with λi ≥ 0 for all i
The conical hull of a set X is the set of all conical combinations of the
vectors in X , denoted by cone(X)




Convex Optimization                                                               16
                                                                            Lecture 2


                                   Affine Hull

An affine combination of vectors x1, . . . , xm is a vector of the form
                                           m
             t1x1 + . . . + tmxm    with   i=1 ti   = 1, ti ∈ R for all i
The affine hull of a set X is the set of all affine combinations of the vectors
in X , denoted aff (X)


The dimension of a set X is the dimension of the affine hull of X
                            dim(X) = dim(aff (X))




Convex Optimization                                                               17
                                                                   Lecture 2


                               Simplex
A simplex is a set given as a convex combination of a finite collection of
vectors v0, v1, . . . , vm:
                            C = conv{v0, v1 . . . , vm}
The dimension of the simplex C is equal to the maximum number of linearly
independent vectors among v1 − v0, . . . , vm − v0.
Examples
  • Unit simplex {x ∈ Rn | x 0, e x ≤ 1}, e = (1, . . . , 1), dim -?
  • Probability simplex {x ∈ Rn | x 0, e x = 1}, dim -?




Convex Optimization                                                      18
                                                                      Lecture 2


   Practical Methods for Establishing Convexity of a Set

Establish the convexity of a given set X
  • The set is one of the “recognizable” (simple) convex sets such as
    polyhedral, simplex, norm cone, etc

  • Prove the convexity by directly applying the definition
    For every x, y ∈ X and α ∈ (0, 1), show that αx + (1 − α)y is also in X

  • Show that the set is obtained from one of the simple convex sets through
    an operation that preserves convexity




Convex Optimization                                                         19
                                                                Lecture 2


                      Operations Preserving Convexity

Let C ⊆ Rn, C1 ⊆ Rn, C2 ⊆ Rn, and K ⊆ Rm be convex sets. Then, the
following sets are also convex:
  • The intersection C1 ∩ C2 = {x | x ∈ C1 and x ∈ C2}
  • The sum C1 + C2 of two convex sets
      • The translated set C + a
  • The scaled set tC = {tx | x ∈ C} for any t ∈ R
  • The Cartesian product C1 × C2 = {(x1, x2) | x1 ∈ C1, x2 ∈ C2}
  • The coordinate projection {x1 | (x1, x2) ∈ C for some x2}
  • The image AC under a linear transformation A : Rn → Rm:
                AC = {y ∈ Rm | y = Ax for some x ∈ C}
  • The inverse image A−1K under a linear transformation A : Rn → Rm:
                        A−1K = {x ∈ Rn | Ax ∈ K}

Convex Optimization                                                   20

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:9
posted:8/14/2011
language:English
pages:21