Quadratic Programming - I

							ESI 6492. Global Optimization


Quadratic Programming - I




                      ESI 6492. Global Optimization Quadratic Programming - I – p.
                Introduction
Consider global optimization problem of the following
form:
               min f (x) = 1 xT Qx + cT x
                             2

               s.t.   x∈D
where D is a polyhedron in Rn , c ∈ Rn and Q is a real
symmetric (n × n)–matrix.




                                       ESI 6492. Global Optimization Quadratic Programming - I – p.
                Introduction
If Q is positive semidefinite, the problem is a convex
minimization problem. In this case, every KKT point is a
local minimum, which is also global.
If Q is negative semidefinite, then f (x) is concave, and
this problem is referred to as a quadratic concave
minimization problem.
When Q has at least one positive and one negative
eigenvalue, i.e., Q is indefinite, then the problem is
referred to as indefinite quadratic programming problem.




                                        ESI 6492. Global Optimization Quadratic Programming - I – p.
                Introduction
As we discussed earlier, concave minimization over
polytopes has the interesting combinatorial property that,
if it has an optimal solution, then an optimal solution is
attained at a vertex of D.
For indefinite quadratic programming problems the global
minimum is not necessarily attained at a vertex of D. But
in this case the global minimum is attained at the
boundary of the feasible domain (Why?).




                                       ESI 6492. Global Optimization Quadratic Programming - I – p.
                Introduction
There are close connections between nonconvex
quadratic optimization problems and combinatorial
optimization.
Many integer linear and integer quadratic optimization
problems can be reformulated as quadratic programming
problems in continuous variables. Such a reformulation
provides some insight into the structure of these
problems.
On the other hand, approximate solutions of quadratic
programs can often be detected using mixed integer
programming.

                                      ESI 6492. Global Optimization Quadratic Programming - I – p.
Quadratic Integer Programming
Integer programming is used to model a variety of very
important problems.
Consider the following linear zero–one programming
problem:

            min     cT x

            s.t.    Ax ≤ b,
                    xi ∈ {0, 1}   (i = 1, . . . , n)

where A is a real (m × n)–matrix, c ∈ Rn and b ∈ Rm .



                                           ESI 6492. Global Optimization Quadratic Programming - I – p.
Quadratic Integer Programming
The 0–1 integer linear programming problem is equivalent
to the following concave minimization problem:

              min f (x) = cT x + µxT (e − x)

              s.t.   Ax ≤ b, 0 ≤ x ≤ e

where µ is a sufficiently large positive number.
The function f (x) is concave since −xT x is concave.
The global minimum is attained only when xT (e − x) = 0,
provided that µ is large enough.



                                         ESI 6492. Global Optimization Quadratic Programming - I – p.
Quadratic Integer Programming
Similar techniques can be applied to reduce nonlinear
zero–one problems to equivalent concave minimization
problems.
Consider the quadratic 0–1 problem of the following form:

             min f (x) = cT x + xT Qx

             s.t.   xi ∈ {0, 1}   (i = 1, . . . , n)

where Q is a real symmetric (n × n) matrix.
Given any real number µ, let Q = Q + µI, where I is the
(n × n) identity matrix, and c = c − µe.

                                            ESI 6492. Global Optimization Quadratic Programming - I – p.
Quadratic Integer Programming
Then the above 0–1 quadratic problem is equivalent to
the problem:

              min f (x) = cT x + xT Qx

              s.t.     xi ∈ {0, 1}   (i = 1, . . . , n)

since f (x) = f (x).
If we choose µ such that Q = Q + µI becomes a negative
semidefinite matrix (e.g., µ = −λ, where λ, is the largest
eigenvalue of Q), then the objective function f (x)
becomes concave and the constraints can be replaced by
0 ≤ x ≤ e.
                                               ESI 6492. Global Optimization Quadratic Programming - I – p.
         Nonlinear Assignment
The Quadratic Assignment Problem (QAP) can be stated
as follows:

Given a positive integer n, and two n × n matrices A =
(aij ) and B = (bij ) with nonnegative entries, find a
permutation p = (p(1), . . . , p(n)) of the set {1, 2, . . . , n}
that minimizes
                              n   n
                    C(p) =             aij bp(i)p(j).
                             i=1 j=1


The QAP is known to be NP–hard.



                                                 ESI 6492. Global Optimization Quadratic Programming - I – p. 1
             Nonlinear Assignment
Facility Location Problem:
    n facilities, each of which can be located at one of n
    locations.
    A = (aij ) is the flow matrix, where for i, j = 1, ..., n, aij
    represents the flow of materials from facility i to facility j.
    B = (bij ) is the distance matrix, where for i, j = 1, . . . , n,
    bij represents the distance from location i to location j.
    The objective function is the cost associated with the
    assignments of the n facilities to the n locations.



                                                ESI 6492. Global Optimization Quadratic Programming - I – p. 1
       Nonlinear Assignment
The QAP may be reformulated as a quadratic 0–1
programming problem:
                   n     n   n    n
          min                           aij bkl xik xjl
                  i=1 j=1 k=1     l=1
                   n
           s.t.         xij = 1   (j = 1, . . . , n)
                  i=1

                   n
                        xij = 1   (i = 1, . . . , n),
                  j=1

                  xij ∈ {0, 1}     (i, j = 1, . . . , n).


                                               ESI 6492. Global Optimization Quadratic Programming - I – p. 1
         Nonlinear Assignment
If we denote the feasible domain of the above problem by
D, then the problem can be written as

                         min xT Sx

                         s.t.   x ∈ D,

where the (n2 × n2 )–matrix S has nonnegative entries.
The entries of S are the products aij bkl , and it is natural to
define a row of S by i and j fixed, and a column of S by k
and l fixed (or vice–versa).



                                          ESI 6492. Global Optimization Quadratic Programming - I – p. 1
          Nonlinear Assignment
Proposition. The QAP is equivalent to the following
concave programming problem:

                            min xT Qx

                            s.t.   x ∈ Ω,

where Q = S − αI, α > ||S||∞ , and Ω is the set of all
                              T   n2
x = (x11 , x12 , . . . , xnn ) ∈ R satisfying

    n
    j=1   xij = 1 (i = 1, . . . , n),

    n
    i=1   xij = 1 (j = 1, . . . , n), xij ≥ 0, (i, j = 1, . . . , n).

                                                ESI 6492. Global Optimization Quadratic Programming - I – p. 1
           Nonlinear Assignment
Proof:
   It is well–known that all vertices of the set Ω are integer
   with components in {0, 1}.
   Moreover, we know that the concave function xT Qx
   achieves its minimum at some vertex of Ω.
   But on the vertex set of Ω we have
                       n   n                   n     n
          αxT Ix = α             (xij )2 = α                xij = αn,
                       i=1 j=1                 i=1 j=1


   which is a constant.


                                                   ESI 6492. Global Optimization Quadratic Programming - I – p. 1
         Maximum Clique Problem
Notations:

G = (V, E) is a simple undirected graph, V = {1, 2, . . . , n}.

G = (V, E), is the complement graph of G = (V, E), where
E = {(i, j) | i, j ∈ V, i = j and (i, j) ∈ E}.
                                         /

For S ⊆ V , G(S) = (S, E ∩ S × S) the subgraph induced by S .




                                             ESI 6492. Global Optimization Quadratic Programming - I – p. 1
       Maximum Clique Problem
Example: V =
{1, 2, 3, 4, 5}       1                            2
E=
{(1,2), (1,3),
(1,4), (2,3),
(2,4), (3,4),
(3,5), (4,5)}     3                                       4



                          5



                              ESI 6492. Global Optimization Quadratic Programming - I – p. 1
         Maximum Clique Problem
S = {1, 3, 5}
                    1                        2




                3                                   4



                        5



                            ESI 6492. Global Optimization Quadratic Programming - I – p. 1
         Maximum Clique Problem
S = {1, 3, 5}
G(S) :              1




                3



                        5



                            ESI 6492. Global Optimization Quadratic Programming - I – p. 1
          Maximum Clique Problem
S = {1, 3, 5}
G(S) :               1




                 3



                         5



                             ESI 6492. Global Optimization Quadratic Programming - I – p. 1
        Maximum Clique Problem
A subset I ⊆ V is called an independent set (stable set, vertex
packing) if G(I) has no edges.

A subset C ⊆ V is called a clique if G(C) is complete, i.e. it
has all possible edges.

An independent set (clique) is said to be
 – maximal, if it is not a subset of any larger independent
   set (clique);
 – maximum, if there is no larger independent set (clique) in
   the graph.




                                            ESI 6492. Global Optimization Quadratic Programming - I – p. 1
     Maximum Clique Problem

Example:         1                            2




             3                                       4



                     5


                         ESI 6492. Global Optimization Quadratic Programming - I – p. 1
      Maximum Clique Problem
A maximal
clique:          1                        2
{3, 4, 5}




             3                                   4



                     5


                         ESI 6492. Global Optimization Quadratic Programming - I – p. 1
      Maximum Clique Problem

The maximum
clique:            1                        2
{1, 2, 3, 4}




               3                                   4



                       5


                           ESI 6492. Global Optimization Quadratic Programming - I – p. 1
         Maximum Clique Problem
α(G) – the independence (stability) number of G.
ω(G) – the clique number of G.
V C ⊆ V is a vertex cover if every edge has at least one
endpoint in V C .
     I is a maximum independent set of G

                                    ¯
           I is a maximum clique of G

     V \ I is a minimum vertex cover of G.

MC, MIS and MVC problems are NP-hard


                                              ESI 6492. Global Optimization Quadratic Programming - I – p. 2
    Maximum Clique Problem
The Maximum Clique Problem is one of the classical
problems in graph theory with many applications in
different fields.
The maximum clique problem can be formulated in
several ways either as an integer programming problem
or as a continuous global optimization problem.
Next, we consider a formulation as an indefinite quadratic
programming problem.




                                      ESI 6492. Global Optimization Quadratic Programming - I – p. 2
    Maximum Clique Problem
Following Motzkin and Strauss (1965), let AG = (aij )n×n
be the adjacency matrix of G defined by
                     
                      1 if (i, j) ∈ E
               aij =
                       0 if (i, j) ∈ E .
                                   /
                     

The matrix AG is symmetric, and hence all its eigenvalues
are real numbers. The sum of the eigenvalues is zero
because the main diagonal elements aii are zero. It
follows that, in general, AG has positive and negative (and
possibly zero) eigenvalues.


                                       ESI 6492. Global Optimization Quadratic Programming - I – p. 2
    Maximum Clique Problem
Consider the indefinite quadratic programming problem

      max fG (x) =               xi xj = 1 xT A G x
                                         2
                       (i,j)∈E
                                                       n
                                              T
      s.t.   x ∈ S = {x = (x1 , . . . , xn ) :                 xi = 1,
                                                     i=1

             xi ≥ 0   (i = 1, . . . , n)}.

Proposition. If α = max{fG (x) : x ∈ S}, then G has a
maximum clique C of size k = 1/(1 − 2α). This maximum
can be attained by setting xi = 1/k if i ∈ C and xi = 0 if
i ∈ C.
  /


                                               ESI 6492. Global Optimization Quadratic Programming - I – p. 2
    Maximum Clique Problem
Corollary. The size k of the maximum clique in an
undirected graph G = (V, E) with | V |= n satisfies
                      1               n
              k≥        2|E|
                               =
                   1−            n − (n − 1)d
                         n2

where d =| E | / n is the density of G. The bound is
                 2
exact when G is complete (d = 1, k = n).
Consider the following formulation of the maximum clique
problem as a quadratic 0–1 program.




                                         ESI 6492. Global Optimization Quadratic Programming - I – p. 2
     Maximum Clique Problem
Proposition. The maximum clique problem in a graph
G = (V, E) with vertex set V = {1, . . . , n} is equivalent to
                              n
            min f (x) = −          xi + 2                   xi xj
                             i=1
                                            (i, j) ∈ E
                                                   /
                                                                                              (1)
                                              i>j

            s.t.   x ∈ {0, 1}n .

A solution x∗ to (1) defines a maximum clique
C = {i ∈ {1, . . . , n} : x∗ = 1} with | C |= −f (x∗ ).
                           i




                                                ESI 6492. Global Optimization Quadratic Programming - I – p. 2
    Maximum Clique Problem
Proof: Let G = (V, E), denote the complement of
G = (V, E), where E = {(i, j) : i, j ∈ V, i = j and
(i, j) ∈ E}
       /
Consider the problem
                                   n
                      min      −         xi
                                   i=1                                                      (2)
    s.t.   xi + xj ≤ 1 for all (i, j) ∈ E and x ∈ {0, 1}n

Each constraint ensures that xi = 1 implies xj = 0 for all
(i, j) ∈ E, and vice versa. Therefore, any feasible solution
of (1) corresponds to a clique C defined by
i ∈ C ⇔ xi = 1.
                                              ESI 6492. Global Optimization Quadratic Programming - I – p. 2
     Maximum Clique Problem
An optimal solution of problem (1) has the maximum
number of components x∗ set to one, and hence it
                          i
corresponds to a maximum clique with clique size
     n
k=         x∗ .
            i
     i=1

Another way of stating the constraints in (2) is to require
xi xj = 0 for all (i, j) ∈ E and x ∈ {0, 1}n , because for
x ∈ {0, 1}n , xi xj = 0 is equivalent to xi + xj ≤ 1.
Finally, in an optimal solution of (1) the terms xi xj must
vanish because xi = 1, xj = 0 (or xi = 0, xj = 1) yields a
smaller objective value than xi = xj = 1.


                                        ESI 6492. Global Optimization Quadratic Programming - I – p. 2