Clark's Theorem on linear programs holds for convex programs

Proc. NatI. Acad. Sci. USA Vol. 75, No. 4, pp. 1624-1626, April 1978 Mathematics Clark's Theorem on linear programs holds for convex programs (dual programs/feasible sets) R. J. DUFFIN Department of Mathematics, Carnegie-Mellon University, Pittsburgh, Pennsylvania 15213 Contributed by R. J. Duffin, January 23, 1978 Given a linear minimization program, then ABSTRACT there is an associated linear maximization program termed the dual. F. E. Clark proved the following theorem. "If the set of feasible points of one program is bounded, then the set of feasible points of the other program is unbounded." A convex program is the minimization of a convex function subject to the constraint that a number of other convex functions be nonpositive. As is well known, a dual maximization problem can be defined in terms of the Lagrange function. The dual objection function is the infimum of the Lagrange function. The feasible Lagrange multipliers are those satisfying: (i) the multipliers are nonnegative and (ii) the dual objective function is not negative infinity. It is found that Clark's Theorem applies unchanged to dual convex programs. Moreover, the programs have equal values. If g, gl,. gp are linear functions and C = X, then Program A becomes the linear program Program A: Seek MA = inf E ajxj subject to E CkjXj + bk < 0, xl1 1 n n k 1,...,p. The dual objective function can be calculated explicitly and Program B reduces to the linear program Program B: Seek MB = sup f Xkbk subject to XkCkj + a; = Feasible points of convex programs Let g(x), gi(x), . . ., gp(x) be real-valued functions that are continuous and convex on a closed convex set C of a normed space X. The space X can be RW, a real Hilbert space, or a reflexive Banach space. Then a basic minimization problem can be stated as a program. Program A: Seek the value MA = inf g(x) subject to the X 0,j = 1,. n and Xk 2 O. k = 1,. p. In this case A and B have the standard form of a pair of dual linear programs and so the following theorem of Clark holds (ref. 1). THEOREM 1: Given a pair of dual programs. If the feasible points of one program form a bounded set, then the feasible points of the other program form an unbounded set. The convention is adopted here that the empty set is neither bounded or unbounded. The first goal of this note is to show that Clark's Theorem holds for a convex program and its Lagrange dual program. For linear programs MA = MB. Unfortunately this duality equality is not true for all convex programs and it can happen that MA > MB. A second goal of this note is to give additional qualifications on the convex functions which will ensure that the duality equality holds. This is expressed in the following theorem. THEOREM 2: Given a pair of dual programs. If one program has a bounded feasible set, then both programs have finite values. Moreover the values are equal. In other words there is no "duality gap" when one of the feasible sets is bounded. We need the following elementary lemma which shows that convex functions can decay at a linear rate at most. LEMMA 0: Let g(x) be a real-valued function that is continuous and convex on a closed convex set C of a normed space X. Then given an e > 0 there is a A > 0 such that for any x E C for k = 1,... , p. constraints x E C and gk(x) This is termed the primal program and g(x) is termed the objective function. The set of feasible points for the primal program is termed a and is defined as a = xlx (E C, gk(x) < 0, k = 1l... , pi. It is well known that there is an associated maximization problem involving the Lagrange function <0 L(x, X) = g(x) + 1 Xkgk(x). Of course the Xk are the Lagrange multipliers. Then a set of multipliers (X,, . . X,p) is thought of as a point X in a dual space RP. Let a function f(X) be defined as inf f(X) = xEEC L(x, X). Then the associated maximization problem is stated as follows, g(x) > g(O)- AlIxII - e [1] Program B: Seek the value MB = sup f(X) subject to the X constraints Xk > Ofor k = 1, ... p. This is termed the Lagrange dual program and f(X) is termed the dual objective function. (It is only rarely that f can be expressed by an explicit formula.) The set of feasible points for the dual program is termed f3 and is defined as = IXIXk > 0, k 1. p andf(X)> -col. 1624 provided C contains the origin. Proof: By the continuity of g there is a > 0 such that if x E C, then llg(x)-g(O) I < Cif l xii <6 [2] If x C C, then by the convexity of g and C Og(x) + (1 - O) g(O) > g(Ox), O < 0 < 1 g(x) > g(O) + jg(Ox) g(O)j/O. - Mathematics: Duffin If Proc. Natl. Acad. Sci. USA 75 (1978) = 1625 g(O)I we l1x <> eaand may take ii O = b/ lx 11. Then li1xil > a so Ig(Ox) - 6. [3] Let A = E/a; then it is seen that 2 and 3 prove 1. In what follows it is assumed, without loss of generality, that the convex set C contains the origin. Superconsistent programs The following basic lemma is easy to establish. LEMMA 1: If the feasible sets a and d are both nonempty, then MA and MB are both finite. Moreover MA > MB. Program A is said to be superconsistent if there is a point x* such that gk(X*) d. Thus consider Program A0: Seek MA' = inf t subject to gk(x) - t < 0, k = 1,. p, x C. This is seen to be a convex program for a space whose points are pairs (x, t). This program is clearly superconsistent because if t is large all relations in 4 become strict inequalities. Moreover MA 0 = d if Lemma 3 is false. Let B0 be the dual of A 0 so by Lemma 2 we have MA° = MB0 = f0(X") for some X". The Lagrange function for B0 is > g(x) g(0) - (E/0)lixii if lIx I LEMMA 4: If the feasible set A3 is bounded, then MA is finite and MA = MB. Proof: It is apparent from Lemma 3 that taking t a negative number shows that Program A is superconsistent. Thus A and B are both consistent so MA is finite by virtue of Lemma 1. Then Lemma 2 completes the proof. LEMMA 5: If the feasible set # is bounded, then the feasible set a is unbounded. Proof: Consider the system of inequalities 4 of Lemma 3. If t - o we see that gk(xt) - -co. By Lemma 0 it is seen that gk(x) > gk(O) - AjXIxI -E, X e C. It follows that IIXt || co as t -c. The points xt are in a so the lemma follows. It is seen that Lemmas 4 and 5 prove Theorems 1 and 2 in case the dual feasible set /3 is bounded. Canonical programs To avoid the duality gap the concept of a canonical program was introduced in ref. 2. Program A is said to be canonical if for some positive constant d the relation x E C and the inequalitites g(x) < a, gl(x) < d,. . . ., gp(x) < d confine x to a bounded set D when a is any large constant. (Correction: In ref. 2 the constant a was incorrectly put equal to d in this definition.) LEMMA 6: If the feasible set a is bounded, then Program A is canonical. Proof: Let a function h(x) be defined in C as h(x) = max (gI(x), g2(x), , gp(x)). Then it is easy to see that h(x) is a continuous convex function for x in C. Moreover, the feasible set a can now be defined , as a = Ex xjx C, h(x) 0. If C is bounded then D is bounded because C contains D. Hence A is canonical. Thus it may be assumed that C is unbounded. Therefore x E C and lIx 1 = r can be satisfied because the origin is in the convex set C. Then let d = inf h(x) for x C C, lx II = r. Of course d > 0 because the spherical shell of radius r does not contain points of a. If the space X is Rn, then compactness shows that there is a point x' such that d = h(x'), x'& C, lIx'll = r. If X is Hilbert space, this relation can be easily established by an argument using the convexity of h (see ref. 2). Because x' is not in a we have d > 0. By the convexity of h Oh(x) + (1 - O)h(O) > h(Ox), O < O < 1. Let 0 = r/ lix Ii for lix I > r. Then because h(0) < 0, 1lOx 1 =, x h(Ox) > d, we have h(x) > d lIxil/r if lIxil > r, x C C. inf [6] f0(X") = xEC i Xkgk(x). 1 Now return to Program B and let A' be in the feasible set /. Let Xs = X' + sX" for s > 0. Then XA > 0 and L(x, Xs) = L(x, X') + s i Xkgk(x). Take the infimum with respect to x and use relation 6 to obtain f(XS) > f(X') + sfP(X"). But f(X') and f0(X") are both finite, sof(Xs) > -C. Thus Xs is > 0. Now observe that j1AsX 1 as s X because of Eq. 5. This is a contradiction because / was assumed to be a bounded set. The contradiction proves Lemma 3. in the feasible set / for any s 1626 Mathematics: Duffin Proc. Natl. Acad. Sci. USA 75 (1978) Thus the set fx Ix E C, h(x) < d} is contained in the sphere lixII < r. This shows that A is canonical. LEMMA 7: If the feasible set a is bounded, then MA is finite and MA = MB. Proof: Lemma 0 applies to g(x), and it is seen thereby that MA = inf g(x) is finite because lix i is bounded. Moreover, A is canonical by Lemma 6. It was shown in refs. 2 and 3 that if MA is finite, then the duality equality, MA = MB, holds for canonical programs. LEMMA 8: If the feasible set a is bounded, then the feasible set ( is unbounded. Proof: If a is bounded then /B is not empty by virtue of Lemma 7. Lemma 5 shows that j3 can not be bounded. Thus # is unbounded. It is seen that Lemmas 7 and 8 prove Theorems I and 2 in case the primal feasible set a is bounded. This completes the proof. Discussion There are other types of dual programs besides the Lagrange dual considered here. The dual of a geometric program is an example. Avriel and Williams (ref. 4) showed that a version of Clark's Theorem applies in geometric programming. Peterson (ref. 5) has made studies of various types of duality. He showed that Clark's Theorem holds for quadratic programming, 1P regression analyses, roadway network analysis, and chemical equilibrium analysis. Peterson found that Clark's Theorem need not hold if equality constraints are used. Interesting refinements of Clark's Theorem in the linear case have been treated by Eckhardt (ref. 6). This study was supported by the Army Research Office, Research Triangle Park, NC, under Grant DAAG29-77-0024. 1. 2. 3. 4. Clark, F. E. (1961) Am. Math. Mon. 68,351-352. Duffin, R. J. (1975) Proc. Natl. Acad. Sci. USA 72, 1778-1781. Duffin, R. J. (1977) Proc. Natl. Acad. Sci. USA 74,26-28. Avriel, M. & Williams, A. C. (1970) J. Math. Anal. Appl. 32, 684-688. 5. Peterson, E. L. (1977) Math. Prog. 12,392-405. 6. Eckhardt, U. (1975) Methods of Operatfons Research XXI, ed. Henn, R. (Verlag Anton Hein, Meisenheim, W. Germany), pp. 69-73.