VIEWS: 0 PAGES: 49 POSTED ON: 10/10/2012
ENGINEERING OPTIMIZATION Methods and Applications A. Ravindran, K. M. Ragsdell, G. V. Reklaitis Book Review Page 1 Chapter 5: Constrained Optimality Criteria Part 1: Ferhat Dikbiyik Part 2:Yi Zhang Review Session July 2, 2010 Page 2 Constraints: Good guys or bad guys? Page 3 Constraints: Good guys or bad guys? reduces the region in which we search for optimum. Page 4 Constraints: Good guys or bad guys? makes optimization process very complicated Page 5 ( x 2) 2 x4 25 20 15 10 5 0 -2 -1 0 1 2 3 4 5 6 Page 6 Outline of Part 1 • Equality-Constrained Problems • Lagrange Multipliers • Economic Interpretation of Lagrange Multipliers • Kuhn-Tucker Conditions • Kuhn-Tucker Theorem Page 7 Outline of Part 1 • Equality-Constrained Problems • Lagrange Multipliers • Economic Interpretation of Lagrange Multipliers • Kuhn-Tucker Conditions • Kuhn-Tucker Theorem Page 8 Equality-Constrained Problems GOAL solving the problem as an unconstrained problem by explicitly eliminating K independent variables using the equality constraints Page 9 Example 5.1 Page 10 What if? Page 11 Outline of Part 1 • Equality-Constrained Problems • Lagrange Multipliers • Economic Interpretation of Lagrange Multipliers • Kuhn-Tucker Conditions • Kuhn-Tucker Theorem Page 12 Lagrange Multipliers Converting constrained problem to an unconstrained problem with help of certain unspecified parameters known as Lagrange Multipliers Page 13 Lagrange Multipliers Lagrange function Page 14 Lagrange Multipliers Lagrange multiplier Page 15 Example 5.2 Page 16 Page 17 Test whether the stationary point corresponds to a minimum positive definite Page 18 Page 19 Example 5.3 Page 20 Page 21 Page 22 max positive definite negative definite Page 23 Outline of Part 1 • Equality-Constrained Problems • Lagrange Multipliers • Economic Interpretation of Lagrange Multipliers • Kuhn-Tucker Conditions • Kuhn-Tucker Theorem Page 24 Economic Interpretation of Lagrange Multipliers The Lagrange multipliers have an important economic interpretation as shadow prices of the constraints, and their optimal values are very useful in sensitivity analysis. Page 25 Outline of Part 1 • Equality-Constrained Problems • Lagrange Multipliers • Economic Interpretation of Lagrange Multipliers • Kuhn-Tucker Conditions • Kuhn-Tucker Theorem Page 26 Kuhn-Tucker Conditions Page 27 NLP problem Page 28 Kuhn-Tucker conditions (aka Kuhn-Tucker Problem) Page 29 Example 5.4 Page 30 Example 5.4 Page 31 Example 5.4 Page 32 Outline of Part 1 • Equality-Constrained Problems • Lagrange Multipliers • Economic Interpretation of Lagrange Multipliers • Kuhn-Tucker Conditions • Kuhn-Tucker Theorem Page 33 Kuhn-Tucker Theorems 1. Kuhn – Tucker Necessity Theorem 2. Kuhn – Tucker Sufficient Theorem Page 34 Kuhn-Tucker Necessity Theorem Let • f, g, and h be differentiable functions • x* be a feasible solution to the NLP problem. • • and for k=1,….,K are linearly independent Page 35 Kuhn-Tucker Necessity Theorem Let h be differentiable functions f, g, andConstraint qualification x* be a feasible solution to the NLP problem. • and for k=1,….,K are linearly independent at the optimum ! is an to verify, since it requires that If x*Hardoptimal solution to the NLP problem, the exists a (u*, v*) such that (x*,u*, then there optimum solution be known v*) solves the KTP given by KTC.! beforehand Page 36 Kuhn-Tucker Necessity Theorem For certain special NLP problems, the constraint qualification is satisfied: 1. When all the inequality and equality constraints are linear 2. When all the inequality constraints are concave functions and equality constraints are linear ! When the constraint qualification is not met at the optimum, there may not exist a solution to the KTP Page 37 Example 5.5 x* = (1, 0) and for k=1,….,K are linearly independent at the optimum Page 38 Example 5.5 x* = (1, 0) No Kuhn-Tucker point at the optimum Page 39 Kuhn-Tucker Necessity Theorem Given a feasible point not that satisfies the constraint qualification optimal optimal If it does not satisfy the KTCs If it does satisfy the KTCs Page 40 Example 5.6 Page 41 Kuhn-Tucker Sufficiency Theorem Let • f(x) be convex • the inequality constraints gj(x) for j=1,…,J be all concave function •the equality constraints hk(x) for k=1,…,K be linear If there exists a solution (x*,u*,v*) that satisfies KTCs, then x* is an optimal solution Page 42 Example 5.4 • f(x) be convex • the inequality constraints gj(x) for j=1,…,J be all concave function •the equality constraints hk(x) for k=1,…,K be linear Page 43 Example 5.4 • f(x) be convex semi-definite Page 44 Example 5.4 • f(x) be convex v • the inequality constraints gj(x) for j=1,…,J be all concave function g1(x) linear, hence both convex and concave negative definite Page 45 Example 5.4 • f(x) be convex v • the inequality constraints gj(x) for j=1,…,J be all concave function • the equality constraints hk(x) for k=1,…,K be linear Page 46 Remarks For practical problems, the constraint qualification will generally hold. If the functions are differentiable, a Kuhn–Tucker point is a possible candidate for the optimum. Hence, many of the NLP methods attempt to converge to a Kuhn– Tucker point. Page 47 Remarks When the sufficiency conditions of Theorem 5.2 hold, a Kuhn–Tucker point automatically becomes the global minimum. Unfortunately, the sufficiency conditions are difficult to verify, and often practical problems may not possess these nice properties. Note that the presence of one nonlinear equality constraint is enough to violate the assumptions of Theorem 5.2 Page 48 Remarks The sufficiency conditions of Theorem 5.2 have been generalized further to nonconvex inequality constraints, nonconvex objectives, and nonlinear equality constraints. These use generalizations of convex functions such as quasi-convex and pseudoconvex functions Page 49