# Cosine Simplex Method

Document Sample

```					                     Cosine Simplex Method
Federico Trigos, Earl Barnes, Juan Frausto,
Irma Hernández, Alma Nieto, Jenny Díaz,
Roberto López

INFORMS - Denver, Co - October 2004

The Goal

To generate an efficient
starting point, and a
simplex like algorithm, to
achieve the optimal
solution of a LP problem.

Large Scale
Optimization Group
ITESM

Federico Trigos / Denver, Co . October 2004                    2 de 32

1
Cosine Simplex in an example
min − x1 − 0.25x2
x2
st
-c-c
x1 ≤ 6        (1

2 x1 + x 2 ≤ 14        (2
3
x1 + 2 x2 ≤ 14        (3
Cosine
10 x1 − 2 x2 ≥ 10       (4                                                                                Vertex
x1 , x 2 ≥ 0                             4
2
Optimal
Smaller              FO= -6.5               Vertex
Angles
1

x1
Large Scale
Optimization Group    2 variables                                           4 basic variables
ITESM                                  4 constraints
4 slacks                                              2 active constraints
Federico Trigos / Denver, Co . October 2004                                   3 de 32

The LP Problem
T
min cT x
min c x
≤
st . bli ≤ Ax ≤ bui                                       st. AIi x   bIi i = 1,..., m '
≥
lj ≤ xj ≤ uj                                        AEi x ( = ) bEi i = 1,..., m − m '

A ∈ R m×n ' ; b ∈ R m ; c ∈ R n '                                          l j ≤ x j ≤ u j j = 1,K n

min cT x                     n           m’
Notation :
 x  A M ± I   x                                                         α1 
st. A   =  I        = b
 s   AE M 0   s                                                            
AI =  M  ; α i ∈ R1×n ;
α 
 m' 
Large Scale
Optimization Group                  l j ≤ x j ≤ u j j = 1,...n
ITESM

Federico Trigos / Denver, Co . October 2004                                   4 de 32

2
Cosine Vector
min cT x
st .                                                                                          Technical
α i x ≥ bi                             α iT c                                        Inequality
co s θ i =
αi c                                          Constraints

cj
xj ≥ 0                    co s θ j =                                               Non
c                              Negativity
(l j = 0)                                                                          Constraints

i = 1,..., m '                   j = 1,..., n

Large Scale
Optimization Group
ITESM

Federico Trigos / Denver, Co . October 2004                                       5 de 32

Building the basis
G = Cosθ1 , Cosθ 2 ,..., Cosθ i ,..., Cosθ m′ , Cosθ m′+1 , Cosθ m′+2 ..., Cosθ m '+ j ,..., Cosθ m′+n 
                                                                                                   

Pθ = [ n + 1, n + 2, ..., n + i, K , n + m′,                               1,      2,    ... ,     j , ... ,    n   ]

m’ inequalities                                                        n variables
G vector is sorted in non
decreasing order.

[G ] = Cosθ[1] ≤ Cosθ[2] ≤ ... ≤ Cosθ[ j ] ≤ ... ≤ Cosθ[m'+n] 
                                                       

[θ ] = θ[1]

≥ θ[2] ≥ ... ≥ θ[ j ] ≥ ... ≥ θ[m ' +n] 

Large Scale
Optimization Group
ITESM                                Pθ = [ Pθ[1] , Pθ[ 2] , ..., Pθ[ m′+n ] ]
Federico Trigos / Denver, Co . October 2004                                       6 de 32

3
Building the basis …
[θ ] = θ[1] ≥ θ[2] ≥ ... ≥ θ[m] ≥ ... ≥ θ[m'+n] 
                                         
Constraints associated with
Slack variables associated                                    Smallest angles are active
with these constraints are
at zero value

participate in the intersection
(Initial Vertex)
These slack variables are
non basic variables

Large Scale
Optimization Group
ITESM
Pθ[ j ] > n    Slack of technical                              Pθ[ j ] ≤ n   Variable at
Zero value
constraint = 0
Federico Trigos / Denver, Co . October 2004                               7 de 32

Building the basis …
[θ ] = θ[1] ≥ θ[2] ≥ ... ≥ θ[m] ≥ ... ≥ θ[m'+n] 
                                         
Number of passive constraints
Equals to Number of basic variables

Slack or technical variables associated
with these constraints ≥ zero

B0 =  APθ [1] , APθ [ 2] ,..., APθ [ m−1], APθ [ m ] 
                                                
Slack of technical
Pθ[ j ] > n     constraint has a value ::
Passive constraint
Large Scale
Optimization Group
ITESM
Pθ[ j ] ≤ n      The technical variable
has a value
Federico Trigos / Denver, Co . October 2004                               8 de 32

4
Is the basis full rank?
Dynamic column LU factorization
x x x                                 x 0 0        
0 x x                                              
B = P × L ×U              U =                                  L = x x 0        
0 0 x                                 x x x        
                                                   
0 0 0
                                 
     x x x
             

If:                                                                 Set of previous factors

x x x                          Replace the column by the
0 x x                          Next in Pθ vector
U =                  
0 0 0                          Until rank (B) = m
Large Scale                                            Or the problem is infeasible.
Optimization Group
ITESM
0 0 0
                  

Federico Trigos / Denver, Co . October 2004                           9 de 32

The first Simplex Tableau
c’ = c – CB       X B-1          xA               Z = CB x B-1 x b
A’ = B-1 x A                                          b’ = B-1 x b

Where:
CB is the vector of basic variables coefficients in the
objective function.
c´ is the vector of the objective function coefficients.
A´ is the matrix of technical constraints coefficients.
b´ is the right hand side of the constraints.
Large Scale
Optimization Group
ITESM
OF is the value of the objective function.
Federico Trigos / Denver, Co . October 2004                           10 de 32

5
Four cases
OPTIMAL

YES                                             NO

CASE I                                         CASE II

c ' optimal                                   c ' non optimal
YES                       and                                             and
lj ≤ xj ≤ uj                                      lj ≤ xj ≤ uj
FEASIBLE

Done!                                   Bounded Simplex

CASE III                                       CASE IV

c ' optimal                                   c ' non optimal
NO                        and                                          and
(some x j ≤ l j ; x j > u j )                     (some x j ≤ l j ; x j > u j )
Large Scale                                       Dual Simplex                               Modified Dual Simplex
MDS+MPPI
Optimization Group                                                                                   MDS+MPPI
ITESM

Federico Trigos / Denver, Co . October 2004                                        11 de 32

Modified Dual Simplex
(Case 4). Leaving variable
Leaving
(3)            Variable                                 Cosine
h3                                   Vertex
(4)
h1                                                        (1)

h2                                                                  (2)

Entering
Variables                                                   Optimal        -c
Vertex

 si 
     
Large Scale
Optimization Group                                                                               min  α i  ; for b'i < 0
ITESM
 x 
 i 
Federico Trigos / Denver, Co . October 2004                                        12 de 32

6
Modified Dual Simplex
(Case 4) Entering Variable

c' j < 0          c' j < 0         c' j > 0              c' j = 0                c' j > 0           c' j = 0      OF

a'rj < 0          a'rj ≥ 0min{cardinality(<+0)}
a'rj µ β              a'rj < 0                a'rj ≥ 0           a'rj ≥ 0     b'r < 0

α                β                 γ                      δ                       ε                  λ

min cT x
Non optimal                                                    Optimal
Large Scale
s.t. Ax = b
Optimization Group       Entering
ITESM                        min cardinality{α + β}
Variable                                                                                             x≥0
Federico Trigos / Denver, Co . October 2004                                  13 de 32

Big M Parametric Phase I (Case 4)
Given an infeasible Basic solution:
Given an infeasible Basic solution:

r’
c´ c1’ c2’ … cj’ … 0 … cn’                                 RHS

Xi´     a’i1   a’i2 … a’ij … 0 … a’in                       b´i              ≥0

xr’ a’r1 a’r2 … a’rj … 1 … a’rn                             b´r              <0

Replace the infeasible variables for Artificial ones:
Replace the infeasible variables for Artificial ones:
r’                                          Ar’
c’ c1’ c2’ … cj’ … 0 … cn’                                     M            RHS

a’i1 a’i2 … a’ij … 0 … a’in                                0           bi’ ≥ 0
Large Scale
Optimization Group      Ar’ - r1 -a’r2 … -a’rj … -1 … -a’rrn
a’                                                              1           -br’ > 0
ITESM

Federico Trigos / Denver, Co . October 2004                                  14 de 32

7
Big M Parametric Phase I
The previous tableau is equivalent to:
The previous tableau is equivalent to:
r’                                    RHS
c’     c1’       c2’ … cs’     … 0 … cn’
c’M    ∑ a' K 1
K ∈r
∑ a' K 2 ∑ a'Ks 1 ∑ a' Kn
K ∈r           K ∈r                    K ∈r

xí´       a’i1      a’i2 … a’is … 0 … a’in                              bi’ ≥ 0

xr’       a’r1      a’r2 … a’rs … 1 … a’rn                              br’ < 0

The entering column s:
The entering column s:
c 'Ms = Min j {c ' j + M ∑ a´kj }
k∈r
The leaving row ::
The leaving row
Large Scale
Optimization Group
Max p ∑ (b´ p −t a´sp ) −
ITESM

Federico Trigos / Denver, Co . October 2004              15 de 32

Numerical Experimentation

Large Scale
Optimization Group
ITESM

Federico Trigos / Denver, Co . October 2004              16 de 32

8
CSM Iteration Count
1 (Cosine Vertex and case definition)
If in Case 4
+Modified Dual Simplex
+Big M Parametric Phase I
+Cplex
Primal Solver (Case 2, Primal Feasible)
Dual Solver (Case 3, Dual Feasible)
Large Scale
Optimization Group
ITESM

Federico Trigos / Denver, Co . October 2004   17 de 32

Experimental Results
• Cosine Simplex
– Net Lib Set
http://www.netlib.org/lp/data
– Klee & Minty Cubes
• Cosine Network Simplex
– NETGEN Transshipment Problems
http://elib.zib.de/pub/Packages/mp-
testdata/mincost/netg/index.html
• [0,00] Flow
• [0,u] Flow
– Assignment Problems
Large Scale
Optimization Group
ITESM

Federico Trigos / Denver, Co . October 2004   18 de 32

9
Net Lib Set

m in c T x
s t . b li ≤ A x ≤ b u i
lj ≤ xj ≤ uj

A ∈ R m×n '; b ∈ R m ; c ∈ R n '
Large Scale
Optimization Group
ITESM

Federico Trigos / Denver, Co . October 2004                                            19 de 32

Net Lib Set                                                            Net Lib Set

Cplex                                                               Cplex
Problem Name   m          n                    CSM      Savings        Problem Name     m        n                  CSM    Savings
(primal)                                                            (primal)
ADLITTLE           57          97       104       57      45.19%       SC205            206       203         79      58     26.58%
AFIRO              28          32        15        7      53.33%       SC50A             51        48         16       9     43.75%
AGG            489         163           68       85     -25.00%       SC50B             51        48         13       1     92.31%
AGG2           517         302          140      130       7.14%       SCAGR7           130       140         74     253   -241.89%
BANDM           306           472       257      333     -29.57%       SCFXM1           331       457        254     309    -21.65%
BEACONFD        174           262          4       1      75.00%       SCFXM2           661       914        573     428     25.31%
BOEING1        351         384          519      547      -5.39%
SCFXM3            991     1371        927    1091    -17.69%
BOEING2        167         143          177      177       0.00%
SCORPION          389      358          57     56      1.75%
BORE3D          234        315          22        36     -63.64%
SCRS8             491     1169         470    418     11.06%
BRANDY          221        249         198       153      22.73%
SCSD1              78      760         255    145     43.14%
CAPRI           272        353         252       349     -38.49%
SCSD6             148     1350         504    314     37.70%
CZPROB          930       3523         925      1391     -50.38%
SCSD8             398     2750        1559   1780    -14.18%
D6CUBE          416       6184       34208      6467      81.10%
SCTAP1            301      480         199    187      6.03%
DEGEN2         445         534        1502      1141      24.03%
SCTAP2           1091     1880         585    691    -18.12%
DEGEN3         1504       1818        1502      1137      24.30%
SCTAP3           1481     2480         906    899      0.77%
E226           224         282         394       321      18.53%
SEBA              516     1028           6      3     50.00%
ETAMACRO        401        688         596       956     -60.40%
SHARE1B           118      225         147    357   -142.86%
FFFFF800        525        854         442       412       6.79%
FINNIS          298        614         352       512     -45.45%
SHARE2B            97       79         124    118      4.84%
FIT1D            25       1026        1162       159      86.32%       SHELL(2)          537     1775         347    188     45.82%
FIT1P           628       1677         796       640      19.60%       SHIP04L           403     2118         228    198     13.16%
FIT2D            26      10500       13334       153      98.85%       SHIP04S           403     1458         163    120     26.38%
FIT2P          3001      13525       18929      5535      70.76%       SHIP08L           779     4283         542    456     15.87%
GANGES         1310       1681         415       798     -92.29%       SHIP08S           779     2387         325    220     32.31%
GROW22          441        946        1186      1008      15.01%       SHIP12L          1152     5427         584    559      4.28%
GROW7           141        301         295       397     -34.58%       SHIP12S(1)       1152     2763         259    167     35.52%
ISRAEL         175         142         170       220     -29.41%       STAIR             357      467         315    168     46.67%
KB2              44         41           39       46     -17.95%       STANDATA          360     1075          27     48    -77.78%
LOTFI           154        308           99       47      52.53%       STANDGUB          362     1184          27     48    -77.78%
NESM            663       2923         3814     2930      23.18%       STANDMPS          468     1075         131    107     18.32%
PEROLD          626       1376         3011     1420      52.84%       STOCFOR1          118      111          31     39    -25.81%
Large Scale       PILOT4          411       1000          945      688      27.20%       STOCFOR2         2158     2031        1015   1430    -40.89%
Optimization Group   PILOTNOV        976       2172         3013     2552      15.30%       TUFF              334      587         292    223     23.63%
ITESM
RECIPE           92        180           20       14      30.00%       VTP-BASE          199      203          29     19     34.48%
SC105           106        103           34       24      29.41%       WOOD1P            245     2594         369    974   -163.96%
Federico Trigos / Denver, Co . October 2004                                            20 de 32

10
Net Lib Summary

Cplex        Cosine Simplex
Iterations          100,371          42,924
Savings                              57.23%
Savings (G.M.)                       21.27%

Large Scale
Optimization Group
ITESM

Federico Trigos / Denver, Co . October 2004   21 de 32

Klee-Minty Cube

Large Scale
Optimization Group
ITESM

Federico Trigos / Denver, Co . October 2004   22 de 32

11
Klee & Minty Cubes
n        Simplex                 CSM                                Savings
1                               1                               1             0.00%
2                               3                               2            33.33%
3                               7                               3            57.14%
4                              15                               4            73.33%
5                              31                               5            83.87%
6                              63                               6            90.48%
7                           127                                 7            94.49%
8                           255                                 8            96.86%
9                           511                                 9            98.24%
10                       1,023                              10               99.02%
11                       2,047                              11               99.46%
12                       4,095                              12               99.71%
13                       8,191                              15               99.82%
14                     16,383                               14               99.91%
15                     32,767                               17               99.95%
16                     65,535                               17               99.97%
17                   131,071                                25               99.98%
18                   262,143                                28               99.99%
Large Scale
19                   524,287                                29               99.99%
Optimization Group
ITESM
Totals            1,048,555                                 223              99.98%
85.55%
Federico Trigos / Denver, Co . October 2004                             23 de 32

Cosine Network Simplex

The Transshipment Problem
-9                                                                 4
3
1                                                             2
5                            1
1                                                  4
17 3                                                       1
1           1
6                                        4
1                                        1
1
5                                                              6
-5                                                                 -8
Large Scale
Optimization Group
ITESM

Federico Trigos / Denver, Co . October 2004                             24 de 32

12
The LP Transhipment Problem

-9                                               4
3
1                                           2
5                  1
1                                   4
17 3                                 1
1         1
Large Scale                                                                                        4
6
Optimization Group                                                              1                         1
ITESM                                                        5                 1
6
-5                                           -8

Federico Trigos / Denver, Co . October 2004                                    25 de 32

NETGEN Non Negative Flow Transhipment Library

Name    Nodes    Arcs      CNS            CPLEX(netopt)             Savings
stndrd1         200    1308          232                 461                49.67%
stndrd2         200    1511          238                 481                50.52%
stndrd3         200    2000          265                 597                55.61%
stndrd4         200    2200          250                 647                61.36%
stndrd5         200    2900          282                 718                60.72%
stndrd6         300    3174          393                 935                57.97%
stndrd7         300    4519          379                1253                69.75%
stndrd8         300    5168          370                1540                75.97%
stndrd9         300    6075          373                1655                77.46%
stndrd10        300    6320          456                1572                70.99%
stndrd50        350    4500          465                1488                68.75%
stndrd51        350    4508          363                1275                71.53%
stndrd52        350    6000          323                2649                87.81%
stndrd53        350    9000          347                4447                92.20%
netgen11        400    1500          344                 958                64.09%
netgen12        400    2250          345                1068                67.70%
netgen13        400    3000          352                1500                76.53%
netgen14        400    3750          354                1736                79.61%
netgen15        400    4500          351                2070                83.04%
transp9         400   10000          517                1560                66.86%
transp11        600   10020          864                2256                61.70%
transp12        600   20000          912                3573                74.48%
transp13        600   30000          956                4651                79.45%
transp14        600   40000          996                5723                82.60%
transp1         800   10028         1102                2756                60.01%
transp2
transp3
800
800
20000
30000
1125
1227
3929
5750
71.37%
78.66%         61.12% Savings
61.12% Savings
transp4         800   40002         1228                6945                82.32%
stndrd28       1000    2900         1141                1754                34.95%
stndrd29
stndrd30
1000
1000
3400
4400
1374
1419
1639
2513
16.17%
43.53%
Flows [0.00]
Flows [0.00]
stndrd31       1000    4800         1501                2382                36.99%
transp5        1000   20049         1441                4873                70.43%
transp6        1000   30049         1578                6398                75.34%
transp7        1000   40025         1495                8499                82.41%
transp8        1000   50055         1619                9566                83.08%
stndrd54       1400   12084         2153                4207                48.82%
stndrd32       1500    4342         1937                3006                35.56%
stndrd33       1500    4385         1801                3190                43.54%
stndrd34       1500    5107         2079                3450                39.74%
stndrd35       1500    5730         1893                4173                54.64%
stndrd41       2000   10000         2044                6993                70.77%
stndrd48       3000   15441         3726                7951                53.14%
stndrd38       3000   35000         8380               19306                56.59%
p117           5000   12816         5532               10005                44.71%
stndrd37       5000   23000        12034               22778                47.17%
p116           5000   25304         7081               14062                49.64%
p118           5000   37797         7931               18218                56.47%
Large Scale         p119           5000   50301         8496               21929                61.26%
stndrd36       8000   15000         9114               19430                53.09%
Optimization Group     stndrd47       8000   35539        11173               23360                52.17%
ITESM             stndrd42      10000   30000        10226               31387                67.42%
Totals                            122577              315262                61.12%
62.62%

Federico Trigos / Denver, Co . October 2004                                    26 de 32

13
NETGEN Upper Bounded Flow Transhipment Library

Name      Nodes     Arcs            CNS            Cplex(netopt)      Savings
stndrd16           400     1306                468                   820      42.93%
stndrd18           400     1306                547                   809      32.39%
stndrd24           400     1382                690                   851      18.92%
stndrd26           400     1382                482                   561      14.08%
stndrd20           400     1400                409                   872      53.10%
stndrd22           400     1416                444                   867      48.79%
stndrd17           400     2443                585                  1017      42.48%
stndrd19           400     2443                529                   996      46.89%
stndrd25           400     2676                894                  1295      30.97%
stndrd27           400     2676                541                   850      36.35%
stndrd21           400     2836                578                  1057      45.32%
stndrd23           400     2836                483                  1017      52.51%   49.45% Savings
cap1              1000    10000               2293                  3005      23.69%    49.45% Savings
cap2              1000    30000               1178                  4677      74.81%
cap3              1000    40000               1396                  6094      77.09%     Flows [0.u]
Flows [0.u]
stndrd40          3000    23000               7212                 14057      48.69%
stndrd39          5000    15000               8624                 18204      52.63%
cap4              5000    30000              11902                 12970       8.23%
cap5              5000    40000              13752                 17967      23.46%
big7              5000    40105              13814                 42573      67.55%
cap7              5000    60000              13618                 25121      45.79%
big6              5000    60092              15332                 52817      70.97%
big5              5000    80101              15098                 59291      74.54%
stndrd46          8000    35561              11726                 23866      50.87%
cap8             10000    40000              27897                 29596       5.74%
Large Scale
Optimization Group   cap9             10000    50000              26086                 28083       7.11%
ITESM           Totals                                      176578                349333      49.45%
42.15%
Federico Trigos / Denver, Co . October 2004                          27 de 32

Assignment Problem

Large Scale
Optimization Group
ITESM

Federico Trigos / Denver, Co . October 2004                          28 de 32

14
Random Assignment Problems

Name   Nodes      Arcs        CNS        CPLEX(netopt)        Savings
asg_25         50       625            43                186          76.88%
asg_50        100      2500            65                489          86.71%
asg_75        150      5625           114               1555          92.67%
asg_100       200     10000           162               1562          89.63%
asg_125       250     15325           191               2925          93.47%
asg_150       300     22500           265               4175          93.65%
asg_175       350     30625           270               7512          96.41%
asg_200       400     40000           347               9444          96.33%
asg_225       450     50625           401              12387          96.76%
asg_250       500     62500           460              15864          97.10%
asg_275       550     75625           488              22095          97.79%
asg_300       600     90000           505              24513          97.94%
asg_325       625    105625           586              14208          95.88%
asg_350       700    122500           637              16845          96.22%
asg_375       750    140625           664              21841          96.96%
asg_400       800    160000           724              29124          97.51%
asg_425       850    180625           796              27122          97.07%
asg_450       900    202500           884              35819          97.53%
asg_475       950    225625           978              36618          97.33%
asg_500
asg_525
1000
1050
250000
275625
1018
922
43962
45754
97.68%
97.98%
98.41% Savings
98.41% Savings
asg_550      1100    302500          1044              51124          97.96%
asg_575      1150    330625          1144              54709          97.91%
asg_600      1200    360000          1036              65223          98.41%
asg_625      1250    390625          1175              73051          98.39%
asg_650      1300    422500          1369              70866          98.07%
asg_675      1350    455625          1354              82103          98.35%
asg_700      1400    490000          1339              73927          98.19%
asg_725      1450    525625          1446              88543          98.37%
asg_750      1500    562500          1511              94000          98.39%
asg_775      1550    600625          1424             117279          98.79%
asg_800      1600    640000          1585             111401          98.58%
asg_825      1650    680625          1761             125074          98.59%
asg_850      1700    722500          1689             124944          98.65%
asg_875      1750    765625          1774             151301          98.83%
asg_900      1800    810000          1889             133832          98.59%
asg_925      1850    855625          1857             162904          98.86%
Large Scale          asg_950      1900    902500          1928             175284          98.90%
Optimization Group      asg_975      1950    950625          2102             184895          98.86%
ITESM              asg_999      1998    998001          2008             195425          98.97%
Totals                              39955            2509885          98.41%
96.53%
Federico Trigos / Denver, Co . October 2004                          29 de 32

Remarks:

Theorem : If the Problem is non
redundant there is a feasible
Cosine Vertex

Large Scale
Optimization Group
ITESM

Federico Trigos / Denver, Co . October 2004                          30 de 32

15
Final Remarks
• The computational cost of an iteration:
– CSM Vs. Simplex
– CNS Vs. Network Simplex
• We must pay some attention:
– To get well condition basis
– Robust LU factorization
– Degeneracy (Bland´s Rule)
– Sparse Matrix Technology
Large Scale
Optimization Group
ITESM

Federico Trigos / Denver, Co . October 2004              31 de 32

Large Scale Optimization Group
at ITESM

Questions…?
http://dii.tol.itesm.mx
ftrigos@itesm.mx

Large Scale
Optimization Group
ITESM

Federico Trigos / Denver, Co . October 2004              32 de 32

16

```
DOCUMENT INFO
Shared By:
Categories:
Stats:
 views: 103 posted: 12/30/2009 language: English pages: 16
How are you planning on using Docstoc?