Introduction to Simulated Annealing - PowerPoint

Document Sample
Introduction to Simulated Annealing - PowerPoint Powered By Docstoc
					   Introduction to
Simulated Annealing
        gw
        Inspiration: Metallurgy
 Solid is heated to melting point
  – High-energy, high-entropy state
  – Removes defects/irregularities


 Temp is very slowly reduced
  – Recrystallization occurs (regular structure)
  – New internal state of diffused atoms
  – Fast cooling induces fragile structure
         Simulated Annealing
 Method of sampling search space
  – State: feasible solution
  – High energy: many solutions still possible
  – Cooling: perform local search
  – Ground state: optimal solution
Basic Algorithm
  Create initial solution



    Modify to create
     new solution


   Evaluate solutions


  Stochastically select
       a solution


  Reduce temperature
                     Acceptance Criteria
                                         1
                     P          eval( current)  eval( new)
                          1 e               T

As a function of Temp                  As a function of evaluation difference

  T         e-10/T         P             Δ(eval)       eΔ(eval)/10        P
 1010     0.99999         0.50             50          148.41           0.01
 100      0.90484         0.52             10            2.72           0.27
  10      0.36788         0.73               0           1.00           0.50
  1       0.00004         0.99             -10           0.37           0.73
                                           -50           0.01           0.99
         Cooling Schedule




 Fast            Slow

                 http://en.wikipedia.org/wiki/Simulated_annealing
     Acceptance Characteristics
 At high temperatures, more likely to accept
  “worse” solutions
 At low evaluation differences, more likely to accept
  “worse” solution
 Result  wide search of solution space

 Moves towards a region of the search space
  containing good solutions
 Moves towards low-energy regions
 Moves downhill (gradient descent)
                Parameters
 Initial solution: random
 New solution: select - Gaussian distribution
 Temperature schedule
  – Initial
  – Decay function: Ti+1 = cTi
  – Final
 #Iterations at temp:
  – Proportional to neighborhood size
                     Usage
 Search heuristic
  – Global optima in large solution space
 Best for discrete-valued solutions
  – Traveling Salesman
  – VLSI circuit design
             References / Resources
 Optimization by Simulated Annealing
   Kirkpatrick, S.; Gelatt, C. D.; and Vecchi, M. P.
   Science 220:671-680, 1983.

 Simulated annealing: a proof of convergence
   Granville, V., Krivanek, M., and Rasson, J.-P.
   IEEE Trans. Pattern Analysis and Machine Intelligence, 16:652-656, 1994.

    Simulated Annealing applet for Traveling Salesman problem:
    http://www.heatonresearch.com/articles/64/page1.html