# Heuristic Optimization Methods: Simulated Annealing

## Comments

## Transcription

Heuristic Optimization Methods: Simulated Annealing

Zavod za telekomunikacije Sveučilište u Zagrebu, Hrvatska Department of Telecommunications University of Zagreb, Croatia Heuristic Optimization Methods: Simulated Annealing Nina Skorin-Kapov Academic year 2014/2015 Masters program Information and communication technologies Outline Simmulated Annealing concept Main idea Role of temperature parameter Pseudocode Main design issue: Cooling schedule Initial Temperature Schedule type ((in)homogeneous) Decrement function Termination criterion (Final temperature) SA design elements Example Advanced techniques Zavod za telekomunikacije 2 Optimization methods Exact methods A* Approximate methods Constraint programming Simplexbased for LP Branch and Bound, Cut , Price (B&B, B&C, B&P) Approximation algorithms Heuristic algorithms Dynamic programming Improvement (meta)heuristics Constructive heuristics • Local search • Greedy algorithms •Nature inspired Hybrid methods • GRASP •Problem specific heuristics Zavod za telekomunikacije • Tabu search, •Genetic algorithms, •Simmulated Annealing, •Ant colony 3 Simmulated Annealing (SA) Iterative, single-solution based metaheuristic Developed in the 1980‟s through the works of S. Kirkpatrick and V. Cerny Applies principles of statistical mechanics Considers the annealing process where a metal is heated and then slowly cooled to obtain a strong crystalline structure the strength depends on the cooling rate; imperfections appear if the initial temperature is too high or the cooling rate is too fast SA is based on the Monte Carlo model which simulates the energy changes in the system during the cooling process until it converges to an equilibrium state developed by Metropolis et al. in 1953 Zavod za telekomunikacije 4 Analogy Physical system Optimization problem System state Solution Molecular positions Decision variables Energy Objective function Ground state Global optimum Metastable state Local optimum Rapid quenching Local search Temperature Control parameter T Careful annealing Simmulated annealing From: El-Ghazali Talbi, “Metaheuristics: From Design to Implementation”, Wiley 2009 Zavod za telekomunikacije 5 Main idea Main idea: stochastic search the search can accept non-improving solutions and moves with a certain probability and in that way escape from local optima and delay convergence Under certain conditions allows solution degradation Memoryless (only records incumbent solution) In contrast to LS and TS, SA does not generate an entire neighborhood but creates one neighbor at random and evaluates it Zavod za telekomunikacije 6 Neighbor selection – most common method In each iteration, a random neighbor is generated: If it improves the current solution accept as the new current solution Improving moves always selected If it does not improve the current solution accept with a certain probability which depends on: Control parameter T (temperature) Their relative fitness: Δf = |f(current solution)-f(neighbor)| Usually follows a Boltzman distribution for non-improving neighbors. Zavod za telekomunikacije 1, for improving neighbor P(T , f ) f T e , for non - improving neighbor 7 Move acceptance The search escapes local optima by probabilistic acceptance of non-improving moves P, proportional to control parameter T At high temperatures the probability of accepting non-improving moves is high At low temperatures the probability of accepting non-improving moves is low T=∞, all moves are accepted; random local search T=0, no non-improving moves are accepted; local search (with first improving neighbor selection) P, inversley proportional to the relative fitness Δf=|f(current solution)-f(neighbor)| The worse the neighbor, the smaller the selection probability Law of thermodynamics: at temperature T, the probability of an increase in energy of magnitude ΔE is: (k is Boltzmann’s constant) Zavod za telekomunikacije P(T , E ) f e kT 8 Alternative method Accept improving neighbors also with a certain probability Δf is not absolute value but: For maximization: Δf = f(current solution)-f(neighbor) For minimization: Δf = f(neighbor)-f(current solution) Improving neighbors give negative Δf while nonimproving neighbor give positive Δf Acceptance probability: P(T , f ) Zavod za telekomunikacije 1 f 1 e T 9 Solution quality influence Example: Consider a maximization problem The current solution x has a fitness f(x)=100 The current temperature T is T=10 2 selection strategies: 1, improving neighbor P1 f T e , non - improving neighbor P2 1 f 1 e T f(x’) Δf P1 P2 10 50 90 100 110 150 200 90 50 10 0 -10 -50 -100 0.0001234098 0.0067379470 0.3678794412 1 1 1 1 0.0001233946 0.0066928509 0.2689414214 0.5000000000 0.7310585786 0.9933071491 0.9999546021 Zavod za telekomunikacije 10 Control parameter T Control parameter T determines the probability of accepting non-improving neighbors For larger values of T, the relative fitness of the current solution and neighbor (Δf) is not as influential acceptance if a non-improving neighbor does not depend so much on its quality but on chance, random Local Search (choose random neighbor) If T is very small, the stochasticity of the search decreases, i..e for e.g. T=0, behaves like first improving Local Search (never accept a nonimproving neighbor) T is analogous to the temperature in the cooling process Zavod za telekomunikacije 11 Dependency of selection probability on T Example: Consider a minimization problem The current solution x has a fitness f(x)=13 The neighbor generated in the next iteration x‟ has a fitness of f(x‟)=23 P(T , f ) f e T ( 2313) e T Temperature T P(T, 10)=e(-10/T) 1010 0.9999999990 100 0.9048374180 20 0.6065306597 10 0.3678794412 5 0.1353352832 1 0.0000453999 Zavod za telekomunikacije 10 eT 12 Role of parameter T T is decreased during the search (according to a cooling schedule)i.e. as the search progresses, the probability of accepting non-improving moves decreases Motivation: at the beginning, the search is almost random (diversify!) and never stays too long in a local neighborhood (around a local optimum). Eventually, when we are in a good area, possibly near a/the global optimum, converge towards local search (intensify!) There is a guarantee of asymptotic convergence to the global optimum for infinite iterations Zavod za telekomunikacije 13 Pseudocode (Min; nbr. selection method 1) Start Simmulated Annealing Generate initial solution s ; 0 Select initial temperatu re T ; 0 s s0 ; s best s0 ; T T0 while (! terminatio n criterion) Select a neighbor s' N(s) if ( f ( s' ) f ( s )) then s s' f elseif [( f ( s' ) f ( s )) and ( random[0,1) e T )] then s s' endif if ( f ( s ) f ( s best )) then s best s Update T , endwhile return s best ; End Zavod za telekomunikacije 14 Main design issue: Cooling schedule Cooling schedule Initial Temperature Schedule type ((in)homogeneous) Decrement function Termination Zavod za telekomunikacije criterion (Final temperature) 15 Initial Temperature Ideally, the search should start so that many, but not all, non-improving neighbors are accepted (usually around 40-60%), First improving Local search T0 Initial temperature T0 Random Local search T∞ 3 main strategies for setting the Initial Temperature: Set T0 high enough to accept all neighbors during initial phase of search; -: high computational cost; +:easiest. Determined experimentally based on the deviation of the objective function of obtained solutions; several runs Initialize T0 to achieve a predetermined acceptance rate e.g. 40-60% Zavod za telekomunikacije 16 Cooling schedule types 2 basic types with analogies to (in)homogeneous Markov chains Homogeneous cooling schedule: T is decreased in each iteration (move) by a very small amount according to the decrement function Simple; most commonly used in practice Inhomogeneous cooling schedule: At each level of temperature T, multiple solutions are (can be) explored until an equilibrium is reached, after which the temperature is reduced according to the decrement function Zavod za telekomunikacije 17 Reaching equilibrium state For inhomogeneous cooling schedules, reaching the equilibrium state must be defined Usually a sufficient number of moves is defined according to the size of the problem instance and neighborhood size Static strategy: the number of moves to equilibrium is predetermined (a given proportion of the neighborhood) Adaptive strategy: the number of moves depends on the characteristics of the search E.g. it may not be necessary to reach equilibrium at all temperatures; equilibrium may be when we reach an improving move Zavod za telekomunikacije 18 Decrement function Trade-off: between solution quality and cooling speed (faster cooling usually gives poor solutions but less computational time) β is a specified constant T T Linear: Geometric: T T , [0,1] Very slow decrease: T T , 1 T Logarithmic: Zavod za telekomunikacije Titer T0 log(iter ) Most commonly used. Usually, α is between 0.5 and 0.99 β is a constant near 0 Too slow for practical purposes, but has a convergence proof 19 Termination criteria (final temperature) Reaching a final temperature Tf (most common) In theory, the search should continue until Tf=0, but in practice it is sufficient to terminate the search when the acceptance probability is negligible, e.g. Tf =0.01 Note: Tf, T0, and the decrement function determine the number of iterations Inhomogeneous case: depends how equlibrium Homogeneous case: Geometric decrement: Very slow decrease: N it is reached log T f log T0 N it log T0 T f T0T f Alternative termination criterion: The number of iterations without improvement of the incumbent solution Zavod za telekomunikacije 20 SA design elements Solution representation (e.g. binary vector) Initial solution (e.g. random, greedy) Objective function Fitness function Neighborhood definition („move‟ operator) Termination criterion (final temperature or other) Initial temperature Decrement function (in)homogeneous cooling Equilibrium Zavod za telekomunikacije state for inhomogeneous 21 Example Objective and fitness function: 3 2 max f ( x) x 60 x 900 x 50 x [0,31], integer Representation: x in binary form with 5 bits Initial solution: 1 0 0 1 1 f(s)=2349 Neighborhood (move operator): “1-flip” Zavod za telekomunikacije 22 Example: Cooling schedule Initial temperature: T0=100 Homogeneous cooling: reduce T after each iteration Termination criterion: Final temperature Tf = 0.01 (NOTE: we will show only the first few iterations) Decrement function: geometric T T , 0.9 Zavod za telekomunikacije 23 Example: Start SA: Current solution s 1 0 0 1 1 f(s)=2349 Incumbent solution sbest 1 0 0 1 1 f(sbest)=2349 Current temperature T Zavod za telekomunikacije T=100 24 Example: T0=100 , the first 6 iterations Current temp T Nbr. x’ flipped f(x’) bit (move) Δf (or 0) P(T, Δf)= e(-Δf/T) or 1 Accept move? New x f(xbest) 100 00011 1 2237 112 0,3262797946 NO 10011 2349 90 10111 3 1177 1172 0,0000022107 NO 10011 2349 81 10010 5 2642 YES 10010 2642 72.9 11010 2 466 10010 2642 65.61 1000 4 3186 10000 3186 59.0 10100 3 2050 10000 3186 Zavod za telekomunikacije 0 1,0000000000 2176 1,08817E-13 0 1,0000000000 1136 0,0000000043 NO YES NO 25 Example: T0=100 After 6 iterations: 1 0 0 0 0 f(sbest)=3186 The initial temperature T0=100 seems too small nonimproving moves are very unlikely to be chosen (the acceptance probability for non-improving moves is very small, none have been chosen) Incumbent solution sbest It seems that with this temperature, for many more iterations we would be stuck in this local optimum s=10000, f(s)=3186 Note: This example has an objective function with 3 local optima, with a global optimum of 4050 for solution x=10 (binary: 010101) Let‟s try with T0=500 Zavod za telekomunikacije 26 Example: T0=500 Current temp T Nbr. x’ flipped f(x’) bit (move) Δf P(T, Δf)= e(-Δf/T) or 1 Accept move? New x f(xbest) 500 00011 1 2237 112 0,7993151344 YES 00011 2349 450 00111 3 3753 0 1,0000000000 YES 00111 3753 405 00110 5 3506 247 0,5434179536 YES 00110 3753 364,5 01110 2 3634 0 1,0000000000 YES 01110 3753 328 01100 4 3938 0 1,0000000000 YES 01100 3938 295,2 01000 3 3922 16 0,9472421256 YES 01000 3938 265,7 01010 4 4050 0 1,0000000000 YES 01010 4050 239,1 01011 5 4021 29 0,8857786733 YES 01011 4050 215,2 11011 1 293 3728 0,0000000300 NO 01011 4050 Zavod za telekomunikacije 27 Example: T0=500 After 9 iterations: Current solution s Incumbent solution sbest Current temperature T 0 1 0 1 1 f(s)=4050 0 1 0 1 1 f(sbest)=4050 T=215.2 Better! The initial temperature T0=500 was high enough to leave the local optimum However, if the search continued now for another cca 90 iterations (the search continues until T reaches Tf), all the visited solutions would probably be near the found optimum (01011) or in its local neighborhood Zavod za telekomunikacije 28 Advanced techniques Re-heating: The search records the temperature at which the incumbent solution was obtained Repeat the search from the recorded temperature until the final temperature is reached Repeat multiple times (like a multi-start algorithm). Since the search is stochastic, its trajectory in the search space is different each time Additional advanced techniques: Restricted neighborhoods Some moves are so unlikely to be accepted that they are ignored to save computational time Order of searching Sampling the neighborhood in a systematic manner (instead of randomly generating a neighbor in each iteration) Zavod za telekomunikacije 29 TS vs. SA Simmulated Annealing Tabu search Mechanism to escape local optima: nonimproving move acceptance ratio Mechanism to escape local optima: accept non-improving best neighbor, no backtracking, tabu list Intesifies the search around „good‟ solutions The search always moves forward (except when specifically applying the quality/influence dimensions) Move selection depends on the quality of the neighbor (and T) Move selection depends on the neighborhood and history of the search Can accept a non-improving move at any time Accepts non-improving moves ONLY when the search is in a local optimum Stochastic Deterministic Iterative single-solution local searchbased with additional parameters: Temperature , decrement function,… Iterative single-solution local searchbased with additional parameters: Tabu list, other memory structures,… Zavod za telekomunikacije 30