Optimization Hill Climbing Simulated Annealing

1. Informed Search

2. Hill Climbing
• Back – Tracking
• Use Enqued List
• Informed Search
• Very similar to DFS except instead of using
lexical order to break ties, it break ties
according to which node is closer to the goal.
• Hill Climbing: Front Sorted

3. Hill Climbing
• It starts with random (potentially poor)
solution, and iteratively makes small changes
to the solution, each time improving it a little.
When the algorithm cannot see any
improvement anymore, it terminates.
• Ideally, at that point the current solution is
close to optimal, but it is not guaranteed that
hill climbing will ever come close to the
optimal solution.

4. Hill Climbing
• Hill climbing can be applied to the travelling
salesman problem.
• It is easy to find a solution that visits all the
cities but will be very poor compared to the
optimal solution.

5. Hill Climbing
The hill climbing can be described as follows:
1. Start with current-state = initial state
2. Until current-state = goal-state OR there is no
change in current-state do:
– Get the successors of the current state and use the
evaluating function to assign a score to each
successor.
– If one of the successors has a better score than the
current-state then set the new current-state to be the
successor with the best score.
• Hill climbing terminates when there are no
successors of the current state which are better
than the current state itself.

6. Hill Climbing
• Hill climbing is depth-first search with a
heuristic measurement that orders choices as
nodes are expanded. It always selects the
most promising successor of the node last
expanded.

7. Hill Climbing
• For instance, consider that the most
promising successor of a node is the one
that has the shortest straight-line
distance to the goal node G.
• In figure below, the straight line
distances between each city and goal G
is indicated in square brackets, i.e. the
heuristic.

8. Hill Climbing
• Problems with Hill Climbing
– Gets stuck at local maxima when we reach a
position where there are no better neighbors, it is
not a guarantee that we have found the best
solution. Ridge is a sequence of local maxima.
– Another type of problem we may find with hill
climbing searches is finding a plateau. This is an
area where the search space is flat so that all
neighbors returns the same evaluation.

9. Hill Climbing
S
A
A
G
E
FB
88.5
8.5 6
6 3
D
The hill climbing search from S to G proceeds as
follows:
S
A
D
B
E
C
F
G
3
4 4
5 5
42
4
3
[8]
[8.5] [6]
[6] [3]
[3]
[10]

10. Hill Climbing
• Apply the hill climbing algorithm to find a path
from S to G, considering that the most promising
successor of a node is its closest neighbor.
S
A
D
B
E
C
F
G
3
4 4
5 5
42
4
3

11. Hill Climbing
•the best first search method selects for expansion the
most promising leaf node of the current search tree
•the hill climbing search method selects for expansion the most
promising successor of the node last expanded.
• In depth-first search: front of the queue (a stack).
• In breadth-first search: back of the queue.
• Now, in hill-climbing search, you sort[1] the current node's
children before adding them to the queue.
• In best-first search, you add the current node's children to
the queue in any order, then sort[1] the entire queue.
• [1]: sort according to some problem-specific evaluation of the
solution node, for example "distance from destination" in a
path-finding search

12. Hill Climbing
AI, Subash Chandra Pakhrin 12
S
G
D
E
B
A
C
5
4
4
6
3
5
7+
6
7+
S
A B
G
A C
D
7+ 6
5
7+
7+ 3
A and C are both equally far from goal, so now we are going to use
lexical order to break the tie

13. Beam Search
S
A B
B D
C G
A C
Beam width (w) = 2
Beam: Keep w Best

14. Annealing
• In metallurgy, annealing is the process used to
temper or harden metals and glass by heating
them to a high temperature and then
gradually cooling them, thus allowing material
to reach a low energy crystalline state.
• This can be applied to computer science
problems.

15. Simulated Annealing
• It is applied to solve problems like
– Travelling salesman problem
– Designing printed circuit boards
– Solving VLSI layout problems
– Factory scheduling
– Large-scale optimization task
– Planning of path for a robot
– Bioinformatics to design three dimension
structures of protein molecules.

16. Simulated Annealing
• A hill climbing algorithm that never makes “down
hill” moves towards states with lower value is
guaranteed to be incomplete, because it can get
stuck on a local maximum.
• In contrast, a purely random walk – that is,
moving to a successor chosen uniformly at
random from the set of successors- is complete
but extremely inefficient.
• Simulated Annealing = hill climbing + random
walk hence, it is efficient as well as completeness.

17. Simulated Annealing
• At the beginning you don’t care if you are
actually moving towards the good solution
and you accept bad moves as well, you accept
bad configurations as well but as you progress
towards the solution we become more careful
and we try to get closer to the solution by
selecting only the good moves.

18. Simulated Annealing
• Switch the perspective from hill climbing to gradient
descent.
• Imagine the task of getting a ping-pong ball into he
deepest crevice in a bumpy surface.
• If we just let the ball roll, it will get come to rest at a
local minimum.
• If we shake the surface, we can bounce the ball out of
local minimum.
• The trick is to shake just hard enough to bounce the
ball out of local minimum but not hard enough to
dislodge it from the global minimum.
• Simulated Annealing solution is to start by shaking hard
(i.e., at a high temperature) and then gradually reduce
the intensity of shaking (i.e., lower the temperature)

19. Simulated Annealing
• E = Energy of the system
• ΔE = Change in energy
• We need to have a mechanism to alter the
configuration
All possible configurations of a given system
E
n
e
r
g
y
C N
Move

20. Simulated Annealing
C = C init
for T = T max to T min
E c = E(C)
N = next (C)
EN = E(N)
ΔE = EN – EC
If (ΔE > 0)
C = N
else if (eΔE/T > rand (0, 1))
C = N
epoch

21. Simulated Annealing
• If the change in energy is negative, or in other
words if we are making bad move.
– In this case we compute probability
• If probability is very high then we accept the move
even if it is a bad move.
• If probability is very low then we have low probability
to accept the bad move.
• Probability depends upon two variables
change in energy ΔE and the temperature
factor T.

22. Simulated Annealing
• When temperature is very high then
probability for accepting bad move is very
high. In other words, at high temperature we
are exploring the solution space or we are
exploiting the configuration and we’re
accepting bad moves as well.
• When the temperature is low this probability
becomes very low and we have very low
probability to accept bad moves.

23. Simulated Annealing
• Case 1: T = 1000, ΔE = 10
ΔE/T = 0.01
eΔ E/T = 0.99004
A number that is close to one has a very high
(correlation) probability to become greater than a
random number between 0 and 1, which means
we get a very high probability, so when
temperature is very high we have high probability
to accept bad moves as well.

24. Simulated Annealing
ΔE influences probability
• ΔE is very high = a low probability to accept the
move
• ΔE is small = high probability to accept the move.
• We repeat the process for certain number of
times usually known as the number of epochs,
usually 100 to 200 times and for every value of
temperature we repeat the process and finally
expecting the solution to converge towards the
global minimum.

25. Simulated Annealing
• Now on either extremes are 2 other algorithms
1. Hill Climbing
2. Random Walk
• If we remove the probability factor or the
temperature factor and always accept the good
moves only then that’s hill climbing or it’s like a
greedy algorithm which always go towards a
better solution. Such, algorithms are prone to be
easily stuck in local minima.

26. Simulated Annealing
• Random walk: It doesn’t care about how good
a move we are making every time but we just
explore, continue to explore, the space. Such
algorithm never converge and will probably
never give you the best optimal solution.

27. Simulated Annealing
• Parameters T max to T min and number of epochs
are dependent on the problem that we are
solving, usually we start with high temperature
like few thousands, let’s say 5000 or 3000 and
then the minimum temperature is set to a small
value like 0 or 10 or something like that.
• Number of epoch is usually a hundred, or 200
depending on the problem
• If you run multiple times you will have an idea
whether you’re being stuck in the local minima or
global minimum