Dr. Ali Abdalla Lectures

1. What is Search?
Dr. Ali Abdallah
(dr_ali_abdallah@hotmail.com)

2. What is Search?
• Search is something we do all the time
without realizing it.
– Finding this room required search.
– So does finding the best insurance deal.
– Or finding your keys.
• Computers are not, naturally good at
search.
• But if they could do it, they would be very
fast!

3. Computers Bad At Search?
• Yes, they can’t decide for themselves how
to search.
• We have to tell them how to search.
– Even when we do tell them how, they will still
not understand what they are doing!
• If computers are to be efficient at search,
we must very carefully tell them how to do
it.

4. What is Search For?
• All AI is search!
– Not totally true (obviously) but more true than
you might think.
– Finding a good/best solution to a problem
amongst many possible solutions.
• Ways of methodically deciding which
solutions are better than others.

5. Three Types of Search
• Blind Search:
– Searching the possibilities, in some predefined
order. The types of search we do in a maze.
• Heuristic Search:
– Searching the possibilities in order of their
perceived value.
• Stochastic Search:
– Randomly searching in a cleaver way.

6. Generalized Search algorithm
function GeneralSearch(QueuingFunction)
queue = MakeInitialQueue()
loop
if queue is empty then return failure
node = RemoveFromFront( queue )
if GoalTest( node ) succeeds then return node
queue = QueuingFunction( queue, Expand(node) )
end loop
The choice of queuing function is the main feature

7. Example: Arad to Bucharest
start
end

8. Representing Search
Arad
Zerind Sibiu Timisoara
Arad Oradea Fagaras Rimnicu Vilcea
Sibiu Bucharest

9. Search Terminology
• Completeness
– solution will be found, if it exists
• Time complexity
– number of nodes expanded
• Space complexity
– number of nodes in memory
• Optimality
– least cost solution will be found

10. Blind (Uninformed)
Search
(Where we systematically explore
alternatives)
R&N: Chap. 3, Sect. 3.3–5

11. Simple Problem-Solving-Agent
Agent Algorithm
 s0  sense/read state
 GOAL?  select/read goal test
 SUCCESSORS  read successor function
 solution  search(s0, G, Succ)
 perform(solution)

12. Searching the State Space
Search tree
Note that some states are
visited multiple times

13. Basic Search Concepts
 Search tree
 Search node
 Node expansion
 Fringe of search tree
 Search strategy: At each stage it
determines which node to expand

14. Search Nodes ≠ States
8 2
3 4 7
5 1 6
8 2 7
3 4
5 1 6
8 2 8 2 8 4 2 8 2
3 4 7 3 4 7 3 7 3 4 7
5 1 6 5 1 6 5 1 6 5 1 6

15. Search Nodes ≠ States
8 2
3 4 7
5 1 6
8 2 7
If states are allowed to be revisited,
3 4
the search tree may be infinite even
5 1 6 when the state space is finite
8 2 8 2 8 4 2 8 2
3 4 7 3 4 7 3 7 3 4 7
5 1 6 5 1 6 5 1 6 5 1 6

16. Data Structure of a Node
8 2
STATE
3 4 7 PARENT-NODE
5 1 6
BOOKKEEPING
CHILDREN
Action Right
Depth 5
... Path-Cost 5
yes
Expanded
Depth of a node N = length of path from root to N
(Depth of the root = 0)

17. Node expansion
The expansion of a node N of the search
tree consists of:
– Evaluating the successor function on
STATE(N)
– Generating a child of N for each
state returned by the function

18. Fringe and Search Strategy
 The fringe is the set of all search nodes
that haven’t been expanded yet
8 2 Is it identical
3 4 7 to the set of
5 1 6 leaves?
8 2 7
3 4
5 1 6
8 2 8 2 8 4 2 8 2
3 4 7 3 4 7 3 7 3 4 7
5 1 6 5 1 6 5 1 6 5 1 6

19. Fringe and Search Strategy
 The fringe is the set of all search nodes
that haven’t been expanded yet
 It is implemented as a priority queue
FRINGE
• INSERT(node,FRINGE)
• REMOVE(FRINGE)
 The ordering of the nodes in FRINGE
defines the search strategy

20. Search Algorithm
 If GOAL?(initial-state) then return initial-state
 INSERT(initial-node,FRINGE)
 Repeat:
 If empty(FRINGE) then return failure
 n  REMOVE(FRINGE)
 s  STATE(n)
 For every state s’ in SUCCESSORS(s)
 Create a new node n’ as a child of n
 If GOAL?(s’) then return path or goal state
 INSERT(n’,FRINGE)

21. Performance Measures
 Completeness
A search algorithm is complete if it finds a
solution whenever one exists
[What about the case when no solution exists?]
 Optimality
A search algorithm is optimal if it returns an
optimal solution whenever a solution exists
 Complexity
It measures the time and amount of memory
required by the algorithm

22. Important Parameters
• Maximum number of successors of any state
 branching factor b of the search tree
• Minimal length of a path between the initial
and a goal state
 depth d of the shallowest goal node in the
search tree

23. Blind vs. Heuristic Strategies
 Blind (or un-informed) strategies do not
exploit state descriptions to select
which node to expand next
 Heuristic (or informed) strategies
exploits state descriptions to select the
“most promising” node to expand

24. Example
For a blind strategy, N1 and N2
8 2 are just two nodes (at some
depth in the search tree)
STATE
3 4 7
N1
5 1 6
1 2 3
1 2 3 4 5 6
STATE
4 5 7 8
N2
7 8 6 Goal state

25. Example
For a heuristic strategy counting
the number of misplaced tiles,
8 2
N2 is more promising than N1
STATE
3 4 7
N1
5 1 6
1 2 3
1 2 3 4 5 6
STATE
4 5 7 8
N2
7 8 6 Goal state

26. Important Remark
 Some search problems, such as the (n2-1)-
puzzle, are NP-hard
 One can’t expect to solve all instances of
such problems in less than exponential
time
 One may still strive to solve each instance
as efficiently as possible

27. Blind Strategies
 Breadth-first
• Bidirectional
 Depth-first
• Depth-limited
• Iterative deepening
Arc cost
 Uniform-Cost = c(action) ≥ ε > 0
(variant of breadth-first)

28. Breadth-First Strategy
New nodes are inserted at the end of FRINGE
1
2 3 FRINGE = (1)
4 5 6 7

29. Breadth-First Strategy
New nodes are inserted at the end of FRINGE
1
2 3 FRINGE = (2, 3)
4 5 6 7

30. Breadth-First Strategy
New nodes are inserted at the end of FRINGE
1
2 3 FRINGE = (3, 4, 5)
4 5 6 7

31. Breadth-First Strategy
New nodes are inserted at the end of FRINGE
1
2 3 FRINGE = (4, 5, 6, 7)
4 5 6 7

32. Evaluation
 b: branching factor
 d: depth of shallowest goal node
 Breadth-first search is:
• Complete
• Optimal if step cost is 1
 Number of nodes generated:
1 + b + b2 + … + bd = (bd+1-1)/(b-1) = O(bd)
  Time and space complexity is O(bd)

33. Time and Memory Requirements
d # Nodes Time Memory
2 111 .01 msec 11 Kbytes
4 11,111 1 msec 1 Mbyte
6 ~106 1 sec 100 Mb
8 ~108 100 sec 10 Gbytes
10 ~1010 2.8 hours 1 Tbyte
12 ~1012 11.6 days 100 Tbytes
14 ~1014 3.2 years 10,000 Tbytes
Memory requirements vs Time requirements
Assumptions: b = 10; 1,000,000 nodes/sec; 100bytes/node

34. Remark
If a problem has no solution, breadth-first may
run for ever (if the state space is infinite or
states can be revisited arbitrary many times)
1 2 3 4 1 2 3 4
5 6 7 8 ? 5 6 7 8
9 10 11 12 9 10 11 12
13 14 15 13 15 14

35. Bidirectional Strategy
2 fringe queues: FRINGE1 and FRINGE2
s
Time and space complexity is O(bd/2) << O(bd)
if both trees have the same branching factor b

36. Bidirectional Strategy
• What happens if the branching factor
is different in each direction?
Major difficulties:
• Need to have a predecessor function
• What about multiple goals?
• What about implicitly defined goals?
(e.g. n-queens problem)

37. Depth-First Strategy
New nodes are inserted at the front of FRINGE
1
2 3
FRINGE = (1)
4 5

38. Depth-First Strategy
New nodes are inserted at the front of FRINGE
1
2 3
FRINGE = (2, 3)
4 5

39. Depth-First Strategy
New nodes are inserted at the front of FRINGE
1
2 3
FRINGE = (4, 5, 3)
4 5

40. Depth-First Strategy
New nodes are inserted at the front of FRINGE
1
2 3
4 5

41. Depth-First Strategy
New nodes are inserted at the front of FRINGE
1
2 3
4 5

42. Depth-First Strategy
New nodes are inserted at the front of FRINGE
1
2 3
4 5

43. Depth-First Strategy
New nodes are inserted at the front of FRINGE
1
2 3
4 5

44. Depth-First Strategy
New nodes are inserted at the front of FRINGE
1
2 3
4 5

45. Depth-First Strategy
New nodes are inserted at the front of FRINGE
1
2 3
4 5

46. Depth-First Strategy
New nodes are inserted at the front of FRINGE
1
2 3
4 5

47. Depth-First Strategy
New nodes are inserted at the front of FRINGE
1
2 3
4 5

48. Evaluation
 b: branching factor
 d: depth of shallowest goal node
 m: maximal depth of a leaf node
 Depth-first search is:
 Complete only for finite search tree
 Not optimal
 Number of nodes generated:
1 + b + b2 + … + bm = O(bm)
 Time complexity is O(bm)
 Space complexity is O(bm) [or O(m)]
[Reminder: Breadth-first requires O(bd) time and space]

49. Depth-Limited Search
 Depth-first with depth cutoff k (depth
below which nodes are not expanded)
 Three possible outcomes:
• Solution
• Failure (no solution)
• Cutoff (no solution within cutoff)

50. Iterative Deepening Search
Provides the best of both breadth-first
and depth-first search
Main idea:
IDS
For k = 0, 1, 2, … do:
Perform depth-first search with
depth cutoff k

51. Iterative Deepening

52. Iterative Deepening

53. Iterative Deepening

54. Performance
 Iterative deepening search is:
• Complete
• Optimal if step cost =1
 Time complexity is:
(d+1)(1) + db + (d-1)b2 + … + (1) bd = O(bd)
 Space complexity is: O(bd) or O(d)

55. Calculation
db + (d-1)b2 + … + (1) bd
= bd + 2bd-1 + 3bd-2 +… + db
= (1 + 2b-1 + 3b-2 + … + db-d)×bd
≤ (Σi=1,…,∞ ib(1-i))×bd = bd (b/(b-1))2

56. Number of Generated Nodes
(Breadth-First & Iterative Deepening)
d = 5 and b = 2
BF ID
1 1x6=6
2 2 x 5 = 10
4 4 x 4 = 16
8 8 x 3 = 24
16 16 x 2 = 32
32 32 x 1 = 32
120/63 ~ 2
63 120

57. Number of Generated Nodes
(Breadth-First & Iterative Deepening)
d = 5 and b = 10
BF ID
1 6
10 50
100 400
1,000 3,000
10,000 20,000
100,000 100,000
111,111 123,456 123,456/111,111 ~ 1.111

58. Comparison of Strategies
 Breadth-first is complete and optimal,
but has high space complexity
 Depth-first is space efficient, but is
neither complete, nor optimal
 Iterative deepening is complete and
optimal, with the same space complexity
as depth-first and almost the same time
complexity as breadth-first

59. Revisited States
No Few Many
1 2 3
search tree is finite search tree is infinite
4 5
7 8 6
8-queens assembly 8-puzzle and robot navigation
planning

60. Avoiding Revisited States
 Requires comparing state descriptions
 Breadth-first search:
• Store all states associated with generated
nodes in CLOSED
• If the state of a new node is in CLOSED, then
discard the node

61. Avoiding Revisited States
 Depth-first search:
Solution 1:
– Store all states associated with nodes in
current path in CLOSED
– If the state of a new node is in CLOSED, then
discard the node
 Only avoids loops
Solution 2:
– Store of all generated states in CLOSED
– If the state of a new node is in CLOSED, then
discard the node
 Same space complexity as breadth-first !

62. Uniform-Cost Search (Optimal)
 Each arc has some cost c ≥ ε > 0
 The cost of the path to each fringe node N is
g(N) = Σ costs of arcs
 The goal is to generate a solution path of minimal cost
 The queue FRINGE is sorted in increasing cost
A S
0
1 10
S 5 B 5 G A B C
1 5 15
5
15 C
G G
11 10
 Need to modify search algorithm

63. Search Algorithm
 If GOAL?(initial-state) then return initial-state
 INSERT(initial-node,FRINGE)
 Repeat:
 If empty(FRINGE) then return failure
 n  REMOVE(FRINGE)
 s  STATE(n)
 For every state s’ in SUCCESSORS(s)
 Create a new node n’ as a child of n
 If GOAL?(s’) then return path or goal state
 INSERT(n’,FRINGE)

64. Modified Search Algorithm
 INSERT(initial-node,FRINGE)
 Repeat:
 If empty(FRINGE) then return failure
 n  REMOVE(FRINGE)
 s  STATE(n)
 If GOAL?(s) then return path or goal state
 For every state s’ in SUCCESSORS(s)
 Create a node n’ as a successor of n
 INSERT(n’,FRINGE)

65. Avoiding Revisited States in
Uniform-Cost Search
 When a node N is expanded the path to N is
also the best path from the initial state to
STATE(N) if it is the first time STATE(N) is
encountered.
 So:
• When a node is expanded, store its state
into CLOSED
• When a new node N is generated:
– If STATE(N) is in CLOSED, discard N
– If there exits a node N’ in the fringe such that
STATE(N’) = STATE(N), discard the node – N or
N’ – with the highest-cost path