3. Divide-and-Conquer
The whole problem we want to solve may too big to understand or solve at
once.
We break it up into smaller pieces, solve the pieces separately, and
combine the separate pieces together.
Divide-and conquer is a general algorithm design paradigm:
1. Divide: divide the input data S in two or more disjoint subsets S1,
S2, …
2. Recur: solve the subproblems recursively
3. Conquer: combine the solutions for S1, S2, …, into a solution for S
The base case for the recursion are subproblems of constant size
Analysis can be done using recurrence equations
3 Eng: Mohammed Hussein
4. Recurrence Equation Analysis
The conquer step of merge-sort consists of merging two sorted sequences, each with
n/2 elements and implemented by means of a doubly linked list, takes at most bn steps,
for some constant b.
Likewise, the basis case (n < 2) will take at b most steps.
Therefore, if we let T(n) denote the running time of merge-sort:
We can therefore analyze the running time of merge-sort by finding a closed form
solution to the above equation.
That is, a solution that has T(n) only on the left-hand side.
2if)2/(2
2if
)(
nbnnT
nb
nT
4 Eng: Mohammed Hussein
5. Solving recurrences
Recurrences are like solving integrals, differential equations, etc.
Should learn a few tricks.
Three ways of solving recurrences
1. Guess-and-Test Method(If we know the answer)
2. RecursionTree Method (Very useful !)
3. MasterTheorem (Save our effort)
5 Eng: Mohammed Hussein
6. Guess-and-Test Method
In the guess-and-test method, we guess a closed form solution and then try to prove it is
true by induction:
Guess that:T(n) < c n log n.
Wrong: we cannot make this last line be less than cn log n
ncn
nbncnncn
nbnncn
nbnnnc
nbnnTnT
log
loglog
log)2log(log
log))2/log()2/((2
log)2/(2)(
2iflog)2/(2
2if
)(
nnbnnT
nb
nT
6 Eng: Mohammed Hussein
Log(x/y)= log x – log y
Log 2 =1
بالتعويض
7. Guess-and-Test Method, Part 2
Recall the recurrence equation:
Guess #2 that:T(n) < cn log2 n.
if c > b.
So,T(n) is O(n log2 n).
In general, to use this method, you need to have a good guess and you need to be good at
induction proofs.
ncn
nbncnncnncn
nbnnncn
nbnncn
nbnnnc
nbnnTnT
2
2
22
2
2
log
loglog2log
log))2(log2loglog2(log
log)2log(log
log))2/(log)2/((2
log)2/(2)(
2iflog)2/(2
2if
)(
nnbnnT
nb
nT
7 Eng: Mohammed Hussein
(x-y) 2= x2 -2xy +y2
Log 2 =1
nj <= nd if j <= d
Big-Oh of highest degree
8. Recurrences
In Mathematic a recurrence is a function that is defined in
terms of one or more base cases, and itself, with smaller
arguments
Examples:
8 Eng: Mohammed Hussein
9. Master Method (tree)
The master theorem concerns the recurrence
a≥1 , b> 1
n is the size of the problem.
a is the number of subproblems in the recursion. ()الفزوع
n/b is the size of each subproblem. (Here it is assumed that
all subproblems are essentially the same size.)
f (n) is the cost of the work done outside the recursive calls
)()/()( nfbnaTnT
9 Eng: Mohammed Hussein
merge-sort خوارسمية
االستدعاء عمليات عندالذاتيالدمج عملية تنفيذ مستويات مختلف في
10. The Tree Definition
The root of this tree is node A.
Definitions:
Parent (A)
Children (E, F)
Siblings (C, D)
Root (A)
Leaf / Leaves
K, L, F, G, M, I, J…
The degree of a node is the number of sub-trees of the node.
The level of a node:
Initially letting the root be at level one
the level of the node’s parent plus one. Level H=level(Parent (H)+1));
The height or depth of a tree is the maximum level of any node in the tree.
height =Max(level(tree))
10 Eng: Mohammed Hussein
11. Divide and conquer (tree)
We analyze this in some generality:
suppose we have a pieces, each of size n/b and merging takes time f(n).
(In this example a=b=2 and f(n)=O(log n) but it will not always be true that
a=b sometimes the pieces will overlap.)
For simplicity, let's assume n is a power of b, and that the recursion stops
when n is 1.
Notice that the size of a node depends only on its level: size(i) = n/(bi).
What is time taken by a node at level i? : time(i) = f(n/ bi)
How many levels can we have before we get down to n=1?
For bottom level, n/bi =1, so n=bi and i=(log n)/(log b).
How many items at level i?: ai.
11 Eng: Mohammed Hussein
)()/()( nfbnaTnT
time(i) = f(n/ bi)
12. The Recursion Tree
Draw the recursion tree for the recurrence relation and look for a pattern:
At each successive level of recursion the sub problems get halved in size (n/2).
At the (log n) th level the sub problems get down to size 1, and so the recursion ends.
Level/
depth
T’s size
0 1 n
1 2 n/2
k 2k n/2k
… … …
2if)2/(2
2if
)(
nbnnT
nb
nT
time
bn
bn
bn
…
Total time = bn + bn log n
(last level plus all previous levels)
2k =n k= log n for each level.
log n
Notice that the size of a node depends only on its level
12 Eng: Mohammed Hussein
time=b x size = bn
13. Steps using recursion tree.
Height = k=log n
Levels = log n + 1
Cost/level = cn
Bottom node number n = 2log n for
height log n dividing problem by 2
each time.
Recall that a balanced binary tree of
height k has k + 1 levels.
Recall: a
log an = n
so 2
log 2n = 2
log n =n
Bottom cost = T(1) * n = cn
Note that T(1) is problem size n=1,
there are n subproblems at bottom.
13 Eng: Mohammed Hussein 2k =n k= log n for each level.
14. Merge Sort - Finding O
Recall that a balanced binary tree of height k has k + 1 levels.
A problem of size n=16 divided as described by the following recurrence equation
would then be represented by the tree.
Use the implied coefficient c for arithmetic purposes:
14 Eng: Mohammed Hussein
16=2
4
15. Master Method
Many divide-and-conquer recurrence equations have the form:
The MasterTheorem:
dnnfbnaT
dnc
nT
if)()/(
if
)(
.1somefor)()/(provided
)),((is)(then),(is)(if3.
)log(is)(then),log(is)(if2.
)(is)(then),(is)(if1.
log
1loglog
loglog
nfbnaf
nfnTnnf
nnnTnnnf
nnTnOnf
a
kaka
aa
b
bb
bb
15 Eng: Mohammed Hussein
very small f(n)
moderate f(n)
very large f(n)
16. Analysis of Merge Sort
Analysis of Merge Sort
Divide:Trivial.
Conquer:Recursively sort 2subarrays.
Combine:Linear-time merge.
Using Master theorem
16 Eng: Mohammed Hussein