Basics of algorithm analysis and frequently asked design problems are covered in this presentation.

1. DR. K. ANITHA KUMARI, ME.,MBA.,PHD
ASSOCIATE PROFESSOR
DEPT. OF INFORMATION TECHNOLOGY
PSG COLLEGE OF TECHNOLOGY
EMAIL ID: KAK.IT@PSGTECH.AC.IN
Design and Analysis of Algorithms

2. Agenda
 Order of Growth
 Time Complexity Analysis
 Activity selection problem
 Job Sequencing Problem
 Minimize Cash Flow
 Minimum number of platforms required for a Railway/Bus
Station
 Minimum time to finish all jobs with given constraints
 Longest Common Subsequence
 Longest Increasing Subsequence
 Job Assignment Problem

3. What is algorithm?
3
• A sequence of unambiguous instructions for solving a
problem, i.e. for obtaining the required output for any
legitimate input in a finite amount of time.
• Range of inputs
• The same algorithm can be represented in different
ways
• Several algorithms for solving the same problem
AlgorithmInput Output

4. Algorithm principles
• Sequence
- One command at a time
- Parallel and distributed computing
• Condition
- IF
- CASE
• Loops
- FOR
- WHILE
- REPEAT
• Issues
• Range of inputs
• The same algorithm can be represented in different ways
• Several algorithms for solving the same problem
• Hard to design algorithms that are correct, efficient and
implementable

5. 5
Understand the problem
Decide on computational means
Exact vs approximate solution
Data structures
Algorithm design technique
Design an algorithm
Prove correctness
Analyze the algorithm
Code the algorithm

6. Some Important Problem Types
 Sorting
 a set of items
 Searching
 among a set of items
 String processing
 text, bit strings, gene
sequences
 Graphs
 model objects and their
relationships
 Combinatorial
 find desired permutation,
combination or subset
 Geometric
 graphics, imaging, robotics
 Numerical
 continuous math: solving
equations, evaluating
functions
6

7. Algorithm Design Techniques
 Brute Force &
Exhaustive Search
 follow definition / try all
possibilities
 Divide & Conquer
 break problem into distinct
subproblems
 Transformation
 convert problem to
another one
 Dynamic Programming
 break problem into overlapping
subproblems
 Greedy
 repeatedly do what is best now
 Branch and Bound
 Backtracking
 Randomization
 use random numbers
 Space and time tradeoffs
7

8. Complexity
8
Time complexity:
– How much time it takes to compute
– Measured by a function T(N)
Space complexity:
– How much memory it takes to compute
– Measured by a function S(N)

9. Order of Growth

10. Basic asymptotic efficiency classes
1 constant
log n logarithmic
n linear
n log n n-log-n
n2 quadratic
n3 cubic
2n exponential
n! factorial

11. Sequential search
 Worst case
 Best case
 Average case

12. Maximum element

13. Time efficiency of nonrecursive algorithms
General Plan for Analysis
 Decide on parameter n indicating input size
 Identify algorithm’s basic operation
 Set up a sum for the number of times the basic
operation is executed
 Simplify the sum using standard formulas and rules

14. Counting binary digits

15. Recursive evaluation of n!
Recursive definition of n!: F(n) = F(n-1)  n for n ≥ 1
and
F(0) = 1
Size:
Basic operation:
Recurrence relation:
n
multiplication
M(n) = M(n-1) + 1
M(0) = 0

16. Fibonacci numbers
The Fibonacci numbers:
0, 1, 1, 2, 3, 5, 8, 13, 21, …
The Fibonacci recurrence:
F(n) = F(n-1) + F(n-2)
F(0) = 0
F(1) = 1

17. Greedy Technique
Constructs a solution to an optimization problem piece by
piece through a sequence of choices that are:
 feasible, i.e. satisfying the constraints
 locally optimal (with respect to some neighborhood
definition)
 Irrevocable
 Immediate benefit
For some problems, it yields a globally optimal solution for
every instance.

18. Input: Weight of N items {w1, w2, ..., wn}
Cost of N items {c1, c2, ..., cn}
Knapsack limit S
Output: Selection for knapsack: {x1,x2,…xn}
where xi {0,1}.
Sample input:
wi={1,1,2,4,12}
ci ={1,2,2,10,4}
S=15
Knapsack problem
Problem definition
Binary
choice

19. Generalized Knapsack
problem
ci/wi = {1, 2, 1, 2.5, 0.33}
Sorted = {2.5, 2, 1, 1, 0.33}
Select = 1.0, 1.0, 1.0, 1.0, 0.58
Input: Weight of N items {w1, w2, ..., wn}
Cost of N items {c1, c2, ..., cn}
Knapsack limit S
Output: Selection for knapsack: {x1,x2,…xn}
where xi[0,1]. Any real value
between 0 and 1 !
Sample input:
wi={1,1,2,4,12}
ci ={1,2,2,10,4}
S=15 10 2 2 1 2.3
4 1 2 1 7.0Weights
Costs

20. Activity Selection problem
You are given n activities with their start and finish times. Select the
maximum number of activities that can be performed by a single person,
assuming that a person can only work on a single activity at a time.
Example 1 : Consider the following 3 activities sorted by finish time.
start[] = {10, 12, 20};
finish[] = {20, 25, 30};
A person can perform at most two activities. The maximum set of activities
that can be executed is {0, 2} [These are indexes in start[] and finish[] ]
Example 2 : Consider the following 6 activities sorted by finish time.
start[] = {1, 3, 0, 5, 8, 5};
finish[] = {2, 4, 6, 7, 9, 9};
A person can perform at most _______activities. The maximum set of
activities that can be executed is _______

21. Note:
The greedy choice is to always pick the next activity whose finish time
is least among the remaining activities and the start time is more
than or equal to the finish time of previously selected activity. We
can sort the activities according to their finishing time so that we
always consider the next activity as minimum finishing time
activity.
Algorithm:
1) Sort the activities according to their finishing time
2) Select the first activity from the sorted array and print it.
3) Do following for remaining activities in the sorted array.
…….a) If the start time of this activity is greater than or equal to the
finish time of previously selected activity then select this activity
and print it.

22.  Time Complexity : It takes O(n log n) time if input
activities may not be sorted. It takes O(n) time when
it is given that input activities are always sorted.

23. Job Sequencing Problem
Given an array of jobs where every job has a deadline
and associated profit if the job is finished before the
deadline. It is also given that every job takes single
unit of time, so the minimum possible deadline for
any job is 1. How to maximize total profit if only one
job can be scheduled at a time.
Example:

24.  Input: Five Jobs with following deadlines and profits
JobID Deadline Profit
a 2 100
b 1 19
c 2 27
d 1 25
e 3 15
Output: ?

25. Algorithm:
 1) Sort all jobs in decreasing order of profit.
 2) Initialize the result sequence as first job in sorted
jobs.
 3) Do following for remaining n-1 jobs .......
 a) If the current job can fit in the current result sequence
without missing the deadline, add current job to the result.
Else ignore the current job.
Time Complexity of the above solution is O(n2)

26. Minimize Cash Flow
Given a number of friends who have to give or take
some amount of money from one another. Design an
algorithm by which the total cash flow among all the
friends is minimized.

27. Above debts can be settled in following optimized way
Example:
Following diagram shows input debts to be settled.

28. Algorithm
Do following for every person Pi where i is from 0 to n-1.
1) Compute the net amount for every person. The net amount for person ‘i’
can be computed be subtracting sum of all debts from sum of all credits.
2) Find the two persons that are maximum creditor and maximum debtor.
Let the maximum amount to be credited maximum creditor be maxCredit
and maximum amount to be debited from maximum debtor be maxDebit.
Let the maximum debtor be Pd and maximum creditor be Pc.
3) Find the minimum of maxDebit and maxCredit. Let minimum of two be x.
Debit ‘x’ from Pd and credit this amount to Pc
4) If x is equal to maxCredit, then remove Pc from set of persons and recur
for remaining (n-1)persons.
5) If x is equal to maxDebit, then remove Pd from set of persons and recur for
remaining (n-1) persons.
Time Complexity: O(N2) where N is the number of persons.

29. Minimum number of platforms required for a
Railway/Bus Station
Given arrival and departure times of all trains that
reach a railway station, find the minimum number of
platforms required for the railway station so that no
train waits.
 arr[] = {9:00, 9:40, 9:50, 11:00, 15:00, 18:00}
 dep[] = {9:10, 12:00, 11:20, 11:30, 19:00, 20:00}
Output: 3 -> There are at-most three trains at a time
(time between 11:00 to 11:20)

30. All events sorted by time.
Total platforms at any time can be obtained by subtracting total departures from total
arrivals by that time.
Time Event Type Total Platforms Needed at this Time
9:00 Arrival 1
9:10 Departure 0
9:40 Arrival 1
9:50 Arrival 2
11:00 Arrival 3
11:20 Departure 2
11:30 Departure 1
12:00 Departure 0
15:00 Arrival 1
18:00 Arrival 2
19:00 Departure 1
20:00 Departure 0
 Minimum Platforms needed on railway station = Maximum platforms needed at any
time = 3
 Time Complexity: O(nLogn)

31. Minimum Time To Finish All Jobs With
Given Constraints
Given an array of jobs with different time requirements.
There are K identical assignees available and we are also
given how much time an assignee takes to do one unit of
the job. Find the minimum time to finish all jobs with
following constraints.
Extension:
 An assignee can be assigned only contiguous jobs. For
example, an assignee cannot be assigned jobs 1 and 3, by
skipping 2.
 Two assignees cannot share (or co-assigned) a job, i.e., a
job cannot be partially assigned to one assignee and
partially to other.

32. Input :
 K: Number of assignees available. T: Time taken by
an assignee to finish one unit of job job[]: An array
that represents time requirements of different jobs.
 Input: k = 2, T = 5,
job[] = {4, 5, 10}
Output: 50 -> The minimum time required to finish
all the jobs is 50.

33. Dynamic Programming
Dynamic Programming is a general algorithm design technique
for solving problems defined by or formulated as recurrences with
overlapping sub instances
• Invented by American mathematician Richard Bellman in the
1950s to solve optimization problems and later assimilated by CS
• “Programming” here means “planning”
• Main idea:
- set up a recurrence relating a solution to a larger instance to
solutions of some smaller instances
- solve smaller instances once
- record solutions in a table
- extract solution to the initial instance from that table

34. Knapsack Problem by DP
Given n items of
integer weights: w1 w2 … wn
values: v1 v2 … vn
a knapsack of integer capacity W
find most valuable subset of the items that fit into the
knapsack
Consider instance defined by first i items and capacity j (j 
W).
Let V[i,j] be optimal value of such an instance. Then
max {V[i-1,j], vi + V[i-1,j- wi]} if j- wi  0
V[i,j] =
V[i-1,j] if j- wi < 0
Initial conditions: V[0,j] = 0 and V[i,0] = 0
{

35. Knapsack Problem by DP (example)
Example: Knapsack of capacity W = 5
item weight value
1 2 $12
2 1 $10
3 3 $20
4 2 $15 capacity j
0 1 2 3 4 5
0
w1 = 2, v1= 12 1
w2 = 1, v2= 10 2
w3 = 3, v3= 20 3
w4 = 2, v4= 15 4
0 0 0
0 0 12
0 10 12 22 22 22
0 10 12 22 30 32
0 10 15 25 30 37
Backtracing
finds the
actual optimal
subset, i.e.
solution.

36. Longest Common Subsequence (LCS)
 A subsequence of a sequence/string S is obtained by
deleting zero or more symbols from S. For example,
the following are some subsequences of “president”:
pred, sdn, predent. In other words, the letters of a
subsequence of S appear in order in S, but they are
not required to be consecutive.
 The longest common subsequence problem is to find
a maximum length common subsequence between
two sequences.

37. LCS
For instance,
Sequence 1: president
Sequence 2: providence
Its LCS is priden.
president
providence

38. LCS
Another example:
Sequence 1: algorithm
Sequence 2: alignment
One of its LCS is algm.
a l g o r i t h m
a l i g n m e n t

39. How to compute LCS?
 Let A=a1a2…am and B=b1b2…bn .
 len(i, j): the length of an LCS between
a1a2…ai and b1b2…bj
 With proper initializations, len(i, j) can be computed as follows.

41. i j 0 1
p
2
r
3
o
4
v
5
i
6
d
7
e
8
n
9
c
10
e
0 0 0 0 0 0 0 0 0 0 0 0
1 p
2
0 1 1 1 1 1 1 1 1 1 1
2 r 0 1 2 2 2 2 2 2 2 2 2
3 e 0 1 2 2 2 2 2 3 3 3 3
4 s 0 1 2 2 2 2 2 3 3 3 3
5 i 0 1 2 2 2 3 3 3 3 3 3
6 d 0 1 2 2 2 3 4 4 4 4 4
7 e 0 1 2 2 2 3 4 5 5 5 5
8 n 0 1 2 2 2 3 4 5 6 6 6
9 t 0 1 2 2 2 3 4 5 6 6 6
Running time and memory: O(mn) and O(mn).

42. i j 0 1
p
2
r
3
o
4
v
5
i
6
d
7
e
8
n
9
c
10
e
0 0 0 0 0 0 0 0 0 0 0 0
1 p
2
0 1 1 1 1 1 1 1 1 1 1
2 r 0 1 2 2 2 2 2 2 2 2 2
3 e 0 1 2 2 2 2 2 3 3 3 3
4 s 0 1 2 2 2 2 2 3 3 3 3
5 i 0 1 2 2 2 3 3 3 3 3 3
6 d 0 1 2 2 2 3 4 4 4 4 4
7 e 0 1 2 2 2 3 4 5 5 5 5
8 n 0 1 2 2 2 3 4 5 6 6 6
9 t 0 1 2 2 2 3 4 5 6 6 6
Output: priden

43. 
/* Returns length of LCS for X[0..m-1], Y[0..n-1] */
int lcs( char *X, char *Y, int m, int n )
{
if (m == 0 || n == 0)
return 0;
if (X[m-1] == Y[n-1])
return 1 + lcs(X, Y, m-1, n-1);
else
return max(lcs(X, Y, m, n-1), lcs(X, Y, m-1, n));
}
/* Utility function to get max of 2 integers */
int max(int a, int b)
{
return (a > b)? a : b;
}
/* Driver program to test above function */
int main()
{
char X[] = “PRESIDENT";
char Y[] = “PROVIDENCE";
int m = strlen(X);
int n = strlen(Y);
printf("Length of LCS is %d", lcs( X, Y, m, n ) );
return 0; }

44. Longest Increasing Subsequence
 The Longest Increasing Subsequence (LIS) problem is to find the length of
the longest subsequence of a given sequence such that all elements of the
subsequence are sorted in increasing order. For example, the length of LIS
for {10, 22, 9, 33, 21, 50, 41, 60, 80} is 6 and LIS is {10, 22, 33, 50, 60, 80}.
Examples:
Input : arr[] = {3, 10, 2, 1, 20}
Output : Length of LIS = 3
The longest increasing subsequence is 3, 10, 20
Input : arr[] = {10, 22, 10, 33, 9, 60}
Output : Length of LIS = ?
The longest increasing subsequences are ?
Input : arr[] = {50, 3, 10, 7, 40, 80}
Output : Length of LIS = ?
The longest increasing subsequence is ?

45. Optimal Substructure
Let arr[0..n-1] be the input array and L(i) be the length of
the LIS ending at index i such that arr[i] is the last
element of the LIS.
Then, L(i) can be recursively written as:
L(i) = 1 + max( L(j) ) where 0 < j < i and arr[j] < arr[i]; or
L(i) = 1, if no such j exists.
To find the LIS for a given array, we need to return
max(L(i)) where 0 < i < n.

46. int _lis( int arr[], int n, int *max_ref)
{
/* Base case */
if (n == 1)
return 1;
// 'max_ending_here' is length of LIS ending with arr[n-1]
int res, max_ending_here = 1;
/* Recursively get all LIS ending with arr[0], arr[1] ...
arr[n-2]. If arr[i-1] is smaller than arr[n-1], and
max ending with arr[n-1] needs to be updated, then
update it */
for (int i = 1; i < n; i++)
{
res = _lis(arr, i, max_ref);
if (arr[i-1] < arr[n-1] && res + 1 > max_ending_here)
max_ending_here = res + 1;
}
// Compare max_ending_here with the overall max. And
// update the overall max if needed
if (*max_ref < max_ending_here)
*max_ref = max_ending_here;
// Return length of LIS ending with arr[n-1]
return max_ending_here;
}

47. Branch and Bound
 Branch and bound is an algorithm design paradigm
which is generally used for solving combinatorial
optimization problems. These problems typically
exponential in terms of time complexity and may require
exploring all possible permutations in worst case based
on bounds.
[1] It is used to solve optimization problem.
[2] It may traverse the tree in any manner, DFS or BFS.
[3] It realizes that it already has a better optimal solution
that the pre-solution leads to so it abandons that pre-
solution.
[4] It completely searches the state space tree to get
optimal solution.
[5] It involves bounding function.

48. Job Assignment Problem
 Let there be N workers and N jobs. Any worker can be
assigned to perform any job, incurring some cost that may
vary depending on the work-job assignment. It is required to
perform all jobs by assigning exactly one worker to each job
and exactly one job to each agent in such a way that the total
cost of the assignment is minimized.

49. Solution 1: Brute Force
complexity is O(n!).
Solution 2: Hungarian Algorithm
complexity of O(n^3).
Optimal Solution using Branch and Bound
 There are two approaches to calculate the cost function:
 For each worker, we choose job with minimum cost from
list of unassigned jobs (take minimum entry from each
row).
 For each job, we choose a worker with lowest cost for that
job from list of unassigned workers (take minimum entry
from each column).

50.  Assume Job 2 is assigned to worker A.
 Since Job 2 is assigned to worker A (marked in
green), cost becomes 2 and Job 2 and worker A
becomes unavailable (marked in red).

51.  Similarly assign low cost job to other workers.
 Finally, job 1 gets assigned to worker C as it has
minimum cost among unassigned jobs and job 4 gets
assigned to worker C as it is only Job left. Total cost
becomes 2 + 3 + 5 + 4 = 14.

52. Work out
 Huffman codes
 Simplex method
 Closest-pair and convex-hull algorithms
 Ford-Fulkerson algorithm for maximum flow problem
 Maximum matching of graph vertices
 Gale-Shapley algorithm for the stable marriage problem
 Local search heuristics

53. Thank you
53