1. Big O Notation

2. outline
 Performance vs Complexity
 Algorithm Efficiency
 Big O Notation

3. Performance vs Complexity
• Performance:
 How much time/memory/disk/… is used by the algorithm
 Depends on the machine, compiler, code,…
• Complexity:
 How do the resource requirements of a program/algorithm scale
Complexity affects performance
but not the other way around
If I have an algorithm with high complexity, it will take longer time to run.
However, if I have a very slow program, that does not mean that my algorithm is very complex;
that could be all other kind of issues that made the program run slow.

4. Complexity of an Algorithm
 We are going to count the number of the steps that are taken during the
execution
 Assumption (simplification):
 Each step costs the same amount
 +, -, return a value, access an array element, ….

5. Complexity of an Algorithm (cont.)
• Example 1:
• private int getLastElement (int [] numbers)
• {
return numbers[numbers.length -1];
}
Number of steps: 4
Constant time function (independent of array size)
Notes: Let’s look at each step that need to be executed. It is always 4 steps to access the last element,
no matter the number of elements.

6. Complexity of an Algorithm (cont.)
• Example 2:
• private static int sum(int [] numbers)
• {
int sum = 0;
for (int i = 0; i < number.length; i++){
sum = sum + number[i];
}
return sum;
}
Outside loop: 3 + 2 Number of steps:6n + 5
Linear time functionLoop body: 6
INSTRUCTOR-NOTES:This time we use a loop.
We will start by the steps that will be executed only once and the are outside the loop.
Then we count the steps that are going to be repeated over and over again.

7. Algorithm Efficiency
 Let’s look at the following algorithm for initializing the values in an array:
final int N = 500;
int [] counts = new int[N];
for (int i=0; i<counts.length; i++)
counts[i] = 0;
 The length of time the algorithm takes to execute depends on the value of N

8. Algorithm Efficiency (cont.)
 In that algorithm, we have one loop that processes all of the elements in the array
 Intuitively:
– If N was half of its value, we would expect the algorithm to take half the time
– If N was twice its value, we would expect the algorithm to take twice the time
 That is true and we say that the algorithm efficiency relative to N is linear

9. Algorithm Efficiency (cont.)
 Let’s look at another algorithm for initializing the values in a different array:
final int N = 500;
int [] [] counts = new int[N][N];
for (int i=0; i<counts.length; i++)
for (int j=0; j<counts[i].length; j++)
counts[i][j] = 0;
 The length of time the algorithm takes to execute still depends on the value of N

10. Algorithm Efficiency (cont.)
 However, in the second algorithm, we have two nested loops to process the
elements in the two dimensional array
 Intuitively:
– If N is half its value, we would expect the algorithm to take one quarter the time
– If N is twice its value, we would expect the algorithm to take quadruple the time
 That is true and we say that the algorithm efficiency relative to N is quadratic

11. Big O notation
Why do we need Big-O?
 In computer science, we use Big-O to classify algorithms based on their scalability.
 Basically, it tells you how fast a function grows or declines.
 The letter O is used because the rate of growth of a function is also called its order.
 It also allows us to estimate the worst case running time of an algorithm as a
function of the input size.

12. Big O notation
 Examples:
– We can say that the first algorithm is O(n)
– We can say that the second algorithm is O(n2)
 For any algorithm that has a function g(n) of the parameter n that describes its
length of time to execute, we can say the algorithm is O(g(n))
 We only include the fastest growing term and ignore any multiplying by or adding of
constants (which makes sense because those depend on the particular hardware the
program is run on)

13. Big O notation (cont.)
Growth Functions:
Eight functions O(n) that occur frequently in the analysis of algorithms (in order of
increasing rate of growth relative to n):
– Constant  1
– Logarithmic  log n
– Linear  n
– Log Linear  n log n
– Quadratic  n2
– Cubic  n3
– Exponential  2n
– Exhaustive Search  n!

14. Big O notation (cont.)
Growth Rates Compared:

15. Big O notation (cont.)

16. Big O notation (cont.)
 The growth list is useful because of the following fact:
 if a function f(n) is a sum of functions, one of which grows faster than the
others, then the faster growing one determines the order of f(n).
 Example:
 If f(n) = 10 log(n) + 5 (log(n))3 + 7 n + 3 n 2 + 6 n 3,
 then f(n) = O(n 3).

17. Big O notation (cont.)
Big-O for a Problem:
 O(g(n)) for a problem means there is some O(g(n)) algorithm that solves the
problem
 Don’t assume that the specific algorithm that you are currently using is the
best solution for the problem
 There may be other correct algorithms that grow at a smaller rate with
increasing n
 Many times, the goal is to find an algorithm with the smallest possible
growth rate

18. Big O notation (cont.)
Role of Data Structures:
 That brings up the topic of the structure of the data on which the algorithm
operates
 If we are using an algorithm manually on some amount of data, we
intuitively try to organize the data in a way that minimizes the number of
steps that we need to take
 Publishers offer dictionaries with the words listed in alphabetical order to
minimize the length of time it takes us to look up a word

19. Big O notation (cont.)
Role of Data Structures:
 We can do the same thing for algorithms in our computer programs
 Example: Finding a numeric value in a list
 If we assume that the list is unordered, we must search from the beginning
to the end
 On average, we will search half the list
 Worst case, we will search the entire list
 Algorithm is O(n), where n is size of array

20. Big O notation (cont.)
Example:
int [] list = {7, 2, 9, 5, 6, 4};
for (int i=0; i<list.length, i++)
if (value == list[i])
statement; // found it
// didn’t find it

21. Big O notation (cont.)
 If we assume that the list is ordered, we can still search the entire list
from the beginning to the end to determine if we have a match
 But, we do not need to search that way
 Because the values are in numerical order, we can use a binary search
algorithm
 Algorithm is O(logn), where n is size of array

22. Big O notation (cont.)
Role of Data Structures:
 The difference in the structure of the data between an unordered list and an
ordered list can be used to reduce algorithm Big-O
 This is the role of data structures and why we study them
 We need to be as clever in organizing our data efficiently as we are in
figuring out an algorithm for processing it efficiently

23. Big O notation (cont.)
Example:
 Find an order for
 f(x) = 7 x5 + 5 x3 – x + 4
 O(x5)

24. Big O notation (cont.)
Sorting Algorithms Big-O:
http://bigocheatsheet.com/