Hi: This is the first slide of my class on analysis of algorithms based in Cormen's book. In this slides, we define the following concepts: 1.- What is an algorithm? 2.- What problems are solved by algorithms? 3.- What subjects will be studied in this class? 4.- Cautionary tale about complexities

1. Analysis of Algorithms
Introduction
Andres Mendez-Vazquez
September 27, 2015
1 / 36

2. Outline
1 Motivation
What is an Algorithm?
Instance of a Problem
Kolmogorov’s Definition
2 Problems Solved By Algorithms
What Problems Are Solved By Algorithms?
3 Syllabus
Subjects for the Class
4 Some Notes in Notation
Before Anything, The Notation for Pseudo-Code
5 What abstraction of a Computer to use?
The Random-Access Machine (RAM) Machine
6 Analyzing Algorithms
How Input Size affects Running Time
The First Method: Counting Number of Operations
Counting Equation For Insertion Sort
The Analysis of the Best and Worst Case Inputs
2 / 36

3. Outline
1 Motivation
What is an Algorithm?
Instance of a Problem
Kolmogorov’s Definition
2 Problems Solved By Algorithms
What Problems Are Solved By Algorithms?
3 Syllabus
Subjects for the Class
4 Some Notes in Notation
Before Anything, The Notation for Pseudo-Code
5 What abstraction of a Computer to use?
The Random-Access Machine (RAM) Machine
6 Analyzing Algorithms
How Input Size affects Running Time
The First Method: Counting Number of Operations
Counting Equation For Insertion Sort
The Analysis of the Best and Worst Case Inputs
3 / 36

4. Introduction
Informal definition
Informally, an algorithm is any well defined computational procedure that
It takes some value, or set of values, as input.
Then, it produces some value, or set of values, as output.
Examples
4 / 36

5. Introduction
Informal definition
Informally, an algorithm is any well defined computational procedure that
It takes some value, or set of values, as input.
Then, it produces some value, or set of values, as output.
Examples
4 / 36

6. Introduction
Informal definition
Informally, an algorithm is any well defined computational procedure that
It takes some value, or set of values, as input.
Then, it produces some value, or set of values, as output.
Examples
4 / 36

7. Introduction
Informal definition
Informally, an algorithm is any well defined computational procedure that
It takes some value, or set of values, as input.
Then, it produces some value, or set of values, as output.
Examples
ALGORITHMINPUT Cluster 1 (7)
OUTPUT
4 / 36

8. Example
Sorting Problem
Input: A sequence of N numbers a1, a2, ..., aN
Output: A reordering of the input sequence a(1), a(2), ..., a(N)
Actually
We are dealing with instances of a problem.
5 / 36

9. Example
Sorting Problem
Input: A sequence of N numbers a1, a2, ..., aN
Output: A reordering of the input sequence a(1), a(2), ..., a(N)
Actually
We are dealing with instances of a problem.
5 / 36

10. Outline
1 Motivation
What is an Algorithm?
Instance of a Problem
Kolmogorov’s Definition
2 Problems Solved By Algorithms
What Problems Are Solved By Algorithms?
3 Syllabus
Subjects for the Class
4 Some Notes in Notation
Before Anything, The Notation for Pseudo-Code
5 What abstraction of a Computer to use?
The Random-Access Machine (RAM) Machine
6 Analyzing Algorithms
How Input Size affects Running Time
The First Method: Counting Number of Operations
Counting Equation For Insertion Sort
The Analysis of the Best and Worst Case Inputs
6 / 36

11. Instance of a Problem
Instance of the problem
For example, we have
9, 8, 5, 6, 7, 4, 3, 2, 1
Then, we finish with
1, 2, 3, 4, 5, 6, 7, 8, 9
7 / 36

12. Although Instances are Important
Nevertheless
The way we use those instances is way more important
For example
Look at Recursive Fibonacci!!!
8 / 36

13. Although Instances are Important
Nevertheless
The way we use those instances is way more important
For example
Look at Recursive Fibonacci!!!
8 / 36

14. Example: Fibonacci
Fibonacci rule
Fn =



Fn−1 + Fn−2 if n > 1
1 if n = 1
0 if n = 0
Time Complexity
1 Naive version using directly the recursion - exponential time.
2 A more elegant version - linear time.
9 / 36

15. Example: Fibonacci
Fibonacci rule
Fn =



Fn−1 + Fn−2 if n > 1
1 if n = 1
0 if n = 0
Time Complexity
1 Naive version using directly the recursion - exponential time.
2 A more elegant version - linear time.
9 / 36

16. Outline
1 Motivation
What is an Algorithm?
Instance of a Problem
Kolmogorov’s Definition
2 Problems Solved By Algorithms
What Problems Are Solved By Algorithms?
3 Syllabus
Subjects for the Class
4 Some Notes in Notation
Before Anything, The Notation for Pseudo-Code
5 What abstraction of a Computer to use?
The Random-Access Machine (RAM) Machine
6 Analyzing Algorithms
How Input Size affects Running Time
The First Method: Counting Number of Operations
Counting Equation For Insertion Sort
The Analysis of the Best and Worst Case Inputs
10 / 36

17. Kolmogorov’s Definition
A bound for each sub-step
An algorithmic process splits into steps whose complexity is bounded in advance
i.e., the bound is independent of the input and the current state of the
computation.
Transformations done at each step
Each step consists of a direct and immediate transformation of the current state.
This transformation applies only to the active part of the state and does not alter
the remainder of the state.
Size of the steps
The size of the active part is bounded in advance.
Ending the Process
The process runs until either the next step is impossible or a signal says the
solution has been reached.
11 / 36

18. Kolmogorov’s Definition
A bound for each sub-step
An algorithmic process splits into steps whose complexity is bounded in advance
i.e., the bound is independent of the input and the current state of the
computation.
Transformations done at each step
Each step consists of a direct and immediate transformation of the current state.
This transformation applies only to the active part of the state and does not alter
the remainder of the state.
Size of the steps
The size of the active part is bounded in advance.
Ending the Process
The process runs until either the next step is impossible or a signal says the
solution has been reached.
11 / 36

19. Kolmogorov’s Definition
A bound for each sub-step
An algorithmic process splits into steps whose complexity is bounded in advance
i.e., the bound is independent of the input and the current state of the
computation.
Transformations done at each step
Each step consists of a direct and immediate transformation of the current state.
This transformation applies only to the active part of the state and does not alter
the remainder of the state.
Size of the steps
The size of the active part is bounded in advance.
Ending the Process
The process runs until either the next step is impossible or a signal says the
solution has been reached.
11 / 36

20. Kolmogorov’s Definition
A bound for each sub-step
An algorithmic process splits into steps whose complexity is bounded in advance
i.e., the bound is independent of the input and the current state of the
computation.
Transformations done at each step
Each step consists of a direct and immediate transformation of the current state.
This transformation applies only to the active part of the state and does not alter
the remainder of the state.
Size of the steps
The size of the active part is bounded in advance.
Ending the Process
The process runs until either the next step is impossible or a signal says the
solution has been reached.
11 / 36

21. Kolmogorov’s Definition
A bound for each sub-step
An algorithmic process splits into steps whose complexity is bounded in advance
i.e., the bound is independent of the input and the current state of the
computation.
Transformations done at each step
Each step consists of a direct and immediate transformation of the current state.
This transformation applies only to the active part of the state and does not alter
the remainder of the state.
Size of the steps
The size of the active part is bounded in advance.
Ending the Process
The process runs until either the next step is impossible or a signal says the
solution has been reached.
11 / 36

22. Kolmogorov’s Definition
A bound for each sub-step
An algorithmic process splits into steps whose complexity is bounded in advance
i.e., the bound is independent of the input and the current state of the
computation.
Transformations done at each step
Each step consists of a direct and immediate transformation of the current state.
This transformation applies only to the active part of the state and does not alter
the remainder of the state.
Size of the steps
The size of the active part is bounded in advance.
Ending the Process
The process runs until either the next step is impossible or a signal says the
solution has been reached.
11 / 36

23. BTW
How do they look this machines, this algorithms?
After all we like to see them!!!
12 / 36

24. An Example of an Algorithm
Insertion Sort Algorithm
Data: Unsorted Sequence A
Result: Sort Sequence A
Insertion Sort(A)
for j ← 2 to lenght(A) do
key ← A[j];
// Insert A[j] into the sorted sequence A[1, ..., j − 1]
i ← j − 1;
while i > 0 and A[i] > key do
A[i + 1] ← A[i];
i ← i − 1;
end
A[i + 1] ← key
end
13 / 36

25. Outline
1 Motivation
What is an Algorithm?
Instance of a Problem
Kolmogorov’s Definition
2 Problems Solved By Algorithms
What Problems Are Solved By Algorithms?
3 Syllabus
Subjects for the Class
4 Some Notes in Notation
Before Anything, The Notation for Pseudo-Code
5 What abstraction of a Computer to use?
The Random-Access Machine (RAM) Machine
6 Analyzing Algorithms
How Input Size affects Running Time
The First Method: Counting Number of Operations
Counting Equation For Insertion Sort
The Analysis of the Best and Worst Case Inputs
14 / 36

26. Problems Solved By Algorithms
Single-Source
Shortest Path.
Solving systems
of linear
equations.
Huffman codes
Matrix
Multiplication
Convex hull
Short Paths in Maps
These algorithms allows to solve the
problem of finding the shortest path
in a map between two addresses.
@Copyright 2010-2011 Daniel Kastl,
Frédéric Junod. Mis à jour le Apr 02, 2012.
15 / 36

27. Problems Solved By Algorithms
Single-Source
Shortest Path.
Solving systems
of linear
equations.
Huffman codes
Matrix
Multiplication
Convex hull
Inverting Matrices
Normally, given the system Ax = y,
we do not use the inverse A−1, but
the LUP decomposition
L D U R
@Copyright From Wikimedia Commons, the
free media repository
15 / 36

28. Problems Solved By Algorithms
Single-Source
Shortest Path.
Solving systems
of linear
equations.
Huffman codes
Matrix
Multiplication
Convex hull
Compression Trees
This method is part of the greedy
methods. They are used for
compression, they can achieve 20%
to 90% compression.
8
17
41
24
109
5
3 2 2 2 2
6
2 3 4
7
14
3
22
1 1
2
4
7
2
1111
4 4
' '
'U' 'A' 'L' 'M'
'E'
'D' 'F' 'N'
'S'
'B' 'H'
'O'
'J' 'P' 'R' 'T'
@Copyright From Wikimedia Commons, the
free media repository
15 / 36

29. Problems Solved By Algorithms
Single-Source
Shortest Path.
Solving systems
of linear
equations.
Huffman codes
Matrix
Multiplication
Convex hull
Fast Multiplication of Matrices
In many algorithms, we want to
multiply different n × n matrices
A11 A12 A21 A22
B11
B12
B21
B22
C11
1
1
C12
1
1
C21
1
1
C22
1
1
1
11
1
1 1
11
1 1
-1-1
1
-1
1
-1
1
-1
1
-1
1 1
-1-1
1
1
-1
-1
1
1
-1
-1
M1 M2 M3 M4 M5 M6 M7
STRASSEN'S ALGORITHM
@Copyright From Wikimedia Commons, the
free media repository
15 / 36

30. Problems Solved By Algorithms
Single-Source
Shortest Path.
Solving systems
of linear
equations.
Huffman codes
Matrix
Multiplication
Convex hull
Computational Geometry
Given the points in a plane, we want
to find the minimum convex hull that
encloses them.
p0
p1
p2
p3
@Copyright From Wikimedia Commons, the
free media repository
15 / 36

31. Problems Solved By Algorithms
Synthetic Biology
Pattern Recognition
Databases
Face Recognition
Computational Molecular Engineering
In this field the engineers and biologist try to use the
basis of life to create complex molecular machines.
All these machines will requiere complex algorithms
@Copyright 2002-2013 SYNTHETIC
COMPONENTS NETWORK University of Bristol
16 / 36

32. Problems Solved By Algorithms
Synthetic Biology
Pattern Recognition
Databases
Face Recognition
Machine Learning
In pattern recognition, we try to find specific
patterns in data.
@2013Copyright Bülent Üstün, Department of
Analytical Chemistry, University of Nijmegen
16 / 36

33. Problems Solved By Algorithms
Synthetic Biology
Pattern Recognition
Databases
Face Recognition
Partition of the Database Space
A grid file or bucket grid is a point access method:
Query A1 = 'x' and A2 = 6
a f k p z
A2
A1
Grid Array
Pointers to buckets
0 2 7 10 15
Intervals Define
Search Areas
A1
Bucket Accessed
@Copyright Michael Unwalla: A mixed transaction
cost model for coarse grained multi-column
partitioning in a shared-nothing database machine
16 / 36

34. Problems Solved By Algorithms
Synthetic Biology
Pattern Recognition
Databases
Face Recognition
Person Identification
Facial Recognition measure face landmarks
to indentify different features in the face.
@Copyright 2013 Wouter Alberda, Olaf Kampinga
16 / 36

35. Outline
1 Motivation
What is an Algorithm?
Instance of a Problem
Kolmogorov’s Definition
2 Problems Solved By Algorithms
What Problems Are Solved By Algorithms?
3 Syllabus
Subjects for the Class
4 Some Notes in Notation
Before Anything, The Notation for Pseudo-Code
5 What abstraction of a Computer to use?
The Random-Access Machine (RAM) Machine
6 Analyzing Algorithms
How Input Size affects Running Time
The First Method: Counting Number of Operations
Counting Equation For Insertion Sort
The Analysis of the Best and Worst Case Inputs
17 / 36

36. What will you learn in this class?
1 Growth Functions
2 Solving
Recursions
3 Probabilistic
Analysis
4 Sorting
5 Median and
Order Statistics
6 Review of Basic
Data Structures
7 Advanced Data
Structures
8 Advanced
Techniques
Basic Complexities
Asymptotic Notation - Ω, O and
Θ
Standard notation and common
functions.
18 / 36

37. What will you learn in this class?
1 Growth Functions
2 Solving
Recursions
3 Probabilistic
Analysis
4 Sorting
5 Median and
Order Statistics
6 Review of Basic
Data Structures
7 Advanced Data
Structures
8 Advanced
Techniques
For finding Complexities
The substitution method.
The recursive three method.
The master method.
18 / 36

38. What will you learn in this class?
1 Growth Functions
2 Solving
Recursions
3 Probabilistic
Analysis
4 Sorting
5 Median and
Order Statistics
6 Review of Basic
Data Structures
7 Advanced Data
Structures
8 Advanced
Techniques
Using Probability for finding
Complexities
Indicator Random Variables.
Randomization Algorithms.
18 / 36

39. What will you learn in this class?
1 Growth Functions
2 Solving
Recursions
3 Probabilistic
Analysis
4 Sorting
5 Median and
Order Statistics
6 Review of Basic
Data Structures
7 Advanced Data
Structures
8 Advanced
Techniques
Sorting Thecniques
Heapsort.
Quicksort.
Sorting in linear time.
18 / 36

40. What will you learn in this class?
1 Growth Functions
2 Solving
Recursions
3 Probabilistic
Analysis
4 Sorting
5 Median and
Order Statistics
6 Review of Basic
Data Structures
7 Advanced Data
Structures
8 Advanced
Techniques
Finding Order
Minimum and Maximum.
Selection.
Worst Case Selection.
18 / 36

41. What will you learn in this class?
1 Growth Functions
2 Solving
Recursions
3 Probabilistic
Analysis
4 Sorting
5 Median and
Order Statistics
6 Review of Basic
Data Structures
7 Advanced Data
Structures
8 Advanced
Techniques
Basic Data Structures
Elementary data structures.
Hash tables.
Applications of Hash Tables
Linear Counting
Binary search trees.
18 / 36

42. What will you learn in this class?
1 Growth Functions
2 Solving
Recursions
3 Probabilistic
Analysis
4 Sorting
5 Median and
Order Statistics
6 Review of Basic
Data Structures
7 Advanced Data
Structures
8 Advanced
Techniques
Advanced Data Structures
B-Trees.
Fibonacci Heaps.
Data Structures for Disjoint
Sets.
18 / 36

43. What will you learn in this class?
1 Growth Functions
2 Solving
Recursions
3 Probabilistic
Analysis
4 Sorting
5 Median and
Order Statistics
6 Review of Basic
Data Structures
7 Advanced Data
Structures
8 Advanced
Techniques
Advanced Techniques
Dynamic Programming.
Greedy Algorithms.
Amortized Analysis.
Introduction to Analytic
Combinatronic
18 / 36

44. What will you learn in this class?
1 Graph Algorithms
2 Selected Topics
3 NP-Completeness
4 Approximation
Algorithm
Graph Algorithms
Elementary Graph Algorithms.
Minimum Spanning Trees.
Strongly Connected
components.
Single-Source Shortest Paths.
All-Pairs Shortest Paths.
19 / 36

45. What will you learn in this class?
1 Graph Algorithms
2 Selected Topics
3 NP-Completeness
4 Approximation
Algorithm
Applications
Multithreaded Algorithms.
String Matching.
Computational Geometry
19 / 36

46. What will you learn in this class?
1 Graph Algorithms
2 Selected Topics
3 NP-Completeness
4 Approximation
Algorithm
The Hard Problem
Encodings.
Polynomial Time Verification.
Polynomial Reduction.
NP-Hard.
NP-Complete proofs.
A family of NP-Problems
19 / 36

47. What will you learn in this class?
1 Graph Algorithms
2 Selected Topics
3 NP-Completeness
4 Approximation
Algorithm
Dealing with NP
If almost everything is NP, What to
do?
Backtracking
Branch and Bound
Approximation Algorithms
19 / 36

48. Now, what we are going to look at!!!
First
Some stuff about notation!!!
Second
What abstraction of a Computer to use?
Third
A first approach to analyzing algorithms!!!
20 / 36

49. Now, what we are going to look at!!!
First
Some stuff about notation!!!
Second
What abstraction of a Computer to use?
Third
A first approach to analyzing algorithms!!!
20 / 36

50. Now, what we are going to look at!!!
First
Some stuff about notation!!!
Second
What abstraction of a Computer to use?
Third
A first approach to analyzing algorithms!!!
20 / 36

51. Outline
1 Motivation
What is an Algorithm?
Instance of a Problem
Kolmogorov’s Definition
2 Problems Solved By Algorithms
What Problems Are Solved By Algorithms?
3 Syllabus
Subjects for the Class
4 Some Notes in Notation
Before Anything, The Notation for Pseudo-Code
5 What abstraction of a Computer to use?
The Random-Access Machine (RAM) Machine
6 Analyzing Algorithms
How Input Size affects Running Time
The First Method: Counting Number of Operations
Counting Equation For Insertion Sort
The Analysis of the Best and Worst Case Inputs
21 / 36

52. Please follow these simple rules
Insertion Sort(A)
1 for j ← 2 to length(A)
2 do
3 key ← A[j]
4 Insert A[j] into the sorted sequence A[1, ..., j − 1]}
5 i ← j − 1
6 while i > 0 and A[i] > key
7 do
8 A[i + 1] ← A[i]
9 i ← i − 1
10 A[i + 1] ← key
Rule
Always put the name of the algorithm at the top.
Together with the input.
22 / 36

53. Please follow these simple rules
Insertion Sort(A)
1 for j ← 2 to length(A)
2 do
3 key ← A[j]
4 Insert A[j] into the sorted sequence A[1, ..., j − 1]}
5 i ← j − 1
6 while i > 0 and A[i] > key
7 do
8 A[i + 1] ← A[i]
9 i ← i − 1
10 A[i + 1] ← key
Rule
Always initialize all the variables.
The a ← b means that the value b is passed to a. You also can use
"=."
22 / 36

54. Please follow these simple rules
Insertion Sort(A)
1 for j ← 2 to length(A)
2 do
3 key ← A[j]
4 Insert A[j] into the sorted sequence A[1, ..., j − 1]}
5 i ← j − 1
6 while i > 0 and A[i] > key
7 do
8 A[i + 1] ← A[i]
9 i ← i − 1
10 A[i + 1] ← key
Rule
Use identation to preserve the block structure avoiding clutter
22 / 36

55. Please follow these simple rules
Insertion Sort(A)
1 for j ← 2 to length(A)
2 do
3 key ← A[j]
4 Insert A[j] into the sorted sequence A[1, ..., j − 1]}
5 i ← j − 1
6 while i > 0 and A[i] > key
7 do
8 A[i + 1] ← A[i]
9 i ← i − 1
10 A[i + 1] ← key
Rule
It corresponds to comments. You can also use "//"
22 / 36

56. Outline
1 Motivation
What is an Algorithm?
Instance of a Problem
Kolmogorov’s Definition
2 Problems Solved By Algorithms
What Problems Are Solved By Algorithms?
3 Syllabus
Subjects for the Class
4 Some Notes in Notation
Before Anything, The Notation for Pseudo-Code
5 What abstraction of a Computer to use?
The Random-Access Machine (RAM) Machine
6 Analyzing Algorithms
How Input Size affects Running Time
The First Method: Counting Number of Operations
Counting Equation For Insertion Sort
The Analysis of the Best and Worst Case Inputs
23 / 36

57. The Random-Access Machine
Definition
A Random Access Machine (RAM) is an abstract computational-machine
model identical to a multiple-register counter machine with the addition of
indirect addressing.
Instructions
Instructions are executed one after another.
It contains arithmetic instructions found in low level languages.
It has control instructions: Conditional and unconditional branches,
return and call functions.
It is able to do data movement: load, store, copy.
It posses data types: integer and floating point.
Memory Model
A single block of memory is assumed.
24 / 36

58. The Random-Access Machine
Definition
A Random Access Machine (RAM) is an abstract computational-machine
model identical to a multiple-register counter machine with the addition of
indirect addressing.
Instructions
Instructions are executed one after another.
It contains arithmetic instructions found in low level languages.
It has control instructions: Conditional and unconditional branches,
return and call functions.
It is able to do data movement: load, store, copy.
It posses data types: integer and floating point.
Memory Model
A single block of memory is assumed.
24 / 36

59. The Random-Access Machine
Definition
A Random Access Machine (RAM) is an abstract computational-machine
model identical to a multiple-register counter machine with the addition of
indirect addressing.
Instructions
Instructions are executed one after another.
It contains arithmetic instructions found in low level languages.
It has control instructions: Conditional and unconditional branches,
return and call functions.
It is able to do data movement: load, store, copy.
It posses data types: integer and floating point.
Memory Model
A single block of memory is assumed.
24 / 36

60. The Random-Access Machine
Definition
A Random Access Machine (RAM) is an abstract computational-machine
model identical to a multiple-register counter machine with the addition of
indirect addressing.
Instructions
Instructions are executed one after another.
It contains arithmetic instructions found in low level languages.
It has control instructions: Conditional and unconditional branches,
return and call functions.
It is able to do data movement: load, store, copy.
It posses data types: integer and floating point.
Memory Model
A single block of memory is assumed.
24 / 36

61. The Random-Access Machine
Definition
A Random Access Machine (RAM) is an abstract computational-machine
model identical to a multiple-register counter machine with the addition of
indirect addressing.
Instructions
Instructions are executed one after another.
It contains arithmetic instructions found in low level languages.
It has control instructions: Conditional and unconditional branches,
return and call functions.
It is able to do data movement: load, store, copy.
It posses data types: integer and floating point.
Memory Model
A single block of memory is assumed.
24 / 36

62. The Random-Access Machine
Definition
A Random Access Machine (RAM) is an abstract computational-machine
model identical to a multiple-register counter machine with the addition of
indirect addressing.
Instructions
Instructions are executed one after another.
It contains arithmetic instructions found in low level languages.
It has control instructions: Conditional and unconditional branches,
return and call functions.
It is able to do data movement: load, store, copy.
It posses data types: integer and floating point.
Memory Model
A single block of memory is assumed.
24 / 36

63. The Random-Access Machine
Definition
A Random Access Machine (RAM) is an abstract computational-machine
model identical to a multiple-register counter machine with the addition of
indirect addressing.
Instructions
Instructions are executed one after another.
It contains arithmetic instructions found in low level languages.
It has control instructions: Conditional and unconditional branches,
return and call functions.
It is able to do data movement: load, store, copy.
It posses data types: integer and floating point.
Memory Model
A single block of memory is assumed.
24 / 36

64. RAM Model
We have that
Control System
Memory With Random Access
Infinite
Register 1
Register 2
Register N
Inst
Inst
Inst
Inst
Inst
Inst
Inst
Inst
25 / 36

65. Although there are other equivalent models
Von Neumann architecture scheme
Arithmetic
Logic
Unit
Control
Unit
Memory
Input Output
Accumulator
26 / 36

66. Outline
1 Motivation
What is an Algorithm?
Instance of a Problem
Kolmogorov’s Definition
2 Problems Solved By Algorithms
What Problems Are Solved By Algorithms?
3 Syllabus
Subjects for the Class
4 Some Notes in Notation
Before Anything, The Notation for Pseudo-Code
5 What abstraction of a Computer to use?
The Random-Access Machine (RAM) Machine
6 Analyzing Algorithms
How Input Size affects Running Time
The First Method: Counting Number of Operations
Counting Equation For Insertion Sort
The Analysis of the Best and Worst Case Inputs
27 / 36

67. Input Size and Running Time
Definition
The Input Size depends on the type of problem. We will indicate which
input size is used per problem.
Definition
The Running Time of an algorithm is the number of of primitives
operations or steps executed. For now, we will assume that each line in an
algorithm takes ci a constant time.
28 / 36

68. Input Size and Running Time
Definition
The Input Size depends on the type of problem. We will indicate which
input size is used per problem.
Definition
The Running Time of an algorithm is the number of of primitives
operations or steps executed. For now, we will assume that each line in an
algorithm takes ci a constant time.
28 / 36

69. Even Babbage cared about how many turns of the crank
were necessary!!!
Look at the crank!!!
29 / 36

70. Outline
1 Motivation
What is an Algorithm?
Instance of a Problem
Kolmogorov’s Definition
2 Problems Solved By Algorithms
What Problems Are Solved By Algorithms?
3 Syllabus
Subjects for the Class
4 Some Notes in Notation
Before Anything, The Notation for Pseudo-Code
5 What abstraction of a Computer to use?
The Random-Access Machine (RAM) Machine
6 Analyzing Algorithms
How Input Size affects Running Time
The First Method: Counting Number of Operations
Counting Equation For Insertion Sort
The Analysis of the Best and Worst Case Inputs
30 / 36

71. Counting the Operations
Insertion Sort(A) length(A):=N
1 for j← 2 to length(A)
2 do
3 key ← A[j]
4 Insert A[j] into the sorted sequence A[1, ..., j − 1]}
5 i ← j − 1
6 while i > 0 and A[i] > key
7 do
8 A[i + 1] ← A[i]
9 i ← i − 1
10 A[i + 1] ← key
Count Value
c1N
31 / 36

72. Counting the Operations
Insertion Sort(A) length(A):=N
1 for j← 2 to length(A)
2 do
3 key ← A[j]
4 Insert A[j] into the sorted sequence A[1, ..., j − 1]}
5 i ← j − 1
6 while i > 0 and A[i] > key
7 do
8 A[i + 1] ← A[i]
9 i ← i − 1
10 A[i + 1] ← key
Count Value
c2(N − 1)
31 / 36

73. Counting the Operations
Insertion Sort(A) length(A):=N
1 for j← 2 to length(A)
2 do
3 key ← A[j]
4 Insert A[j] into the sorted sequence A[1, ..., j − 1]}
5 i ← j − 1
6 while i > 0 and A[i] > key
7 do
8 A[i + 1] ← A[i]
9 i ← i − 1
10 A[i + 1] ← key
Count Value
c3(N − 1)
31 / 36

74. Counting the Operations
Insertion Sort(A) length(A):=N
1 for j← 2 to length(A)
2 do
3 key ← A[j]
4 Insert A[j] into the sorted sequence A[1, ..., j − 1]}
5 i ← j − 1
6 while i > 0 and A[i] > key
7 do
8 A[i + 1] ← A[i]
9 i ← i − 1
10 A[i + 1] ← key
Count Value
c4
N
j=2 j
31 / 36

75. Counting the Operations
Insertion Sort(A) length(A):=N
1 for j← 2 to length(A)
2 do
3 key ← A[j]
4 Insert A[j] into the sorted sequence A[1, ..., j − 1]}
5 i ← j − 1
6 while i > 0 and A[i] > key
7 do
8 A[i + 1] ← A[i]
9 i ← i − 1
10 A[i + 1] ← key
Count Value
c5
N
j=2(j − 1)
31 / 36

76. Counting the Operations
Insertion Sort(A) length(A):=N
1 for j← 2 to length(A)
2 do
3 key ← A[j]
4 Insert A[j] into the sorted sequence A[1, ..., j − 1]}
5 i ← j − 1
6 while i > 0 and A[i] > key
7 do
8 A[i + 1] ← A[i]
9 i ← i − 1
10 A[i + 1] ← key
Count Value
c6
N
j=2(j − 1)
31 / 36

77. Counting the Operations
Insertion Sort(A) length(A):=N
1 for j← 2 to length(A)
2 do
3 key ← A[j]
4 Insert A[j] into the sorted sequence A[1, ..., j − 1]}
5 i ← j − 1
6 while i > 0 and A[i] > key
7 do
8 A[i + 1] ← A[i]
9 i ← i − 1
10 A[i + 1] ← key
Count Value
c7(N − 1)
31 / 36

78. Outline
1 Motivation
What is an Algorithm?
Instance of a Problem
Kolmogorov’s Definition
2 Problems Solved By Algorithms
What Problems Are Solved By Algorithms?
3 Syllabus
Subjects for the Class
4 Some Notes in Notation
Before Anything, The Notation for Pseudo-Code
5 What abstraction of a Computer to use?
The Random-Access Machine (RAM) Machine
6 Analyzing Algorithms
How Input Size affects Running Time
The First Method: Counting Number of Operations
Counting Equation For Insertion Sort
The Analysis of the Best and Worst Case Inputs
32 / 36

79. Building a function for Analyzing Complexities
What can we do with all these quantities?
Sum them all to get a step count that can be related to computational
complexities!!!
Counting Equation
T(N) = c1N + c2(N − 1) + c3(N − 1) + c4
N(N+1)
2 − 1 + ...
c5
N(N−1)
2 + c6
N(N−1)
2 + c7 (N − 1)
This can be reduced to something like...
T(N) = aN2 + bN + c
33 / 36

80. Building a function for Analyzing Complexities
What can we do with all these quantities?
Sum them all to get a step count that can be related to computational
complexities!!!
Counting Equation
T(N) = c1N + c2(N − 1) + c3(N − 1) + c4
N(N+1)
2 − 1 + ...
c5
N(N−1)
2 + c6
N(N−1)
2 + c7 (N − 1)
This can be reduced to something like...
T(N) = aN2 + bN + c
33 / 36

81. Building a function for Analyzing Complexities
What can we do with all these quantities?
Sum them all to get a step count that can be related to computational
complexities!!!
Counting Equation
T(N) = c1N + c2(N − 1) + c3(N − 1) + c4
N(N+1)
2 − 1 + ...
c5
N(N−1)
2 + c6
N(N−1)
2 + c7 (N − 1)
This can be reduced to something like...
T(N) = aN2 + bN + c
33 / 36

82. Building a function for Analyzing Complexities
What can we do with all these quantities?
Sum them all to get a step count that can be related to computational
complexities!!!
Counting Equation
T(N) = c1N + c2(N − 1) + c3(N − 1) + c4
N(N+1)
2 − 1 + ...
c5
N(N−1)
2 + c6
N(N−1)
2 + c7 (N − 1)
This can be reduced to something like...
T(N) = aN2 + bN + c
33 / 36

83. Outline
1 Motivation
What is an Algorithm?
Instance of a Problem
Kolmogorov’s Definition
2 Problems Solved By Algorithms
What Problems Are Solved By Algorithms?
3 Syllabus
Subjects for the Class
4 Some Notes in Notation
Before Anything, The Notation for Pseudo-Code
5 What abstraction of a Computer to use?
The Random-Access Machine (RAM) Machine
6 Analyzing Algorithms
How Input Size affects Running Time
The First Method: Counting Number of Operations
Counting Equation For Insertion Sort
The Analysis of the Best and Worst Case Inputs
34 / 36

84. The Worst and The Average
1 The Worst Case
2 The Average
Case
Worst Sequence
Upper bound on the running
time of an algorithm.
In case of insertion sort, it will
be the permutation:
N, N − 1, N − 2, ..., 3, 2, 1
35 / 36

85. The Worst and The Average
1 The Worst Case
2 The Average
Case
Average Sequence
In the case of insertion sort,
when half of the elements of
A[1, 2, ..., j − 1] are less than
A[j] and half are greater.
Then, there is only the need to
check half of the elements i.e.
j
2.
35 / 36

86. WHY TO LOOK FOR EFFICIENT ALGORITHMS?
Example
1 Insertion Sort
2 Merge Sort
3 Constraints
for the
example
4 Final Result
Example
T(N) = c1N2
36 / 36

87. WHY TO LOOK FOR EFFICIENT ALGORITHMS?
Example
1 Insertion Sort
2 Merge Sort
3 Constraints
for the
example
4 Final Result
Example
T(N) = c2N log2 N
36 / 36

88. WHY TO LOOK FOR EFFICIENT ALGORITHMS?
Example
1 Insertion Sort
2 Merge Sort
3 Constraints
for the
example
4 Final Result
Example
Assume, we have 106 numbers to
sort, c1 = 2 for a supercomputer
and c2 = 50 for a PC.
The supercomputer can do 1010
instructions per second.
The PC can do 107 instructions per
second.
36 / 36

89. WHY TO LOOK FOR EFFICIENT ALGORITHMS?
Example
1 Insertion Sort
2 Merge Sort
3 Constraints
for the
example
4 Final Result
Example
Insertion Sort in the Supercomputer
2(106)2 ins
1010 ins/sec
= 200sec
Merge Sort in the Humble PC
2(106) log(106) ins
107 ins/sec
= 3.9sec
36 / 36