Lecture 2 role of algorithms in computing

1. Lecture 2 : Role of Algorithms in
Computing
Jayavignesh T
Asst Professor
SENSE

2. What is an Algorithm?
• An algorithm is a set of rules for carrying out calculation
either by hand or on a machine.
• An algorithm is a finite step-by-step procedure to
achieve a required result.
• An algorithm is a sequence of computational steps that
transform the input into the output.
• An algorithm is a sequence of operations performed on
data that have to be organized in data structures.
• An algorithm is an abstraction of a program to be
executed on a physical machine (model of
Computation).

3. What is an Algorithm contd..?
• An algorithm is a sequence of unambiguous
instructions for solving a problem,
– i.e., for obtaining a required output for any
legitimate input in a finite amount of time

4. Requirements
• Recipe, process, method, technique, procedure, routine,… with
following requirements:
1. Finiteness
– terminates after a finite number of steps
2. Definiteness
– rigorously and unambiguously specified
3. Input
– valid inputs are clearly specified
4. Output
– can be proved to produce the correct output given a valid
input
5. Effectiveness
– steps are sufficiently simple and basic

5. Design and Analysis
• Algorithmic is a branch of computer science that
consists of designing and analyzing computer
algorithms
• The “design” pertain to
– The description of algorithm at an abstract level
by means of a pseudo language, and
– Proof of correctness that is, the algorithm solves
the given problem in all cases.
• The “analysis” deals with performance
evaluation(complexity analysis).

6. Design and Analysis
• It is not depended on programming language,
machine.
• Are mathematical entities, which can be thought of
as running on some sort of idealized computer with
an infinite random access memory
• Algorithm design is all about the mathematical
theory behind the design of good programs.

7. Why Analyze?
• Why analyze algorithms?
– Evaluate Algorithm performance
– Compare Different Algorithms
• Analyze what about them?
– Running time, Memory usage, Solution quality
– Worst-case and “typical” case
• Computational complexity
– Understanding the intrinsic difficulty of computational
problems – Classifying problems according to difficulty
– Algorithms provide upper bound to show problem is
hard, must show that any algorithm to solve it requires at
least a given amount of resources

8. Algorithm Problem solving
Understand the problem
Decision Making
Design an Algorithm
Prove Correctness
Analyze Algorithm
Code the Algorithm

9. Understand the problem
• Necessary Inputs to solve the
problem?
• Input – Instances
• Decide range of inputs
• To Fix Boundary values
• Work correctly for all Valid inputs

10. Decision Making
• Analyze the required inputs and decide on
– Computational Means
– Exact vs Approximate Solving
– Data Structures
– Algorithm Design Technique
 Computational Means
 Computational capability of running devices
 Sequential Algorithm ( Random Access Machine RAM)
 Parallel Algorithm (Piece executed on multiple processing
devices and put back together to get correct result)
 Proper choice of Computational device (Space/Time
efficient)

11. Decision Making contd..
• Exact vs Approximate Solving
• Exact Algorithm
– Problem needs to be solved exactly or correctly
• Approximation Algorithm
– Solve a complex problem, we will not get exact
solution
– Ex : Travelling salesman problem
• Data Structures
– Algorithm + Data Structures – Programming Language
– Proper Choice of Data Structures required before
designing actual algorithm

12. Algorithm Design Technique
• Strategy or Paradigm
– General approach to solve program
algorithmically
– Different problems from Different areas of
computing
• Brute Force :
– Straight forward technique with naïve approach
• Divide and Conquer :
– Problem is divided into smaller instances

13. Algorithm Design Technique contd..
• Decrease and Conquer :
– Instance size is decreased to solve the problem
• Transform and Conquer :
– Instance is modified and then solved
• Dynamic Programming :
– Results of smaller, reoccurring instances are
obtained to solve problem
• Greedy Technique :
– Solve the problem by making locally optimal
decisions

14. Specification of Algorithm
• Natural Language
– Drawback : Specification by this means not clear
• Pseudo Code
– Mixture of Natural Language + Programming constructs
ALGORITHM Sum(a,b)
// Problem Description : This algorithm performs addition of two
numbers
// Input : Two integers a and b
// Output : Addition of two integers
c a + b
return c
• Flow Chart
– Using Geometric Shapes containing descriptions of algorithm’s steps.

15. Proving Algorithm’s Correctness
• Prove algorithm yields required results
• For Every Legitimate Input in finite amount of
time
• If one instance of valid input, the algorithm
performs incorrectly !!
– Ex : Mathematical Induction

16. Analysis of Algorithm
• Time Efficiency
– Indicates how fast algorithm runs
• Space Efficiency
– How much Extra memory the algorithm needs to
complete its execution
• Simplicity
– Generating Sequence of instructions which are easy to
understand
• Generality
– Range of inputs it can accept
– Generality of problem which algorithm can solve

17. Coding an Algorithm
• Implementation of algorithm done by
suitable programming language
– C
– C++
– Java,
– Verilog etc etc…

18. Problem Types to solve
• Sorting Algorithms
–Arranging elements in increasing
(ascending) or decreasing (descending)
order
–Numbers, Characters, Strings
–Sorting makes easier to search elements
–Ex : Dictionary, Telephone books, Class
lists, Employee database etc..

19. Problem Types to solve
• Sorting Algorithms
–Properties
• Stable Property
–Preserves the relative order of any two
equal elements. Position i , j where i < j ,
sorted list i’ and j’ such that i’ < j’
• In Place Property
–Does not require extra memory, except for
possibly a few memory unit

20. Searching
• To find the desired element from the list
• Element to be Searched – Search Key
• Sequential Search
• Binary Search etc.
• Some Search algorithms work faster than
others but
– Requires more memory
– Works only in sorted arrays etc..

21. Linear Search Algorithm
• Input: An array of n numbers and an element which is required to
be searched in the given list
• Output: Number exists in the list or not.
Algorithm:
1. Input the size of list i.e. n
2. Read the n elements of array A
3. Input the item/element to be searched in the given list.
4. for each element in the array i=1 to n
5. if A[i]==item
6. Search successful, return
7. if i==n+1
8. Search unsuccessful.
9. Stop

22. Graph Problems
• Collection of points called vertices
• Some are connected by line segments – edges
• Graph traversal algorithms, shortest path
algorithms etc…
• String Processing
– String – sequence of characters
– Text Strings = Letters, numbers, special
characters, bit strings (0 & 1)

23. Combinatorial Problems
• Computing Permutations and combinations
• Complex computing due to
– As problem size grows the combinatorial objects
grow rapidly and reach to huge value
– No algorithm available which solves these
problems in finite amount of time.
– Many fall under unsolvable problems

24. Geometric Problems
• Deal with geometric objects such as
points, lines and polygons
• Applications in Computer Graphics,
Robotics and topography

25. Time Complexity
• Amount of computer time required by an algorithm
to run on completion
• Difficult to compute time complexity in terms of
physically clocked time.
• Drawbacks of measuring running time in-terms of
seconds, millisecond etc are
– Dependence of speed of a underlying hardware
– Number of other programs running (System load)
– Dependence of compiler used in generating
machine code

26. How to calculate running time then?
• Time complexity given in terms of FREQUENCY
COUNT
• Count denoting number of times of execution of
statement.
For (i=0; i <n;i++) { // St1 : 1, St 2 : n+1 , St 3 : n times
sum = sum + a[i]; // n times
}
3n + 2 ; O(n) neglecting constants and lower order terms

27. How to calculate running time then?
for (i=0; i < n ; i ++) // 1 ; n+1 ; n times
{
for (j=0; j < n ; j ++) // n ; n(n+1) ; n(n)
{
c[i][j] = a[i][j] + b[i][j];
}
}
3n2+4n+ 2 = O(n2)

28. How to calculate running time then?
• All Algorithms run longer on larger inputs
• Algorithm’s efficiency - f(n)
• Identify the most important operations of the
algorithm – BASIC Operation
• Basic operation – contributing to most of total
running time
• Compute the number of times basic operation is
executed (mostly in inner loop)
Ex : Sorting Algorithms – Comparison (< >)
Matrix Multiplication, Polynomial evaluation – Arithmetic Operations ( *, +)
= (assignment), ==(equality) etc..

30. Order of Growth of Algorithm
• Measuring the performance of an algorithm
in relation with input size n

31. Rate of Growth of Algorithm as fn of i/p size

32. Efficiency comparisons