1.
Unix Shell Scripting

2.
Instructor
Mustafa Qasim
Certified Ethical Hacker (CEH)
Oracle Certified Expert Solaris Security Administrator
Oracle Certified Professional Solaris System Administrator
Oracle Certified Associate Solaris 10 Operating System
Oracle Certified Associate MySQL 5.5/5
Cisco Certified Network Associate (CCNA)

3.
Catch Me
LinkedIn pk.linkedin.com/in/mustafaqasim
Google+ gplus.to/mustu
Twitter twitter.com/mustafaqasim

4.
Introduction
●
Name
●
Job Description
●
Professional Experience
●
Expectations from Unix Shell Scripting course

5.
Ideal Participants
Hardworking
?

6.
Ideal Participants
?
Hardworking
ŊÖ

7.
Ideal Participants
LaaazzZZzzyyy....

8.
Let's Start...

9.
Unix
●
AT&T Employees at Bell Labs in 1969
●
Ken Thompson, Dennis Ritchie, Brian Kernighan, Douglas McIlroy,
Michael Lesk and Joe Ossanna
●
Unix Family OS
– University of California, Berkeley's BSD
– Solaris
– HP-UX
– AIX
– Sequent and as well as Darwin

10.
GNU
●
GNU Not Unix
●
Richard Stallman, 1983
●
GNU General Public License (GPL)
●
GNU Compiler Collection (GCC), the GNU C library (glibc), and
GNU Core Utilities (coreutils),[1] but also the GNU Debugger
(GDB), GNU Binary Utilities (binutils),[18] and the bash shell.
●
GNU Hurd Kernel
●
Free Software Foundation (FSF)

11.
GNU/Linux
●
In April 1991, Linus Torvalds, a 21-year-old
student at the University of Helsinki, Finland
started working on some simple ideas for an
operating system
●
Kernel vs OS
Linus Torvalds

12.
To Do
Make yourself familiar with Unix, Unix Derivatives, GNU, Linux and
software license GPL and Free & Open Source Software (FOSS).
http://en.wikipedia.org/wiki/Unix
http://www.gnu.org/philosophy/philosophy.html
http://en.wikipedia.org/wiki/GNU
http://www.gnu.org/gnu/why-gnu-linux.html
http://en.wikipedia.org/wiki/Linux

13.
To Do
Make yourself familiar with Unix, Unix Derivatives, GNU, Linux and
software license GPL and Free & Open Source Software (FOSS).
http://en.wikipedia.org/wiki/Unix
http://www.gnu.org/philosophy/philosophy.html
http://en.wikipedia.org/wiki/GNU
http://www.gnu.org/gnu/why-gnu-linux.html
http://en.wikipedia.org/wiki/Linux

14.
So, Where comes the Shell?

15.
Shell?

16.
Shell?

17.
Compiler vs Interpreted
Complete Program > Compiler > Executable
Program Fragment > Interpreter > Instant Output

18.
Compiler vs Interpreted (Contd.)
●
Compiled Languages
– C/C++
– Pascal
– FORTRAN
●
Interpreted Languages
– Unix Shell
– Perl
– Tcl/Tk

19.
Unix Shell
●
First shell for Version 6 Unix was written by Ken Thompson
(Bell Labs) in 1971
●
It was only Interactive Shell

20.
Bourne Shell
●
Created by Stephen Bourne at AT&T Bell Labs for V7 UNIX
●
Goals
– Interpreted Language
– Scripting
●
New Features
– Control Flows, Loops, Variables
●
Lacked Functions

21.
Bourne Shell (Contd.)
●
Bourne Shell Derivatives
– Korn Shell (ksh)
– Almquist shell (ash)
– Bourne Again Shell (Bash)

22.
C Shell (csh)
●
Bill Joy in 1978 for BSD
●
Create a scripting language similar to C
●
Prominent Feature
– Command History
●
Tenex Enhancements (tcsh)
– Command completion
– Command line editing

23.
Korn Shell (ksh)
●
By David Korn
●
Backward compatibility with Bourne Shell (sh)
●
Derivative of Bourne Shell (sh)

24.
Bourne Again Shell (Bash)
●
GNU Project
●
Superset of sh and ksh

25.
POSIX
●
Institute of Electrical and Electronics Engineering (IEEE)
●
International Organization for Standardization (ISO)
●
POSIX Compliance
●
Bash, gawk

26.
Responsibilities of Shell
1. Reading input and parsing the command line
2. Evaluating special characters, such as wildcards and the
history character
3. Setting up pipes, redirection, and background processing
4. Handling signals
5. Setting up programs for execution

27.
To Do
●
Perform Login and Logout on a Unix shell
●
Perform and understand the following commands
– ls, dir, touch, mkdir, cd, rm, mv, cat, less, more, which, pwd, man,
info
●
Use vi editor to create and edit files
●
Understand Unix File System hierarchy especially the
following directories
– /bin, /sbin, /home, /root, /var

28.
Unix Startup
●
OS is booted and very first process is born
– Init, PID 1
●
OS authenticates the user and provides access to shell
– /bin/login
●
Data Streams are set
– Stdout, stderr, stdin
●
A process is forked to run Graphical Desktop

29.
Parsing Commands
●
Lexical Analysis
– The process of breaking the line up into tokens is called lexical
analysis.
●
Tokens
●
Command?
– Built-in command => Execute Internally
– Executable command => Fork a Child Shell
●
Forking Child Shel

30.
Types of Commands
1. Aliases
2. Keywords
3. Functions
4. Built-in commands
5. Executable programs

31.
Shell Processes
●
PID
●
ps
●
Linux: pstree
●
Solaris: ptree

32.
System Calls
}
fork
exit
Kernel
exec
wait

33.
fork System Call
●
Parent/Child
●
Inheritance of environment, open files, real and user
identifications, umask, current working directory, and
signals.

34.
fork System Call

35.
wait System Call
●
Waits for Child's PID & Exit Code
●
Zombie Process
●
Orphan Zombie Process

36.
exec System Call
●
exec call for executable
●
Executable becomes the child

37.
exit System Call
●
sigchild signal to parent
●
Exit code 0 - 255

38.
To Do
●
Perform and Understand the following commands
– top, ps, pgrep, pstree/ptree, kill
●
Learn to hunt a specific process and kill it
●
Learn to hunt a specific instance of a binary running multiple
times simultaneous

39.
Environment and Inheritance
●
UID, GID
●
Ownership
●
Permission
●
Variables
●
Working Directory

40.
User & Group IDs
$ id
uid=502(mustu) gid=502(mustu)

41.
File Creation Mask
●
Umask
777 (Directory) 666 (File)
–022 (umask value) –022 (umask value)
------- ---------
755 644
drwxr-xr-x -rw-r--r--

42.
Set Ownership & Permission
●
chown
●
chmod

43.
Variables
●
Local Variables
●
Environment Variables
– $env

44.
File Descriptor
●
0 stdin
●
1 stdout
●
2 stderr

45.
I/O Redirection

46.
I/O Redirection

47.
I/O Redirection

48.
Pipe

49.
Signals

50.
To Do
●
Understand & Practice the following
– Bash Environmental variables
– Check your current environment variables (hint: $env)
– Check your user's UID & GID
– Check a file's permission set for owner, group and others
– Change a file/folder permission using both numerical and character format
– Change a file/folder owner and group
– Redirect stdout and stderr into a file
– Pipe or Redirect output of programs

51.
Let's write a Bash script ;-)

52.
Writing Bash Script
●
Shebang
– #!/bin/bash
●
Comments
– #Getting input for value of age
●
Make script executable
– Chmod +x myscript.sh
●
Execute Script
– ./myscript.sh

53.
Simple Bash Script
#!/bin/bash
# This is a comment
echo “Khamoshi Ka Boycot”

54.
Use Variable
#!/bin/bash
name=”Telenor”
echo “$name Khamoshi Ka Boycot”

55.
Pass Arguments
./myscript ali 15 Lahore
#!/bin/bash
echo “Mr. $1 is $2 years old and belongs to $3”

56.
Get Input
#!/bin/bash
echo "What is your name?"
read uname
echo ” Welcome $uname”

57.
Arithmetic Operators
== Equal to
!= Not equal to
> Greater then
>= Greater then, equal to
< Less then
<= Less then, equalt o

58.
Logical Operators
&& and
|| or
! not

59.
Conditional Statements
if Construct:
if command
then
block of statements
fi

60.
Conditional Statements
if/else Construct:
if command
then
block of statements
else
block of statements
fi

61.
Conditional Statements
if/else if/else Construct:
if command
then
block of statements
elif command
then
block of statements
else
block of statements
fi

62.
Iterative Statements
for variable in word_list
do
block of statements
done

63.
Schedule Script
●
Cron
– crontan -l #List Exiting Crons
– crontab -e #Edit Cron Scheduling File

64.
REMEMBER
●
Environment Variable Differ in Cron
●
Use the MAIL parameter to receive cron errors
●
Use the PATH parameter to set environment variables in cron
●
If your program needs to use neighbor files to run properly
then in cron script first change directory (cd) to your
program's path then run the program.

65.
Tip :)
Your best buddy
http://www.corntab.com/

66.
Regular Expressions
●
Pattern of characters used to match the same characters in a
search
●
Enclosed by forward slash /telenor/

67.
Regular Expressions
Hi Asad,
We've a upcoming training on Unix shell Programming later this week.
As you've to deal with programming tasks frequently I would highly
recommend you to participate in this training. It will improve your
programming skills and help you perform daily tasks efficiently.
Thanks

68.
Regular Expressions
:1,$s/programming/scripting/g

69.
Regular Expression Metacharacters
:1,$s/<[Pp]rogramming>/scripting/g
1,$ First Line to Last Line
s Substitute
<, > Beginning and End of Word
[Pp] Match one of the word
<[Pp]rogramming> Programming OR programming
g Globally

70.
Regular Expression Metacharacters
Meta Character
^
Function
Beginning of line anchor
Example
/^lahore/
What It Matched
Matches all lines beginning with lahore

71.
Regular Expression Metacharacters
Meta Character
$
Function
End of line anchor
Example
/lahore$/
What It Matched
Matches all lines ending with lahore

72.
Regular Expression Metacharacters
Meta Character
.
Function
Matches one character
Example
/la...e/
What It Matched
Matches all starting with la, followed by three characters, followed by e

73.
Regular Expression Metacharacters
Meta Character
*
Function
Matches zero or more of the preceding characters
Example
/ *lahore/
What It Matched
Matches lines with zero or more spaces, followed by the pattern lahore

74.
Regular Expression Metacharacters
Meta Character
[]
Function
Matches one in the set
Example
/[Ll]ahore/
What It Matched
Matches lines containing Lahore or lahore

75.
Regular Expression Metacharacters
Meta Character
[x-y]
Function
Matches one character within a range in the set
Example
/[A-Z]ahore/
What It Matched
Matches letters from A through Z followed by ahore

76.
Regular Expression Metacharacters
Meta Character
[^]
Function
Matches one character not in the set
Example
/[^A-Z]ahore/
What It Matched
Matches any character not in the range between A and Z

77.
Regular Expression Metacharacters
Meta Character
Function
Used to escape a metacharacter
Example
/lahore./
What It Matched
Matches lines containing lahore, followed by a literal period; Normally the period matches one of any
character

78.
Regular Expression Metacharacters
Meta Character
<
Function
Beginning-of-word anchor
Example
/<lahore/
What It Matched
Matches lines containing a word that begins with lahore (supported by vi and grep)

79.
Regular Expression Metacharacters
Additional
Meta Character
>
Function
Ending-of-word anchor
Example
/lahore>/
What It Matched
Matches lines containing a word that ends with lahore (supported by vi and grep)

80.
Regular Expression Metacharacters
Additional
Meta Character
(..)
Function
Tags match characters to be used later
Example
/(love)able 1er/
What It Matched
May use up to nine tags, starting with the first tag at the leftmost part of the pattern. For example, the pattern love
is saved as tag 1, to be referenced later as 1. In this example, the search pattern consists of lovable followed by lover
(supported by sed, vi, and grep)

81.
Regular Expression Metacharacters
Additional
Meta Character
(..)
Function
Tags match characters to be used later
Example
/(love)able 1er/
What It Matched
May use up to nine tags, starting with the first tag at the leftmost part of the pattern. For example, the pattern love
is saved as tag 1, to be referenced later as 1. In this example, the search pattern consists of lovable followed by lover
(supported by sed, vi, and grep)

82.
Regular Expression Metacharacters
Additional
Meta Character
x{m} or x{m,} or x{m,n}
Function
Repetition of character x, m times, at least m times, at least m and not more than n times
Example
o{5,10}
What It Matched
Matches if line contains between 5 and 10 consecutive occurrences of the letter o (supported by vi and grep)

83.
Sed
●
Streamlined, non-interactive editor

84.
How Sed Works
abc abc abc
Text File Sed Buffer Sed Output

85.
Awk
●
UNIX programming language used for manipulating data and
generating reports

86.
How Awk Works
$0
Ahmad 33 LHR
Shakir 35 ISB Ahmad 33 LHR
Qasim 28 KHI
Awk takes a line and put in an internal variable $0
Text File
$1 $2 $3
Ahmad 33 LHR
Line is broken into fields separated by spaces and stored
into internal numbered variables starting from $1

87.
How Awk Works
Ahmad 33 LHR
Shakir 35 ISB
Qasim 28 KHI
data.txt
#awk '{print $1, $3}' data.txt
Ahmad LHR
Shakir ISB
Qasim KHI

88.
To Do
●
Practice Regular Expressions with grep, sed and awk.