3. Outline
● Introduction to linux
● Using essential tools
● Essential File Management Tools
● Working with text files
● Connecting to server
● User and Group management
● Configuring Permissions
● Network fundamentals
● Managing Processes
● Managinx Software
● Scheduling tasks
● Configuring Logging
● Managing Partitions
● Managing LVM Logical Volumes
6. Introduction on Linux Operating System
● Linux History
○ AT&T - Unix (not free source code (1989))
○ Richard Stallman - Free Software Foundation (FSF)
○ GNU (1990) - “GNU’s Not Unix!”
■ Kernel (?)
■ Applications (GCC, …)
■ License: GPL (General Public License) - You can share, access to source code and modify and
redistribute it (MUST BE in GPL license and open source)
○ Tanenbaum - minix
○ Linus Torvalds - Linux (1991)
○ Berkeley - BSD
○ Linux license is GPL but BSD is public domain :(
○ BSD used in many operating system or product like ‘Apple’, ‘VMware’, ...
7. Introduction on Linux Operating System
● GNU/Linux distributions
○ Distro: GNU (Applications) and Linux (Kernel)
○ Debian, RedHat (RHEL), Slackware
■ Debian: deb base
■ RedHat: rpm base
■ Slackware: source base
○ Fedora, Centos, Ubuntu, Mint, Suse, …
○ Some Enterprise Linux (It’s company updates it continuously)
○ LFS (Linux From Scratch)
8. Introduction on Linux Operating System
● What is an Operating System?
○ A software that manages all of the hardware
resources associated with your desktop or
laptop
○ It manages the communication between
your software and your hardware
9. Linux Architecture
● Kernel space is where the kernel (i.e., the core of the
operating system) runs and provides its services. Its
something that the user is not allowed to interfere with.
● User space is that portion of system memory in which
user processes run. The irony is that even those
processes are managed by the kernel. ;)
12. Executing Commands
● The purpose of the Linux shell is that it provides an environment in which
commands can be executed.
● shell makes a difference between three kinds of commands:
○ Aliases
■ alias newcommand=’old command’
■ alias ll=’ls -l --color=auto’
○ Internal commands
■ a command that is a part of the shell itself.
○ External commands
■ a command that exists as an executable file on disk of the computer.
13. Understanding the Shell Environment
● When you are working from a shell, an environment is created to ensure
that all that is happening is happening the right way.
14. Shell Environment (variables)
● The Linux shell environment consists of many variables.
● Variables are fixed names that can be assigned dynamic values.
15. Shell Environment (Configuration Files)
When a user logs in, an environment is created for that user automatically.
This happens based on four different files and where variables can be defined
for use by one specific user:
● /etc/profile: This is the generic file that is processed by all users upon
login.
● /etc/bashrc: This file is processed when subshells are started.
● ~/.bash_profile: In this file, user-specific login shell variables can be
defined.
● ~/.bashrc: In this user-specific file, subshell variables can be defined.
17. File System Hierarchy
/ The root directory. This is where the file system tree starts.
/bin In here, you find executable programs that are needed to repair a system
in a minimal troubleshooting mode. This directory is essential during boot.
/boot Contains all files and directories that are needed to boot the Linux kernel.
/dev Device files that are used for accessing physical devices. This directory is
essential during boot.
/ect Contains configuration files that are used by programs and services that
are used on your server. This directory is essential during boot.
/home Used for local user home directories.
/lib, /lib64 Shared libraries that are used by programs in /boot, /bin and /sbin.
/media/, /mnt Directories that are used for mounting devices in the file system tree.
18. File System Hierarchy
/opt This directory is used for optional packages that may be installed on your
server.
/proc This directory is used by the proc file system. This is a file system structure that
gives access to kernel information.
/root The home directory of the root user.
/run Contains process and user specific information that has been created since the last
boot.
/sbin Like /bin, but for system administration commands that are not necessarily needed
by regular users.
/sys Used as an interface to different hardware devices that is managed by the Linux
kernel and associated processes.
/tmp Contains temporary files that may be deleted without any warning during boot.
19. File System Hierarchy
/usr Directory that contains subdirectories with program files, libraries for
these program files and documentation about them. Typically, many
subdirectories exist in this directory that mimic the contents of the /
directory. The contents of /usr are not required during boot.
/var Directory that contains files which may change in size dynamically, such as
log files, mail boxes, and spool files.
20. Basic exploration commands ( mount )
● Mount: The mount command gives an overview of all mounted devices.
● You can use ( $ cat /proc/mount ) too ;)
22. Managing Files
● Most common file management task should be done by server admin:
○ Working with wildcards
○ Managing and working with directories
○ Working with absolute and relative pathnames
○ Listing files and directories
○ Copying files and directories
○ Moving files and directories
○ Deleting files and directories
23. Working with Wildcards
● A wildcard is a shell feature that helps you referring to multiple files in an
easy way.
Wildcard Use
* Refers to an unlimited number of all characters. ls * , for instance, shows all files
in the current directory (except those that have a name starting with a dot).
? Used to refer to one specific character that can be any character. ls c?t would
match cat as well as cut.
[auo] Refers to one character that may be selected from the range that is specified
between square brackets. ls c[auo]t would match cat, cut, and cot.
24. Working with Absolute and Relative Pathnames
● An absolute filename, or absolute pathname, is a complete path
reference to the file or directory you want to work with:
○ /usr/bin/python2.7
● A relative filename is relative to the current directory as shown with the
pwd command
○ $ pwd
■ /home/
○ $ cd liza is the same as $cd /home/liza
25. Listing Files and Directories ( ls )
ls -l Shows a long listing, which includes information about file properties, such
as creation date and permissions.
ls -a Shows all files, including hidden files.
ls -lrt This is a very useful command. It shows commands sorted on modification
date. You’ll see the most recently modified files last in the list.
ls -d Shows the names of directories, not the contents of all directories that
match the wildcards that have been used with the ls command.
26. Copying Files ( cp )
● Copy single file:
○ $ cp /path/to/file /path/to/destination
○ $ cp /etc/hosts /tmp
● Copy and entire subdirectory (copy directories recursively) :
○ Cp -R /etc/ /tmp
● Copy with exact permissions from home to /tmp directory:
○ $ cp -aR /home/$USER /tmp
29. Editing files with vim ( vi improved )
● you often need to change the contents of a configuration file with a text
editor.
● Over the years, many text editors have been created for Linux. One editor
really matters, though, and that is vi. Even if some other text editors are
easier to use, vi is the only text editor that is always available.
● Vim uses different modes:
○ Command mode: you cannot change the contents of a text file
○ Input mode: you can edit file
30. Vim command explanation
Esc Switches from input mode to command mode. Use this before typing any command.
i, a Switches from command mode to input mode at (i) or after (a) the current cursor
position.
:wq Writes the current file and quits.
:q! Quits the file without applying any changes.
dd Deletes the current line.
yy Copies the current line.
p Pastes the current selection.
/text Searches for text from the current cursor position forward.
!ls Adds the output of ls (or any other command) in the current file.
:%s/old/new/g Replaces all occurrences of old with new .
32. Essential Tools for Managing Text File Contents
Command Explanation
less Opens the text file in a pager, which allows for easy reading of the text file
cat Dumps the contents of the text file on the screen
head Shows the first 10 lines of the text file
tail Shows the last 10 lines of the text file
cut Used to filter specific columns or characters from a text file
sort Sorts contents of a text file
wc Counts the number of lines, words, and characters in a file
33. I/O Redirection
● when a command is executed it shows its results on the screen of the
computer you are working on
● The computer monitor is the so-called standard output, which is also
referred to as the STDOUT
● The shell also has default destinations to send error (STDERR) messages
to and to accept input (STDIN)
34. Standard Input, Output, and Error Overview
Name Default destination Use in Redirection File Descriptor Number
STDIN Computer keyboard < (same as 0<) 0
STDOUT Computer monitor > (same as 1>) 1
STDERR Computer monitor 2> 2
35. Using Pipes
● Where an I/O redirector is used to use alternatives for keyboard and
computer monitor, a pipe can be used to catch the output of one
command and use that as input for a second command.
● $ <cmd1> <file> | <cmd> <file2>
● $ cat txt1 | grep ‘var’
36. less
● $ less filename
● use the PageUp and PageDown keys on your keyboard to browse through
the file contents.
● /sometext - Search forward in the file
● ?sometext - Search backward in the file
● Repeat the last search by using n
37. cat
● cat - concatenate files and print on the standard output
● $ cat filename.txt
● $ cat -n filename.txt - number all output lines
38. head & tail
● Using head on a text file will show by default the first 10 lines of that file.
● Using tail on a text file shows the last 10 lines by default.
● $ head/tail textfile - print first/last 10 line number
● $ head/tail -n 5 textfile - print first/last 5 line number
39. Filtering Specific Columns with cut
● Imagine that you need to see a list of all users in the /etc/passwd file
○ root:x:0:0:root:/root:/bin/bash (( 8 field with : seperated ))
○ …
● $ cut -d : -f 1 /etc/passwd
○ -d : specify the field delimiter
○ -f : number of the specific field
40. sort
● $ sort /etc/passwd
○ The content of the /etc/passwd file is sorted in alphabetic order.
● You can use the sort command on the output of a command also, as in:
○ $ cut -f 1 -d : /etc/passwd | sort
● Sort by selecting specific field
○ $ sort -k3 -t : /etc/passwd
41. wc
● wc - print newline, word, and byte counts for each file
○ $ wc /etc/passwd
○ 33 45 1762 /etc/passwd
42. Grep
● grep - print lines matching a pattern
○ $ grep <pattern> <filename>
○ $ grep root /etc/passwd
Option Use
-i Not case sensitive. Matches uppercase as well as lowercase.
-v Only show lines that do not contain the regular expression.
-r Search files in the current directory and all subdirectories.
-A <number> Show <number> of lines after the matching regular expression.
-B <number> Show <number> of lines before the matching regular expression.
43. tar - an archiving utility
● This command originally designed to stream files to a tape.
● Create an archive:
○ $ tar -cvf /root/homes.tar /home
● List the contents of an archive:
○ $ tar -tvf /root/homes.tar
● Extract an archive:
○ $ tar -xvf homes.tar -C /tmp ( use -C to specify the target directory where you want to
extract the file in)
44. Wget - The non-interactive network downloader.
● Download file with <URL> path:
○ $ wget <URL>
● Set number of tries:
○ $ wget -t 10 <URL>
● Download file and directories recursively:
○ $ wget -r <URL>
50. User & group concepts
● On Linux, there are two ways to look at system security.
○ Privileged users: normal users
○ Unprivileged user: full access to everything < root >
● Use command <id> to get information from user
○ $ id root
○ uid=0(root) gid=0(root) groups=0(root)
51. Methods to Run Tasks with Elevated Permissions
su Opens a subshell as a different user, with the advantage that only in the subshell
commands are executed as root
sudo Allows you to set up an environment where specific tasks are executed with
administrative privileges
52. Manage users (adduser - deluser)
● adduser <username>
○ $ adduser iman - create user with name of iman and home directory of /home/iman
● Deluser <username>
○ $ userdel iman - delete user named iman
○ $ userdel -r iman - delete user named iman with /home/iman directory
53. /etc/passwd & /etc/shadow
● Both user accounts (privileged & unprivileged) share common properties,
which are kept in the files /etc/passwd and /etc/shadow.
● $ head -n 1 /etc/passwd
○ root:x:0:0:root:/root:/bin/bash
○ <username> : <password> : <UID> : <GID> : <commentField> : <Directory> : Shell
● $ head -n 1 /etc/shadow
○ root:$6$0UY8UDC1xW5DktJELlDlEKbTlE8pBTCpGSaM4DbzSzLA2i1:18514:0:99999:7:::
○ <username> : <encryptedPassword> : <Days since Jan 1, 1970, that the password was last
changed> …
55. Displaying Ownership
● every file and every directory has two owners (user and a group owner)
● These owners are set when a file or directory is created
● On creation, the user who creates the file becomes the user owner, and
the primary group of that user becomes the group owner
● Checking are as follows:
○ IF you are user owner THEN get owner’s permission
○ ELSE IF you are group owner THEN get group’s permission
○ ELSE get the permissions of others.
56. Changing User Ownership (chown)
● Chown <new-user> <file/dir>
● $ chow iman myfile
○ $ ls -l
○ -rw-r--r--. 1 iman root 0 Sep 9 15:14 myfile
● $ chow -R <new-user> <dir>
○ operate on files and directories recursively
57. Changing Group Ownership
● Chown :<new-group> <file/dir>
● $ chow :iman myfile
○ $ ls -l
○ -rw-r--r--. 1 iman iman 0 Sep 9 15:14 myfile
● $ chow -R :<new-group> <dir>
○ operate on files and directories recursively
● Change both user and group
○ $ chown iman:iman myfile
61. Applying Read, Write and Execute permissions
● chmod - change file mode bit
● $ chmod <bit format permission> <filename>
● $ chmod 775 somefile
● $ chmod +x somefile - adds the execute permission for all users
● $ chmod g+w,o-r some file - adds the write permission to the group and
remove read for others
63. Internet Protocol
● IP addresses were assigned to computers and routers ( computer ~ node ).
● Every IP address belongs to a specific network.
● Routers are used to connect networks.
● Public IP address, allow device to direct access over the Internet (5.9.201.150).
● Private IP address, is the address space allocated by InterNIC to allow
organizations to create their own private network.
○ 10.0.0.0/8 ( a single Class A network)
○ 172.16.0.0/12 (16 Class B networks)
○ 192.168.0.0/16 (256 Class C networks)
● private nodes cannot directly communicate with public networks, but require
Network Address Translation at a routing gateway for this purpose.
65. Network Masks
● The subnet mask defines which part of the network address indicates the network
and which part indicates the node.
● Example:
○ Address: 192.168.10.100 11000000.10101000.00001010 .01100100
○ Netmask: 255.255.255.0 = 24 11111111.11111111.11111111 .00000000
○ Network: 192.168.10.0/24 11000000.10101000.00001010 .00000000
(Class C)
○ Broadcast: 192.168.10.255 11000000.10101000.00001010 .11111111
○ HostMin: 192.168.10.1 11000000.10101000.00001010 .00000001
○ HostMax: 192.168.10.254 11000000.10101000.00001010 .11111110
66. Validating Network Configuration
1. IP address and subnet mask
a. Use ip addr to configure and monitor network addresses
2. Routing
a. Use ip route to configure and monitor routing information
3. Availability of ports and services
a. Use ip link to configure and monitor network link state
4. Examples:
a. $ ip addr show (or $ip a)
b. $ ip link show
c. $ ip route show
68. Ip assignment
Not persistent
$ Ip addr add 192.168.50.5 dev eth0
$ ifconfig eth0 192.168.50.5 up
(Although net-tools is depricated)
Persistent
1. Use /etc/network/interfaces
(ifupdown)
2. Nmcli (desktop)
3. Netplan
70. Process monitoring
Ps, pstree, pgrep - process monitor
● To see every process on the system - standard syntax
○ $ ps -ef
● To see every process on the system - BSD syntax
○ $ ps aux
● Print a process tree:
○ $ ps -ejH
○ $ pstree
● look up or signal processes based on name and other attributes
○ $ pgrep syslog
72. top
Load Average
● every process that needs to be served enters a run queue before the
kernel scheduler can allocate it to run on a CPU core.
● average number of processes that is waiting to be served at any given
moment.
● the number indicated as the load average should not be much higher
than the total number of CPU cores
● if server has four cores, four processes can be handled at the same time,
and the CPU load should not be higher than four.
73. Top - CPU Performance Parameters
us Percentage of time the CPU spends handling processes in user mode.
sy Percentage of time the CPU spends in kernel mode.
id Percentage of time the processor spends in the idle loop.
wa
Time the processor spends waiting for noninterruptible I/O, such as requests to disks, hard-
mounted NFS, and tape units.
hi
Time the processor spends handling hardware interrupts. A high value may indicate faulty
hardware.
74. Top - Memory Usage
KiB Mem Total amount of physical memory in KiB (1 KiB = 1024 bytes)
used Total amount of RAM that is used for any purpose
free Total amount of RAM that is not used for anything
buffers
Total amount of used memory that is used for storing unstructured Data
cached
Mem
Total amount of memory that is used to cache files that have recently been fetched from
disk
76. ● The default utility used to manage software on Red Hat based linux is
yum
● Software on Red Hat based linux is provided in the RPM format.
● Yum is managing package dependencies
● Syntax:
○ $ yum <task> <package-name>
77. Using yum
Task Explanation
search Search for the exact name of a package
info Provide more information about the package
install Install the package
remove Remove the package
list [all | installed ] List all or installed packages
group list List package groups
group install Install all packages specified
clean all Remove all stored metadata
78. Yum sample
● $ yum search user
● $ yum info nmap
● $ yum install nmap
● $ yum list kernel
● $ yum groups info “Basic Web Server”
80. How to repeat task periodically
● configure each process with a process-specific solution to handle
recurring tasks
● Use generic service to run processes automatically at specific times.
● cron service consists of two major components
○ cron daemon crond: looks every minute to see whether there is work to do
○ cron configuration: consists of multiple files working together to provide the right
information to the right service at the right time.
81. Managing the cron service
● It does not need much management, where other services need to be
reloaded or restarted to activate.
● Cron daemon wakes up every minute and checks it’s configuration.
● Monitoring the current state of the crond service
○ $ systemctl status crond -l
82. Understanding cron Timing
● After scheduling services through cron, you need to specify when exactly
the services need to be started.
Field Values
minute 0-59
hour 0-23
Day of month 1-31
month 1-12 (or names which are better avoided)
Day of week 0-7 (Sunday is 0 or 7, or names (which are better avoided)
84. ● In any of these fields, you can use an * to refer to any value. Ranges of
number are allowed, as are lists and patterns.
○ * 11 * * * - any minute between 11:00 and 11:59
○ 0 11 * * 1-5 - every day at 11 a.m. on weekdays only
○ 0 7-18 * * 1-5 - every hour on weekdays on the hour
● Instead of modifying /etc/crontab, different cron configuration files are
used:
○ Cron files in /etc/cron.d
○ Scripts in /etc/cron.hourly, cron.daily, cron.weekly, and cron.monthly
○ User-specific files that are created with crontab -e
85. ● Do not edit /etc/crontab directly
● cron jobs can be started for specific users by running crontab -e
command.
● After editing the cron configuration, the temporary file is moved to its
final location in the directory /var/spool/cron.
● You can add cron jobs to /etc/cron.d directory. Just put cron file in that
directory
86. ● If the exact time of running cron job is not matter, put scripts in the
following directories:
○ /etc/cron.hourly
○ /etc/cron.daily
○ /etc/cron.weekly
○ /etc/cron.monthly
● When opening these scripts, notice that no information is included about
the time when the command should be executed
88. Understanding MBR
● Master Boot Record(MBR) partitioning scheme:
○ The system to define hard disk layout, invented in early 1982.
● While booting a computer the Basic Input Output System (BIOS) was
loaded to access hardware devices by using MBR
● The MBR contains all that is needed to start a computer, including a boot
loader and a partition table
● MBR:
○ Defined as the first 512 bytes on a computer hard drive
○ Operating system boot loader
○ Partition table
89. Understanding MBR
● In MBR, just four partitions could be created because the size that was
used for partition table was relatively small (just 64 bytes).
● As maximum four partition table limits exists, so a solution of extended
partition was created.
90.
91. Understanding GPT
● Current computer hard drives have become too big to be addressed by
MBR partitions.
● That is why new partitioning scheme was needed
● This partitioning scheme is the GUID Partition Table (GPT) partitioning
scheme.
92. GPT improvements
● The maximum partition size in GPT is 8 zebibyte (ZiB), which is 1024 *
1024 * 1024 *1024 gibibytes.
● In GPT, up to a maximum number of 128 partitions can be created.
● The 2 TiB limit no longer exists.
● here is no longer a need to distinguish between primary, extended, and
logical partitions.
● A backup copy of the GUID partition table is created by default at the end
of the disk, which eliminates the single point of failure that exists on MBR
partition tables.
93. Managing partitions and file systems
● To match the different partition types, there are also two different
partitioning utilities.
● fdisk : this utility is used to create MBR partitions.
● Gdisk: this utility is used to create GPT prtitions
94. Common disk device types
Device Name Description
/dev/sda A hard disk that uses the SCSI driver. Used for SCSI and SATA disk
devices
/dev/hda The (legacy) IDE disk device type. You will seldom see this device type on
modern computers.
/dev/vda A disk in a KVM virtual machine that uses the virtio disk driver. This is the
common disk device type for KVM virtual machines.
/dev/xvda A disk in a Xen virtual machine that uses the Xen virtual disk driver.
95. Creating File Systems
File system Description
XFS The default file system in RHEL 7
ext4 evolution of the most used Linux filesystem, Ext3.
ext3 was mostly about adding journaling to Ext2
ext2 second extended file system is a file system for the Linux kernel.
replacement for the extended file system (ext)
NTFS NT file system; sometimes New Technology File System
VFAT virtual file allocation table
97. mount/unmount file systems (manually)
● To mount a file system, some information is needed:
○ What to mount
○ Where to mount it
○ What file system to mount
○ Mount options
● $ mount -t <filesystem> <device-name> <folder> -o <options>
○ $ mount -t ext4 /dev/vda5 /mnt
● $ unmount <folder>
○ $ unmount /mnt
98. mount/unmount file systems (automatic)
● Mount filesystem with device name is not good solution. Device names
can be rename ( /dev/sdb5 ⇒ /dev/vdb5 )
● Every file system by default has a UUID associated to it
● use the blkid command to get an overview of the current file systems
○ [root@greentech iman]# blkid
○ /dev/vda1: UUID="f9dca4a3-09de-4ebb-b0b7-0a8f5ee9866a" TYPE="ext4"
○ /dev/vda2: UUID="aa9fee4b-fb07-4b8c-ac05-5a88d3f6b8e6" TYPE="swap"
99. Automatic File system mounts through /etc/fstab
● The classical way to automatic mount is through the /etc/fstab file
○ #
○ # /etc/fstab
○ # Created by anaconda on Mon Nov 6 16:03:45 2017
○ #
○ # Accessible filesystems, by reference, are maintained under '/dev/disk'
○ # See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
○ #
○ /dev/vda1 / ext4 defaults 1 1
○ UUID=9dca4a3-09de-4ebb-b0b7-0a8f5ee9866a ext4 defaults 1 1
○ /dev/vda1 swap swap defaults 0 0
100. /etc/fstab fields
Field Description
Device The device that must be mounted. A device name, UUID, or label
can be used
Mount point The directory or kernel interface where the device needs to be
mounted
File System The file system type
Mount Options Mount options
Dump Support Use 1 to enable support to backup using the dump utility. This may
be necessary for some backup solutions
Automatic Check Specifies if the file system should be checked automatically when booting. Use 0 to disable
automated check, 1 if this is the root file system and it has to be checked automatically, and 2 for
all other file systems that need automatic checking while booting. Network file systems should
have this option set to 0.
102. Understanding LVM
● In the early days, storage was handled by creating partitions on disks
○ Cons: disks are so inflexible , where it is not possible to dynamically grow a partition
● In the LVM architecture, several layers can be distinguished.
● several layers can be distinguished:
○ On the lowest layer, the storage devices are used
○ abstraction of all available storage is defined by volume group
■ Volume groups can be resized dynamically
○ On top of the volume group are the logical volumes
■ do not act on disks directly but get their disk space from available disk space in the
volume group
104. LVM Features
● flexible solution for managing storage
○ Volumes are no longer bound to the restrictions of physical hard drives
○ It is also possible to reduce the size of a logical volume, but only if the file system that was
created on that volume supports resizing
● snapshot
○ keeps the current state of a logical volume and can be used to revert to a previous
situation or to make a backup of the file system on the logical volume if the volume is
open
● logical volumes is the option to replace failing hardware easily