Linux 104 subjects: - Packaging - Job scheduling

1. (Practical) Linux 104
Arie Bregman
Senior Software Engineer @RedHat

2. Agenda
● Package Management
● Job Scheduling
● Interview Questions Examples

3. Package Management
● What is it good for?
○ Better distribution of software
○ Dependency resolution
○ Updates

4. Types of package managers
● RPM
○ Red Hat Operating Systems. Also the base for other package managers such as zypper
● dpkg
○ Past: debian. Today: Ubuntu
● pip
○ Python packages
● Google Play
○ Android OS

5. How it (usually) works

6. Base vs Tools
● If you are using Red Hat OSs you might ask yourself:
○ Should I use RPM or dnf/yum?”
● If you are using Ubuntu you might ask yourself:
○ Should I use dpkg or apt?
● Short answer: you’ll probably use both

7. Enough talking. Practice Time!
Open your terminal

8. List packages
● List all installed packages on the system
● Do you know how to count how many packages installed on the system?
● Query specific package
[arie@fedora ~]$ rpm -qa [arie@ubuntu ~]$ dpkg -l
# or you can use apt list
[arie@fedora ~]$ rpm -q at
at-3.1.20-10.fc28.x86_64
[arie@ubuntu ~]$ dpkg -l at
ii at 3.1.18-2ubuntu1

9. Package name
● at 3.1.20-10 fc28 x86_64
Software name Version Release Architecture
● More examples
○ python-requests-2.3.4-1.el7.i686
○ zlib-6.2-1.noarch

10. Package Installation
● Install a new package on your system
● Will it work in a script?
[arie@fedora ~]$ sudo dnf install python-requests [arie@ubuntu ~]$ sudo apt-get install python-requests
[arie@fedora ~]$ sudo dnf install -y python-requests [arie@ubuntu ~]$ sudo apt-get install -y python-requests

11. Package Removal
● Remove installed package from the system
[arie@fedora ~]$ sudo dnf remove python-requests [arie@ubuntu ~]$ sudo apt-get remove python-requests

12. Update system packages
● Updates all the installed packages on your system
● Why?
○ Keeps your system up to date
○ Usually Mandatory step for upgrading the system
■ Upgrade = increase in major version
■ Update = increase in minor version
[arie@fedora ~]$ sudo dnf update [arie@ubuntu ~]$ sudo apt-get update

13. Searching for packages
● Check if package is available for installation by specifying name or description
[arie@fedora ~]$ sudo dnf search ansible [arie@ubuntu ~]$ sudo apt-cache search ansible

14. Tips
● You can install or remove multiple packages by specifying their names separated
by a space
● Don’t forget to specify ‘-y’ in scripts
● Each command we saw so far, can be used with additional parameters
Explore the options with ‘man’
[arie@fedora ~]$ sudo dnf install -y python-requests zlib vim
[arie@fedora ~]$ sudo dnf install -v zlib
[arie@ubuntu ~]$ sudo apt-get install -d zlib

15. Exercise
● Make sure the packages python-virtualenv, git and groovy are installed on your
system
● If they are not installed, install them
● Remove the package groovy
● Check if the package ‘ansible’ is available for installation

16. Solution
[arie@fedora ~]$ rpm -qa | grep -E -w 'virtualenv|^git|^groovy'
[arie@fedora ~]$ sudo dnf install -y groovy git python-virtualenv
[arie@fedora ~]$ sudo dnf remove -y groovy
[arie@fedora ~]$ sudo dnf search ansible # or install without -y :)

17. Where the packages are coming from?
● Repository = a directory which contains one or more packages
● Ubuntu
○ /etc/apt/sources.list
○ /etc/apt/sources.list.d/*
● Red Hat Based OSs
○ /etc/yum.repos.d/*
[arie@fedora ~]$ sudo cat /etc/yum.repos.d/google-chrome.repo
[google-chrome]
name=google-chrome
baseurl=http://dl.google.com/linux/chrome/rpm/stable/x86_64
enabled=1

18. List repositories
● Ubuntu
● Red Hat Based OSs
[arie@fedora ~]$ sudo dnf repolist
repo id repo name status
Fedora-27-Workstation Fedora-27-Workstation 3,004
docker-ce-edge Docker CE Edge - x86_64 4
docker-ce-stable Docker CE Stable - x86_64 2
[arie@ubuntu ~]$ sudo apt-cache policy
500 http://security.ubuntu.com/ubuntu xenial-security/multiverse amd64 Packages release v=16.04

19. Creating your own custom repository - Ubuntu
● Install the package that allows you to create the repository
● Create a directory for storing the packages and move some packages there
● Create Packages.gz which “describes” the repository
● Add it to repositories list
[arie@ubuntu ~]$ sudo apt-get install - y dpkg-dev
[arie@ubuntu ~]$ sudo mkdir -p /usr/local/my_repo
[arie@ubuntu ~]$ sudo chmod 777 /usr/local/my_repo
[arie@ubuntu ~]$ dpkg-scanpackages . /dev/null | gzip -9c > Packages.gz
[arie@ubuntu ~]$ sudo vi /etc/apt/sources.list
deb file:/usr/local/my_repo ./

20. Creating your own custom repository - Red Hat
● Install the package that allows you to create the repository
● Create a directory for storing the packages
● Run createrepo
● Add it to repositories list
[arie@fedora ~]$ sudo dnf install -y createrepo
[arie@fedora ~]$ sudo mkdir -p /usr/local/my_repo
[arie@fedora ~]$ sudo chmod 777 /usr/local/my_repo
[arie@fedora ~]$ sudo createrepo /usr/local/my_repo
[arie@fedora ~]$ vi /etc/yum.repos.d/some.repo
[my_repo]
name=my_repo
baseurl=file:///usr/local_my_repo
enabled=1

21. Packaging - Suggestions for Next Steps
● Package Building
● Explore base limitations compared to extended tools
● Cache

22. Job Scheduling
● What is it good for?
○ Running tasks at any given date and time
○ Periodic task execution
● Use cases
○ Removing old data
○ Performing updates
○ Configuring systems

23. Types of job schedulers
● Cron
○ Most known and used job scheduler
○ Has many flavors
● Anacron
○ Works well for systems that may be turned
off some of the time
● Systemd Timers
○ Relatively new
○ Supports calendar time events

24. Schedule your first job
● We’ll use crontab to install the cron files which define what to run and when
[arie@fedora ~]$ crontab -e
*/2 * * * * /bin/ls
● Save the file and exit

25. What just happened?
● We created a cron file which tell crond what to run and when
○ Ubuntu: /var/spool/cron/crontabs/<user>
○ Red Hat: /var/spool/cron/<user>
● crond daemon is responsible for executing the jobs
● List your crontab tasks with the following command
[arie@fedora ~]$ crontab -l
*/2 * * * * /bin/ls

26. Cron Syntax
* * * * * [command to execute]
Minutes Hours Day of month Month Day of week
0-59 0-23 1-31 1-12 0-6
*/2 Every two _____ (minutes, hours, …)
1,6,8 Specific values
0-5 Range

27. More examples
● Do you understand what is the interval in each of the following examples?
0 0 * * * Every night
0 0 1 */3 * Every quarter
30 15 * 2 5 15:30 on Friday in February
* * * 1,6,10 * Every minute at January, June and October

28. Exercise
● Schedule the following jobs:
○ Runs every second month and executes ls
○ Runs every hour between 14:00 and 17:00 and executes ‘w’
○ Run every hour on Tuesday and saves the output of date to /tmp/date.log

29. Solution
* * * */2 * /bin/ls Runs every second month and executes ls
0 14-17 * * * /usr/bin/w Runs every hour between 14:00 and 17:00 and executes ‘w’
0 * * * 2 /bin/date > /tmp/date.log Runs every hour between 14:00 and 17:00 and
executes ‘w’

30. Remove Cron Jobs
● crontab can be used for creating, listing and removing cron jobs
Note: be careful, in some flavors it will not ask for confirmation
[arie@fedora ~]$ crontab -r

31. Job Scheduling - Suggestions for Next Steps
● Explore systemd timers
● Explore anacron
● Cron
○ How to load cron jobs from a file
○ Special keywords (e.g. @reboot)

32. Break
● Any questions on Packaging?
● Any questions on Job Scheduling?
● Good. Now let’s see a couple of questions you should be able to easily
answer after practicing and gaining more experience in the mentioned
subjects

33. Interview Questions Examples
● Given a file path, how do you know which package owns it?
○ If no package owns it, what does it mean?
● Did you build a package in the past? Can you describe the process?
● How do you create your own repository of packages?
● How to schedule a job to run every Sunday at 18:30?
● A scheduled job was supposed to run yesterday at morning but the server was
down. How can you overcome such situation in the future?

34. Thank You
● You can find this presentation on GitHub and SlideShare
● Questions?