ICT role in 21st century education and it's challenges.
Lecture1 Introduction
1.
2. Open Source Community.
Why should we care about OS Programming?
What is ArabBSD?
OS Categories.
FreeBSD Vs. Linux.
FreeBSD Kernel Design Overview.
FreeBSD File Hierarchy.
FreeBSD Booting Process.
FreeBSD Kernel Basics.
5. Kernel has no scheduler.
Scheduling policies and synchronization protocols
are defined by user-level schedulers.
Mutable Protection Domains (MPD) instead of IPC.
6.
7.
8. Kernel development.
Different Systems Emulation Support (Binary
Emulation).
Levels of Security .
Maintainers.
License.
16. Paged Virtual Address Space.
Software Interrupts.
Timers and Counters.
Identifiers for accounting, protection,
scheduling, ..etc.
Descriptors
Higher layers provide “system” facilities such as
filesystem, terminals and sockets.
17. CPU Time.
Asynchronous Events.
Memory.
IO Descriptors.
Processes are the fundamental service provided
by UNIX.
18.
19. User process
Blocking Wall
Top half of Kernel
Bottom Half of Kernel
20. Multiple User
Processes
Blocking Wall
Top half of Kernel
Bottom Half of Kernel
21.
22. bin
sbin
boot
cdrom
dev
etc
lib
mount
media
/
proc
(Root sys kern
Directory) tmp home
src
ports
usr games
bin
bin
Local
sbin
share
crash
run
var
tmp
spool
23. Directory Description
bin Essential command binaries
boot Static files of the boot loader
dev Device files
etc Host-specific system configuration
lib Essential shared libraries and kernel modules
media Mount point for removable media
mnt Mount point for mounting a filesystem temporarily
sbin Essential system binaries
tmp Temporary files
usr Secondary hierarchy
var Variable data
24. home User home directories
lib Alternate format essential shared libraries
/sys/kern Kernel Directory.
25. /bin contains tools required system to work (like 'ls',
'rm', 'mkdir' etc.) which would be available at any time
(for example if system crashes, they were available in
single user mode). Used by both root and users.
/usr/bin contains tools which are not needed in
single user mode. Used by both root and users.
/usr/local/bin is for local use - you install your
binaries there. Used by both root and users (Third
party binaries).
26. /sbin : System binaries
Purpose
Utilities used for system administration are stored
in /sbin, /usr/sbin, and /usr/local/sbin.
/sbin contains binaries essential for booting, restoring,
recovering, and/or repairing the system in addition to
the binaries in /bin. Programs executed after /usr is
known to be mounted (when there are no problems) are
generally placed into /usr/sbin.
Locally-installed system administration programs
should be placed into /usr/local/sbin.
27. /boot : Static files of the boot loader
Purpose
This directory contains everything required
for the boot process except configuration
files not needed at boot time. Thus /boot
stores data that is used before the kernel
begins executing user-mode programs. This
may include saved master boot sectors and
sector map files.
Also, the compiled kernel should be located
in /boot/kernel/.
28. /dev : Device files
Purpose
The /dev directory is the location of special
or device files.
29. /etc : Host-specific system configuration
Purpose
The /etc hierarchy contains configuration
files.
A "configuration file" is a local file used to
control the operation of a program; it must
be static and cannot be an executable binary.
30. File Description
fstab Static information about filesystems
ftpusers FTP daemon user access control list
gateways File which lists gateways for routed
host.conf Resolver configuration file
inetd.conf Configuration file for inetd
31. /etc/X11 : Configuration for the X Window
System
Purpose
/etc/X11 is the location for all X11 host-
specific configuration.
32. /home : User home directories (optional)
Purpose
/home is a fairly standard concept, but it is
clearly a site-specific filesystem.
33. /media : Mount point for removable media
Purpose
This directory contains subdirectories which
are used as mount points for removable
media such as floppy disks, cdroms and zip
disks.
34. /mnt : Mount point for a temporarily
mounted filesystem.
Purpose
This directory is provided for temporarily
mounted filesystem. For example, partitions
from harddrives or network shares.
35. /tmp : Temporary files
Purpose
The /tmp directory must be made available
for programs that require temporary files.
36. The /usr Hierarchy
Purpose
This directory includes all the user related
stuff.
37. /usr/include : Directory for standard include
files.
Purpose
This is where all of the system's general-use
include files for the C programming language
should be placed.
38. /usr/lib : Libraries for programming and
packages
Purpose
/usr/lib includes object files, libraries, and
internal binaries that are not intended to be
executed directly by users or shell scripts.
39. /usr/local : Local hierarchy
Purpose
The /usr/local hierarchy is for use by the
system administrator when installing software
locally. It needs to be safe from being
overwritten when the system software is
updated. It may be used for programs and
data that are shareable amongst a group of
hosts, but not found in /usr.
40. /usr/local/share
Purpose
The requirements for the contents of this
directory are the same as /usr/share. The
only additional constraint is
that /usr/local/share/man and /usr/local/m
an directories must be synonomous (usually
this means that one of them must be a
symbolic link).
41. /usr/sbin : Non-essential standard system
binaries
Purpose
This directory contains any non-essential
binaries used exclusively by the system
administrator.
42. /usr/share : Architecture-independent data
Purpose
This hierarchy is intended to be shareable
among all architecture platforms of a given
OS.
Any program or package which contains or
requires data that doesn't need to be
modified should store that data
in /usr/share (or /usr/local/share, if installed
locally).
43. /usr/share/man : Manual pages
Purpose
This section details the organization for
manual pages throughout the system,
including /usr/share/man.
44. The /var Hierarchy
Purpose
/var contains variable data files. This includes
spool directories and files, administrative and
logging data, and transient and temporary files.
Some portions of /var are not shareable between
different systems. For instance, /var/log
and /var/lock. Other portions may be shared,
notably /var/mail, /var/cache/man, /var/cache/fo
nts, and /var/spool/news.
45. /var/cache : Application cache data
Purpose
/var/cache is intended for cached data from
applications. Such data is locally generated as
a result of time-consuming I/O or
calculation.
46. /var/crash : System crash dumps (optional)
Purpose
This directory holds system crash dumps.
47. /var/mail : User mailbox files (optional)
Purpose
The mail spool must be accessible
through /var/mail.
User mailbox files in this location must be
stored in the standard UNIX mailbox format.
48. /var/run : Runtime variable data
Purpose
This directory contains system information
data describing the system since it was
booted. Files under this directory must be
cleared (removed or truncated as appropriate)
at the beginning of the boot process.
49. /proc : Kernel and process information virtual
filesystem
Purpose
The proc filesystem is the standard method
for handling process and system information.
52. When a computer is powered on, there is
nothing running on the CPU. For a program to
be set running, the binary image of the
program must first be loaded into memory
from a storage device.
Bootstrapping: cold start to the point at
which user-mode programs can be run.
In FreeBSD, the binary image is in /vmunix or
/boot/kernel/kernel
53. Initial bootstrapping is machine dependant.
Usually 512 bytes.
Used to load boot program.
The boot program's task is to load and
initialize the executable image of a program
and to start that program running.
Not required to be on the same storage
device.
55. Binary file reading in terms of Assembly
language.
FreeBSD Assembly Tools (as, nasm).
56. The boot program reads the binary image of
a program to be bootstrapped into main
memory, and then initializes the CPU so that
the loaded program can be started.
Boot -> several stages of hardware and
software initialization in preparation for
normal system operation
57. The first stage is responsible for initializing
the state of the CPU, including the run-time
stack and virtual-memory mapping.
Once virtual-memory mapping is enabled,
the system does machine-dependent
initializations, and then machine-
independent initializations.
58. The machine-dependent operations include
setting up virtual-memory page tables and
configuring I/O devices; the machine-
independent actions include mounting the
root filesystem and initializing the myriad
system data structures. This order is
necessary because many of the machine-
independent initializations depend on the I/O
devices being initialized properly.
Following the setup of the machine-
independent portions
59. Turn on PC
boot0
Read the value 0xfffffff0 from instruction pointer register
boot1
Hardware translates the value to read BIOS Memory Block
boot2
Jump to BIOS’s POST routines loader
POST performs different checks init
One of the POST instructions is INT 0x19 instruction which
reads 512 bytes from the first sector of boot device into
the memory at address 0x7c00
60. File •sys/boot/i386/boot0/boot0.S
Location
•INT 0x19 instruction loads an MBR, i.e. the boot0 content, into the
Memory memory at address 0x7c00
Location
•Starting from 0x1be, called the partition table, It has 4 records of 16 bytes each,
MBR called partition record
Structure
•the 1-byte filesystem type
•the 1-byte bootable flag
Partition
Record •the 6 byte descriptor in CHS format
Structure •the 8 byte descriptor in LBA format
61. LBA (Logical Block Addressing)
• has the starting sector for the
partition and the partition's length.
CHS (Cylinder Head Sector)
• has coordinates for the first and last
sectors of the partition.
62. The boot manager scans the partition table and prints the menu on the
screen so the user can select what disk and what slice to boot. By
pressing an appropriate key, boot0 performs the following actions:
modifies the bootable flag for the selected partition to make it
bootable, and clears the previous
saves itself to disk to remember what partition (slice) has been
selected so to use it as the default on the next boot
loads the first sector of the selected partition (slice) into memory and
jumps there
63. Actually, there is a 512-byte file
called boot1 in the directory /boot as well. It is
used for booting from a floppy. When booting
from a floppy, boot1 plays the same role
as boot0 for a harddisk which is
locating boot2 and runs it.
64. You may have realized that a
file /boot/mbr exists as well. It is a simplified
version of boot0. The code in mbr does not
provide a menu for the user, it just blindly
boots the partition marked active (i.e. No
Boot Manager Used).
65. Configure boot file using boot0cfg.
Note: Both boot0 and boot2 are invisible to
listing commands because they are not under
filesystem’s control.
66. It’s the first specific file for FreeBSD since it’s
used for locating FreeBSD slice.
67. File • sys/boot/i386/boot2/boot2.c
Location
• It must scan the harddisk.
• knowing about the filesystem structure
• finds the file /boot/loader
• reads it into memory using a BIOS service
Procedure • passes the execution to the loader's entry point.
• boot2 prompts for user input so the loader can be booted
from different disk, unit, slice and partition
• passes the execution to BTX
68. • switches the processor to protected mode
• prepares a simple environment before calling the client
• boot0 passes the execution to BTX's entry point. BTX then switches the
processor to protected mode, and prepares a simple environment before
calling the client. This includes:
• virtual v86 mode. That means, the BTX is a v86 monitor. Real mode
BTX instructions like pushf, popf, cli, sti, if called by the client, will work.
Procedure
• Interrupt Descriptor Table (IDT) is set up so all hardware interrupts are routed
to the default BIOS's handlers, and interrupt 0x30 is set up to be the syscall
gate.
• Two system calls: exec and exit.
• Finally, BTX creates a Global Descriptor Table (GDT)
• boot2 defines an important structure, struct bootinfo. This structure is
initialized by boot2 and passed to the loader, and then further to the kernel.
Some nodes of this structures are set by boot2, the rest by the loader. This
Back to
structure, among other information, contains the kernel filename, BIOS
Boot2
harddisk geometry, BIOS drive number for boot device, physical memory
available, envp pointer etc.
69.
70.
71. Open Boot2 file.
Take a look in Boot2 make file.
How do the two system calls exec and exit
are created within boot2? .
Take a look in Global Descriptor Table.
Bootinfo header file.
Choosing the most appropriate choice in 10
seconds!!
72. File • sys/boot/i386/boot/loader
Location
• kernel bootstrapping final stage
• When the kernel is loaded into memory, it is being called by
Procedure the loader
• The main task for the loader is to boot the kernel.
• It provides a scripting language that can be used to automate tasks, do
Functionality
pre-configuration or assist in recovery procedures.
75. • sys/i386/i386/machdep.c
File
Location
•Initialize the kernel tunable parameters, passed from the bootstrapping
program.
•Prepare the GDT.
•Prepare the IDT.
•Initialize the system console.
•Initialize the DDB, if it is compiled into kernel.
Functionality
•Initialize the TSS.
•Prepare the LDT.
•Set up proc0's pcb
76. • sys/kern/init_main.c
File
Location
• This function performs a bubble sort of all the
system initialization objects and then calls the
entry of each object one by one.
Functionality • Call System Scheduler which represents a
process with PID 0.
77.
78. boot0 : The first stage is written entirely in
assembly language and does the work
necessary for non-assembly-language code
to operate.
boot1: Used by floppy disks..
boot2: The third stage does machine-
independent operations, completing its work
by starting up the system-resident processes
(Filesystems and kernel loading).
79. Init : kernel starts Init as the first process. Init
starts shell and runs Resource Configuration
(rc) files. rc config files (for start and set
options of services) are /etc/rc ,
/etc/defaults/rc.conf , /etc/rc.conf
85. Some modules should be compiled in the
kernel and can’t be loaded during run time.
Those modules are ones that used before control
turned to the user.
Kernel Loadable Modules result large
flexibility in the kernel size since only needed
KLD’s are loaded.
We can use Binary file for total KLD’s indexing and
make it easier for the user to determine what KLD’s
can be loaded at boot time without the need for
loading each one separately.
87. • <sys/sysctl.h>
• <sys/socket.h>
• <vm/vm_param.h>
File
• <netinet/in.h>
Location • <netinet/icmp_var.h>
• <netinet/udp_var.h>
• get or set kernel state
• The sysctl utility retrieves kernel state and allows processes with
appropriate privilege to set kernel state. The state to be retrieved
or set is described using a “Management Information Base''
Functionality
(“MIB'') style name, described as a dotted set of components.
• To retrieve the maximum number of processes allowed in the system, one
would use the following request sysctl kern.maxproc
• Information about the system clock rate may be obtained with sysctl
Example kern.clockrate
89. Q. How do we can realize object oriented
approach in FreeBSD Kernel?
A. Kernel Objects (Kobj)
90. Terminology Description
A set of data - data structure - data
Object
allocation.
Method An operation – function
Class One or more methods.
Interface A standard set of one or more methods
91. • Kobj works by generating descriptions of methods. Each description holds
a unique id as well as a default function. The description's address is used
Kobj to uniquely identify the method within a class' method table.
• A class is built by creating a method table associating one or more functions
with method descriptions. Before use the class is compiled. The compilation
allocates a cache and associates it with the class. A unique id is assigned to
each method description within the method table of the class if not already
Class
done so by another referencing class compilation
• To retrieve the maximum number of processes allowed in the system, one
would use the following request sysctl kern.maxproc
• Information about the system clock rate may be obtained with sysctl
Example kern.clockrate
92. New Method
Use Method
New Method
Use script to
generate function
associated function to qualify
The default within the class is arguments
function associated used
with the method
description is used
Found
Not Found
automatically
reference the
generated function proceeds to use the class'
method
table to find the method
description for a
lookup
looks up the method by using the
Not Cached unique id associated with the method
description as a hash into the cache
associated with the object's class
95. FreeBSD Handbook Web Page
http://www.freebsd.org/doc/en_US.ISO8859-
1/books/handbook/
McKusick, M., et al. The design and
implementation of the 4.4BSD operating system,
Addison-Wesley, 1996.
ArabBSD Web Page
https://sites.google.com/site/arabbsd/
This presentation is available on ArabBSD Website.