The session: https://coscup.org/2022/en/session/AGCMDJ After Linux kernel boots, it will try to launch first process “init” in User Space. Then, the system begins the featured journey of the Linux distribution. This sharing takes Busybox as the example and shows that how does Linux kernel find the “init” which directs to the Busybox. And, what will Busybox do and how to get the console. Try to make it like a simple Linux system. Before Linux kernel launches “init” process, the file system and storage corresponding drivers/modules must be loaded to find the “init”. Besides, to mount the root file system correctly, the kernel boot command must include the root device and file system format parameters. On the other hand, the Busybox directed from “init” is a lightweight program, but has rich functions, just like a Swiss Army Knife. So, it is usually used on the simple environment, like embedded Linux system. This sharing will have a demo on a virtual machine first, then on the Raspberry Pi. Drafts: * https://hackmd.io/@starnight/Busbox_as_the_init * https://hackmd.io/@starnight/Build_Alpines_Root_Filesystem_Bootstrap Relate idea: https://hackmd.io/@starnight/Systems_init_and_Containers_COMMAND_Dockerfiles_CMD

1. Launch the First Process in
Linux System
潘建宏 Jian-Hong Pan (StarNight)
@ COSCUP 2022

2. Who am I
潘建宏 / Jian-Hong Pan (StarNight)
Endless OS Foundation
You can find me at
● http://www.slideshare.net/chienhungpan/
● GitHub: starnight
● Email:
jhp [AT] endlessos.org
chienhung.pan [AT] gmail.com

3. Outline
● Linux kernel Boots
● The Init Process
● Busybox as the Init Process
● Build Busybox
● Boot on a QEMU aarch64 VM
○ Build a System Image
○ Have the Linux kernel
○ Boot the QEMU VM with the system image
● Boot on Raspberry Pi 3B
● Use Alpine’s Root Filesystem
● Reference

4. Linux Kernel Boots
Chip
x86(_64)
arm(64)
RISC-V
MIPS
…
Firmware
BIOS/UEFI
firmwares
…
Bootloader
GRUB
LILO
SYSLINUX
U-Boot
…
Kernel
Linux
(initramfs)
Init
PID 1

5. Build the Kernel
$ git clone https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git linux-stable
$ cd linux-stable
$ make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- defconfig
$ make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- menuconfig
$ make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu-

6. Boot QEMU arm64 VM with an Empty Root Partition
$ qemu-system-aarch64
-smp 2
-M virt
-cpu cortex-a57
-m 1G
-kernel ~/linux-stable/arch/arm64/boot/Image
--append "console=ttyAMA0 root=/dev/vda2 rw rootfstype=ext4"
-hda ~/qemu-images/simple-busybox.img
-serial stdio

7. Linux Kernel Tries to Find init
…
[ 2.146201] Run /sbin/init as init process
[ 2.148061] Run /etc/init as init process
[ 2.149846] Run /bin/init as init process
[ 2.150521] Run /bin/sh as init process
[ 2.151871] Kernel panic - not syncing: No working init found. Try passing init=
option to kernel. See Linux Documentation/admin-guide/init.rst for guidance.
…

8. https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/init/main.c?h=v5.18#n1558
static int __ref kernel_init(void *unused)
{
int ret;
…
if (!try_to_run_init_process("/sbin/init") ||
!try_to_run_init_process("/etc/init") ||
!try_to_run_init_process("/bin/init") ||
!try_to_run_init_process("/bin/sh"))
return 0;
panic("No working init found. Try passing init= option to kernel. "
"See Linux Documentation/admin-guide/init.rst for guidance.");
}

9. The Init Process
In Unix-based computer operating systems, init (short for initialization) is the first
process started during booting of the computer system. Init is a daemon process
that continues running until the system is shut down. ~ from init on Wiki
● SysVInit
● Upstart
● Systemd
● OpenRC
● …
● Busybox
Reference: Differences between SysVinit, Upstart and Systemd

12. Busybox as the Init Process
Busybox: The Swiss Army Knife of Embedded Linux
● BusyBox combines tiny versions of many common UNIX utilities into a single
small executable. It provides replacements for most of the utilities you usually
find in GNU fileutils, shellutils, etc.
● BusyBox provides a fairly complete environment for any small or embedded
system.
~ # ls -l /sbin/init
lrwxrwxrwx 1 root 0 14 Apr 5 14:39 /sbin/init -> ../bin/busybox

13. Build Busybox
$ git clone https://git.busybox.net/busybox
$ cd busybox
$ make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- defconfig
$ make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- menuconfig
$ make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu-
$ make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- install
$ ls -l _install/
total 12
drwxr-xr-x 2 zack zack 4096 Jun 5 12:06 bin
lrwxrwxrwx 1 zack zack 11 Jun 5 12:06 linuxrc -> bin/busybox
drwxr-xr-x 2 zack zack 4096 Jun 5 12:06 sbin
drwxr-xr-x 4 zack zack 4096 Jun 5 12:06 usr
PS. When install Busybox into the root partition, have to set
CONFIG_PREFIX="<root partition mount path>"
Enable CONFIG_STATIC to
avoid shared libary issue

14. $ ls -l _install/sbin/
total 0
lrwxrwxrwx 1 zack zack 14 Apr 5 20:38 acpid -> ../bin/busybox
lrwxrwxrwx 1 zack zack 14 Apr 5 20:38 adjtimex -> ../bin/busybox
lrwxrwxrwx 1 zack zack 14 Apr 5 20:38 arp -> ../bin/busybox
lrwxrwxrwx 1 zack zack 14 Apr 5 20:38 blkid -> ../bin/busybox
lrwxrwxrwx 1 zack zack 14 Apr 5 20:38 blockdev -> ../bin/busybox
lrwxrwxrwx 1 zack zack 14 Apr 5 20:38 bootchartd -> ../bin/busybox
lrwxrwxrwx 1 zack zack 14 Apr 5 20:38 depmod -> ../bin/busybox
…
lrwxrwxrwx 1 zack zack 14 Apr 5 20:38 init -> ../bin/busybox
lrwxrwxrwx 1 zack zack 14 Apr 5 20:38 insmod -> ../bin/busybox
lrwxrwxrwx 1 zack zack 14 Apr 5 20:38 ip -> ../bin/busybox
…

15. Boot on a QEMU aarch64 VM

16. Build a System Image (for QEMU VM)
1. Have a raw image
2. Format the raw image with designed partition layout
3. Mount the root partition
4. Install the built Busybox into the root partition
5. Prepare more fundamental folders/pathes into the root partition
6. Prepare config files for the init process into the root partition
7. Prepare config files for other processes into the root partition: Network,
DHCP, DNS …
8. Unmount the root partition

17. System Storage Layout
Boot Partition:
● Boot loader
● Kernel, Initial RAM disk, Device Tree Blobs
Root Partition:
● /boot
● /sbin, /bin
● /usr, /lib
● /etc
● /dev, /proc, /sys
● /root
● /tmp, /var, /mnt
● …
init

18. Prepare the Image and Partitions
$ dd if=/dev/zero of=~/qemu-images/simple-busybox.img bs=8M count=16
16+0 records in
16+0 records out
134217728 bytes (134 MB, 128 MiB) copied, 0.086382 s, 1.6 GB/s
$ fdisk -l ~/qemu-images/simple-busybox.img
Disk /home/zack/qemu-images/simple-busybox.img: 128 MiB, 134217728 bytes, 262144 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x7ac31012
Device Boot Start End Sectors Size Id Type
/home/zack/qemu-images/simple-busybox.img1 * 2048 206847 204800 100M b W95 FAT32
/home/zack/qemu-images/simple-busybox.img2 206848 262143 55296 27M 83 Linux
# Some Linux distributions’ mkfs cannot be used in this way. Fallback to kpartx, then mkfs
$ mkfs.vfat -v --offset=2048 ~/qemu-images/simple-busybox.img $((100*1024*1024/1024))
$ mkfs.ext4 -E offset=$((512*206848)) ~/qemu-images/simple-busybox.img
Boot Partition
Root Partition

19. Mount Root Partition & Build a Root Filesystem
$ ROOTPART_PATH=/mnt
$ sudo mount -o offset=$((512*206848)) ~/qemu-images/simple-busybox.img $ROOTPART_PATH
$ cd ~/busybox && sudo make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- install
$ sudo mkdir -p $ROOTPART_PATH/etc/init.d
$ sudo mkdir -p $ROOTPART_PATH/{proc, sys, dev, tmp, root, var, lib, mnt, boot}
$ ls $ROOTPART_PATH
bin boot dev etc lib linuxrc lost+found mnt proc root sbin sys tmp usr var
Reference:
● The Linux Bootdisk HOWTO - 4. Building a root filesystem
● rootfs 淺談 by Carl Su @COSCUP 2020

20. Install the /etc/inittab
$ sudo install ~/busybox/examples/inittab $ROOTPART_PATH/etc/inittab
● The QEMU VM need ttyAMA0::respawn:/sbin/getty -L 0 ttyAMA0 vt100 for
serial console.
● Disable "askfirst" shell on the console by commenting ::askfirst:-/bin/sh

21. Create the /etc/fstab
$ cat $ROOTPART_PATH/etc/fstab
# Begin /etc/fstab
# file system mount-point type options dump fsck
# order
/dev/vda2 / ext4 rw,relatime 0 0
/dev/vda1 /boot vfat rw,relatime 0 0
proc /proc proc nosuid,noexec,nodev 0 0
#sysfs /sys sysfs nosuid,noexec,nodev 0 0
#devpts /dev/pts devpts gid=5,mode=620 0 0
#tmpfs /run tmpfs defaults 0 0
#devtmpfs /dev devtmpfs mode=0755,nosuid 0 0
# End /etc/fstab

22. Have the Network
$ sudo install -d $ROOTPART_PATH/usr/share/udhcpc
$ sudo install ~/busybox/examples/udhcp/simple.script
$ROOTPART_PATH/usr/share/udhcpc/default.script
$ echo "busybox-arm64" | sudo tee -a $ROOTPART_PATH/etc/hostname

23. The Initial Script
$ cat $ROOTPART_PATH/etc/init.d/rcS
#!/bin/sh
mount -a
ip link set eth0 up
udhcpc
hostname -F /etc/hostname
ntpd -n -q -p time.stdtime.gov.tw
$ sudo chmod +x $ROOTPART_PATH/etc/init.d/rcS

24. Create the Root
$ echo "root:x:0:0::/root:/bin/sh" | sudo tee -a $ROOTPART_PATH/etc/passwd
root:x:0:0::/root:/bin/sh
$ echo "root::18541::::::" | sudo tee -a $ROOTPART_PATH/etc/shadow
root::18541::::::

25. Boot the QEMU VM with the System Image
Unmount the $ROOTPART_PATH, before start the QEMU VM
$ qemu-system-aarch64
-smp 2
-M virt
-cpu cortex-a57
-m 1G
-kernel ~/linux-stable/arch/arm64/boot/Image
--append "console=ttyAMA0 root=/dev/vda2 rw rootfstype=ext4"
-hda ~/qemu-images/simple-busybox.img
-serial stdio

26. Get into the Real World
Raspberry Pi 3B for example

27. Boot on Raspberry Pi 3B
● Install Raspberry Pi’s boot firmware files from Raspberry Pi OS into the boot
partition.
● Prepare the config.txt and cmdline.txt into the boot partition.
● Install built kernel into the boot partition.
● Have interactive consoles by listing in /etc/inittab:
○ Have tty1::respawn:-/bin/sh for normal console
○ Have ::respawn:/sbin/getty -L ttyS1 115200 vt100 to replace the original
ttyAMA0 console. Becasue, Raspberry Pi 3B’s serial console is ttyS1.
● The mircro SD is mmcblk0 as the block. So, the boot partition is mmcblk0p1
and the root partition is mmcblk0p2.
● Tips: Add sleep 1 second before network interface “up” in /etc/init.d/rcS to
wait network devices becoming ready.

28. arch/arm/boot/dts/bcm2837-rpi-3-b.dts
/ {
compatible = "raspberrypi,3-model-b", "brcm,bcm2837";
model = "Raspberry Pi 3 Model B";
chosen {
/* 8250 auxiliary UART instead of pl011 */
stdout-path = "serial1:115200n8";
};
…
ttyS1

29. Use Alpine’s Root Filesystem
● Alpine is a lightweight Linux distribution based on musl libc and Busybox.
○ uses OpenRC as the init.
○ uses its own package manager called apk.
Build Alpine’s root filesystem
● Bootstrapping Alpine Linux with apk-tools-static to build root filesystem.
● Edit /etc/fstab for mounting.
● Edit /etc/inittab to get the serial console.
● Set network interface by editing /etc/network/interfaces. Also, enable the
networking service with rc-update add networking.
Detail: Build Alpine’s Root Filesystem (Bootstrap)
CI: https://github.com/starnight/build-image/blob/main/.github/workflows/image.yml

30. Reference
● QEMU ARM guest support
● Getting started with embedded-linux
● Linux Documentation/admin-guide/init.rst
● fdisk, mkfs.vfat, mkfs.ext4 and mount
● Busybox
● Alpine
● OpenRC and OpenRC Service Script Writing Guide
● The Linux Bootdisk HOWTO - 4. Building a root filesystem
● rootfs 淺談 by Carl Su @COSCUP 2020
● Bootstrapping Alpine Linux
● A Journey to Boot Linux on Raspberry Pi