= Please note that the code in the slides may not be working
= Refer to the repository for working examples: https://github.com/platisd/meta-dimitriOS
Welcome to a beginner-friendly deep dive on how to create your own Linux distribution!
But why would you want to create one yourself?
** BACKGROUND **
Mainstream Linux distributions (e.g. Ubuntu, ArchLinux) enable their users to perform a plethora of tasks and often include the means for further customization of the system via a package manager, development tools, a desktop environment etc.
However, if you are creating a device that has a very specific set of use cases and needs to operate under strict constraints in regards to resources, power consumption, performance, or robustness, then using a full-fledged desktop operating system is infeasible. This is very common when developing embedded systems, meant to be used in an IoT application, the automotive or telecom industry, and so forth.
To create an Embedded Linux operating system, there are two paths you can follow:
1. Create a "golden image"; often the first choice when prototyping with a development board, e.g. a Raspberry Pi.
An off-the-shelf operating system, such as Debian, is hacked until it fits the purpose. Then clones of that "golden image" are created and installed in more devices.
This is good enough for creating a proof-of-concept but quickly falls short when the project becomes more complicated, larger, or the need to develop product variants arises.
2. Create a configuration-based distribution; the industry-proven way to create an Embedded Linux operating system.
Instead of maintaining the operating system as a big binary, its components are specified in version controllable configuration files. Setting everything up may require a steeper learning curve since things are not already conveniently in place. However, it is the only sustainable way to go forward when it comes to serious Embedded Linux development.
More on the topic: https://www.linuxjournal.com/content/linux-iot-development-adjusting-binary-os-yocto-project-workflow
** WORKSHOP **
In this workshop, we are going to demonstrate the "ingredients" needed to build your own Linux distribution using the Yocto project. Then we will run our distribution on a Raspberry Pi board.
Yocto is a collection of tools and processes enabling the creation of configurable Embedded Linux distributions. You define which components are to be included in your operating system with software- and hardware-specific configuration files. "Recipes" determine how these components are built and also what they depend upon. Finally, these recipes are utilized by "bitbake" (a tool offered by Yocto) to build your custom Linux image.
During the workshop, we will go over the setup needed to create a custom operating system for a typical IoT device. It boots up, automatically connects to the internet, and launches a C++ demon that fetches information from a cloud API.
4. Topics
● Why create your own Linux distribution?
● What is Yocto?
● How to configure hardware & software?
● How to include your own component in an image?
● How to launch your component on boot?
● How to connect to WiFi on boot?
● How to cross compile for your distro?
● How to configure the Linux kernel?
6. Why
dimitriOS?
platisd/meta-dimitriOS
● Got tired of following the golden image
approach
● As projects get complex, maintaining a
modified image becomes unsustainable
● Reproducibility is really low
● Maintaining variants would be a nightmare
● Updating to a new operating system version
would require major rework
● Need a standardized, version controllable
method to configure and create embedded
operating systems
7. About Yocto ● "Umbrella open source project that builds and
maintains validated open source tools and
components associated with embedded Linux"
● A collection of tools and processes enabling the
creation of configurable Linux distributions
● Used a lot for embedded systems
○ IoT
○ Automotive
○ Wearables
○ Industry
● Open source
● Little to no assumptions on hardware architecture,
programming languages, init system etc
● Good documentation (but too much for beginners)
8. Basic
glossary
Recipe: Specifies
how a particular
package is built.
They include all the
dependencies,
source code
locations,
configuration,
compilation, build,
install and remove
instructions. They
are stored in layers.
(employee-getter_git.bb)
Layer: A collection
of related recipes.
Layers allow you to
customize your
build, and isolate
information for
multiple
architecture builds.
Layers are
hierarchical in their
ability to override
previous
specifications.
(meta-dimitriOS)
Bitbake: a scheduler
and execution
engine which
parses recipes. It
then creates a
dependency tree to
order the
compilation,
schedules the
compilation of the
included code, and
finally, executes the
building of the
specified, custom
Linux image.
9. Basic glossary .conf files: Files which hold
global definitions of
variables, user defined
variables and hardware
configuration information.
They tell the build system
what to build and put into
the image to support a
particular platform.
(local.conf, dimitrios.conf)
OpenEmbedded-Core:
oe-core is meta-data
comprised of foundation
recipes, classes and
associated files that are
meant to be common
among many different
OpenEmbedded-derived
systems, including the
Yocto Project.
(meta-openembedded)
BSP: The Board Support
Package. A layer
containing
hardware-specific drivers
and utilities, necessary for
the Operating System to
function.
(meta-raspberrypi)
10. Basic glossary Packages: The output of
the build system used to
create your final image.
(.rpm, .deb etc)
Distribution: A
hardware-agnostic
(abstract) representation
of an operating system.
(dimitriOS.conf)
Image: A binary form of a
Linux distribution
(operating system)
intended to be flashed onto
a device.
(.bin, .sdcard etc)
Read more
11. Poky
Poky: A reference embedded
distribution and a reference test
configuration created to provide a
base level functional distro which
can be used to illustrate how to
customize a distribution
12. (Over)Simplified
layered system view
Vendor BSP
(RPi drivers etc)
Linux kernel
Your Distro
(dimitriOS common functionality)
Your product variant
(workshop-image)
employee_getter
24. Environment
variables
(BB_ENV_EXTRAWHITE)
The build does not have access to your host's environment
variables unless they are stated in BB_ENV_EXTRAWHITE!
1. Export variables
● export WIFI_SSID=your-ssid
● export WIFI_PASSWORD=your-password
2. Register them through BB_ENV_EXTRAWHITE
● export BB_ENV_EXTRAWHITE="WIFI_SSID WIFI_PASSWORD"
25. Cross-compile
for your
embedded OS
To cross compile an executable that will run on your
embedded OS, you need:
1. Compiler toolchain for the target system
2. All dependencies compiled for the target system
This is very cumbersome to set up manually!
To cross-compiling employee_getter, what about:
⁇ Compiler for Raspberry Pi
⁇ Compile-time dependencies:
○ #include <curl/curl.h>
○ #include <nlohmann/json.hpp>
26. Cross-compile
for your
embedded OS
Yocto can produce an SDK for you!
Cross compiling for your image now becomes trivial:
1. Build SDK
● bitbake workshop-image -c populate_sdk
2. Install SDK
● ./dimitriOS-glibc-x86_64-workshop-arm1176jzfshf-vfp-raspberrypi0-wifi-toolchain-3.1.4.sh
3. Source SDK
● . ~/opt/dimitriOS/3.1.4/environment-setup-arm1176jzfshf-vfp-poky-linux-gnueabi
4. Configure and build (e.g. with CMake and make)
● mkdir build
● cd build
● cmake ..
● make
5. Verify employee_getter is cross-compiled:
● file employee_getter
> employee_getter: ELF 32-bit LSB shared object, ARM
27. Configure
Linux kernel
with
fragments
If you need to load or remove kernel modules, you can use
the menuconfig tool to produce fragments (.cfg) and
include them to your image.
● Run menuconfig for RPi
○ bitbake linux-raspberrypi -c menuconfig
● diff the new configuration and place the delta into a
.cfg file
○ CONFIG_ATH9K_AHB=y
● Append to the kernel recipe
○ linux-raspberrypi_%.bbappend
For more on how to get started, look at the
QuickAndDirtyKernelConfig guide.
28. Building the
image
● Takes a long time at first!
● You are building the entire OS from source
● Bitbake will try to utilize all your CPU cores
● Tip: Let it run over night
$ bitbake workshop-image
Flashable image found inside build folder at:
tmp/deploy/images/raspberrypi0-wifi/workshop-raspberrypi0-wifi.rpi-sdimg
Root filesystem found inside build folder at:
tmp/work/raspberrypi0_wifi-poky-linux-gnueabi/workshop-image/1.0-r0/rootfs
