From Zero to Hero: Contribute to Linux Kernel in 15 Minutes

From Zero to Hero:
Contribute to Linux Kernel in 15 minutes
Ruslan Bilovol
Associate Manager
Kharkiv 2019

2
Agenda
• Linux Kernel - History
• Role in the system
• Download, configure, build
• Linux versioning scheme and development process
• Contribution process
• Demo
• Questions

4
Linux Kernel: history
• The Linux kernel is one component of a system, which also requires
libraries and applications to provide features to end users
• The Linux kernel was created just for fun in 1991 by a Finnish student,
Linus Torvalds
– Only i386 was supported
– There were only about 10,000 lines of code
– Linux quickly started to be used as the kernel for free software operating
systems
– Ported to many other architectures
• Linus Torvalds has been able to successfully create a large and
dynamic developer and user community around Linux

5
Linux Kernel: history
• Nowadays: changing at an unprecedented speed
• 2017 Linux Kernel Development Report by The Linux Foundation:
– 24,7M lines of code
– 15600 developers from >1500 companies have contributed since 2005
– 4319 developers from nearly 519 companies in the past 15 months
– 8.5 patches merged per hour
– nearly 15 files and 7500 lines of code added every day
– a new kernel every 9-10 weeks

7
Role in the system
• Manages all system resources: CPU, memory, I/O
• A hardware-abstraction layer for userspace applications and libraries
– provides a set of architecture and hardware independent APIs
• Handles concurrent accesses and usage of hardware resources from
different applications.
– Example: a single Hard Disk Drive is used by multiple user space
applications. The kernel is responsible to “multiplex” the hardware
resource.

10
Download, configure, build
• The official versions of the Linux kernel are available at
http://www.kernel.org
– Work-in-progress (mainline), stable and long-term kernels
– May not contain latest development for specific subsystems (not ready for
inclusion yet)
• Kernel sub-communities maintain their own kernel repos (like
architecture-specific, drivers, kernel infrastructure etc)
– Only development trees available

11
• The kernel sources from http://kernel.org are available as full tarballs
(complete kernel sources) and patches (differences between two
kernel versions).
• However, usually people use the git version control system. Must have
for kernel development!
– Fetch the entire kernel sources and history
• git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
– Create a branch that starts at a specific kernel version
• git checkout -b v5.1
– Web interface available at
http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/

12
• The kernel configuration and build system is based on multiple
Makefiles
• Configure kernel using make
– cd linux-5.1/
– make menuconfig
– make defconfig
• Build kernel
– make
• Produces
– vmlinux, the raw uncompressed kernel image in the ELF format
– arch/<arch>/boot/*Image, the final, usually compressed, kernel image
– all kernel modules, spread over the kernel source tree as .ko files.

13
Linux versioning scheme and
development process

14
Linux versioning scheme and development process
• Early days (1990s)
– Linus reads/audits every patch
• each patch was audited and often rewritten by Linus before applying
• during 2.1 Linus burned out
– Beginning of Maintainers era
• Linus’ audit/rewrite tasks were delegated to maintainers
• made Linus' job more manageable
• Linus still spends a lot of his time reading and auditing code

15
• “Linus doesn’t scale”
• Maintainers era
– Patches are collected by maintainers of various subsystems
– Linus trusts to top-level maintainers, and accepts pull requests from
– Top-level maintainers trust to submaintaners
– Sometimes Linus still audits patches in pull requests

16
• Linux kernel
maintainers
structure
subsystem
maintainer
Andrew
Morton
linux-next
Linus
Torvalds
subsystem
maintainer
subsystem
maintainer
sub-
maintainer
sub-
maintainer
sub-
maintainer
sub-
maintainer
developer developer developer developer developer

17
• Until Linux 2.6
– Coexistence of stable and development branches
– Stable major branch every 2 or 3 years
• Identified by an even middle number
• Examples: 1.0.x, 2.0.x, 2.2.x, 2.4.x
– Development branch to integrate new functionalities and major changes
• Identified by an odd middle number
• Examples: 2.1.x, 2.3.x, 2.5.x
• After stabilization, a development version becomes the new base version for
the stable branch
– Minor releases once in while: 2.2.23, 2.5.12, etc.

18
• Until Linux 2.6
2.4.22.4.0 2.4.3 2.4.4
2.5.12.5.0 2.5.2 2.6.02.5.1 2.5.2
stable
development stable

19
• Changes since Linux 2.6
– Since 2.6.0, new features were added without disruptive changes to
existing subsystems
– There was no need anymore to create a development branch breaking
compatibility with the stable branch
– Features are released faster, no breaking kernel switch

20
• Versions since 2.6.0
– From 2003 to 2011, the official kernel versions were named 2.6.x.
– Linux 3.0 was released in July 2011
– Linux 4.0 was released in April 2015
– Linux 5.0 was released in March 2019
– Only a change to the numbering scheme, no breaking changes in Linux
Kernel
• Linus just tired to count big numbers

21
• Current development model using merge and bug fixing windows
Merge window Bug fixing and kernel stabilization
2 weeks 7-10 weeks
4.16 4.17-rc1 4.17-rc2 4.17-rc3 4.17-rcN 4.17
4.16.1 4.16.2 4.16.3 4.16.4 4.16.5 4.16.6
4.17.1

22
• Linux Stable
– Issue: bug and security fixes only released for most recent kernel
versions.
• but many people want to have long-term support with security fixes only
– Solution: Linux Stable kernels maintained for many years
• The https://kernel.org shows which versions will be supported for some time
(up to 5 years)
• Some distributions support Linux Kernel for more time (RHEL, Ubuntu)
– Civil Infrastructure Project
• Linux Kernel v4.4 maintained for more than 20 years

24
Contribution process
Identify a problem
in Linux Kernel
Write a
patch
Verify the
patch
Submit the
patch to
community
Patch review by
maintainer and
community
Patch
accepted?
You are
a Hero!
Reasonable
review
comments?
Rework
your patch
no
yes
no

25
Problem identification
• Linux is written by different people
– may have missing features
– some subsystems need refactoring
– new/existing bugs that live in the kernel
• New hardware is coming
• New software requirements
• Experienced Linux Kernel developer can identify many of them

26
Problem identification - easy way
• Linux Kernel Bugzilla - https://bugzilla.kernel.org/
– bugs, reported by users
• Do grep sources for “TODO|FIXME|REVISIT” comments
– often marks a work that still needs to be done
– 10962 occurrences in Linux 4.17-rc6!

27
Problem identification - easy way
• Run code analyzing tools
– sparse (semantic checker for C programs)
– smatch (built on top of sparse, extends its funcitonality)
– coccinelle (utility for matching and transforming C source code)
– cppcheck (a tool for static C/C++ code analysis)
• Run kernel sanitizers
– KASAN, kmemleak, UBSAN, …

28
Identify a problem
in Linux Kernel
Write a
patch
Verify the
patch
Submit the
patch to
community
Patch review by
maintainer and
community
Patch
accepted?
You are
a Hero!
Reasonable
review
comments?
Rework
your patch
no
yes
no

29
Write a patch
• Search over mailing lists, to see if anybody already works on the same
topic
– there may be already a work in progress
– but you can help with testing
• Notify community about work you’ve started doing
– useful in case of huge features, difficult bugs, hot topics
– because somebody else can start to do the same work in the same time
• this may save somebody’s else time
– other contributors may want to align with your on changes you are doing

30
Write a patch
• Be based on latest sources
• Respect Linux Kernel coding style
– Documentation/process/coding-style.rst
• slightly modified K&R indentation style
• naming, writing functions, commenting
• line length (80 symbols) rule
– Some subsystems may have slightly different coding style
• due to historical or other reasons
• Don’t reinvent the wheel (use existing macro, helpers)
– Linux Kernel has rich set of such helpers
– But you can write a new one if needed
– See Andy’s presentation “Typical mistakes when submitting a new code to
Linux kernel” from RLC2018

31
Write a patch
• Write a good commit message (1/2)
– first line (summary) - where and what you are doing
• “media: tm6000: avoid casting just to print pointer address”
– describe why you are doing this change
• don’t be too short or too long
• don’t repeat your code change
• ideally, external people should understand what and why you are doing
without looking into code
– sign your work
• certifies that you wrote the patch or otherwise have the right to pass it on as
an open-source patch
– remember “50/72" rule
• https://medium.com/@preslavrachev/what-s-with-the-50-72-rule-8a906f61f09c

32
Write a patch
• Write a good commit message (2/2)
– add to Cc people who can be interested in your patch
– if you fix a known patch that introduces bug, put “Fixes:” tag
– if you fix a bug that exists in Linux Stable branches, don’t forget to put
“stable” mailing list to Cc.
– more recommendations can be found in
Documentation/process/submitting-patches.rst

33
Identify a problem
in Linux Kernel
Write a
patch
Verify the
patch
Submit the
patch to
community
Patch review by
maintainer and
community
Patch
accepted?
You are
a Hero!
Reasonable
review
comments?
Rework
your patch
no
yes
no

34
Verify your patch
• At least build it
– build also for other CPU architectures
– build also for different kernel configuration
• Run patched kernel on a real hardware
– do some unit-testing
– inspect kernel log, make sure there is no new crashes/traces/kernel panic
– run automated tests (like LTP)
– run this kernel on other HW, other CPU architectures
• your change can be used another way
– ask community to test your patch
• Remember: we do not break userspace!
– Linux Kernel <-> Userspace API must be backward compatible

36
Verify your patch
• Run ./script/checkpatch.pl to see obvious patch issues
• Run static analyzers
• Run kernel sanitizers

37
Identify a problem
in Linux Kernel
Write a
patch
Verify the
patch
Submit the
patch to
community
Patch review by
maintainer and
community
Patch
accepted?
You are
a Hero!
Reasonable
review
comments?
Rework
your patch
no
yes
no

38
Submit the patch to community
• Found maintainer in MAINTAINERS to get emails of maintainers,
reviewers and mailing lists
– preferred way - use ./scripts/get_maintainer.pl
• Send your patch using git-send-email
– Don’t send your patch from web email interface, often patch will be
mangled

39
Identify a problem
in Linux Kernel
Write a
patch
Verify the
patch
Submit the
patch to
community
Patch review by
maintainer and
community
Patch
accepted?
You are
a Hero!
Reasonable
review
comments?
Rework
your patch
no
yes
no

40
Patch review by community
• If the patch isn’t an obvious fix, Maintainer often doesn’t accept it
quickly
– like whiskey - it should be aged to become good
– this gives some time for other to review and test it
• If there is no feedback in a few weeks - ping community
– don’t do it too often - don’t produce extra noise to busy mailing lists
– reviewers may be busy during release cycle
– there may be no people who can review your change
• and you can become a maintainer
– if months passed, it may be needed to rebase/resend your patch, to
remind community

41
Patch review by community
• Discussion may be quick (ACK) or long - alternative approaches may
be proposed
• You may need to argument your point of view
• But you may also need to accept relevant comments
• Be polite!

42
Rework your patch
• Your patch may contain bugs, there may be alternative approach
proposed during discussion
– write a new version of your patch
– describe what’s changed comparing to previous version
– do all needed verifications and preparations
• It may take years and many versions to be accepted
• Examples:
– arm64 kdump support: 36 versions, years to complete
– virtio-balloon: 32 versions
– KPTI (for Meltdown fixes) 196 versions, not all were published

43
Identify a problem
in Linux Kernel
Write a
patch
Verify the
patch
Submit the
patch to
community
Patch review by
maintainer and
community
Patch
accepted?
You are
a Hero!
Reasonable
review
comments?
Rework
your patch
no
yes
no

44
Patch acceptance
• You may be notified or not - depends on Maintainer
– some do manually reply “thanks, applied”
– some have scripts to notify patch author
– some apply it silently

46
You are a Hero - what’s next?
• Watch mailing lists for some time - your patch may cause regressions
• Watch mailing lists for people who can modify your code
– they may need some help understanding your changes
• Start working on a new amazing feature for Linux Kernel :)

49
Bonus slides - misc stories

50
Stories
• “From commit to revert”
• Noralf and DRM refactoring
• Maintainers and beer

51
Sources
kernel.org
https://lkml.org/lkml/2011/11/3/110

From Zero to Hero: Contribute to Linux Kernel in 15 Minutes

Recomendados

Recomendados

Mais conteúdo relacionado

Mais de GlobalLogic Ukraine

Mais de GlobalLogic Ukraine (20)

Último

Último (20)

From Zero to Hero: Contribute to Linux Kernel in 15 Minutes