Unikernels offer a exciting opportunity to rethink kernel design choices and experiment with new low-level features that may affect the performance and security of applications in the cloud and ultimately change the way they are used. This talk is about my experience building Solo5, an open-source kernel library that runs directly on virtual hardware, at the lowest layer of a unikernel. The goal of Solo5 is to better understand the effect of the lowest layer of a unikernel on its behavior (e.g., performance), bring MirageOS to more hypervisors, and also provide a platform for further experimentation with unikernel architectures. Many of the most popular Unikernels (including MirageOS and ClickOS) rely on Xen Project's Mini-OS as a thin kernel library between the (para)virtual hardware and the rest of the unikernel. These unikernels are reported to have impressive performance, especially boot time (~20ms), which challenges traditional notions of the cost of virtualization. With Solo5, we first ask the question: what role does Mini-OS (or paravirtualization) play in achieving this performance? Like Mini-OS, Solo5 is a thin kernel library. Unlike Mini-OS, Solo5 runs on fully virtualized hardware rather than paravirtualized hardware. In particular, Solo5 runs on KVM/QEMU (or other x86_64 virtualization environments that expose virtio devices). It currently supports MirageOS unikernels and therefore can can be thought of as an alternative to Xen Project's Mini-OS that runs underneath OCaml in a typical MirageOS stack. Solo5 is very much a work in progress. I will describe some of the interesting directions going forward, show a demo of a MirageOS/Solo5 unikernel running on KVM/QEMU, and detail the steps for others to get involved and try it out!

1. Building a unikernel base from scratch
Dan Williams, IBM Research
2016 Unikernels and More: Cloud Innovators Forum
January 22, 2016, Pasadena, CA
Solo5

2. ©2016 IBM Corporation2 20 January 2016
§ For the purposes of this talk, think of MirageOS unikernels
– Are tiny VMs running on Xen
– Run one program (no more, no less)
– Are written in OCaml
§ Many potential benefits
– Security
– Performance
– Ops
Unikernels are great
Xen-based Cloud
OCaml
Program

3. ©2016 IBM Corporation3 20 January 2016
Inside a unikernel
Unikernel
application code
libraries and
runtime
unikernel base
OCamlC
Hypervisor

4. ©2016 IBM Corporation4 20 January 2016
§ Low-level hypervisor-interfacing code
§ Example: Mini-OS
– Demonstrates Xen PV interface
– Used by MirageOS, ClickOS,
HalVM, LING, etc.
Inside a unikernel
Unikernel
application code
libraries and
runtime
unikernel base
OCamlC
Hypervisor

5. ©2016 IBM Corporation5 20 January 2016
§ Built from scratch
§ Available on Github
– https://github.com/djwillia/solo5
Solo5: a new unikernel base
Unikernel
application code
libraries and
runtime
Solo5
OCamlC
Hypervisor

6. ©2016 IBM Corporation6 20 January 2016
§ Where a unikernel can run
§ How fast a unikernel can boot
§ What higher layers can do
Why focus on the unikernel base?
Unikernel
application code
libraries and
runtime
unikernel base
OCamlC
Hypervisor

7. ©2016 IBM Corporation7 20 January 2016
§ Different hypervisors expose different abstractions
– Full virtualization (e.g., KVM/QEMU)
– Paravirtualization (e.g., Xen PV)
– Mini-OS was designed for Xen PV
§ Device interfaces
– PV device access (Xen, virtio)
– Physical device access (SR-IOV)
§ Defined by interaction between hypervisor and unikernel base
Where a unikernel can run
Mini-OS
Xen PV
Solo5
KVM/QEMU

8. ©2016 IBM Corporation8 20 January 2016
§ 20ms boot time
– ClickOS and Jitsu
– Both built on mini-OS
§ Is PV essential?
§ What is the role of the
hypervisor toolstack vs.
the unikernel base?
How fast a unikernel can boot
Image from: https://github.com/mirage/jitsu
§ Defined by interaction between hypervisor and unikernel base

9. ©2016 IBM Corporation9 20 January 2016
§ Base for language runtime
– MirageOS (OCaml), LING (Erlang), HalVM (Haskell), etc.
§ Base for native applications
– ClickOS (Click router), etc.
§ Exposing primitives
– Memory protection or tracing
– Address space layout randomization
– Support for thread/event model
What higher layers can do

10. ©2016 IBM Corporation10 20 January 2016
§ The unikernel base is fundamentally important!
§ The best way to really understand (and then innovate on) this
layer is to build one (Solo5)
§ But hopefully it can be useful to others
– Ensure existing higher layers still work à MirageOS
– Broaden where MirageOS can run à KVM/QEMU
§ Solo5 runs MirageOS on KVM/QEMU
Summary

11. ©2016 IBM Corporation11 20 January 2016
§ Why focus on the unikernel base?
§ How to build a unikernel base (Solo5) from scratch
§ How you can try it out
Roadmap

12. ©2016 IBM Corporation12 20 January 2016
MirageOS in a bit more detail
§ Application (OCaml)
Config
files
App
Code

13. ©2016 IBM Corporation13 20 January 2016
MirageOS in a bit more detail
§ Application (OCaml)
§ OCaml libraries
TCP/IP
HTTP
serving
Lwt FS
Config
files
App
Code

14. ©2016 IBM Corporation14 20 January 2016
MirageOS in a bit more detail
§ Application (OCaml)
§ OCaml libraries
§ Platform bindings
– OCaml runtime
– Calls out to a subset
of libc
– Calls out to some
Xen-specific functions
TCP/IP
HTTP
serving
Lwt FS
Config
files
App
Code
mirage-platform
bindings

15. ©2016 IBM Corporation15 20 January 2016
MirageOS in a bit more detail
§ Application (OCaml)
§ OCaml libraries
§ Platform bindings
§ Drivers
– Written in OCaml
– Xen PV split model
– Call out to platform
TCP/IP
HTTP
serving
Lwt FS
Config
files
App
Code
mirage-
net-xen
mirage-
blk-xen
mirage-
console-xen
mirage-platform
bindings

16. ©2016 IBM Corporation16 20 January 2016
MirageOS in a bit more detail
Mini-OS kernel
Low-level Xen PV primitives
§ Application (OCaml)
§ OCaml libraries
§ Platform bindings
§ Drivers
§ Unikernel base
– Contains some libc
– Low-level Xen info
TCP/IP
HTTP
serving
Lwt FS
Config
files
App
Code
Xen PV
mirage-
net-xen
mirage-
blk-xen
mirage-
console-xen
mirage-platform
bindings

17. ©2016 IBM Corporation17 20 January 2016
MirageOS in a bit more detail
Mini-OS kernel
Low-level Xen PV primitives
§ Application (OCaml)
§ OCaml libraries
§ Platform bindings
§ Drivers
§ Unikernel base
§ Tooling
VM
TCP/IP
HTTP
serving
Lwt FS
Config
files
App
Code
Xen PV
mirage-
net-xen
mirage-
blk-xen
mirage-
console-xen
mirage-platform
bindings

18. ©2016 IBM Corporation18 20 January 2016
MirageOS on Solo5
Mini-OS kernel
Low-level Xen PV primitives
§ Application (OCaml)
TCP/IP
HTTP
serving
Lwt FS
Config
files
App
Code
mirage-
net-xen
mirage-
blk-xen
mirage-
console-xen
mirage-platform
bindings

19. ©2016 IBM Corporation19 20 January 2016
MirageOS on Solo5
Mini-OS kernel
Low-level Xen PV primitives
§ Application (OCaml)
§ OCaml libraries
– No changes!
TCP/IP
HTTP
serving
Lwt FS
Config
files
App
Code
mirage-
net-xen
mirage-
blk-xen
mirage-
console-xen
mirage-platform
bindings

20. ©2016 IBM Corporation20 20 January 2016
MirageOS on Solo5
Mini-OS kernel
Low-level Xen PV primitives
§ Application (OCaml)
§ OCaml libraries
§ Platform bindings
– OCaml runtime
– Calls out to a subset
of libc
– Rewrite Xen-specific
functions
TCP/IP
HTTP
serving
Lwt FS
Config
files
App
Code
mirage-
net-xen
mirage-
blk-xen
mirage-
console-xen
mirage-platform
bindings

21. ©2016 IBM Corporation21 20 January 2016
MirageOS on Solo5
Mini-OS kernel
Low-level Xen PV primitives
mirage-platform
bindings
§ Application (OCaml)
§ OCaml libraries
§ Platform bindings
§ Drivers
– virtio instead of Xen
– Access PCI bus
– Solo5 drivers do most
of the work in C with
wrappers in OCaml
TCP/IP
HTTP
serving
Lwt FS
Config
files
App
Code
mirage-
net-solo5
mirage-
blk-solo5
mirage-
console-solo5
virtio net
driver
virtio blk
driver
console driver

22. ©2016 IBM Corporation22 20 January 2016
MirageOS on Solo5
Solo5 kernel
Low-level HW primitives
mirage-platform
bindings
§ Application (OCaml)
§ OCaml libraries
§ Platform bindings
§ Drivers
§ Unikernel base
– Some libc
– HW initialization
– Memory, Interrupts
– No threads, address
spaces
TCP/IP
HTTP
serving
Lwt FS
Config
files
App
Code
mirage-
net-solo5
mirage-
blk-solo5
mirage-
console-solo5
virtio net
driver
virtio blk
driver
console driver
KVM/QEMU

23. ©2016 IBM Corporation23 20 January 2016
MirageOS on Solo5
§ Application (OCaml)
§ OCaml libraries
§ Platform bindings
§ Drivers
§ Unikernel base
§ Tooling
– mirage tool
– Makefile
VM
TCP/IP
HTTP
serving
Lwt FS
Config
files
App
Code
mirage-
net-solo5
mirage-
blk-solo5
mirage-
console-solo5
mirage-platform
bindings
Solo5 kernel virtio net
driver
virtio blk
driver
console driver
Low-level HW primitives
KVM/QEMU

24. ©2016 IBM Corporation24 20 January 2016
§ Why focus on the unikernel base?
§ How to build a unikernel base (Solo5) from scratch
§ How you can try it out
Roadmap

25. ©2016 IBM Corporation25 20 January 2016
§ On a Linux host with the KVM module
§ Build and run from a Docker container
– Fetch the image
– Start a privileged container
– Enter the container
– Build and run!
How you can try it out
docker pull djwillia/solo5-mirage
docker run –d privileged –name solo5-mirage –t djwillia/solo5-mirage
docker exec –it solo5-mirage /bin/bash -l
cd ~/solo5
make config_console
make kvm

26. ©2016 IBM Corporation26 20 January 2016
§ Boot time investigation
– A bootable iso in KVM/QEMU will be too slow
– What about KVM/lkvm?
§ How much of Solo5 can be pushed:
– Down into the hypervisor?
– Up into MirageOS (OCaml)?
§ What should the hypervisor/unikernel base interface be?
Next steps with Solo5

27. ©2016 IBM Corporation27 20 January 2016
§ Bare unikernel base to build from
– https://github.com/djwillia/solo5
§ MirageOS on Solo5 on KVM/QEMU
– https://github.com/djwillia/solo5/tree/mirage
§ Contact me!
– djwillia@us.ibm.com
Thank you!
| ___|
__| _ | _ __
__ ( | | ( | ) |
____/___/ _|___/____/
hello world

28. ©2015 IBM Corporation