1. Introduction Network path Bootloader Device model Xen Conclusion
Securing Your Cloud with Xen Project’s
Advanced Security Features
Russell Pavlicek, Xen Project Evangelist
CloudOpen North America 2013
2. Introduction Network path Bootloader Device model Xen Conclusion
Who is the Old, Fat Geek Up Front?
Xen Project Evangelist
Employed by Citrix, focused entirely on the Xen Project
History with Open Source begins in 1997
Former columnist with Infoworld, Processor magazines
Former panelist on The Linux Show webcast, repeat guest on
The Linux Link Tech Show
Over 150 pieces published, plus one book on Open Source
development and several blogs
CloudOpen North America 2013 Securing Your Cloud with Xen Project’s Advanced Security Features 2 / 32
3. Introduction Network path Bootloader Device model Xen Conclusion
Introduction: Xen Project and Security
Xen Project is an enterprise-grade Type I hypervisor
Built for the Cloud before it was called the Cloud
A number of advanced security features
Driver Domains, Stub Domains, FLASK, and more
Most of them are not (or cannot) be turned on by default
Although they are simple to use, sometimes they can appear
to be complicated
CloudOpen North America 2013 Securing Your Cloud with Xen Project’s Advanced Security Features 3 / 32
4. Introduction Network path Bootloader Device model Xen Conclusion
Presentation Goals
Introduce you to key Xen Project Security Tools
Discuss some key Xen security features
Get you started in the right direction toward securing your
Xen installation
CloudOpen North America 2013 Securing Your Cloud with Xen Project’s Advanced Security Features 4 / 32
5. Introduction Network path Bootloader Device model Xen Conclusion
Presentation Outline
A few thoughts on the problem of securing the Cloud
Overview of the Xen architecture
Brief introduction to principles of security analysis
Consider some attack surfaces and Xen features we can use to
mitigate them:
Driver Domains
PVgrub
Stub Domains
Paravirtualization (PV) mode vs Hardware Virtualization
(HVM) mode
FLASK example policy
CloudOpen North America 2013 Securing Your Cloud with Xen Project’s Advanced Security Features 5 / 32
6. Introduction Network path Bootloader Device model Xen Conclusion
The Cloud Security Conundrum
Security: The 800 pound gorilla of the Cloud world
Nothing generates more fear in specific, and FUD in general
Probably the single greatest barrier to Cloud adoption
Immediately behind it is the inability to get out of a 20th
century IT mindset (i.e., ”Change is Bad”)
The good news: we don’t need to fear it – we just need to
solve it
CloudOpen North America 2013 Securing Your Cloud with Xen Project’s Advanced Security Features 6 / 32
7. Introduction Network path Bootloader Device model Xen Conclusion
Cloud Security: New Visibility to Old Problem
Security has always been an issue
Putting a truly secure system in the open does not reduce its
security, just increases the frequency of attack
Unfortunately, system security behind the firewall has not
always been comprehensive
Having solutions in Clouds forces us to solve the security
issues we should have already solved
Security through obscurity is no longer sufficient
CloudOpen North America 2013 Securing Your Cloud with Xen Project’s Advanced Security Features 7 / 32
8. Introduction Network path Bootloader Device model Xen Conclusion
Security by Design, not by Wishful Thinking
Security by Wishful Thinking is Officially Dead
Merely hoping that your firewall holds off the marauding
hordes is NOT good enough
Addressing security in one area while ignoring others is NOT
good enough
Saying, ”We’ve never had a problem before” is NOT good
enough
Comprehensive security starts with design
It needs to planned and carefully thought through
It needs to be implemented at multiple levels
It needs components which are themselves securable
CloudOpen North America 2013 Securing Your Cloud with Xen Project’s Advanced Security Features 8 / 32
9. Introduction Network path Bootloader Device model Xen Conclusion
Xen Project: Security by Design
Xen Project was designed for Clouds before the term ”Cloud”
was coined in the industry
Designers foresaw the day of an ”infrastructure for wide-area
distributed computing” which we now call ”the Cloud”
http://www.cl.cam.ac.uk/research
/srg/netos/xeno/publications.html
Xen is designed to thwart attacks from many attack vectors,
using different defensive techniques
CloudOpen North America 2013 Securing Your Cloud with Xen Project’s Advanced Security Features 9 / 32
10. Introduction Network path Bootloader Device model Xen Conclusion
Xen Architecture: A Basic Picture
Xen Hypervisor
Hardware
device model
(qemu)
toolstack
dom 0
Hardware
Drivers
I/O Devices CPU Memory
Paravirtualized
(PV)
Domain
Fully
Virtualized
(HVM)
Domainnetback
blkback
netfront
blkfront
CloudOpen North America 2013 Securing Your Cloud with Xen Project’s Advanced Security Features 10 / 32
11. Introduction Network path Bootloader Device model Xen Conclusion
Security Overview
Threat Models:
Attacker can access network
Attacker controls one Guest VM
Security considerations to evaluate:
How much code is accessible?
What is the interface like? (e.g., pointers vs scalars)
Defense-in-depth
Then combine security tactics to secure the installation
There is no single ”magic bullet”
Individual tactics reduce danger; combined tactics go even
farther
CloudOpen North America 2013 Securing Your Cloud with Xen Project’s Advanced Security Features 11 / 32
12. Introduction Network path Bootloader Device model Xen Conclusion
Example System, for our Discussion
Hardware setup
Two networks: one Control network, one Guest network
IOMMU with interrupt remapping (AMD or Intel VT-d v2) to
allow for full Hardware Virtualization (HVM)
Default configuration
Network drivers in the Control Domain (aka ”Domain 0” or
just ”Dom0”)
Paravirtualized (PV) guests using PyGrub (grub-like boot
utility within context of Guest Domain)
Hardware Virtualized (HVM) guests using Qemu (as the device
model) running in the Control Domain
CloudOpen North America 2013 Securing Your Cloud with Xen Project’s Advanced Security Features 12 / 32
13. Introduction Network path Bootloader Device model Xen Conclusion
Attack surface: Network path
Xen Hypervisor
Hardware
toolstackdom 0
NIC
Driver
Control NIC
Rogue
Domain
netback netfront
Guest NIC
bridgeiptables
Domain
netfront
Where might an exploit focus?
Bugs in hardware driver
Bugs in bridging / filtering
Bugs in netback (via the ring protocol)
CloudOpen North America 2013 Securing Your Cloud with Xen Project’s Advanced Security Features 13 / 32
14. Introduction Network path Bootloader Device model Xen Conclusion
Attack surface: Network path
Xen Hypervisor
Hardware
toolstackdom 0
NIC
Driver
Control NIC
Rogue
Domain
netback netfront
Guest NIC
bridgeiptables
Domain
netfront
What could a successful exploit yield?
Control of Domain 0 kernel
Pretty much control of the whole system
CloudOpen North America 2013 Securing Your Cloud with Xen Project’s Advanced Security Features 14 / 32
15. Introduction Network path Bootloader Device model Xen Conclusion
Security feature: Driver Domains
Xen Hypervisor
Hardware
toolstack
dom 0
NIC
Driver
Control NIC
Rogue
Domain
netback
netfront
Guest NIC
bridgeiptables
Domain
netfront
NIC
Driver
Driver Domain
What is a Driver Domain?
Unprivileged VM which drives hardware, provides access to
guests
CloudOpen North America 2013 Securing Your Cloud with Xen Project’s Advanced Security Features 15 / 32
16. Introduction Network path Bootloader Device model Xen Conclusion
Security feature: Driver Domains
Xen Hypervisor
Hardware
toolstack
dom 0
NIC
Driver
Control NIC
Rogue
Domain
netback
netfront
Guest NIC
bridgeiptables
Domain
netfront
NIC
Driver
Driver Domain
Now a successful exploit could yield:
Control of the Driver Domain (PV hypercall interface)
Control of that guest’s network traffic
Control of NIC
An opportunity to attack netfront of other guests
CloudOpen North America 2013 Securing Your Cloud with Xen Project’s Advanced Security Features 16 / 32
17. Introduction Network path Bootloader Device model Xen Conclusion
HowTo: Driver Domains
Create a VM with appropriate drivers
Use any distribution suitable as a Control Domain
Install the Xen-related hotplug scripts
Just installing the Xen tools in the VM is usually good enough
Give the VM access to the physical NIC with PCI pass-through
Configure the network topology in the Driver domain
Just like you would for the Control Domain
Configure the guest Virtual Network Interface (vif) to use the
new domain ID
Add backend=domnet to vif declaration
vif = [ ’type=pv, bridge=xenbr0, backend=domnet’ ]
http://wiki.xenproject.org/wiki/Driver Domain
CloudOpen North America 2013 Securing Your Cloud with Xen Project’s Advanced Security Features 17 / 32
18. Introduction Network path Bootloader Device model Xen Conclusion
Attack surface: PyGrub
Xen Hypervisor
toolstackdom 0
Paravirtualized
(PV)
Domain
domain
builder
pygrub
guest
disk
What is PyGrub?
grub implementation for PV guests
Python program running in Control Domain
Reads guest filesystem, parses grub.conf, shows menu
Passes resulting kernel image to domain builder
CloudOpen North America 2013 Securing Your Cloud with Xen Project’s Advanced Security Features 18 / 32
19. Introduction Network path Bootloader Device model Xen Conclusion
Attack surface: PyGrub
Xen Hypervisor
toolstackdom 0
Paravirtualized
(PV)
Domain
domain
builder
pygrub
guest
disk
Where might an exploit focus?
Bugs in file system parser
Bugs in menu parser
Bugs in domain builder
CloudOpen North America 2013 Securing Your Cloud with Xen Project’s Advanced Security Features 19 / 32
20. Introduction Network path Bootloader Device model Xen Conclusion
Attack surface: PyGrub
Xen Hypervisor
toolstackdom 0
Paravirtualized
(PV)
Domain
domain
builder
pygrub
guest
disk
kernel
What could a successful exploit yield?
Control of Domain 0 user space
Pretty much control of the whole system
CloudOpen North America 2013 Securing Your Cloud with Xen Project’s Advanced Security Features 20 / 32
21. Introduction Network path Bootloader Device model Xen Conclusion
Security practice: Fixed kernels
Xen Hypervisor
toolstackdom 0
Paravirtualized
(PV)
Domain
domain
builder
kernel
image
guest
disk
What is a fixed kernel?
Passing a known-good kernel from Control Domain
Removes attacker avenue to domain builder
CloudOpen North America 2013 Securing Your Cloud with Xen Project’s Advanced Security Features 21 / 32
22. Introduction Network path Bootloader Device model Xen Conclusion
Security practice: Fixed kernels
Xen Hypervisor
toolstackdom 0
Paravirtualized
(PV)
Domain
domain
builder
kernel
image
guest
disk
Disadvantages
Host admin must keep up with kernel updates
Guest admin can’t pass kernel parameters, custom kernels,
CloudOpen North America 2013 Securing Your Cloud with Xen Project’s Advanced Security Features 22 / 32
23. Introduction Network path Bootloader Device model Xen Conclusion
Security feature: PVgrub
Xen Hypervisor
toolstackdom 0
domain
builder
guest
disk
MiniOS
pvgrub
What is PVgrub?
MiniOS + PV port of grub running in a guest context
PV equivalent of HVM “BIOS + grub”
Now a successful exploit could yield:
Control of the attacked guest domain alone
CloudOpen North America 2013 Securing Your Cloud with Xen Project’s Advanced Security Features 23 / 32
24. Introduction Network path Bootloader Device model Xen Conclusion
HowTo: PVgrub
Make sure that you have the PVgrub image
pvgrub-$ARCH.gz
Normally lives in /usr/lib/xen/boot
Included in Fedora Xen packages
Debian-based: need to build yourself
Use appropriate PVgrub as bootloader in guest configuration
kernel="/usr/lib/xen/boot/pvgrub-x86_32.gz"
http://wiki.xenproject.org/wiki/Pvgrub
CloudOpen North America 2013 Securing Your Cloud with Xen Project’s Advanced Security Features 24 / 32
25. Introduction Network path Bootloader Device model Xen Conclusion
Attack surface: Device model (Qemu)
Where might an exploit focus?
Bugs in NIC emulator parsing packets
Bugs in emulation of virtual devices
What could a successful exploit yield?
Control Domain privileged userspace
Pretty much control of the whole system
CloudOpen North America 2013 Securing Your Cloud with Xen Project’s Advanced Security Features 25 / 32
26. Introduction Network path Bootloader Device model Xen Conclusion
Security feature: Qemu stub domains
What is a stub domain?
Stub domain: a small “service” domain running just one
application
Qemu stub domain: run each Qemu in its own domain
What could a successful exploit yield?
Control only of the stub domain VM (which, if FLASK is
employed, is a relatively small universe)
You need to devise another attack entirely to do anything
more significant
CloudOpen North America 2013 Securing Your Cloud with Xen Project’s Advanced Security Features 26 / 32
27. Introduction Network path Bootloader Device model Xen Conclusion
HowTo: Qemu stub domains
Make sure that you have the PVgrub image:
ioemu-$ARCH.gz
Normally lives in /usr/lib/xen/boot
Included in Fedora Xen packages
On Debian (and offshoots), you will need to build it yourself
Specify stub domains in your guest config
device_model_stubdomain_override = 1
http://wiki.xenproject.org/wiki/Device Model Stub Domains
CloudOpen North America 2013 Securing Your Cloud with Xen Project’s Advanced Security Features 27 / 32
28. Introduction Network path Bootloader Device model Xen Conclusion
Attack Surface: Xen Hypervisor itself
Where might an exploit focus?
On Paravirtualized (PV) Guests:
PV Hypercalls
On full Hardware Virtualized (HVM) Guests:
HVM hypercalls (Subset of PV hypercalls)
Instruction emulation (MMIO, shadow pagetables)
Emulated platform devices: APIC, HPET, PIT
Nested virtualization
Security practice: Use PV VMs whenever possible
CloudOpen North America 2013 Securing Your Cloud with Xen Project’s Advanced Security Features 28 / 32
29. Introduction Network path Bootloader Device model Xen Conclusion
Security feature: FLASK example policy
What is FLASK?
Xen Security Module (XSM): Xen equivalent of LSM
FLASK: Framework for XSM developed by NSA
Xen equivalent of SELinux
Uses same concepts and tools as SELinux
Allows a policy to restrict hypercalls
What can FLASK do?
Basic: Restricts hypercalls to those needed by a particular
guest
Advanced: Allows more fine-grained granting of privileges
FLASK example policy
This contains example roles for the Control Domain (dom0),
User/Guest Domain(domU), stub domains, driver domains,
etc.
CloudOpen North America 2013 Securing Your Cloud with Xen Project’s Advanced Security Features 29 / 32
30. Introduction Network path Bootloader Device model Xen Conclusion
HowTo: Use the example FLASK policy
Build Xen with XSM enabled
Build the example policy
Add the appropriate label to guest config files
seclabel=[foo]
stubdom label=[foo]
Make sure you TEST the example policy in your environment
BEFORE putting it into production!
NOTE: As an example policy, it is not as rigorously tested as
other parts of Xen during release, and it may not be suitable
as-is if you are doing unusual things
http://wiki.xenproject.org/wiki
/Xen Security Modules : XSM-FLASK
CloudOpen North America 2013 Securing Your Cloud with Xen Project’s Advanced Security Features 30 / 32
31. Introduction Network path Bootloader Device model Xen Conclusion
ARM: Right solution for security
Stays in ARM Hypervisor Mode
The ARM architecture has separate Hypervisor and Kernel
modes
Because Xen’s architecture maps so well to the ARM
architecture, Xen never has to use Kernel mode
Other hypervisors have to flip back and forth between modes
If a hypervisor has to enter Kernel mode, it loses the security
of running in a privileged mode, isolated from the rest of the
system
This is a non-issue with the Xen Hypervisor on ARM
Does not need to use device emulation
No emulation means a smaller attack surface for bad guys
CloudOpen North America 2013 Securing Your Cloud with Xen Project’s Advanced Security Features 31 / 32
32. Introduction Network path Bootloader Device model Xen Conclusion
For More Information...
Details at http://wiki.xenproject.org/wiki/Securing Xen
Thanks to George Dunlap for supplying much of the information
presented here, and Stefano Stabellini for ARM information
Check out our blog: http://blog.xenproject.org/
Contact me at russell.pavlicek@xenproject.org
——————————–
Thank You!
CloudOpen North America 2013 Securing Your Cloud with Xen Project’s Advanced Security Features 32 / 32
