"Session ID: BUD17-301
Session Name: KVM/ARM Nested Virtualization - BUD17-301
Speaker: Christoffer Dall
Track: Virtualization
★ Session Summary ★
Nested virtualization, the ability to run a virtual machine inside another virtual machine, is increasingly important because of the need to deploy virtual machines running software stacks on top of virtualized cloud infrastructure, as well as for prototyping and testing. As ARM servers make inroads in various deployment scenarios, being able to support nested virtualization on ARM is a key requirement, which has been met recently with the introduction of nested virtualization support in the latest ARMv8.3 revision of the architecture. I will present the initial effort to introduce ARM nested virtualization support to KVM/ARM, which involves adding significant logic to core KVM/ARM code, MMU support, timers, and the GIC emulation. I will also briefly discuss a paravirtualization approach we have used to prototype and evaluate the implementation on current ARMv8 hardware without hardware support for nested virtualization.
---------------------------------------------------
★ Resources ★
Event Page: http://connect.linaro.org/resource/bud17/bud17-301/
Presentation: https://www.slideshare.net/linaroorg/bud17301-zenos-paradox-optimizing-kvmarm
Video: https://youtu.be/Mqh9J83xFxo
---------------------------------------------------
★ Event Details ★
Linaro Connect Budapest 2017 (BUD17)
6-10 March 2017
Corinthia Hotel, Budapest,
Erzsébet krt. 43-49,
1073 Hungary
---------------------------------------------------
Keyword: virtualization, KVM, ARM
http://www.linaro.org
http://connect.linaro.org
---------------------------------------------------
Follow us on Social Media
https://www.facebook.com/LinaroOrg
https://twitter.com/linaroorg
https://www.youtube.com/user/linaroorg?sub_confirmation=1
https://www.linkedin.com/company/1026961"
4. Use Cases
1. IaaS hosting private clouds
2. Test your hypervisor in a VM
3. Debug your hypervisor in a VM
4. Develop hypervisors using a cloud
5. Theorem 2
“A conventional third generation computer
is recursively virtualizable if it is: (a) virtualizable, and
(b) a VMM without any timing dependencies can be
constructed for it.”
Formal requirements for virtualizable third generation architectures
[Popek and Goldberg ’74]
6. Recursively Virtualizable
• Only applies to virtualizable architectures
• ARM and x86 are not virtualizable
• Hardware support for virtualization
27. Shadow Stage 2
Page Tables
• Translate IPA to PA
• Entries are created by host
KVM by walking guest
hypervisor stage 2 page
tables in software
VM
EL0
EL1
EL2 Host KVM
Kernel
User Space
vEL2 Guest KVM
IPA -> VM PA
VA -> IPA
VM PA -> PA
29. • ARM provides a virtual and physical timer in EL1
• EL2 provides a separate EL2 “hyp” timer
• KVM must emulate a VM with EL2 and the hyp
timer
Nested Timer Virtualization
32. ARM Generic Interrupt
Controller (GIC)
GIC
CPU 0
CPU 1
CPU
Interface
CPU
Interface
Dist.
IRQ
ACK
Virtual CPU
Interface
Virtual CPU
Interface
VIRQ
ACK
List Registers (LRs)
List Registers (LRs)
33. VM
Nested VM
• Deliver both virtual and
nested virtual interrupts
using the GIC
• Multi-level virtualization
using single-level
virtualization hardware
[Turtles - OSDI ‘10]
Nested Interrupt
Virtualization
HostVMM
Kernel
User
Space
Guest
VMM
Virtual CPU
Interface
LRs
34. • Shadow LRs
• Guest hypervisor traps when attempting to
program virtual LRs
• Host hypervisor handles traps by writing to shadow
LRs
• Hardware uses shadow LRs when running the
nested VM
Nested Interrupt
Virtualization
35. Implementation Status
• RFC v1 on @kvmarm by Jintack Lim
(Columbia University)
• CONFIG_KVM_ARM_NESTED_HYP
• vcpu->arch.features &
KVM_ARM_VCPU_NESTED_VIRT
36. To Do
• Must expose EL2 registers to user space
• Mostly scattered out over existing files. Should we try to isolate
more?
• Hard-coded addresses and interrupt numbers
• Reverse map for shadow stage 2 page tables
• More efficient emulation of TLBI instructions
• Get rid of config option and use command line parameter instead
• Hypercalls from the VM vs. virtual self-hypercalls and PSCI
39. KVM/ARM Nested Virtualization
• VHE is fun with nested virtualization
• We don’t set the E2H bit
• The VM thinks it runs in EL2 using VHE so uses EL1
register accesses to access EL2 registers
• But really does run in EL1 and doesn’t need to trap
except on a few registers with different bit
configuration
CPU Virtualization - VHE
40. KVM/ARM Nested Virtualization
• EL2 is separate translation regime
• No ASIDs - cannot alias with EL1 translations
• VMID not used
• Emulating virtual EL2 in EL1
• Separate VMID for virtual EL2
• Always use ASID 0
Memory Virtualization - VMIDs and ASIDs