Mais conteúdo relacionado

Apresentações para você(20)

Similar a XPDDS19: Argo and Hypervisor-Mediated Data eXchange (HMX) - Christopher Clark, OpenXT Project(20)


Mais de The Linux Foundation(20)



XPDDS19: Argo and Hypervisor-Mediated Data eXchange (HMX) - Christopher Clark, OpenXT Project

  1. Argo Hypervisor-Mediated data eXchange Christopher Clark OpenXT Project Xen Design and Developer Summit, 10th July, 2019
  2. Argo ● What it is ○ Overview ○ Origin ● Why ○ Key properties ○ What is different, distinctive about it ● How it relates to other hypervisors ○ HMX in other hypervisors ● A bit more about what it is ○ Details of the interface ○ Examples of it in use ● References
  3. Argo : an interdomain communication transport ● An OpenXT technology, originally developed for XenClient. ○ Derived from v4v deployed in uXen in Bromium’s vSentry on millions of machines. ● Hypervisor-mediated data copies via private ring buffers with notifications. ○ Prioritizes isolation, access control and mandatory conformance to the transfer protocol. ○ Simple but powerful primitive to build upon. ● Included in Xen 4.12 ○ Set the KConfig option CONFIG_ARGO=y to build it in, then turn it on at the Xen boot command line: argo=1,mac_permissive=1
  4. Argo : an Odyssey The longest patch upstreaming process in Xen’s history. 2011 - 2019.
  5. Argo : an Odyssey The longest patch upstreaming process in Xen’s history. 2011 - 2019. ● 2010: XenClient project begins ● 2011: Ross reserves the hypercall number (39) in Xen. ← The timer starts here with the first patch integrated. ● 2012: Jean’s v4v patch series posted ● 2013: Phil’s Xen Summit presentation, Ross’s v4v patch series posted ● 2014: OpenXT project begins ● 2016: Clark presentation at Xen Summit ● 2017: HP ships laptops with Bromium micro-virtualization for Hyper-V and Windows 10 VBS ● 2018 - 2019: Argo patch series submitted, iterated, reviewed and accepted into Xen ● 2019: Design activity on Argo’s next development HMX: ● 2019: Microsoft ships Windows Sandbox and SystemGuard in Windows 10 ● 2019: IBM/RedHat microvm QEMU machine type inspired by Intel NEMU and AWS Firecracker
  6. Argo : an Odyssey One new hypercall, with four operations. ○ Register ring ○ Unregister ring ○ Sendv ○ Notify
  7. Argo : Key Properties ● No memory is shared between guest VMs. ● Data is copied from the source to the destination. ○ Maintains spatial isolation between guest VMs. ● The Hypervisor performs the data transfer into guest-accessible memory. ○ Ensures that the transfer protocol in the ring structure is strictly adhered to. ○ Enables enforcement of mandatory access control policy by the hypervisor. These properties will be maintained.
  8. MILS Architecture Foundational Security Principals (2005) ● Data Isolation Information in a partition is accessible only by that partition and private data remains private. ● Control of Information Flow Information flow between partitions is from an authenticated source to authenticated recipients; the source of information is authenticated to the recipient, and information only goes where intended. ● Periods Processing / Temporal Separation Resources may be used by different components by time-slicing, where the system enforces that the resource is cleaned to remove any trace of its previous use before being reassigned. ● Fault Isolation Failure within a partition is prevented from cascading to any other partition. Failures are detected, contained and recovered locally.
  9. Argo : an interdomain communication transport ● Strong isolation between communicating domains ○ No memory is shared between VMs. ○ Does not use or require grant tables or XenStore. ● Strong mechanism for data transfer ○ Hypervisor performs the data movement. Memory accesses conform to the protocol. ● High performance suitable for sustained throughput ● A clean mapping to Linux and Windows native I/O primitives ● Clear separation from guest Operating System networking stacks ● A foundation for the future work that we intend to do
  10. HMX : Hypervisor Mediated data eXchange A term to describe: Asynchronous authenticated message passing between VM partitions, performed by the hypervisor. Channels use no shared memory between the source and the receiver. Enable enforcement of Control of Information Flow between domains and preserve Data Isolation.
  11. HMX: pattern for data delivery Hypervisor Message Data Receive memory buffer Delivery performed by the hypervisor: ● data delivered with context (size, origin) ● writes to the receive buffer, will conform to protocol / structure VM : Sender VM : Receiver DataContext Hypervisor invoked to send message
  12. Inter-partition Communication: Xen (Grants) Traditional PV communication channels using Xen’s primitives: Grants + Events eg: PV network front and back ends. Grants are most commonly used to share or copy pages between domains. ● Grant copy operations can ask hypervisor to write to a remote page, but: ○ Conformance to a ring access protocol is not enforced. ○ Hypervisor does not provide message header with metadata about the source. => HMX: no. Aside: Grant table code has excessive complexity: Attributed responsibility for 5% of Xen’s XSA/CVEs. Most recent grant table XSA was issued... yesterday. Two API versions, switchable at runtime plus legacy operations... so OpenXT adds KCONFIGs to reduce it.
  13. Inter-partition Communication: Hyper-V VMBus : HvPostMessage Messages are copied by the hypervisor into a private per-receiver message buffer in hypervisor-owned private memory. For delivery, individual messages are then copied by the hypervisor out into a Message Page shared between the hypervisor and the receiver partition, when free slots within the page are available. => HMX: yes. since at least 2006.
  14. Inter-partition Communication: uXen Bromium vSentry uses uXen, which uses v4v. ● Open Source ● Has no legacy PV interfaces. ○ no grant tables ○ no XenStore ● All virtual devices use the simple, copy-based primitive. ○ No memory sharing of any kind. Demonstrates that Xen’s problematic legacy interfaces can be replaced with an approach built upon Argo. => HMX: yes.
  15. Inter-partition Communication: Xen (Argo, since 4.12) => HMX : yes.
  16. Argo: ring registration Hypercall operations to establish and tear down a communication channel: Register ring: ● Guest supplies memory to be used for receiving incoming messages. ● Specifies either: a specific other domain as sender, or receive from anyone. ● Access control policy check: is this channel allowed? ● Ring is mapped into the hypervisor address space. Unregister ring ● Removes the hypervisor state. Message delivery ceases.
  17. Argo: sendv Sendv: ● Guest indicates the destination ring it wants to send a message to ○ and provides the data to be sent ● Access control check: is this transmission allowed? ● Is there sufficient space in the destination ring? ○ If not, interest is recorded and a notification will occur when sufficient space is free. ● Hypervisor synchronously copies the data into the ring. ○ Message header is written by the hypervisor, indicating the size and source.
  18. Argo: notify Notify: ● Queries for space availability in remote rings ○ Access control check: can this domain send to the specified ring? ○ Returns ring state indicator flags: ■ Exists ■ Shared ■ Empty ■ Sufficient space exists for the specified message size ■ Busy ○ Register sender’s interest: notification issued when available space becomes sufficient ● Initiates checking of space within own rings to trigger pending notifications
  19. Argo: XSM/Flask controls ● enable ○ governs whether a domain may access any Argo operations at all ● ring registration ○ single-sender rings: allow / disallow registration of rings with the specified sender ○ any-sender rings: allow / disallow a domain to register any-sender rings ● send ○ allow sending a message to a specified domain ○ if the send permission is not granted, notify query operations will also be disallowed
  20. Argo: use cases in OpenXT Examples: ● User interface export from the control domain to the User Interface renderer VM ● Stubdomain toolstack helper services ● Interdomain DBUS service ○ Enables use of existing components maintained by other Open Source communities, deployed in a multi-domain configuration ■ eg. NetworkManager and its user interface ● VM provisioning services for remote management ● Debugging support: ssh access to remote VM terminals
  21. Argo: simple to use: ssh example eg. Using the Linux device driver and userspace library to communicate between VMs: # VM1: load the Argo kernel module: insmod /lib/modules/*/extra/xen-argo.ko # use the interposer to run the ssh server on port 8022 export INET_IS_ARGO=1 LD_PRELOAD=/usr/lib/ -p 8022 # VM2: load the Argo kernel module: insmod /lib/modules/*/extra/xen-argo.ko # use the interposer to run the ssh client to connect to the server export INET_IS_ARGO=1 LD_PRELOAD=/usr/lib/ -p 8022
  22. ssh example ssh server process libargo interposer library process env: INET_IS_ARGO=1 Linux kernel driver: xen-argo.ko /dev/argo_stream Xen hypervisor System calls using the Linux Argo device Hypercalls: argo_op : REGISTER_RING, SENDV, NOTIFY Argo VIRQ ssh client process libargo interposer library process env: INET_IS_ARGO=1 Linux kernel driver: xen-argo.ko /dev/argo_stream System calls using the Linux Argo device Hypercalls: argo_op: SENDV, NOTIFY Argo VIRQ
  23. ssh example Xen Message Data Receive memory buffer Delivery performed by Argo in Xen: ● data delivered with context (size, origin) ● writes to the receive buffer, will conform to protocol / structure VM with ssh client VM with ssh server DataContext Hypervisor invoked to send message
  24. Development items ● Sender domain context: hypervisor to provide metadata to message recipient ○ message header conveys sender’s XSM sid and bits indicating stubdomain relationship state ● Connection state: enforce and track ring pairings for bidirectional communication ○ Tracking bidirectional connections enables fine-grained access policy control ● Access control: run-time configurable firewall ○ Uses sender context and connection state. Replaces OpenXT’s existing v4vtables port. ● Nesting data transport ○ Sending data between VMs at different levels of nesting, with firewall policy enforcement ● Wildcard rings: protection of client forward progress ○ Increases resilience of the clients of any-sender rings to DoS by other clients ● Shutter rings: mechanism and policy controls for reducing hypervisor mappings
  25. Research items ● Accelerated transport options leveraging available hardware ● Asynchronous message send primitive ● Reducing scope of notify wakeups to replace existing broadcast ● Integration with memory encryption technologies
  26. References Ian Pratt, Founder of the Xen Project, CTO of Bromium, at PSEC 2018: Hypervisor Security : Lessons Learned Essential viewing. Strongly recommended.
  27. References ● PSEC 2018: Hypervisor-Mediated data eXchange (HMX): Primitives for Authentic Communication ● Xen Summit 2013: XenClient Mechanisms to Strengthen Guest Separation ● Microsoft Hyper-V: Hypercall APIs Explained WinHEC 2006 presentation, slide 45 describes HvPostMessage. ○ Hypervisor Top Level Functional Specification, v5.0C, see HvPostMessage references, inc. section 11.11. ● OpenXT Summit 2016: Secure Virtual Platform ● Argo source code: ○ Hypervisor: argo.c in Xen 4.12 ○ Linux drivers: current driver, and under-development vsock-based driver ○ Linux userspace: libargo, including LD_PRELOAD interposer enabling transparent use by socket applications ○ For a Windows driver, a port from v4v is required: see the xenv4v source code in OpenXT as a starting point. ● Argo documentation: argo.pandoc in Xen 4.12 ● Argo next development: OpenXT wiki on Argo development items ● OpenXT: about the Project, project events, v4v documentation, v4vtables port for basic access control, with tooling ● uXen: v4v.c in uXen 4.1.6. uXen hypervisor source code repository: Any questions? Please get in touch: #OpenXT on freenode IRC