OSv on BitVisor and Physical Machine with Intel 10GbE Driver
•
2 gostaram•1,556 visualizações
The document discusses the evolution of OSv from running on BitVisor (OSb) to running directly on hardware (OSp). It provides details on implementing a 10GbE network driver for the Intel X540 NIC in OSp. The driver implementation covers initialization, TX and RX operations using descriptor rings, and the interface with OSv. Performance is improved over OSb but there is still work needed on interrupts and advanced features. Additional scripts are discussed that allow OSv to directly access PCI devices when running on KVM/Linux for further optimization.
Recomendados
Mais conteúdo relacionado
Mais procurados
Mais procurados (18)
Semelhante a OSv on BitVisor and Physical Machine with Intel 10GbE Driver
Semelhante a OSv on BitVisor and Physical Machine with Intel 10GbE Driver (20)
Último
Último (20)
OSv on BitVisor and Physical Machine with Intel 10GbE Driver
- 1. OS*: OSv on * @ysmoo
- 4. BitVisor
- 5. BitVisor OSv
- 9. OSb Design BitVisor Hardware OSv App Virtio NIC With a small fix, OSv can handle chipsets OSv has an AHCI driver BitVisor providesVirtio NIC
- 10. OSb Design BitVisor Hardware OSv App Virtio NIC With a small fix, OSv can handle chipsets OSv has an AHCI driver BitVisor providesVirtio NIC
- 11. One thing to fix Skipped the code! (drivers/acpi.cc:early_init())
- 12. OSb Design BitVisor Hardware OSv App Virtio NIC With a small fix, OSv can handle chipsets OSv has an AHCI driver BitVisor providesVirtio NIC
- 13. Para Pass-through Virtio BitVisor Hardware Virtio Ring PRO/1000 Ring OSv App MSI-X etc. Handle minimal operations. PCI Function Pass-through! For more details, see: http://www.slideshare.net/yushiomote/osb-osv-on-bitvisor-2
- 14. for (12-billion times) val++; 0.00 6.50 13.00 19.50 26.00 OSb Linux (Host, Bare-metal) 25.51 20.75 ElapsedTime(sec)
- 15. Netperf TCP_STREAM 0.00 237.50 475.00 712.50 950.00 OSb Linux (Host, Bare-metal) 941.39921.83 Throughput(Mbps)
- 16. Ping 0.00 0.04 0.08 0.12 0.16 OSb Linux (Host, Bare-metal) 0.16 0.12 Latency(msec)
- 17. Thinner and thinner Hardware KVM/Linux OSv Hardware BitVisor OSv12.8M LOC 0.3M LOC virtio sched. driver http://cdn-ak.f.st-hatena.com/images/fotolife/j/jovi1kamiya/20140607/20140607114847.jpg
- 19. OSv on Physical Machine Hardware KVM/Linux OSv Hardware BitVisor OSv Hardware OSv OSp
- 20. OSp Design Hardware OSv App Physical NIC Driver Implemented Intel X540 10GbE Driver in C++11 With a small fix, OSv can handle chipsets OSv has an AHCI driver
- 21. About implementation • Intel NIC TX/RX Basics • Device initialization • TX initialization/operations • RX initialization/operations
- 22. Intel NIC TX/RX Basics Descriptor 0 Descriptor 2 Descriptor 3 Descriptor 4 Descriptor 5 Descriptor 6 Descriptor 7 Ring Buffer Descriptor 1 Producer I/O Register Consumer I/O Register OSv I/O Registers
- 23. Intel NIC TX Basics Descriptor 0 Descriptor 2 Descriptor 3 Descriptor 4 Descriptor 5 Descriptor 6 Descriptor 7 Ring Buffer Descriptor 1 Producer I/O Register Consumer I/O Register OSv I/O Registers
- 24. Intel NIC TX Basics Descriptor 0 Descriptor 2 Descriptor 3 Descriptor 4 Descriptor 5 Descriptor 6 Descriptor 7 Ring Buffer Descriptor 1 Producer I/O Register Consumer I/O Register OSv I/O Registers
- 25. Intel NIC TX Basics Descriptor 0 Descriptor 2 Descriptor 3 Descriptor 4 Descriptor 5 Descriptor 6 Descriptor 7 Ring Buffer Descriptor 1 Producer I/O Register Consumer I/O Register OSv I/O Registers Xmit!
- 26. Intel NIC RX Basics Descriptor 0 Descriptor 2 Descriptor 3 Descriptor 4 Descriptor 5 Descriptor 6 Descriptor 7 Ring Buffer Descriptor 1 Consumer I/O Register Producer I/O Register OSv I/O Registers Recv!
- 27. Device Init.
- 28. TX Init.
- 29. TX Interface (OSv => Driver) • OSv’s upper layer (TCP/IP) calls: • int try_xmit_one_locked(void *mbuf_etc) • Then, driver puts mbuf on ring buffer. • bool kick_hw() • Then, driver updates producer reg. • … EOP EOP mbuf
- 30. try_xmit_one_locked() to put mbuf on ring buffer
- 31. kick_hw() to update producer reg.
- 32. RX Init.
- 33. RX Interface (Driver => OSv) • struct ifnet *_ifn = if_alloc (IFT_ETHER); • On init., driver creates ifnet and… • ether_ifattach(_ifn, macaddr); • …passes it to the upper layer. (Then, the upper layer puts an callback function on ifnet.if_input) • (*_ifn->if_input)(_ifn, mbuf); • On packet reception, driver calls the callback function (if_input) by passing mbuf as an argument.
- 34. Under improvement (A lot of TODOs) • No interrupts… (bad polling impl.) • No TSO, LRO, checksum assist… • No advanced descriptor… (legacy descriptor seems slow) • …
- 35. Netperf TCP_STREAM 0.00 2000.00 4000.00 6000.00 8000.00 OSp OSv 7229.00 4200.00 Throughput(Mbps)
- 36. Future Work 0.00 2000.00 4000.00 6000.00 8000.00 OSp OSv 7229.00 4200.00 Throughput(Mbps) Revenge of the OSp
- 37. One more thing KVM/Linux Hardware OSv App ./scripts/bindctrl bind 8086 1528 0000:07:00.0 ./scripts/run.py -a 07:00.0 Additional scripts & options to start OSv with PCI pass-through config.
- 38. Output $ sudo ./scripts/run.py -n -v -c 8 -a 07:00.0 ixgbe [00:04.0] 8086:1528 Address: a0:36:9f:14:5f:84 PHY Status: State: Connected Type: MDI-X Speed: 10Gbps Duplex: Full MAC Status: Link: UP Speed: 10Gbps ixgbe [00:04.0] 8086:1528 eth0: 192.168.122.15 ixgbe0: 192.168.100.2 /# /# /# /# exit Goodbye
- 41. References • The Ugly Patch of Intel X540 10GbE Driver and Additional scripts for PCI Pass-through Configuration • http://goo.gl/0yRcV4 • git diff -r 5e5c227b2aec23418e5e3fd104cc21e98f31729c • Usage & True Motivation • http://www.yushiomote.org/blog/?p=428