LinuxCon/ContainerCon Japan 2016 "Networking Containers in Ultra-Low Latency Environments"
•
0 gostou•611 visualizações
Presentation by Avi Deitcher at LinuxCon/ContainerCon Japan 2016. "Networking Containers in Ultra-Low Latency Environments".
Recomendados
Recomendados
Mais conteúdo relacionado
Último
Último (20)
Destaque
Destaque (20)
LinuxCon/ContainerCon Japan 2016 "Networking Containers in Ultra-Low Latency Environments"
- 4. אכסניא Akh-san-ya ?aksnaja? n. (ancient Aramaic, from Ancient Greek xénos) 1: Hospitality, lodging; 2: Host. :אכסניא בכבוד פותחים Ancient Jewish custom to begin public speaking by honouring or thanking the hosts. Avi Deitcher avi@atomicinc.com
- 7. Who Am I? • Life in tech business: – 10 yrs financial services IT – 10+ yrs consulWng & training – Some startups on the way • Avid (if not very good) ice hockey player • Long-Wme lover of great engineering…. when used to make a real difference • Atomic Inc: – ConsulWng – Training Avi Deitcher avi@atomicinc.com (not 24601)
- 9. A Lile History Summer 2015 • Fintech X: “Help us containerize!” – Hint: It is harder than you think… and worth it – Culture/process > technology • QuesWon: Networking? • Answer: ScienWfic method Avi Deitcher avi@atomicinc.com
- 10. A Lile History Summer 2015 • Fintech X: “Help us containerize!” – Hint: It is harder than you think… and worth it – Culture/process > technology • QuesWon: Networking? • Answer: ScienWfic method Summer 2016 • Good pracWce demands: 1. Redo tests with new opWons and versions 2. Make tests available 3. Explain it all well Avi Deitcher avi@atomicinc.com
- 22. Our Tests What We Tested • netperf ⇒ netserver • UDP & TCP round-robin • Sizes: 300, 500, 1024, 2048 • No orchestraWon = complete control • 50000 iteraWons – Law of large numbers • Latency (Avg, %iles), CPU • DifferenRals, not absolutes How We Tested • .net – Because it had to be metal – Wicked smart team • Complete test run – Network changes – Hardware variaWons, errors hps://github.com/deitch/network-tests Avi Deitcher avi@atomicinc.com
- 27. Local Networking Summary • SR-IOV horrible latency but great CPU – Hold that thought… • net=host on par with metal • macvlan closest virtualized to metal • Rest in same range: – Latency: 5-10 𝓊-sec overhead – CPU: negligible difference • Calico (IPIP & naWve) & Docker overlay slightly more performant • Watch out for very large TCP packets Avi Deitcher avi@atomicinc.com
- 31. Remote Networking Summary • Weave (sleeve) adds latency and CPU – Reason for “fast datapath” • Again, macvlan best virtualized • All the rest: – Latency: within 50 𝓊-sec of each other, except SR- IOV with very large TCP packets – CPU: similar, but keep an eye on Flannel (UDP) Avi Deitcher avi@atomicinc.com
- 33. SR-IOV SR-IOV does not automaRcally mean beXer • Switch in network card • Trades host CPU for card processor • Quality varies drama5cally – Even Mellanox far worse locally • My 2¥: SR-IOV falls further behind due to: – Speed of iteraWon – Open-source – Sowware + CPU Avi Deitcher avi@atomicinc.com
- 34. Headaches (and Thanks) • Headaches – Weave SYN-(nothing) – etcd is “touchy” – Packet L3 network is powerful but… unique • Macvlan, weave, flannel: all required pings for mac • Se{ng up bridge w/o NAT, Calico, macvlan was “different” – SR-IOV is complicated and flaky, especially Mellanox – netperf with UDP packets can get stuck (Calico-ipip) – And a whole lot more (ask me offline) • And thanks: – Bryan Boreham, Adam Harrison at weave.works – Zac Smith, Adam, Aaron, Andy, Lucas, everyone at Packet Avi Deitcher avi@atomicinc.com
- 36. Conclusions • SR-IOV: most of the Wme, just not worth it • Performance: – Metal (+ net=host): always performs best – Direct network++: macvlan is your friend – Others: Roughly similar, careful of Weave (sleeve) • What’s your use case? – ULL: Metal/net=host > macvlan > calico > overlay – Everything else: Focus on your architecture and skills Pick intelligently: easier, not simple Avi Deitcher avi@atomicinc.com
- 37. Conclusions • SR-IOV: most of the Wme, just not worth it • Performance: – Metal (+ net=host): always performs best – Direct network++: macvlan is your friend – Others: Roughly similar, careful of Weave (sleeve) • What’s your use case? – ULL: Metal/net=host > macvlan > calico > overlay – Everything else: Focus on your architecture and skills Pick intelligently: easier, not simple Avi Deitcher avi@atomicinc.com