Load Balancing 101
•
2 gostaram•883 visualizações
Introduce the basic concept of load-balancing, common implementations of load-balancing and the detail fo kubernetes service. In the last, demonstrate how to modify the linux iptable kernel module to fulfill the layer-7 load-balcning for kubernetes
Recomendados
Mais conteúdo relacionado
Mais procurados
Mais procurados (20)
Semelhante a Load Balancing 101
Semelhante a Load Balancing 101 (20)
Mais de HungWei Chiu
Mais de HungWei Chiu (20)
Último
Último (20)
Load Balancing 101
- 1. Load Balancing 101 HungWei Chiu
- 2. Who Am I • Hung-Wei Chiu (hwchiu) • Member of Technical Staff at Open Networking Foundation • https://www.hwchiu.com • Co-Organizer of SDNDS-TW/CNTUG
- 3. Outline • What’s Load-Balancing • How it works • Kubernetes
- 4. Load Balancing • Efficiently distribute incoming network traffic across a server pool • Distribute Algorithm
- 5. Functionality • Ensure high availability and reliability • Only send requests to servers that are online • Provides the flexibility to add or subtract servers as demand dictates • Distributes client requests or network load efficiently across multiple servers. https://www.nginx.com/resources/glossary/load-balancing/
- 6. Distribute Algorithm • Round Robin • Header Hash • Layer 3 • Layer 4 • Layer 7 • Server Status • Connection number • …etc
- 7. OSI / TCP-IP OSI – TCP/IP https://techdifferences.com/difference-between-tcp-ip-and-osi-model.html Layer1 Layer2 Layer3 Layer4 Layer5 Layer6 Layer7
- 11. True/False ? • Kubernetes Service (Layer 4) • ClusterIP • NodePort • Kubernetes Ingress (Layer 7) • Different Implementation • TCP/UDP/HTTP
- 13. Client Client Client Client Server Server Server Load Balancer Typical Implementation 1.2.3.4 192.168.1.2 192.168.1.3 192.168.1.4 client_ip 1.2.3.4 1.2.3.4 192.168.1.2 NGINX 1.2.3.4 client_ip 192.168.1.2 1.2.3.4 src dest src dest
- 14. Client Client Client Client Server Server Server Load Balancer Typical Implementation 1.2.3.4 192.168.1.2 192.168.1.3 192.168.1.4 client_ip 1.2.3.4 1.2.3.4 192.168.1.2 NGINX 1.2.3.4 client_ip 192.168.1.2 1.2.3.4 src dest src dest HTTPS HTTPCertificate
- 15. How it works • Type • Client-Side • Middleware • Connection Statue • Proxy • Transparency
- 16. Client-Side • Client directly sends request to backend server. • Query all possible addresses of backend servers • Apply distributed algorithm • Send request to backend server • Someone should provide address list of all backend server
- 20. Client Client Client Client Server Server Server Address list Provider Client-Side LB 192.168.1.2 192.168.1.3 192.168.1.4 Send to 192.168.1.3 client_ip 192.168.1.3 192.168.1.3 client_ip
- 21. Real Case • gRPC is a layer 7 protocol, multiple gRPC request rely on the same TCP connection. • gRPC in Kubernetes environment • Server type: Headless • Return a set of PodIPs • https://medium.com/google-cloud/loadbalancing- grpc-for-kubernetes-cluster-services-3ba9a8d8fc03
- 22. Middleware • Client sends packets to a middleware • Middleware handles the traffic load-balancing • Client doesn’t need to know the IP address of backend servers, only the middleware. • Different implementation • Proxy • Transparency
- 23. Proxy • Usually, a demand daemon to handle the incoming traffic. • Establish a new network connection. • One to one • One to many • SSL-terminator
- 24. Client Client Client Client Server Server Server Load Balancer Proxy 1.2.3.4 192.168.1.2 192.168.1.3 192.168.1.4 client_ip 1.2.3.4 1.2.3.4 192.168.1.2 NGINX 1.2.3.4 client_ip 192.168.1.2 1.2.3.4 Different TCP connections
- 25. Real Case • gRPC is a layer 7 protocol, multiple gRPC request rely on the same TCP connection. • Layer 4 proxy can’t identify gRPC connection. • All gRPC request will be forward to same backend server. • Layer 7 proxy if supports gRPC • One to Many • Identify gRPC format • Envoy/LinkerD https://www.bugsnag.com/blog/envoy
- 30. Transparency • Modify packets to make it be routed to backend server. • Keep the same connection, no additional one. • Client still doesn’t know the IP address of backend servers.
- 31. Client Client Client Client Server Server Server Transparency 1.2.3.4 192.168.1.2 192.168.1.3 192.168.1.4 client_ip 1.2.3.4 client_ip 192.168.1.2 1.2.3.4 client_ip 192.168.1.2 client_ip Same tcp connection
- 32. Client Client Client Client Server Server Server Transparency 1.2.3.4 192.168.1.2 192.168.1.3 192.168.1.4 client_ip 1.2.3.4 client_ip 192.168.1.2 1.2.3.4 client_ip 192.168.1.2 client_ip Same tcp connection
- 33. Real Case • gRPC is a layer 7 protocol, multiple gRPC request rely on the same TCP connection. • gRPC load-balancing can’t work correctly in this mode. • Only single TCP connection. • TCP is a connection based protocol (3 way handshake)
- 34. Client Client Client Client Server Server Server Transparency 1.2.3.4 192.168.1.2 192.168.1.3 192.168.1.4 client_ip 1.2.3.4 client_ip 192.168.1.2 1.2.3.4 client_ip 192.168.1.2 client_ip Same tcp connection Clone packets TCP needs connection status.
- 35. Client Client Client Client Server Server Server Transparency 1.2.3.4 192.168.1.2 192.168.1.3 192.168.1.4 client_ip 1.2.3.4 client_ip 192.168.1.2 1.2.3.4 client_ip 192.168.1.2 client_ip Same tcp connection TCP needs connection status.
- 36. Example • Kubernetes Service (by default) • ClusterIP/NodeIP • Packet are modified by iptables • Connection persistence are maintained by kernel conntrack.
- 37. Node1 Node2 Node3 BackendBackend Backend clusterIP
- 38. Node1 Node2 Node3 BackendBackend Backend clusterIP Kube-proxy Kube-proxy Kube-proxy Insert iptables rules Insert iptables rules Insert iptables rules
- 39. Node1 Node2 Node3 BackendBackend Backend clusterIP Client 1. Choose the backend server 2. Change the packet destination IP address 3. Forward packet Iptables..
- 40. Node1 Node2 Node3 BackendBackend Backend clusterIP 10.104.8.48 10.244.0.4 10.244.0.5 10.244.0.6
- 41. Node3 Backend Client Iptables.. PacketsPacketsPackets Match ClusterIP Choose backend server Change Packet IP ip/protocol Distributed algo Action
- 42. Node3 Backend Client Iptables.. PacketsPacketsPackets Match ClusterIP Choose backend server Change Packet IP ip/protocol Distributed algo Action
- 44. EP1 EP2 EP3 P < 0.2 P < 0.25 EP1 EP2 EP3 EP4 EP5 P < 0.33 P < 0.5 EP4 EP5 P = 4/5 * 1/4 = 1/5 P = 4/5 * 3/4 * 1/3 = 1/5
- 45. Question • How to make the connection consistent? https://medium.com/@chirag.singla/tcp-3-way-handshake-and-latency-aae5a2a6a68b
- 46. Conntrack Linux Kernel helps you
- 47. Answer • Only the first packet meets NAT rules • Kernel modifies the packets for you • You need the conntrack tool to control it. • You should choose the destination before you see the real data packets.
- 48. Kubernetes service • People always said kubernetes only supports layer 4 load-balancing • Is it possible to provide layer 7 load-balancing in ? • Without modifying kubernetes source code
- 49. I think • For normal TCP implementation, No. • Should be Yes for customized TCP protocol • Real zero-RTT TCP. • How about other protocols ? • UDP ?
- 50. DEMO • Source code • https://github.com/cloud-native-taiwan/k8s- course/pull/5/files • Modify the iptables linux kernel module • Statistic module • Parse the UDP payload and try to match