O slideshow foi denunciado.
Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.

Best practises for using Kubernetes and GKE in production

Kubernetes has become the de facto Container orchestration platform over the last few years. For all the power and the scale that Kubernetes provides, it's still a complex platform to configure and manage. Managed Kubernetes service like GKE takes a lot of the pain out in managing Kubernetes. When we are using Kubernetes in production, it makes sense to follow the best practises around distributed system development targeted towards Kubernetes. In this session, I will talk about Kubernetes and GKE best practices around infrastructure, security and applications with specific focus on day-2 operations.

  • Entre para ver os comentários

Best practises for using Kubernetes and GKE in production

  1. 1. Best practises for using Kubernetes in production Sreenivas Makam August 2, 2020
  2. 2. About myself ● Application modernization specialist at Google Cloud. Previously at Cisco and few startups ● Interest areas - Containers, Kubernetes, Networking, Cloud native technologies ● Author of “Mastering CoreOS”, published 2016. Reviewed many technology books ● Docker Captain from Oct 15 - Mar 18 ● Active blogger and Community speaker
  3. 3. Agenda ● What is Kubernetes? ● What makes Kubernetes unique? ● Kubernetes day 2 operations ○ Cluster Management ○ Application Design ○ Security ● Kubernetes ecosystem and tools
  4. 4. What is Kubernetes?
  5. 5. What is Kubernetes Kubernetes is a portable, extensible, open-source platform for managing containerized workloads and services, that facilitates both declarative configuration and automation https://kubernetes.io/docs/concepts/overview/what-is-kubernetes/
  6. 6. Kubernetes market share https://sysdig.com/blog/sysdig-2019-container-usage-report/
  7. 7. https://kubernetes.io/docs/concepts/overview/components/ Kubernetes Architecture
  8. 8. What makes Kubernetes unique?
  9. 9. Kubernetes Uniqueness ● Declarative rather than imperative ● Extensible - custom resource, controllers, schedulers ● Meet the user where they are - (eg) read config, secrets from applications ● Decouple distributed system application development ● Open source ecosystem friendly
  10. 10. Borrowed from: https://www.digitalocean.com/communit y/tutorials/imperative-vs-declarative-ku bernetes-management-a-digitalocean-c omic
  11. 11. Custom authorization , admission control Custom resource(operator pattern. eg: etcd, prometheus) Custom scheduler Custom controller, works with custom resource Network plugin Storage plugin Extend kubectl Kubernetes extensions
  12. 12. Controller(Standard, custom) (Standard eg: Replica set controller, Deployment controller) API server Standard Kubernetes resources Desired State Current State Kubernetes controller
  13. 13. Kubernetes operator (eg: prometheus, etcd, Spark, Airflow) API server Custom resources Desired State Current State Operators manage the lifecycle of the custom application Extensions -Kubernetes Operator
  14. 14. https://coreos.com/blog/the-prometheus-operator.html Prometheus Operator
  15. 15. apiVersion: v1 kind: Pod metadata: name: mypod spec: containers: - name: mypod image: redis volumeMounts: - name: foo mountPath: "/etc/foo" readOnly: true volumes: - name: foo secret: secretName: mysecret apiVersion: v1 kind: Pod metadata: name: secret-env-pod spec: containers: - name: mycontainer image: redis env: - name: SECRET_USERNAME valueFrom: secretKeyRef: name: mysecret key: username - name: SECRET_PASSWORD valueFrom: secretKeyRef: name: mysecret key: password restartPolicy: Never Use Mount paths Use environment variables Applications can consume config/secrets without knowledge of Kubernetes Meet user where they are -Consume secrets in App
  16. 16. https://itnext.io/tutorial-basics-of-kubernetes-volumes-part-2-b2ea6f397402 Kubernetes provides portability by decoupling infrastructure(Storage, networking) from the application manifest Decouple distributed system -Storage provisioning
  17. 17. Init container (Clone git repo and generate config) App container (Web server) Pod Execution sequence Specialized containers that runs to completion before application containers in a pod can get started. This enforces sequence. Pod patterns -Init Containers
  18. 18. Pod patterns -Sidecar Sidecar containers extend and enhance the “main” container Other examples: Istio envoy proxy Monitoring Database config
  19. 19. Pod patterns -Adapter Adapter containers standardize and normalize output so that external services can access interface in a standard way(eg: Prometheus adapter)
  20. 20. Pod patterns - Ambassador Ambassador containers proxies a local connection to the world and hides the complexity to access external service. Examples: Accessing different kinds of cache based on environment Client side service discovery using different mechanisms
  21. 21. Single app defined using Dockerfile and multiple apps done using deployment Config map and secrets Service abstraction and discovery Stateless containers, stateful dataset where needed Services provides different options for port bindings Autoscaler support is comprehensive Centralized log management with third party integrations possible Autohealing Many ways to create and manage clusters(cloud provider, kops, kubeadm) Map Twelve factor apps to Kubernetes
  22. 22. Kubernetes day 2 Operations
  23. 23. Kubernetes Day 2 operations -Best practises ● Cluster management ○ Multi-tenant design(clusters/namespaces, multi-cluster handling, zonal/regional), Upgrade policy(node and containers, pod disruption budget), Ingress(load balancers), External service access policy(db, cache etc) ● Application design ○ Pod design(using pod design patterns), Lifecycle(health check, graceful termination), Scaling(resource request, autoscaling), Application types(stateful/stateless/batch/Big Data/ML), Service mesh ● Security ○ Access control(rbac), Image validation(binary authorization, vulnerability scanning), Secure clusters(private cluster, firewall)
  24. 24. Kubernetes day 2 Operations Cluster management
  25. 25. Use Namespaces and RBAC for isolation https://cloud.google.com/solutions/prep-kubernetes-engine-for-prod
  26. 26. Multicluster handling Need for multiple clusters - Different applications, teams, environments, regions Central policy management using Anthos config management(ACM) Proximity based cluster routing
  27. 27. 4 wayAutoscaling in GKE HPA Autoscales pool of workers on custom metrics VPA Recommends podspec Actuates the adjustment CA NAP+ Scale Nodepools Create right nodes for the job Gate changes by HPA + NAP Workload Infrastructure
  28. 28. Kubernetes day 2 Operations Application design
  29. 29. Readiness and Liveness probe https://cloud.google.com/blog/products/gcp/k ubernetes-best-practices-setting-up-health-ch ecks-with-readiness-and-liveness-probes
  30. 30. Graceful shutdown handling Best practises ● Have handler for Prestop hooks or SIGTERM and handle shutdown gracefully ● Keep readiness check interval aggressive ● Have client retry failed requests https://dzone.com/articles/kubernetes-lifecycle-of-a-pod
  31. 31. Big Data on Kubernetes ● Kubernetes as a replacement for YARN for Big data workloads ● Spark and Flink operators for Kubernetes are available as beta ● Dataproc on GKE is available as beta ● Advantages ○ Single orchestrator for applications and Big Data ○ Better use of cluster resources ○ Big Data application dependency handled through containers ○ Use Kubernetes ecosystem for Big Data
  32. 32. MLworkloads with Kubeflow ● Deploying and managing ML models at scale using Kubernetes ● Build, train and serve models ● Components - Notebooks, UI, training, Serving, Pipelines ● Multiple frameworks supported for training as well as serving ● Advantages ○ Portable ML pipelines ○ Best of Kubernetes features used for Machine learning
  33. 33. Kubernetes day 2 Operations Security
  34. 34. Private Clusters in GKE
  35. 35. Secure Image pipeline
  36. 36. Kubernetes Network policies Topology Defined with Network Policy API: ● A first-class Kubernetes API ● Defines allowed traffic patterns How does it work: ● K8s defines the API. ● User applies a policy. ● Network policy agents watch and enforce. ● Restricts pod to pod traffic. K8sMaster etcd policy.yaml spec: podSelector: matchLabels: run: nginx ingress: - from: - podSelector: matchLabels: run: access Frontend Service A Service B Policy Agent K8sNode
  37. 37. Kubernetes ecosystem and Tools
  38. 38. Kubernetes ecosystem CI/CD (Tekton, Argo) Monitoring (Prometheus) Logging (Fluentd) Service Mesh (Istio, Linkerd) Packaging (Helm, kpt) Infra (Network, storage plugin) Service Discovery (CoreDNS) Serverless (knative, Virtual kubelet) ML (Kubeflow) Registry (Harbor) Security (Falco, Open policy) VM (Kubevirt, Config connector)
  39. 39. Helpers Kubectx kubens Config Mgmt Kustomize Pkg Mgmt Helm Build Dockerfile Kaniko Jib CI/CD Skaffold IDE Cloud code for VSCODE Kubernetes tools(my favorites)
  40. 40. Practical Experiences ● Networking and security for clusters has to be pre-planned. These cannot be changed later. ● Plan IP addresses before-hand. Kubernetes needs lot of addresses(Node, Pod, Service) ● Use managed services when possible ● Keep separate environments for Dev, staging and production ● Isolate helper applications(CI/CD, Monitoring) from primary workloads ● Start with stateless workloads and then expand to stateful, big data and ML ● Invest in monitoring/logging/secret management solution ● Backup and DR is important for Kubernetes ● Make sure that every container has resource requests
  41. 41. References ● Kubernetes design principles video ● Kubernetes patterns video ● Kubernetes patterns slides ● Building Cloud native applications with Kubernetes and Istio - Kelsey ● Designing cloud native applications ● Extending Kubernetes