O slideshow foi denunciado.
Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.

FOSDEM 2019 How Kubernetes used gRPC to encrypt secrets with an external Key Management Service

This presentation focuses on how a gRPC-based implementation was added to Kubernetes to delegate encrypting secrets to an external Key Management Service and the benefits of using a gRPC-based design for this type of problem.

  • Seja o primeiro a comentar

  • Seja a primeira pessoa a gostar disto

FOSDEM 2019 How Kubernetes used gRPC to encrypt secrets with an external Key Management Service

  1. 1. Rita Zhang • Software engineer @ Microsoft, San Francisco • Kubernetes upstream features, Azure Kubernetes Service • Maintainer for K8s KMS plugin for Azure Key Vault @ritazzhang
  2. 2. Rita Zhang • Software engineer @ Microsoft, San Francisco • Kubernetes upstream features, Azure Kubernetes Service • Maintainer for K8s KMS plugin for Azure Key Vault @ritazzhang
  3. 3. The problem… @ritazzhang
  4. 4. @ritazzhang
  5. 5. How could this happen... @ritazzhang
  6. 6. Kubernetes Database ž Uses etcd as its persistent storage for API objects ž Stores secrets as base64 encoded plaintext https://kubernetes.io/docs/concepts/overview/components/#etcd@ritazzhang
  7. 7. https://elweb.co/the-security-footgun-in-etcd/ “Authentication was added in etcd 2.1. … etcd before 2.1 was a completely open system; anyone with access to the API could change keys. In order to preserve backward compatibility and upgradability, this feature is off by default.” Read more from coreos etcd doc @ritazzhang
  8. 8. I did a simple search on shodan and came up with 2,284 etcd servers on the open internet. CREDENTIALS, a lot of CREDENTIALS. Credentials for things like cms_admin, mysql_root, postgres, etc. Passwords for databases of all kinds, AWS secret keys, and API keys and secrets for a bunch of services. GET http://<ip address>:2379/v2/keys/?recursive=true @ritazzhang
  9. 9. @ritazzhang
  10. 10. An attacker who can successfully access your cluster database can compromise your entire cluster and have access to your cloud resources. @ritazzhang
  11. 11. So…How do I keep my etcd data safe? @ritazzhang
  12. 12. With the KMS provider plugin, a gRPC implementation, we can encrypt Kubernetes data stored in etcd at rest with a Key Management Service managed key. @ritazzhang
  13. 13. So…How does this work? @ritazzhang
  14. 14. Secret Kubernetes Master etcd Node API Server Kubelet kubectl create secret generic secret1 Kubernetes Secrets
  15. 15. Secret Kubernetes Master etcd API Server Key Management Service (KMS) Provider for Encryption at Rest • Kubernetes v1.10, v1.13 stable • etcd v3 required • Separate key management from K8s cluster management • Supports encryption using keys stored in external trusted Key Management Service (KMS), e.g. Azure Key Vault, Google Cloud KMS • Hardware Security Modules (HSM)-protected keys @ritazzhang https://kubernetes.io/docs/tasks/administer-cluster/kms-provider/ KMS KMS provider
  16. 16. @ritazzhang V1.13
  17. 17. Let’s look at the implementation… @ritazzhang
  18. 18. gRPC
  19. 19. High Level Design of the KMS Plugin @ritazzhang
  20. 20. Components • The Contract - Public APIs for the the gRPC service: service.proto • The Server – gRPC server • The Client – Kubernetes API Server
  21. 21. A new KMS Provider • gRPC server • how to connect to the external KMS • how to authenticate with the external KMS • which keys to use for encryption and decryption • exposed as a UNIX domain socket connect (e.g. unix:///opt/azurekms.sock) https://github.com/Azure/kubernetes-kms/blob/master/server.go
  22. 22. Demo: K8s cluster with Azure Key Vault data encryption @ritazzhang https://github.com/Azure/kubernetes-kms
  23. 23. Azure Key Vault KMS plugin • Use a key, HSM (hardware security module) -protected, in Key Vault for etcd encryption • Secrets/keys/certs are stored in etcd, managed as part of Kubernetes • Restrict access using K8s concepts: RBAC, Service Accounts, namespaces • Bring your own keys • Available on AKS-engine @ritazzhang Recap
  24. 24. Resources https://ritazh.com/using-azure-key-vault-for-kubernetes-data-encryption- d5eac8daee71 https://kubernetes.io/docs/tasks/administer-cluster/kms-provider/ https://github.com/Azure/aks- engine/blob/master/docs/kubernetes/features.md#azure-key-vault-data-encryption https://github.com/kubernetes/kubernetes/pull/55684 https://github.com/Azure/kubernetes- kms https://github.com/GoogleCloudPlatform/k8s-cloudkms-plugin/ @ritazzhang

×