O slideshow foi denunciado.
Seu SlideShare está sendo baixado. ×

OSMC 2022 | Monitoring multiple Kubernetes Clusters with Thanos by Pascal Fries

Anúncio
Anúncio
Anúncio
Anúncio
Anúncio
Anúncio
Anúncio
Anúncio
Anúncio
Anúncio

Confira estes a seguir

1 de 45 Anúncio

OSMC 2022 | Monitoring multiple Kubernetes Clusters with Thanos by Pascal Fries

Baixar para ler offline

By now, Prometheus has become the defacto standard for monitoring containerised applications. However, when it comes to monitoring multiple Kubernetes clusters through a single plane of glass, additional tools are required. In this talk, I will show how to setup a production monitoring landscape based on Prometheus and Thanos, spanning several Kubernetes clusters. Focussing on examples and best practices, I will also elaborate on how to secure communication between the individual components.

By now, Prometheus has become the defacto standard for monitoring containerised applications. However, when it comes to monitoring multiple Kubernetes clusters through a single plane of glass, additional tools are required. In this talk, I will show how to setup a production monitoring landscape based on Prometheus and Thanos, spanning several Kubernetes clusters. Focussing on examples and best practices, I will also elaborate on how to secure communication between the individual components.

Anúncio
Anúncio

Mais Conteúdo rRelacionado

Mais recentes (20)

Anúncio

OSMC 2022 | Monitoring multiple Kubernetes Clusters with Thanos by Pascal Fries

  1. 1. Pascal Fries ● Kubernetes ● Infra as Code ● “cloud stuff”
  2. 2. who is using Kubernetes?
  3. 3. who is using Kubernetes? single cluster? multi cluster?
  4. 4. who is using Kubernetes? single cluster? multi cluster?
  5. 5. who is using Prometheus?
  6. 6. who is using Prometheus? standalone?
  7. 7. ● service discovery ● prom operator ● kube prom stack ♥
  8. 8. multi cluster?
  9. 9. multi cluster? sure! ♥ ♥ ♥
  10. 10. and for a single common endpoint...
  11. 11. ♥ ♥ ♥ ● remote write ● remote read ● federation
  12. 12. Awesome!
  13. 13. Awesome! but...
  14. 14. ● long term storage? ● redundancy? ● high availability? ● multi tenancy?
  15. 15. how are you using Kubernetes? a customer’s story
  16. 16. multiple teams ● 3-4 dev ● 3-4 ops ● 1 infra
  17. 17. dev A ops A infra ops A dev B infra ● 20+ clusters ● shared ● managed ● owned
  18. 18. dev A ops A infra dev B infra requirement: Single endpoint where everyone can get their metrics.
  19. 19. dev A ops A infra dev B infra other requirements: ● long term storage ● high availability ● push based
  20. 20. ♥ ♥ ♥ ; (
  21. 21. hanos storage layer for prometheus querier | sidecar | store | receiver
  22. 22. hanos ● truly cloud native ● great for tinkering ● remarkably resilient
  23. 23. hanos how to set it up: sample configurations
  24. 24. gRPC and TLS auth instead of REST and basic auth
  25. 25. ♥ ♥ ♥ querier sidecar sidecar sidecar
  26. 26. long term storage
  27. 27. ♥ ♥ ♥ querier sidecar sidecar sidecar S3 store
  28. 28. push based
  29. 29. ♥ ♥ ♥ querier S3 store receiver remote write compactor
  30. 30. microservices ♥
  31. 31. microservices ♥ https://www.bizpacreview.com/wp-content/uploads/2021/01/GettyImages-1206292055-1200x630.jpg
  32. 32. authenticating connections ● Thanos / Thanos: TLS auth ● Prom / Thanos: basic auth
  33. 33. sharding & redundancy ● receiver as short term db ● shard/replicate timeseries by labels ● end up with duplicates in S3 :(
  34. 34. deduplication, compaction, & downsampling ● querier deduplicates on the fly ● compactor deduplicates in S3, downsamples for faster queries
  35. 35. high availability querier: stateless, replicated receiver: stateful, replicated compactor: cron, non-replicated
  36. 36. multi-tenancy: a minimal model ● compactor sets “tenant” label on ingestion via relabeling config ● querier augmented by prom label proxy forces “tenant” on queries
  37. 37. ♥ ♥ ♥ querier S3 store receiver remote write compactor
  38. 38. querier receiver prom label proxy auth proxy - source_labels: - cluster - namespace separator: "/" regex: "dev/(appx|appy)" replacement: "dev-b" target_label: "tenant" action: "replace"
  39. 39. relabeling ● automate generation of config ● use to force labels on untrusted data ● currently only reread on receiver restart :( #5398 - source_labels: - cluster - namespace separator: "/" regex: "dev/(ap replacement: "d target_label: " action: "replac
  40. 40. summary ● long term storage :) ● redundancy :) ● high availability :) ● multi tenancy :|
  41. 41. extra stuff ● Thanos ruler: Prom rules for Thanos API ● Thanos query frontend: shard Thanos queries
  42. 42. Thank you!
  43. 43. questions?

×