Soft multi-tenancy can be hard to achieve and secure. Multiple tenants sharing the same cluster means there are global objects, like Custom Resource Definitions (CRDs), namespaces, and so on, that you don’t want tenants controlling. Platform admins, cluster admins, and tenants, should be separated, with dedicated namespaces, role bindings, node groups, taints and tolerations, etc.
With Flux, tenant isolation is enforced by default, so you don’t have to worry about accidental tenant cross-over / cross-contamination.
In this session, Priyanka “Pinky” Ravi, Developer Experience Engineer at Weaveworks, will walk you through how to set up multi-tenancy on an existing Kubernetes cluster and manage several tenants within the cluster.
Take advantage of the benefits that come with infrastructure as code.
Alfresco TTL#157 - Troubleshooting Made Easy: Deciphering Alfresco mTLS Confi...
Implementing Flux for Scale with Soft Multi-tenancy
1. 1
1
Implementing Flux for Scale
with Soft Multi-tenancy
Russ Parmer
Senior Engineer, Weaveworks
Priyanka Ravi
Developer Experience Engineer, Weaveworks
2. 2
2
Weaveworks is founded on open source
● Flux & Flagger (CNCF): GitOps and Progressive Delivery for k8s
● Weave GitOps: A powerful extension & web UI for Flux
● Weave GitOps Terraform Controller: Flux controller for Terraform
resources
● GitOps Tools for Flux: VS Code extension
weave.works
3. 3
3
● Operating model for cloud native applications such as Kubernetes
● Utilizes a version controlled system (Commonly Git) as the “single
source of truth”
● Enables continuous delivery through automated deployment,
monitoring, and management by a version controlled system
● Managing your infrastructure and applications declaratively
What is GitOps
5. 5
5
Individuals, teams, and organizations who implement GitOps experience
many benefits, including:
● Stronger Security Guarantees
● Increased Developer & Operational Productivity
● Enhanced Developer Experience
● Improved Stability
● Higher Reliability
● Consistency and Standardization
Why GitOps
6. 6
6
● A git centric package manager for your applications
● A set of continuous and progressive delivery solutions for Kubernetes
What is Flux
fluxcd.io
7. 7
7
🤝 Flux provides GitOps for both apps and infrastructure
🤖 Just push to Git and Flux does the rest
🔩 Flux works with your existing tools
☸ Flux works with any Kubernetes and all common Kubernetes tooling
🤹Flux does Multi-Tenancy (and “Multi-everything”)
📞 Flux alerts and notifies
👍 Users trust Flux
💖 Flux has a lovely community that is very easy to work with!
Flux in Short fluxcd.io
8. 8
8
● Reduces developer burden
● Extensible
● Comes with out of the box support for Kustomize and Helm
● Designed For Kubernetes
Benefits of Flux
fluxcd.io
10. 10
10
What Flux’s Controllers do
Source Controller
- Fetch resources and store as
artifacts
Kustomize Controller
- Apply manifests, Run
manifest generation using
kustomize
Helm Controller
- Deployment of Helm Charts
Notification Controller
- Notification Dispatch
Image Reflector Controller
- Reflects Image metadata for
Automation Controller
Image Automation Controller
- Updates YAML when new
container images are
available
fluxcd.io
11. 11
11
● Helm
● Kustomize
● Prometheus
● Grafana
● Jenkins
● EKS
● AKS
● GCP
Flux Works with Other Tools
● Traefik
● Falco
● GitHub, GitLab, Bitbucket,
s3-compatible buckets
● Terraform
● …and more!!!
fluxcd.io
12. 12
12
● Makes life easier
● Multi-tenancy
● DependsOn
● Helm integration
● Notifications and Alerts
● Bootstrap
● Flux CLI
Reasons I and Others Love Flux fluxcd.io
13. 13
13
● There are two different forms of multi-tenancy
○ Hard multi-tenancy
■ Every tenant has their own cluster
○ Soft multi-tenancy
■ A cluster is shared across many different tenants
■ Tenants need to be isolated
What is Multi-Tenancy
14. 14
14
● Multi-tenancy lock down*
○ Ensure relevant controllers have cross namespace references disabled via
`--no-cross-namespace-refs=true`
● Resource Isolation
○ Ensure additional Flux instances are deployed when mission critical
tenants/workloads must be assured.
● Node Isolation
○ Ensure worker nodes are not being shared across tenants and the Flux
components.
● Network Isolation
○ Ensure the Container Network Interface (CNI) being used in the cluster
supports Network Policies.
Best Practices for Multi-Tenancy
15. 15
15
● Add `--no-cross-namespace-refs=true`
flag
○ Allow Flux to only reconcile Flux
resources that exist in the same
namespace
● Add `--default-service-account=default`
flag
○ Default Kustomization and HelmRelease
objects to not use the cluster scoped
service account
● Set the flux-system Kustomization
resource to use correct service account
○ This still needs cluster level access and
should not use the ‘default’ service
account
Enable Tenant Isolation for Flux