See how Kubernetes-native security differs from traditional security approaches.
We'll talk about how you can find and fix blind spots, critical vulnerabilities, and misconfigurations that are unique to Kubernetes to increase protection. And to get your team to adopt this, you'll also see how to help shorten the learning curve for them. Lastly, you'll see how to minimize operational risk by using scalable enforcement functions, while keeping operations simple.
The demo will be on how to use Red Hat Advanced Cluster Security/Stackrox to implement Kubernetes-native security on containers that are running across k8s/OpenShift clusters and implement best practices across use cases like visibility, vulnerability management, and more.
Presented by Krishnan Narayana Swamy, Specialist Solution Architect, Red Hat
2. 2
Accelerate your journey to Kubernetes
with the Konveyor Community
A community of people passionate about
helping others modernize and migrate
their applications to the hybrid cloud by
building tools and best practices on
how to break down monoliths, adopt
containers, and embrace Kubernetes.
www.konveyor.io
4. 4
About Me
▸ Krishnnan Narayana swamy
▸ Specialist Solutions Architect, ANZ
▸ Based in Melbourne, Australia
▸ https://github.com/krnaraya
▸ https://www.linkedin.com/in/nkrishnan/
5. 5
Bake Security into Dev & Ops
What we’ll discuss today...
▸ Kubernetes Security Best Practices
▸ Kubernetes Native Security through
StackRox
▸ StackRox Kubernetes Security
Platform Architecture
▸ Demo
12. 13
KubeLinter
StackRox Shifted Left
▸ Open source Apache 2.0 license
▸ Checks Kubernetes YAML files and
Helm charts
▸ 16 default checks
▸ Extensible with custom checks
▸ Integrates with any CI tool
KubeLinter as a GitHub action
https://github.com/stackrox/kube-linter
13. 14
Welcome to StackRox.io community
Open source version
▸ Join our Slack Channel
▸ Participate
▸ Contribute
Hello All, Welcome and Thank you for taking your time and joining us today
I am Krishnan Narayana swamy, started my career as a software developer and right after my graduation, few bunch of college mates, we Founded a startup named telesto based on open source lucene search project in 2008 and i have had great association and passion with open source technologies since then. I have in the past and continue to help lot of customers to innovate and adopt open source technologies, especially kubernetes and applications that run on kubernetes, in the best way possible and that is exactly my role as a Specialist Solutions Architect at Redhat ANZ. I am based out of Melbourne in Australia. Feel free to connect with me on linked in and explore my github repositories for my experimentations or demos in the kubernetes space.
I’m a developer and I worked in infrastructure before and I don’t think I thought about security a single time. Basically, what I would do was develop code, make sure it worked on docker containers, push it, and hope for the best; and now looking at it from a security point-of-view, it probably wasn’t the best way to go. So, how do we build security into rolling out our software?
How to bake security into your Dev and Ops processes for your applications running in our kubernetes clusters. Kubernetes is Awesome, but the out of the box open source version has little for security. We will discuss why container security is important, some of the kubernetes security best practices and how stackrox kind of bridges the gap here.
Stackrox is a recent acquisition by Red hat and we have decided to open source and we are in the process of making the platform open source soon.
We will look at the architecture of the platform and its components and how it enables cloud native security for kubernetes clusters and the applications that run in them
We will see a demonstration of the product and how it helps in continuous security across your DevOps process and how it integrates into your CI/CD process enabling your Shift Left on security.
Let us assume, you have a kubernetes native application deployed and have a solid DevOps practice set up
Continuous Integration = Development practice in which developers integrate code into a shared repository (main code path/trunk) every commit; Automated build and unit tests are run to validate the changes. In this way, defects and integration errors are verified and remedied earlier in the process.
Continuous Delivery = Orchestrating your build through a series of quality gates with automated/manual approval processes, however the build stops before reaching production - typically additional validations are required (manual, compliance)
Continuous Deployment = And extension of Continuous Delivery where the build is pushed automatically from commit to production if all the quality gates are met
DevOps-driven adoption of new technologies and processes may leave security as an afterthought or, in some instances, expose new gaps in security coverage and risk management.
Let us assume, you have a kubernetes native application deployed and have a solid DevOps practice set up
Continuous Integration = Development practice in which developers integrate code into a shared repository (main code path/trunk) every commit; Automated build and unit tests are run to validate the changes. In this way, defects and integration errors are verified and remedied earlier in the process.
Continuous Delivery = Orchestrating your build through a series of quality gates with automated/manual approval processes, however the build stops before reaching production - typically additional validations are required (manual, compliance)
Continuous Deployment = And extension of Continuous Delivery where the build is pushed automatically from commit to production if all the quality gates are met
DevOps-driven adoption of new technologies and processes may leave security as an afterthought or, in some instances, expose new gaps in security coverage and risk management.
The defaults in upstream Kubernetes can not be assumed to provide adequate protection based on our use case
A common reason for attacking containers today is to abuse compute resources, for example, for cryptocurrency mining. can also offer access to customer or workload data.
But attacking the container chance to propagate to other nodes in the cluster and also gain persistent access to valuable user code, compute and/or data
receives privileged access, they could potentially access information running in the other containers.
The Kubernetes master controls your cluster. An attacker that can compromise the master can control the environment, including the ability to take it offline. And a compromised etcd can mean the ability to modify or destroy the cluster, steal secrets and credentials, or gain enough information about the application it’s running to go recreate it somewhere else.
Lets discuss some of the best practices out there At the infrastructure level, i recommend keeping the kubernetes cluster up to date,
Make sure that your network access is firewalled off correctly. Make sure the Kubernetes API server (which is basically the entry point into all of your infrastructure) is locked down. (I know a lot of people will restrict it to a VPC or VPN. Just ensure that the traffic to the API server is protected.) Lastly, let’s make sure that the actual host itself is locked down.
Leverage a matured CI/CD processes to deploy applications to kubernetes cluster. One service account authorized against your API server. Obviously, you still have people who need access to the cluster, you still need to debug and fix, but now since you have fewer users and groups, you have a much smaller number of people you need to manage in RBAC.
From an application / container stand point,
It starts with building your image, which means looking at the dependencies, the packages, and most importantly analyzing the vulnerabilities to discover what we already know is bad.
Then, you look at how the app is configured:
What privileges does it have on host os?
What privileges does it have against the API server?
Are there weak secrets or other sensitive data (API keys for registries, sensitive databases)?
What labels and annotations are used? This is key because it allows you to to answer the question of who owns a given service (annotation of owner, email, or team) and is very valuable for debugging for operations. This also allows you to route the issue quickly to the right owner as opposed to playing a game of murder mystery and trying to figure out who owns this application and who deployed it.
Some of the most important configurations best practices that I we highly recommend for containers and Kubernetes are as follows:
Mount host-sensitive directories as read-only - this means no one can write to your file system.
Set up appropriate SELinux profiles to limit containers mounting unix socket. This gives permissions to run containers, create images and so on..
Use capabilities to grant fine-grained privileges - use CAP DROP and CAP ADD to limit what Linux capabilities containers are allowed to run.
Do not run ssh services inside a container.
Ensure that there are no unnecessary ports exposed
ensure that the container’s ports are not mapped to host ports
Network policies – use network policies to limit ingress and egress network communication between Pods (east-west) and from outside to Pods (north-south).
Secure the host OS - SELinux
RESTRICT A CONTAINER FROM ACQUIRING NEW PRIVILEGES
So, the last thing to always remember is that security is hard, more akin to a marathon than a sprint. There’s no such thing as perfect security; it’s always about monitoring, iterating, and making sure that the tools are available for people who are building the code to integrate it into their process and drive security.
Throwing another Buzz word here , the DevSecOps.
The idea here is to embed Security early in your development process to prevent any security incidents and enabling continuous security in a cloud native fashion.
Another key is the constant feedback loop between the build/deploy phase and the runtime phase.
Enable Developers and provide them with the security issues during development and educate them on how they can be fixed.
Culture change when it comes to security right, how do we fix a container image that has vulnerability? , we don’t kill the container process and leave it there, we go back and fix them in our source, build and push a new image into production. Cloud native CI/CD processes allow us to do that.
Stackbox runs in kubernetes, supports declarative style and same Cloud native way of doing things, does remediate using kubernetes constructs ..leverages most of the kubernetes out of the box flexibility to achieve things.
protection
Eliminate blinspots
Threats unique kubernetes vulnerabilities
Time to implement security and learning cost
Remediates in the kubernetes way
Operational security - greater uptime
Operational conflict - inconsistent configurations
Ability to bring security to your build phase, deploy phase and Run phase
It supports all kubernetes
Disconnected installs on prem and runs anywhere
Highly flexible and Integrates with some of the platforms
For example, for package scanning scanning for vulnerabilities, it can work with tenable, Quay or anchore or you can use inbuilt scanning. Supports number of different languages including .NET core
Integrate with your CI/CD tools including jenkins,
DevOps notifications for Jira, slack pagerduty and others..
SIEM toold it can
Central
Core function: UI and API server; policy engine
How it works: Runs as a Kube service in cluster/VM/cloud; must be able to receive inbound connections from all monitored clusters
What it does:
Facilitates all UI and customer-facing API interactions
Performs policy evaluation
Facilitates multi-cluster management of all services environments across various providers and data centers
Sensor
Core function: Point of integration with Kubernetes; processing engine
How it works: Runs as a Kube service with permissions to listen to orchestrator events and CRUD rights for enforcement
What it does:
Inventories cloud-native components (namespaces, running services, network policies, and orchestrator secrets)
Performs detection rule processing
Facilitates enforcement
Collector a Daemonset
Core function: syscall-level data collector
How it works: Runs as a container on each host and inserts a kernel module for syscall visibility
What it does:
Facilitates collection of data for runtime detection
Scanner (Optional)
Core function: Vulnerability scanning of images
How it works: Runs as a Kube HPA in the same cluster as Central.
What it does:
Handles requests to scan image layers from Central
Pulls image layers from registry using configured image integrations
Retrieves updated CVE data from stackrox.io or from Central
Scanner-db
Core function: Database for caching image layer data.
How it works: Runs as a single Kube pod with PostgreSQL DB
What it does:
Caches results of layer scanning
Customers do not need to provide redundancy / backup
AdmissionController
Core function: Plugin to Kube AdmissionController to approve / deny API actions
How it works: Runs as a Replicaset, registered as ValidatingWebhookConfiguration
What it does:
Examines Kubernetes API commands that create pods, deployments, daemonsets, etc.
Offer stackrox admins the ability to reject these API requests based on policy.
Can also examine Update API commands to reject changes for already deployed
Can examine & prevent ‘exec’ and ‘port-forward’ API requests
Sensor talks to kube apis to get data , what deployments, what configuration
Collector - one pod at the node level, what process. What network traffic
From a cluster level, most of the kubernetes distributions, you can leverage out of the box functionalities to extend it for applying security best practices.
For example., From OKE, the upstream open source version of OpenShift few security aspects comes default
Secure OS/Infrastructure
The core components of the Linux kernel that are used for containers are cgroups — control groups, which define the resources like CPU and memory which are available to a given process — and namespaces, which are a way of separating processes by restricting what each process can see, so that system resources “appear” isolated to the process. Along with cgroups and namespaces, you can also use a Linux Security Module (LSM) like SELinux to configure a container’s capabilities. SELinux. Both deny undesirable default capabilities, like the ability to write to the proc filesystem
RHEL CoreOS
Cri-o container runtime - provides a smaller footprint and reduced attack surface
SELinux
Security Built-In
Identity & Access Management
Compliance Operator - describe the desired compliance state of a cluster and provides them with an overview of gaps and ways to remediate them
Volume Encryption
File Integrity Operator
Security Context Constraints - allow administrators to control permissions for pods
Secrets Management
All the above from a more cluster level security
Stackrox elevates it to the next level or application level and brings in the ability to customize your policies and gives a better visibility and ability respond to those events through notifications.
Build Security
Vulnerability scanning
Vulnerability management
CI/CD integration to fail builds
Registry integration to block deployments
CI/CD Integration for CIS Docker checks/failures
Image Content Analysis
Deploy Security
Declarative Policy Creation
Multi-factor risk profiling
Compliance benchmarks
RBAC analysis for Kubernetes
Network policy enforcement
Service Configuration
Admission Controls
Posture Management
Kubernetes Events (Execs/Forwards)
Runtime Security
Intrusion detection & behavioral analysis
Network policy visibility/incident detection
Deep data collection and correlation for forensics
Baselines/whitelists of runtime activity profiles
Killing of compromised pods
Detection of real-time intrusions, cryptojacking, and orchestrator threats