Robert Bates, SVP Sales Engineering of Crunchy Data explains how you can tackle Data Gravity, Kubernetes, and strategies/best practices to run, scale, and leverage stateful containers in production.
2. Agenda
● Crunchy Data Overview
● Crunchy PostgreSQL for Kubernetes
○ Crunchy Container Suite
○ Crunchy PostgreSQL Operator
● Container Tech. Roadmap
● Conclusion/Next Steps
2
3. - Crunchy Certified PostgreSQL is open source and Common Criteria EAL 2+ Certified, with
essential security enhancements for enterprise deployments
- Author of DISA Secure Technology Implementation Guide for PostgreSQL and CIS PostgreSQL
Benchmark
- Automated Security and “Authority to Operate (ATO) in a Day”
Crunchy Data Overview
3
Market Leading Data Security
Cloud Ready Data Management
- Crunchy PostgreSQL for Kubernetes - open source & scales to 1000s of PostgreSQL instances
- Cloud Agnostic - Deploy to your choice of public or private clouds, or on-premise infrastructure
Leader in Open Source Enterprise PostgreSQL
- Leading PostgreSQL Contributor, driving PostgreSQL stability, security, and performance
- Open source solutions for high-availability, disaster recovery, and monitoring
4. Crunchy PostgreSQL for Kubernetes
● Group of open source projects to combine the robustness of PostgreSQL
with the deployment flexibility of Kubernetes
○ Crunchy Container Suite
○ Crunchy PostgreSQL Operator
● Works on enterprise Kubernetes based platforms
● Actively developed
● Successfully deployed at Fortune 100 and Government customers
4
5. Container Container Suite
5
https://github.com/CrunchyData/crunchy-containers
● Microservices architecture
● PostgreSQL/PostGIS Containers
(9.5, 9.6, 10.x, 11.x)
● PostgreSQL utility containers that
are the building blocks for running
production environments:
○ High Availability
○ Disaster recovery for
terabyte-scale clusters with
pgBackRest
○ Monitoring
● Open source
● Examples provided
7. When to Use Kubernetes with PostgreSQL
● Value of Kubernetes increases
exponentially as number of containers
increases
● Due to statefulness of databases,
Kubernetes requires more knowledge
to successfully operate a standard
database workload:
○ Avoid scheduling and availability issues for
longer-running database containers
○ Data continues to exist even if container
does not
7
8. Crunchy PostgreSQL Operator Overview
● Open source project to provide a smarter way of provisioning PostgreSQL
clusters on Kubernetes
● Based on the Kubernetes client APIs, written in golang
● Custom Resource Definitions used to capture the definition of PostgreSQL
clusters and administration tasks
● REST API provides client integration, RBAC included for fine grained controls
● Goal is to assist in Day 2 operations as well as provisioning
● Single Deployment per Namespace, PostgreSQL clusters represented as
Deployments
● Leverages the Crunchy Container Suite set of microservice containers
8 https://github.com/CrunchyData/postgres-operator
9. ● Allows an administrator to run PostgreSQL-specific commands to
manage database clusters, including:
○ Creating / Deleting a cluster (your own DBaaS)
○ Scaling up / down replicas
○ Failover
○ Apply user policies to PostgreSQL instances
○ Define what container resources to use (RAM, CPU, etc.)
○ Smart pod deployments to nodes
○ Backup and restore
○ Backup scheduling
● REST API allows for easy integration with other services (e.g. pgo-
osb)
● The Crunchy PostgreSQL Operator enables your own secure,
elastic database-as-a-service
Crunchy PostgreSQL Operator Features
9
10. Crunchy PostgreSQL Operator Architecture
10
● Utilizes Kubernetes
Deployments
● Flexibility in storage
classes
● Flexibility in operating
environments
○ Node affinity
○ Resource (CPU, RAM)
configurations
● Flexibility in database
version runtimes
11. Why Use An Operator With PostgreSQL?
11
● Automation: Complex, multi-step DBA
tasks reduced to one-line commands
● Standardization: Many customizations,
same workflow
● Ease-of-Use: Simple CLI; UI in beta
● Scale
○ Provision & manage clusters quickly
amongst thousands of instances
○ Load balancing, disaster recovery,
security policies, deployment
specifications
● Security: Sandboxed environments,
RBAC, mass grant/revoke policies
12. Operator HA Support
The Crunchy PostgreSQL Operator
supports:
● PostgreSQL Deployments spread
across Kubernetes Nodes.
● Multiple storage backends within a
single PostgreSQL cluster
● High-availability through automated
and manual failover
● Rescheduling if a Kubernetes Node
becomes unavailable
● Storage providers replicating storage
between zones depending upon
storage vendor
12
13. Crunchy PostgreSQL Operator Successes
● Federal Government organizations’ deployments,
automated provisioning process from days to minutes
● Fortune 100 manufacturer deploying the Operator to build
an in-house database-as-a-service offering internally,
reducing provisioning times from weeks to seconds
● Fortune 250 customers using Container Suite and
Operator to simplify their development, test, QA
processes on on-premise and public cloud infrastructure
13
14. Crunchy PostgreSQL Operator Roadmap
● Open service broker implementation (pgo-osb) (underway)
● Web user interface (pgo-ui) (underway)
● pgBackRest integration (underway)
● Backup management
● Advanced Data loading
● Disk level snapshots
● archive/export/import
● Ingress integration
● Operator Federation (cross-namespace and cross-Kube-cluster)
● Additional advanced user management
● Automated Postgres Upgrading
● Sharding/Advanced Data Deployments
14
16. Next steps
Questions to consider
● Do you want to securely provision thousands of databases in a reliable, auditable environment?
● Do you want to be able to deploy your databases to ANY cloud platform, public or private, from a
single interface?
● Do you want to simply create highly-available PostgreSQL clusters with full disaster recovery
capabilities for databases of terabyte scale?
● Do you want to give your developers the ability to instantly provision databases that meet your
compliance requirements?
● Would the ability to leverage cost-effective open source used by Fortune 100 companies to
provision and manage thousands of databases be meaningful to your business?
Who to call?
Robert Bates, robert.bates@crunchydata.com, 770-330-5838
Paul Laurence, paul@crunchydata.com, 843-737-6045
16
Founded in 2012, emphasizing the security, scalability and reliability of PostgreSQL
Early PostgreSQL requirements at US Department of Defense
Common Criteria EAL 2+ Certified PostgreSQL
DISA STIG for PostgreSQL, NIST 800-53 conformance, CIS PostgreSQL Benchmark
World-class PostgreSQL expertise, including Core Developers, Committers and Major Contributors.
Responsible for a number of PostgreSQL enhancements including:
pgJDBC, pgBackRest, pgAudit, pgPartman, pgMonitor, and others
Support from 100% Senior PostgreSQL engineers (not a call center model)
100% Open Source- NO proprietary license fees.
Business model:
Annual Support Subscription on Trusted Distribution of PostgreSQL
Professional Services, Remote DBA & Training Available
Introduce the concepts here: CCS is a set of microservices required for successful production deployment of PostgreSQL
Operator allows stateful PostgreSQL to run in Kubernetes
PGO CLI - REST Client
API Server - REST API server
postgres-operator - core operator logic
Template based, alternative deployment models are possible but not easy!
Configuration is server side
Command set (provision, user mgmt., volume mgmt., scaling, failover, policy mgmt., simple data loading, labeling, status)
Failover (manual and automated) implemented natively within the Operator
Targets Day 2 Operations/Use Cases
PGO CLI - Rest client that comes with the Operator
PGO UI - easy-to-use interface for managing PostgreSQL clusters
PGO OSB - Open Service Broker implementation for communicating across applications
The Operator is deployed to a namespace on a Kube cluster, it operates on PG clusters created in that namespace. The Operator Pod includes up to 3 containers including the REST API server, the operator logic, and the web user interface. The pgo CLI tool runs outside the cluster on your laptop for example where you can access the Operator features.
The Operator deploys a PG cluster which includes a Service, a primary PG deployment, storage, and optionally PG replicas. The operator also can embed a metrics collection container into the PG pod to enable metrics collection. Also, the Operator can deploy a pgpool Pod configured for your PG cluster, also optional.
