Containers and Kubernetes allow for code portability across on-premise VMs, bare metal or multiple cloud provider environments. Yet, despite this portability promise, developers may include configuration and application definitions that constrain or even eliminate application portability. In this meetup Oleg Chunikhin, CTO at Kublr, described best practices for “configuration as code” in a Kubernetes environment. He demonstrated how a properly constructed containerized app can be deployed to both Amazon and Azure using the Kublr platform, and how Kubernetes objects, such as persistent volumes, ingress rules and services, can be used to abstract from the infrastructure.

1. Application Portability with Kubernetes
Oleg Chunikhin |CTO, Kublr

2. Production-Grade K8s

3. Reasons for Portability
• Move load quickly (geography, cost, features)
• Lifecycle (dev/test/staging/production)
• Disaster recovery
• Split-tier architecture (application tiers may reside
in different environments)
• Cloud-bursting

4. Best Practices for Portable Applications
12-factor app is your bible–especially verses 2, 3, 4, 6, 7, 12
2. Explicitly declare and isolate dependencies
3. Store configuration in the environment
4. Treat backing services as attached resources
6. Execute the app as one or more stateless processes
7. Export services via port binding
12. Run admin/management tasks as one-off processes

5. Hard Problems
Ingress traffic
• Routing
• HTTPS/SSL certificates
State persistence
• Shared file storage
• Reliable block storage
• Database
Messaging

6. Application Needs
Application
Database Exclusive FS Shared FS
Routing SSL TermBalancing

7. “Rich” Platform
Application
Database Exclusive FS Shared FS
Routing SSL TermBalancingPlatform
Platform

8. Not Quite as Rich of a Platform
Application
? Database ? Exclusive FS Shared FS
? Routing ? ? SSL Term ?Balancing
Platform
Platform

9. “Poor” Platform
Application
? Database ? Exclusive FS ? Shared FS?
? Routing ? ? SSL Term ?? Balancing ?
Platform

10. Kubernetes to the Rescue
• Extreme ease and flexibility of component configurations and connections
• Configuration templating tools
• Helm
• Abstractions and extensible framework for ingress traffic processing
• Service
• Ingress
• Ingress Controllers
• Abstractions and extensible framework for storage management
• Volumes
• Persistent Volumes

11. Service
External node ports or external load balancer
Kubernetes cluster
Pod A-1
10.0.0.3
Pod A-2
10.0.1.5
Pod B-1
10.0.0.8
SrvB
10.7.0.3
Internal service
SrvA
10.7.0.1
SrvC
10.7.0.5
Ext
Resource

12. Ingress
Kubernetes cluster
SrvA
10.7.0.1
SrvB
10.7.0.3
Ingress rule 1:
abc.com/abc > SrvA
Ingress rule 2:
def.com/def > SrvB
Pod A-1
Pod A-2
Pod B-1

13. Ingress Controller
Edge / entry point / load balancer
Kubernetes cluster
SrvA
10.7.0.1
SrvB
10.7.0.3
Ingress
controller
Ingress rule 1:
abc.com/abc > SrvA
Ingress rule 2:
def.com/def > SrvB
Pod A-1
Pod A-2
Pod B-1

14. Configurability and Persistence
Pod
Container 1 Container 2
Persistent Volume:
NFS, Gluster, ceph,
EBS, dir, etc
Volume
Volume
Claim
Volume
Mount
Volume
Mount
Config Map
Secret
Storage Class
Static or dynamically allocated

15. Demo Application
WordPress
MySql Shared FS MySql Data
Routing SSL TermBalancing

16. Demo Application Helm Package
WordPress Deployment
MySql PVC
MySql Service
WordPress Ingress
WordPress Service
MySql Deployment

17. Demo Environment
AWS Kubernetes
Master
Ubuntu
Worker 1
Ubuntu
Worker 2
Ubuntu
Worker 3
Ubuntu
Azure Kubernetes
Master
RHEL
Worker 1
RHEL
Worker 2
RHEL
Worker 3
RHEL
kubectl --context=aws ...
helm --kube-context=aws ...
kubectl --context=azure ...
helm --kube-context=azure ...

18. Demo Application – Evaluation
WordPress
MySql
Routing SSL TermBalancing
MySql Data emptyDir Shared FS
ephemeral storage
provided by Kubernetes
Accessible from inside the cluster only via HTTP

19. Demo Application – Evaluation – Ingress
AWS
WordPress
MySql MySql Data emptyDir
Ingress Routing
Ingress SSL
Term + LEGO
ELB Balancing
Shared FS
ephemeral storage
provided by Kubernetes
Accessible via HTTPS on ELB with the given host

20. Demo Application – Production – EBS
AWS
WordPress
MySql
Ingress Routing
Ingress SSL
Term + LEGO
ELB Balancing
EBS MySql Data AWS EFS
persistent storage on AWS EBS
allocated by Kubernetes
Accessible via HTTPS on ELB with the given host

21. Demo Application – Production – RDS
AWS
WordPress
AWS RDS MySql
Ingress Routing
Ingress SSL
Term + LEGO
ELB Balancing
AWS EBS AWS EFS
persistent storage on AWS RDS
allocated outside of Kubernetes
Accessible via HTTPS on ELB with the given host

22. Demo Application – Production – Rook/Ceph
Azure
WordPress
MySql
Ingress Routing
Ingress SSL
Term + LEGO
Azure LB
Rook operator and Ceph cluster
MySql Data on Ceph
replica pool Ceph File System
Accessible via HTTPS on Azure LB with the given host
persistent storage on self-hosted
MySql and Ceph

23. Managed / Self-Hosted Options
File system: AWS EFS; Rook; OpenEBS; GlusterFS
Database: AWS RDS; MySql; PostgreSQL
Ingress: AWS ELB/ALB/NLB; Ingress controller
Certificates: AWS Certs; Let’s Encrypt; manual; self-signed
Messaging: AWS SQS; RabbitMQ; ActiveMQ
Azure has equivalent managed services for most app needs

24. Gotchas
• Self-hosted is more difficult to
operate than managed
• Different implementations
have varying functionalities
and QoS
• Performance
• Standards compliance
For example
• AWS EBS is AZ local
• Let’s Encrypt limits certificate
issuance rate
• Managed services may be better
hardware tuned
• Self-hosted services may be
better application tuned

25. Takeaways
• Cloud native Kubernetes applications are
portable, and easy to test, experiment,
and configure
• Portability tools
• Helm configuration templating
• Kubernetes abstractions: PV, PVC,
Ingress, Service etc
• Using self-hosted resources where
managed are not available
• Managed vs self-hosted services
considerations
• Different platforms
• On AWS and Azure
• On Ubuntu and RHEL
• Different ingress options
• With and without reverse-proxy
• With and without SSL
• Different persistence options
• Ephemeral storage
• Managed database
• Managed block storage, self-hosted
database
• Self-hosted cloud native storage

26. Q&A

27. Oleg Chunikhin
Chief Technology Officer
oleg@kublr.com
Kublr | kublr.com
Thank you!