If you're interested in using Kubernetes in the Cloud, then this talk is for you. Learn the deployment and management techniques that will help you manage Kubernetes clusters that you are supporting in the cloud. Apply concepts of CI / CD1/CD2 to your development of container based applications. Key Takeaways * Learn how to deploy Kubernetes with IaC * Learn common methods in developing IaC with a team * Understand patterns in building and testing containers * Standards on branching, project management and release engineering

1. Deploying to Day N
Operations of Kubernetes
and Containerized Apps
Anthony Ramirez
March 2020
© 2019 Nebulaworks, Inc.

2. DEVOPS PRACTITIONERS,
SOFTWARE DEVELOPERS,
ENGINEERING MANAGERS
WHO IS THIS TALK FOR?
© 2019 Nebulaworks, Inc.

3. OPERATIONALIZING KUBERNETES IN A CLOUD
WITH TERRAFORM (IaC)
LEVERAGING AGILE
WHAT IS THIS ABOUT?
© 2019 Nebulaworks, Inc.

4. DIVERSE METHODS/PROCESS IN
DEVELOPMENT
PLETHORA OF TOOLING
WHY?
© 2019 Nebulaworks, Inc.

5. Desirable
1. Deploy cluster in a consistent way
2. Ease of management of cluster, and write efficient
code/configs that can be updated transparently
3. Iterate quickly and safely on container applications, have
stable code releasable every 2 weeks.

6. Managed K8s
Services

8. K8s Managed Service Options
Managed
Service
Offering Multi-AZ Persistent
Volumes
Version RBAC
GKE GA Yes Block 1.15.8 Yes
EKS GA Yes Block 1.15.10 Yes
AKS GA Yes Block and
CIFS
1.16 Yes

9. IaC and K8s

11. Infrastructure as Code (IaC)
Manage and provision resources in cloud platforms with source code
Declarative
Define what the desired
resources and
dependencies that should
exist
Terraform
Cloud agnostic deployment
tool to provision resources
using a declarative language
Source code
as
Source of Truth
Leverage software development
practices to manage and test
IaC
Desired State
Management
Use IaC to increase replicability
and consistency in platform
deployments
© 2019 Nebulaworks, Inc.

12. How we build something
1. Identify the problem to solve, team and timeline
2. Create formal specification for the desired system
3. Kickoff meeting - “Divide and Conquer” - Plan, Build,
Test, Deploy Repeat
4. Issues are tracked on Kanban board
5. Release end of every bi-weekly sprint, regroup,
reorganize
6. At the end of timeline repeat Step 1

14. .
├── terraform
| ├── roots
| │ ├── dev
| │ │ └── k8s-platform
| │ ├── stage
| │ │ └── k8s-platform
| │ └── prod
| │ └── k8s-platform
| └── modules
| └── k8s-platform
└── docs
AWS AccountMultiple Environments Desired

15. Upgrading

16. Upgrading a Cluster
• Assumptions: AWS, EKS Terraform Module
1. Configure Cluster Autoscaler (CA) (github kubernetes/autoscaler)
2. Run Terraform changes to update Launch Config to new AMI
3. kubectl drain --force --ignore-daemonsets --delete-local-data
nodeA
4. Workloads will be rescheduled to untainted nodes
5. CA will create new nodes
6. Repeat steps 3 through 5
7. CA will terminate old nodes automatically
Additionally...
8. Kube-proxy, CoreDNS, Amazon VPC CNI, GPU support etc

17. Containers

18. Container Adoption Model
Holistic view of the evolution of container adoption
Initial Container Adoption
Initial deployment of CaaS,
no standards around
development, no
requirements of apps or
teams
CI/CD1
All groups are aligned with
adoption, standard
branching strategy
enforced, leverage cloud
operating model, CI in place
for apps, identification of
security domains
Standardization &
Modernization
Enforce security best
practices for
container/image creation,
execute securing all
security domains, parallel
app and infra pipelines with
common tools, process for
updating CaaS
End-to-End telemetry, security
High degree of monitoring and
logging capabilities, service
discovery/registration,
end-to-end security policy and
governance, tooling and
systems are flexible and can
accommodate change
TEAM CAPABILITIES
BUSINESS VELOCITY
© 2019 Nebulaworks, Inc.

19. CI / CD1 / CD2

20. Continuous Workflow Maturity
From scripts to CD-II
Script-based Testing
Individual or team-level scripted
assembly and testing of
applications. Entry-level
configuration management. Little
to no automation
Continuous Integration
Application and Infrastructure
pipelines having initial repo-level
automation for assembly and testing.
Continuous Deployment (CD-II)
Parallel application and infrastructure
pipelines supporting modern deployment
methods (blue/green, canary) leveraging a
common set of tools supporting the
infrastructure, platform, and application
services
Continuous Delivery (CD-I)
High degree of automation, leveraging
automated delivery and configuration of
applications and of Infrastructure as
Code. Ability to leverage pipelines in
parameterized builds in any
environment.
© 2019 Nebulaworks, Inc.

21. Master
branch-1
branch-2
The Development
Process
Open PR Update PR Update
PR
WIPWIP READY
Open PR Update PR
WIP READY
test

22. Thank You!