WSO2Con USA 2015 talk on implementing a multi-cloud with Kubernetes and WSO2 Private PaaS

1. Scale into Multi Cloud
with Containers
Imesh Gunaratne
Senior Technical Lead, WSO2

2. Agenda
• WSO2 Carbon Reference Architectures for Kubernetes
• Implementing a Multi-Cloud with Kubernetes
• WSO2 Private PaaS Multi-Cloud Solution
• Kubernetes, WSO2 Private PaaS Feature Comparison
• WSO2 Vision on Kubernetes

3. WSO2 Carbon Reference Architectures
for Kubernetes

4. WSO2 Carbon Worker/Manager
Reference Architecture for Kubernetes
Pod 1 Pod 2
Manager Cluster
Services
Pod 3 Pod 4 Pod n
Worker Cluster
Services
Manager
Replication
Controller
Worker Replication
Controller
Client
Gov
Reg
Conf
Reg
User
Store

5. WSO2 API-M Reference Architecture
for Kubernetes
Pod 1
Gateway Mgr
Services
Pod 2 Pod 3
Gateway Worker
Services
Client
Pod 4 Pod 5
Key Manager
Services
Pod 6 Pod 7
Store Services
Gateway
Mgr RC
Gateway
Worker RC
Key
Manager RC
Store RCGov
Reg
Conf
Reg
User
Store
API-M
DB
A volume mount for
synchronizing API
artifacts
Pod 8 Pod 9
Publisher Services
Publisher
RC

6. Implementing a Multi-Cloud with
Kubernetes

7. Why do we need a Multi-Cloud?
● Capacity overflow
○ Overflow from on-premise clusters to public when run out of
on-premise capacity
● Sensitive workloads
○ Privacy-sensitive workloads to run on on-premise clusters
● Vendor lock-in avoidance
○ Workloads to run across multiple cloud providers
● High availability
○ Multiple availability zones within a single cloud provider
https://github.com/kubernetes/kubernetes/blob/master/docs/proposals/federation.md

8. Kubernetes Multi-Cloud Reference
Architecture 1 (monolithic)
IaaS Provider 1
Kubernetes Cluster 1
Load
Balancer 1
Load
Balancer 2
Load
Balancer n
Central Load
Balancer
N1 N2 N3 NnM N1 N2 N3 NnM N1 N2 N3 NnM
IaaS Provider n
Client
IaaS Provider 2

9. Kubernetes Multi-Cloud Reference
Architecture 2 (decoupled & hierarchical)
IaaS Provider 1
Kubernetes Cluster 1
Load
Balancer 1
Load
Balancer 2
Load
Balancer n
Central Load
Balancer
N1 N2 N3 NnM
Kubernetes Cluster 2
N1 N2 N3 NnM
Kubernetes Cluster n
N1 N2 N3 NnM
IaaS Provider n
Client
Federation
Controller

10. Core Challenges of Implementing a Multi-
Cloud
● Location affinity
○ Distributed applications
■ Strictly coupled
■ Strictly decoupled
■ Preferentially coupled
○ Can pods of a single distributed application partitioned across
more than one cluster?
● Cross-cluster service discovery
○ Use DNS
https://github.com/kubernetes/kubernetes/blob/master/docs/proposals/federation.md

11. Core Challenges of Implementing a Multi-
Cloud
● Cross-cluster scheduling
○ Closely related to location affinity
○ Have a layer on top of k8s
clusters to handle resource
creation
○ Make policy-based decisions
● Cross-cluster migration
○ Moving a distributed application
from one k8s cluster to another
https://github.com/kubernetes/kubernetes/blob/master/docs/proposals/federation.md

12. Kubernetes Cluster Federation Proposed
Architecture (decoupled & hierarchical)
https://github.com/kubernetes/kubernetes/blob/master/docs/proposals/federation.md
Ubernetes Control Plane
Kubernetes Cluster 1
GCE
Kubernetes Cluster 2
AWs
Kubernetes Cluster n
Private
Standard
Kubernetes API
Ubernetes API
Policy Engine
Auth
Desired
Federation
State
Migration
Controllers

13. WSO2 Private PaaS Multi-Cloud Solution
+

14. WSO2 Private PaaS Architecture

15. Network Partitioning
IaaS Provider 1 IaaS Provider n
Region 1 Region 2
Zone A Zone B
Network Partition 1
Partition 1 Partition 2
Network Partition 2 Network Partition 3
Region 1
https://docs.wso2.com/display/PP410/Network+Partitions

16. Composite Application Model
https://docs.wso2.com/display/PP410/WSO2+API+Manager+as+a+Service
Gateway Mgt
Cartridge
Gateway Worker
Cartridge
Key Manager
Cartridge
Publisher/Store
Group
Gateway/KM Group
API-M App
Network Partition 1
Network Partition 2
Publisher Cartridge Store Cartridge
Deployment
Policy
Autoscaling
Policy Deployment
Policy
Autoscaling
Policy

17. Runtime Model (Topology)
https://docs.wso2.com/display/PP410/WSO2+API+Manager+as+a+Service
API-M App
NP1 App Instance NP2 App Instance
Gateway Mgt
Cluster
Gateway Worker
Cluster
Key Manager Cluster
Pub/Store Cluster
Gateway/KM Group
Gateway Mgt
Cluster
Gateway Worker
Cluster
Key Manager Cluster
Pub/Store Cluster
Gateway/KM Group

18. WSO2 Private PaaS Multi-Cloud Solution
IaaS Provider 1
Kubernetes Cluster 1
Load
Balancer 1
Load
Balancer 2
Load
Balancer n
Central Load
Balancer
N1 N2 N3 NnM
Kubernetes Cluster 2
N1 N2 N3 NnM
Kubernetes Cluster n
N1 N2 N3 NnM
IaaS Provider n
Client

19. Kubernetes, WSO2 Private PaaS Feature
Comparison
≅

20. Feature Comparison
Kubernetes WSO2 Private PaaS
Carbon Cluster Discovery Available Available
Artifact Distribution Available (Rolling update) Available (Git based)
Centralized Logging Available (kubectl) Available (Thrift, DAS)
Monitoring & Metering Available (cAdvisor,
heapster, grafana)
Available (DAS)
Multi-Tenancy Available (Namespaces) Available (Carbon)
Load Balancing Available (nginx) Available (nginx, haproxy,
LVS, ec2)

21. Feature Comparison (cont.)
Kubernetes WSO2 Private PaaS
Horizontal Pod
Autoscaling
Available with v1.2.0-
alpha.2
Available
Multi-Cloud Deployment A proposal available Available
Horizontal VM
Autoscaling
Being discussed Available
Composite Application
Model
Being discussed Available

22. WSO2 Vision on Kubernetes

23. WSO2 Vision on Kubernetes
● Contribute to Kubernetes on implementing:
○ Autoscaling (Horizontal Pod/Node + Vertical Pod scaling)
○ Kubernetes Cluster Federation (Ubernetes)
○ Composite Application Model (may be using TOSCA, CAMP,
etc)
● Use Kubernetes in the future:
○ As the WSO2 PaaS
○ As the Microservices solution platform
○ Replace in JVM multi-tenancy using Kubernetes namespaces

24. Thank You