The presentation is about the operator goal from networking perspective and how it is influenced by both swarm and kubernetes on the Docker EE platform

1. v
Networking in Docker EE 2.0
with Kubernetes and Swarm

2. SW Engineer, Docker
Flavio Crisciani
SW Engineer, Docker
Abhinandan Prativadi

3. Objectives
Design
principles
behind
Docker EE
Focus on
operator
deployment
goals
What’s the
best fit for
Kubernetes
and Swarm
Final
takeaways
SWARM KUBERNETES

4. Docker EE Design Principles
Multiple orchestrators
Multiple OSs
Multiple infrastructure
Choice SecurityAgility
Safer apps
Chain of custody
Threat mitigation
Unified operations
Cost efficiency

5. Docker EE Architecture
Secure Cluster Management
App Scheduler
Swarm KubernetesOR
Docker EE Cluster
Universal Control Plane (UCP)
Node
• Each node is K8s and Swarm ready
• Operator chooses the production orchestrator
• Freedom to choose orchestrator
Node Node

6. Operator network design goals
Network Security
Multi-tenancy
Observability
Flexibility
Data-path
Migration

7. Network Security
Practice of preventing unauthorized access, use,
disclosure, disruption, modification, inspection,
recording or destruction of information
Control planeData plane Management
Plane
Information
segregation

8. Manager
Mgmt Plane: Node identities with mutual TLS
Control Plane: Encrypted gossip based DB
Data Plane: Optionally encrypted with IPSec
Manager Manager
Network Security
WorkerWorker

9. Network Security
ucp-bundle-john $docker network create -d overlay john-network
Error response from daemon: access denied:
no access to Network Create, on collection
ucp-bundle-admin $source env.sh
Cluster "ucp_10.1.1.1:6443_admin" set.
User "ucp_10.1.1.1:6443_admin" set.
Context "ucp_10.1.1.1:6443_admin" created.
ucp-bundle-admin $docker network create --opt encrypted --driver overlay net1
l5vtb59oqk6r7fgzrjlou6llg
Jun 11 21:13:57 ubuntu-1 dockerd[4721]: time="2018-06-11T21:13:57.200950113Z"
level=debug msg="Initial encryption keys: [(key: dda14, tag: 0x1eb1) (key: b6b91,
tag: 0x1eb0) (key: 0e8a0, tag: 0x1eb2)]"
Jun 11 21:13:57 ubuntu-1 dockerd[4721]: time="2018-06-11T21:13:57.201251695Z"
level=debug msg="Initial encryption keys: [(key: dda14, tag: 0x1eb1) (key: b6b91,
tag: 0x1eb0) (key: 0e8a0, tag: 0x1eb2)]"

10. Manager
Mgmt Plane: Secure etcd, api-server access control
Control Plane: Calico BGP based Control plane
Data Plane: App to app encryption with service mesh
Manager Manager
Network Security
WorkerWorker

11. Network Security
ucp-bundle-john $source env.sh
Cluster "ucp_10.1.1.1:6443_john" set.
User "ucp_10.1.1.1:6443_john" set.
Context "ucp_10.1.1.1:6443_john" created.
ucp-bundle-john $
ucp-bundle-john $kubectl get pods -n kube-system
Error from server (Forbidden): pods is forbidden: User "john" cannot list
pods in the namespace "kube-system": access denied

12. Multi-tenancy
Concept that refers to the logical isolation of shared
virtual compute, storage, and network resources.
Application
isolation
Traffic
isolation

13. Multi-tenancy
Constraints allow to specify where a workload can
be deployed
Containers in different networks are isolated.
Worker
Worker
Net1 Net2

14. Multi-tenancy
$docker service create --name redis_2 --constraint 'node.labels.type ==
queue' redis:3.0.6
$docker service create --name prod-db --network net1 alpine sleep 9000
X8qnrfhhjrcis5nk6fx6mfc5w
$docker service create --name prod-web --network net2 alpine sleep 9000
T5uwwccffj0qg0zeddfnd5ouu
$docker exec -it prod-web.1.87aa93qtbg1dvxip9cpizjdls sh
/ # ping prod-db.1.87aa93qtbg1dvxip9cpizjdls
ping: bad address 'prod-db.1.87aa93qtbg1dvxip9cpizjdls'

15. Multi-tenancy
Node Affinity, Taints and Tolerations allow to
specify where a workload can be scheduled and
deployed
Policies define network connectivity
between pods
Worker
Worker

16. Multi-tenancy
$kubectl taint nodes node1 Tenant=node.org1:NoSchedule
…..
tolerations:
- key: ”Tenant"
operator: "Equal"
value: ”node.org1"
effect: "NoSchedule"
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata: name: default-deny
spec:
podSelector: {}
policyTypes:
- Ingress

17. Observability
Is a measure of how well internal states of a
system can be inferred from knowledge of its
external outputs.
Control
Plane
Data
Plane
Metrics

18. Observability
Mgmt Plane: Cluster key-value store based on raft
Control Plane: Gossip based datastore
Metrics through swarm events
Data Plane
• Linux: network namespaces, iptables, IPVS
• Windows: Windows Host Network Service

19. Observability
Mgmt Plane: etcd, kubectl
Control Plane: BGP for route distribution
Metrics through Prometheus
Data Plane
• Linux: L3 forwarding, iptables and ipsets, nsenter, iproute
• Windows: WinCNI that configures windows HNS

20. Flexibility
Ability of a system to adapt to different ecosystems
Network
Drivers
Cluster
Configuration

21. Flexibility
Allows multiple drivers,
most used in is overlay
Abstraction on top of
physical infrastructure
Dynamic network
creation
ORIGINAL ETHERNET FRAME
VXLAN FRAME
Available Drivers: Overlay, MacVlan, IPVlan, external drivers

22. Flexibility
Multiple CNI
plugins available
CNI integrated with the
cloud provider
Static network
configuration
ORIGINAL ETHERNET FRAME
IPINIP FRAME
Available Drivers: IPinIP, Native L3 routing

23. Data Path
Data-path traffic ingress and egress out of the
cluster and between workloads
Concept of
Service
Service
Discovery
Cloud
Provider
Performance

24. Data Path
Service is a group of containers sharing the same image
Forwarding performance depends on the driver,
but leverages Linux and Windows native data path
Service discovery is built-in served by the docker
daemon and extensible

25. Data Path
Service is a logical set of pods determined by label selectors.
Forwarding performance depends on the driver,
but leverages Linux and Windows native data path
Service discovery is swappable. kube-dns by default

26. Migration
Process of transferring apps between different systems
Docker EE
nodes are
Swarm and K8s
enabled
Both networking
stacks work
independently
Node

27. What fits best ?
App
Getting started InnovationFirst Project Scale
It depends…

28. Docker EE
allows you to
choose what fits
best for your
purpose.
Leverage your
current
expertise or
investment.
Swarm is simpler
with native
Docker
experience.
Kubernetes
brings the flexibility
and native
integration with
cloud providers.
Final Takeaways

29. Thank you!
Questions??