Talk held on R-ETAIL:CODE in London on 2018-03-15.
- The history of how DevOps evolved at Zalando: from on-premise data centers to autonomous teams, microservices and cluster management in the cloud
- How the developer experience looks like for the application lifecycle from idea to production and what our vision for the future is
- Challenges and learnings from our past experiences: why architecture principles and constraints are important to lead 200+ engineering teams
2. 2
ZALANDO IN NUMBERS
> 4.4billion EUR
2017
> 200
million
visits
per
month
> 14,000
employees in
Europe
> 70%
of visits via
mobile devices
> 23
million
active customers
> 250,000
product choices
~ 2,000
brands
15
countries
3. 3
OUR FOOTPRINT AROUND EUROPE
as of November 2017
1
8
10
11
12
13
BERLIN HEADQUARTERS AND OUTLET
BRIESELANG FULFILLMENT CENTER
ERFURT FULFILLMENT CENTER AND TECH OFFICE
MÖNCHENGLADBACH FULFILLMENT CENTER AND TECH OFFICE
LAHR FULFILLMENT CENTER
DORTMUND TECH HUB
FRANKFURT OUTLET
DUBLIN TECH HUB
HELSINKI TECH HUB
MILAN (STRADELLA) FULFILLMENT CENTER
KÖLN OUTLET
PARIS (MOISSY-CRAMAYEL) FULFILLMENT CENTER
SZCZECIN (GRYFINO) FULFILLMENT CENTER
HAMBURG ADTECH LAB
STOCKHOLM (BRUNNA) FULFILLMENT CENTER
10
9
7
6
5
3
2
1
11
12
13
4
14
15
15
14
9
8
7
6
5
4
3
2
1
4. 4
OUR FOOTPRINT AROUND EUROPE
TECH
as of November 2017
1
8
10
11
12
13
BERLIN HEADQUARTERS AND OUTLET
BRIESELANG FULFILLMENT CENTER
ERFURT FULFILLMENT CENTER AND TECH OFFICE
MÖNCHENGLADBACH FULFILLMENT CENTER AND TECH OFFICE
LAHR FULFILLMENT CENTER
DORTMUND TECH HUB
FRANKFURT OUTLET
DUBLIN TECH HUB
HELSINKI TECH HUB
MILAN (STRADELLA) FULFILLMENT CENTER
KÖLN OUTLET
PARIS (MOISSY-CRAMAYEL) FULFILLMENT CENTER
SZCZECIN (GRYFINO) FULFILLMENT CENTER
HAMBURG ADTECH LAB
STOCKHOLM (BRUNNA) FULFILLMENT CENTER (start winter 2017)
10
9
7
6
5
3
2
1
11
12
13
4
14
15
15
14
9
8
7
6
5
4
3
2
1
5. 5
WE ARE CONSTANTLY INNOVATING TECHNOLOGY
HOME-BREWED,
CUTTING-EDGE
& SCALABLE
technology solutions
~ 1,900
employees from
tech locations
(HQs in Berlin)7
77
nations
help our brand to
WIN ONLINE
12. 12
KPIs
• Adoption
• Deployments per developer
• Time from commit to production
• Developer satisfaction
• Infrastructure cost efficiency
⇒ Measurable OKRs
35. 35
ASSIGNING AWS IAM ROLE TO POD
kind: Deployment
spec:
template:
metadata:
annotations:
# annotation for kube2iam
iam.amazonaws.com/role: "app-myapp-role"
spec:
containers:
- name: ...
...
https://github.com/jtblin/kube2iam
⇒ AWS SDKs just work as expected
36. 36
POSTGRES OPERATOR
Application to manage PostgreSQL clusters
Observes “postgres” manifests (CRDs)
Spawns and modifies new clusters
Syncs and provisions roles
Handles volume resize, incl. Resize2fs
Also responsible for updating Docker images
37. 37
SUMMARY
• Application Bootstrapping
• Git (and GHE) as source of truth and UI
• 4-eyes principle for master/production
• Extensible Kubernetes API as primary interface
• OAuth/IAM credentials
• PostgreSQL
• CloudFormation for proprietary AWS services
41. 41
DOCUMENTATION
"Documentation is hard to find"
"Documentation is not comprehensive enough"
"Remove unnecessary complexity and obstacles."
"Get the documentation up to date and prepare
use cases"
"More and more clear documentation"
"More detailed docs, example repos with more
complicated deployments."
45. 45
LEARNINGS FROM THE PAST
• Platform as a service vs governance
• Platform teams think they are "special",
but lack basic customer insights
• Developers love autonomy, hard to "go back" once granted
• Developers are demanding
46. 46
NOTE TO SELF
• Establish a product mindset
• Listen to your customers
• Exploit unfair advantages
• Invest into documentation & support
48. 48
FUTURE
• Kubernetes adoption
• Multi cloud strategy
• More streamlined end-to-end experience (monitoring, ..)
• Mindshift towards full hands-off operations
• Data science use cases
49. 49
OPEN SOURCE
Running Kubernetes in Production on AWS
http://kubernetes-on-aws.readthedocs.io/en/latest/admin-guide/kubernetes-in-production.html
Kube AWS Ingress Controller
https://github.com/zalando-incubator/kube-ingress-aws-controller
External DNS
https://github.com/kubernetes-incubator/external-dns
PostgreSQL Operator
https://github.com/zalando-incubator/postgres-operator
Zalando Cluster Configuration
https://github.com/zalando-incubator/kubernetes-on-aws