The document discusses the benefits of moving infrastructure to Amazon ECS. It highlights both obvious benefits like cost savings and faster scaling, as well as less obvious benefits like having open source and private infrastructure, consistency across the organization through standardized tools and templates, increased cost clarity through custom monitoring, and empowering teams with better development tools and cost ownership. The presentation provides examples from Mapbox's experience migrating to ECS and emphasizes how the platform has allowed them to separate infrastructure from applications, standardize practices, run more workloads on spot instances to reduce costs, and give teams more autonomy.

1. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS re:INVENT
Moving to Amazon ECS—the Not-So-
Obvious Benefits
F r a n z i s k a S c h m i d t , P l a t f o r m E n g i n e e r , M a p b o x
Y a n i v D o n e n f e l d , B u s i n e s s D e v e l o p m e n t M a n a g e r , A W S
D e c e m b e r 1 , 2 0 1 7

2. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon Elastic Container Service

3. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Much more than container orchestration

4. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

5. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

6. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

7. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Concrete example, please…

8. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Canary Deployments
• New version verification with (a subset of) real users
• Zero down-time
• Seamless rollback

9. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Canary Deployments with ECS
TaskTask
Amazon
Route 53
record set
with weighted
routing policy
0%
100%

10. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Automated Canary Migration

11. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Automated Canary Migration (Cont’d)

12. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Demo

13. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Show us more…

14. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Significant Cost Savings

15. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Spot + On-Demand

16. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon
SQS
Watcher
Container
70,000 Tasks processed in one hour

17. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Hi,
I’m Franziska Schmidt
Platform Engineer at Mapbox
@franschm

18. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Beautiful
Maps

19. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Geocoding and Search

20. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Developer tools
& Mobile sdks

21. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Maps
Directions
Analysis
Teams and Systems @ Mapbox
Geocoding
Developer Tools
Mobile
Maps
Directions
Analysis

22. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Maps
Directions
Analysis
Teams and Systems @ Mapbox
Geocoding
Developer Tools
Mobile
Maps
Directions
Analysis
Platform

23. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Processing jobs
&
Api services

24. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Processing traffic at peak
70.000 tasks within one hour
200 Amazon EC2 instances at peak
Running in 2 AWS Regions

25. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
API traffic at peak
500-1500 tasks
40-200 EC2 instances
Running in 7 AWS Regions

26. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
About a year ago, we decided to migrate
our entire backend infrastructure to
Amazon ECS…

27. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

28. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
We decided to migrate to ECS for:
•Better Cost Savings
•Faster and more flexible Scaling
•Increased Developer productivity
The Obvious Benefits

29. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
The
Not-So-Obvious
Benefits

30. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
The Non-Obvious Benefits
Open source, private infrastructure
Consistency across organisation
Cost Monitoring Tools
Empowering Teams

31. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
What if…
I’m thinking about migrating over?
I’m in the middle of a system migration to ECS?
I’m already running on ECS successfully?
I run my systems on Kubernetes or ContainerSystemXYZ?

32. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Open Source, Private Infrastructure
• Consistency across organisation
• Cost Clarity
• Empowering Teams

33. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
A product developer asks you:
How can I run some code
on a server?

34. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
How can I run some code on a server?
The EC2 story (very simplified)
• Provision an EC2
• Boot the EC2
• Install requirements, set up your Node (/python/etc) environment
• Get your code onto the EC2
• Run your application!

35. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
How can I run some code on a server?
The ECS story (very simplified)
• Provision an EC2
• Boot the EC2
• Install requirements, set up your Node (/python/etc) environment
• Pull your code image onto the EC2
• Run your container!

36. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Infrastructure code separation benefits:
• Developers don’t need to touch infrastructure setup
• Consolidating infrastructure scripts for better security
and control
• You can open source application code easier!

37. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Why Open Source?
• Idea exchange and
collaboration with
larger
communities and
domain
• More eyes on
makes better code
• Developer perks

38. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Two ECS libraries we’ve open-sourced
•github.com/mapbox/ecs-conex
•Automated CI tool to push Docker images from GitHub commits to
Amazon ECR
•github.com/mapbox/ecs-watchbot
•CFN templates for running processing tasks on ECS

39. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Migrating to ECS helped us
separate infrastructure setup from
core domain code.

40. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Open Source, Private Infrastructure
Consistency across organisation
• Cost Clarity
• Empowering Teams

41. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
ecs-api: AWS CloudFormation helper library
for setting up an ECS-based HTTP API
•Establish patterns, conventions and best practises across ECS use
•Reduce copy/paste shared functionalities
•Easier to pick up context in unknown projects

42. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
CloudFormation for consistency
•Version Control: Config changes leave
a paper trail
•Reproducibility
•Standardised usage
`/cloudformation` folder in each
repository
•Tool for building templates:
github.com/mapbox/cloudfriend

43. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Incident response
•Using CloudFormation for version control & reproducibility
•Standardised usage across repositories
•Shared libraries give you access to alarm utilities for standard
needs e.g. Memory / CPU utilisations, error status codes

44. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Big system migrations
provide opportunities for
rethinking system designs.

45. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Open Source, Private Infrastructure
Consistency across organisation
Cost Clarity
• Empowering Teams

46. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Spot by Default
• Running on ECS is making it possible for us to run on
Spot by default
• Faster task start up, outages are easier to mitigate
• Diversify our task placement to limit impact of Spot
price outs

47. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Cost Clarity:
Insight in cost spendings on a
per-service, per-team basis

48. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
EC2: Total cost = Number of EC2s * (instance-type cost
per hour) * number of hours
For example,
If I ran 3 api-maps EC2s, at $16/hour, for 10 hours
= 3 * 16 * 10
= $480 ✅

49. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Cost monitoring on ECS is more
complicated. => What’s the cost of my ECS
service?

50. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
What’s the cost of my ECS service?
1. Find all tasks of my service
=> Tasks are tagged with team, category attributes
2. Calculate task cost through cpu, memory, duration
3. Aggregate usage data in Amazon Athena
4. Surface cost info through GitHub issues, Slack bots

51. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

52. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
We save costs through spot by
default on ECS; we gain clarity through
custom cost monitoring.

53. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Open Source, Private Infrastructure
Consistency across organisation
Cost Clarity
Empowering Teams

54. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Empowering Teams
• Many benefits I’ve talked about can be
summarised as empowering teams
• Better tools to launch code faster
• More ownership over cost and spendings!

55. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Ok, I’m sold.
How can I introduce a large system
shift in my organisation?

56. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
How can I introduce a large system shift in my organisation?
Make the
benefits obvious
to all

57. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
How can I introduce a large system shift in my organisation?
Be empathetic
to other teams’ needs and perspectives

58. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
How can I introduce a large system shift in my organisation?
Introduce plans timely
to get the work you
need on their roadmap

59. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
How can I introduce a large system shift in my organisation?
Incorporate
fun and motivation

60. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

61. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
How can I introduce a large system shift in my organisation?
Offer deliberate
time and support

62. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

63. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
What's next for
Mapbox on Amazon ECS?

64. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
THANK YOU!