Amazon EC2 allows you to bid for and run spare EC2 capacity, known as Spot instances, in a dynamically priced market. On average, customers save 80% to 90% compared to On Demand prices by using Spot instances. Achieving these savings has historically required time and effort to find the best deals while managing compute capacity as supply and demand fluctuate. In this session, we dive into how customers who have designed scalable, cloud friendly application architectures can leverage new Spot features to realize immediate cost savings while maintaining availability. Attendees will leave with practical knowledge of how, via well architected applications, they can run production services on the Spot instances just like IFTTT and Mapbox.

1. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Boyd McGeachie, Business Development
November 29, 2016
Save Up to 90% and
Run Production Workloads on Spot
Featuring IFTTT and Mapbox
CMP307

2. On-Demand
Pay for compute
capacity by the hour
with no long-term
commitments
For spiky workloads,
or to define needs
Amazon EC2 Consumption Models
Reserved
Make a low, one-time
payment and receive
a significant discount
on the hourly charge
For committed
utilization
Spot
Bid for unused
capacity, charged at a
Spot Price which
fluctuates based on
supply and demand
For time-insensitive
or transient
workloads

3. With Spot ,the rules are simple
Markets where the price of
compute changes based on
supply and demand.
You’ll never pay more than your
bid. When the market exceeds
your bid, you get 2 minutes to
wrap up your work.

4. $0.27 $0.29$0.50
1b 1c1a
8XL
$0.30 $0.16$0.214XL
$0.07 $0.08$0.082XL
$0.05 $0.04$0.04XL
$0.01 $0.04$0.01L
C3
$1.76
On
Demand
$0.88
$0.44
$.22
$0.11
Show me the markets!
Each instance family
Each instance size
Each Availability Zone
In every region
Is a separate Spot Market

5. 50% Bid
75% Bid
You pay the
market
price
Bid Price Vs Market Price
25% Bid

6. EC2 Best practices
Fault tolerance
for Spot
Stateless Multi-AZ Loosely coupled
Instance
Flexibility

8. Maps
Directions Geocoding
Mobile
Developer tools
Analysis

9. Mapbox maps power over 5,000
platforms ranging from social to
mobility apps

17. 21,014,573,573 probes per week
605,940,678 miles per week

18. S3 Bucket
Amazon
DynamoDBMobile Apps
Amazon
Route 53
Amazon
Kinesis
Processing
Persistence
API
Auto Scaling
group
Spot Fleet
Auto Scaling
group
Spot Fleet

19. Consumption of compute hours
has increased by 1044% since
last year

20. We will do over 500 million
hours of compute this year

21. But margins increased

23. Q: How much did you have to
change in order to switch to Spot?

24. A: Following general best
practices makes it easy to use
Spot Instances.

25. Best Practices
• Diversify across Availability Zones, regions, and the types of
instances you use to ensure stability
• Reduce your cold start time
• Break up large jobs into smaller pieces

26. Diversify for stability
• 10 completely isolated regions, multiple Availability Zones, and
multiple instance types
• The Spot market is segmented by these dimensions, so each one
adds an additional layer of protection against price spikes and
interruptions
• Use Spot Fleet: bid on multiple types of instances
• ECS makes it easy to deploy applications across multiple types of
instances

27. Spotswap
• Manages Spot priceouts for a Spot Fleet by activating backup On-
Demand capacity.
• Polls the termination notification endpoint on each instance and tags
instances that are about to be terminated
• When enough Spot Instances are about to be terminated, we
automatically launch On-Demand instances to pick up the slack
• When Spot prices go back down, we shut down the more expensive
On-Demand instances

29. Reduce your cold start time
• Bake software into an AMI or Docker image for fast loading
• Minimize state on the EC2 using S3, DynamoDB, or Amazon Kinesis
• Goal should be to have your application bootstrap in less than 2
minutes
• ECS helps us quickly deploy applications and move them around to
different EC2 instances - seconds instead of minutes

30. Break up large jobs

31. Long running job?
Instance 1
15 hours

32. Break it up!
Instance 1
15 hours

33. And finish it faster
Instance 1
Instance 2
Instance 3
5 hours
Use the same number of compute hours, but how many EC2
instances you run decides how quickly the work gets done.

34. Minimize lost work when there is a failure
Long jobs
Short jobs
Failure

35. Example: how to break up a text file
• Take a text file with 10 million lines
• Divide number of lines by 10000
• Generate a SQS message for each
chunk of 10000 lines (1000
messages)
• Have your workers start reading at
the specified point in the file
• Deliver output to S3

36. Impact
• Spot interruptions are relatively rare for the instance types we use,
so the fallback is only triggered a 1-2 times per month.
• We are running on discounted Spot Instances more than 98% of the
time.
• On our maps service alone, this has resulted in an 90% savings on
our EC2 costs each month.

37. $$$ Gotchas
Spot enables you to run many more instances that you
normally would, so watch out for any small costs that add
up
Detailed monitoring at 1-minute granularity costs about
$0.005 per hour, but if you do 2 million hours, it adds up to
almost $10K
EBS volumes are still full price on Spot so use instance
store

38. Amazon EC2 Spot – in the wild
1) We make this easy, using
the Spot Bid Advisor.
2) With deliberate pool
selection and bidding, you
keep your Spot Instance as
long as you need to.
3) And with new features like
Spot Fleet diversified, we do
the heavy lifting for you...

39. Spot Bid Advisor – aws-spot-labs

40. Spot Fleet helps you
Launch Thousands of Spot Instances
with one RequestSpotFleet call.
Get Best Price
Find the lowest priced horsepower that works for you.
or
Get Diversified Resources
Diversify your fleet. Grow your availability.
And
Apply Custom Weighting
Create your own capacity unit based on your application
needs

41. It is easy!
aws ec2 request-spot-fleet --spot-fleet-request-config file://config.json {
"IamFleetRole": "arn:aws:iam::781603563322:role/fleet-role", "TargetCapacity":
"100", "SpotPrice": "0.03", "ValidFrom": "2015-09-15T00:56:19Z", "ValidUntil":
"2016-09-14T07:00:00Z", "TerminateInstancesWithExpiration": true,
"LaunchSpecifications": [ { "ImageId": "ami-0d4cfd66", "InstanceType":
"c3.large", "WeightedCapacity": 2, "SubnetId": "subnet-d0dc51fb" }, { "ImageId":
"ami-0d4cfd66", "InstanceType": "c3.large", "WeightedCapacity": 2, "SubnetId":
"subnet-64531413" }, { "ImageId": "ami-0d4cfd66", "InstanceType": "c3.large",
"WeightedCapacity": 2, "SubnetId": "subnet-0b1b8052" }, { "ImageId": "ami-
0d4cfd66", "InstanceType": "c3.xlarge", "WeightedCapacity": 4, "SubnetId":
"subnet-d0dc51fb" }, { "ImageId": "ami-0d4cfd66", "InstanceType": "c3.xlarge",
"WeightedCapacity": 4, "SubnetId": "subnet-64531413" }, { "ImageId": "ami-
0d4cfd66", "InstanceType": "c3.xlarge", "WeightedCapacity": 4, "SubnetId":
"subnet-0b1b8052" }, { "ImageId": "ami-0d4cfd66", "InstanceType": "c3.4xlarge",
"WeightedCapacity": 16, "SubnetId": "subnet-d0dc51fb" }, { "ImageId": "ami-
0d4cfd66", "InstanceType": "c3.4xlarge", "WeightedCapacity": 16, "SubnetId":
"subnet-64531413" }, { "ImageId": "ami-0d4cfd66", "InstanceType": "c3.4xlarge",
"WeightedCapacity": 16, "SubnetId": "subnet-0b1b8052" }, { "ImageId": "ami-
0d4cfd66", "InstanceType": "c3.8xlarge", "WeightedCapacity": 32, "SubnetId":
"subnet-d0dc51fb" }, { "ImageId": "ami-0d4cfd66", "InstanceType": "c3.8xlarge",
"WeightedCapacity": 32, "SubnetId": "subnet-64531413" }, { "ImageId": "ami-
0d4cfd66", "InstanceType": "c3.8xlarge", "WeightedCapacity": 32, "SubnetId":
"subnet-0b1b8052" }, { "ImageId": "ami-0d4cfd66", "InstanceType": "c3.2xlarge",
"WeightedCapacity": 8, "SubnetId": "subnet-d0dc51fb" }, { "ImageId": "ami-
0d4cfd66", "InstanceType": "c3.2xlarge", "WeightedCapacity": 8, "SubnetId":
"subnet-64531413" }, { "ImageId": "ami-0d4cfd66", "InstanceType": "c3.2xlarge",
"WeightedCapacity": 8, "SubnetId": "subnet-0b1b8052" } ] }

42. An easy to use interface that
lets you launch spare EC2
instances in seconds
Helps you select and bid on the
EC2 instances that meet your
applications requirements
Simple to use dashboard lets
you modify and manage your
application’s compute capacity
EC2 Spot Console

43. Spot Fleet – Plays by your rules

44. Spot Fleet – Your preferences

45. Diversification with EC2 Spot Fleet
Multiple EC2 Spot instances
selected
Multiple Availability Zones
selected
Pick the instances with similar
performance characteristics, e.g.
c3.large, m3.large, m4.large,
r3.large, c4.large

46. One connection, countless possibilities

48. 43 Million
9.5 Million
360+
1 Billion
80 Million
Applets created
Users on the platform
Services launched
Runs per month
Service activations

49. IFTTT Infrastructure 2014
• Ruby on Rails
• Dedicated web instances
• Dedicated worker instances

50. IFTTT Infrastructure 2014
• Experimenting with Spot bidding for workers
• Spot Auto Scaling groups with fixed bid prices
• Started to depend on Spot capacity
• ~50% savings over On-Demand
• Approach vulnerable to market fluctuations

51. Enter Spot Fleet

52. IFTTT on Spot Fleet
• Launch worker AMI via Spot Fleet into a mix of markets
• AMI configured to scale based on instance type
• Moved all workers over to Spot Fleet management

53. Good, but not great...

55. IFTTT Infrastructure Now
• All applications in Docker containers
• Containers scheduled by Mesos and Marathon
• Heterogenous (web containers alongside worker, etc.)

56. IFTTT Infrastructure Today
• Reserved instances for Zookeeper & leaders
• Cluster managed entirely by Spot Fleet
• Initially in 48 different Availability Zones
• Mesos instance AMI auto-boots and connects to cluster
• Mesos nodes advertise available resources

57. IFTTT Instance Weights
• Looked at memory and CPU usage across cluster
• Calculated max containers per instance type
• Set instance weights to relative calculations
• Allocation metrics reported to CloudWatch

58. IFTTT Results
• Estimated cost of 75% less than On-Demand
• Bin-packing adds extra savings over previous system
• Spot Fleet manages capacity automatically
• Infrastructure abstraction frees up developers

59. Thank you!

60. Remember to complete
your evaluations!