Running and managing large-scale applications with microservice architecture is hard and often requires operating complex container management infrastructure. Amazon EC2 Container Service (Amazon ECS) is a highly scalable, high-performance container management service that supports Docker containers and makes it easy to run applications on a managed cluster of Amazon EC2 instances. In this session, we walk through a number of patterns used by our customers to run their microservice platforms. We dive deep into some of the challenges of running microservices – such as load balancing, service discovery, and secrets management – and see how Amazon ECS can help address them.

1. © 2017, Amazon Web Services, Inc. or its Affiliates, All rights reserved.
Microservices on AWS
Steve Sofian, Solutions Architect WWPS

2. What to Expect from the Session
© 2017, Amazon Web Services, Inc. or its Affiliates, All rights reserved.
• Review microservices architecture and how it differs
from monolithic and service-oriented architectures
• Examine the challenges in running microservices at
scale
• Demonstrate how to run microservices on AWS

3. © 2017, Amazon Web Services, Inc. or its Affiliates, All rights reserved.
What are Microservices?
“A software architecture style in which complex
applications are composed of small, independent
processes communicating with each other using
language-agnostic APIs. These services are small, highly
decoupled and focus on doing a small task, facilitating a
modular approach to system-building.” - Wikipedia

4. © 2017, Amazon Web Services, Inc. or its Affiliates, All rights reserved.
The Monolithic Architecture

5. © 2017, Amazon Web Services, Inc. or its Affiliates, All rights reserved.
The Service-Oriented Architecture
Presentation Tier Logic Tier
Data Tier

6. © 2017, Amazon Web Services, Inc. or its Affiliates, All rights reserved.
The Microservices Architecture

7. What are containers?
© 2017, Amazon Web Services, Inc. or its Affiliates, All rights reserved.

8. Docker Datacenter on AWS
© 2017, Amazon Web Services, Inc. or its Affiliates, All rights reserved.
Amazon EC2 instances Amazon EC2 instances

9. © 2017, Amazon Web Services, Inc. or its Affiliates, All rights reserved.
Challenges in running
Microservices

10. © 2017, Amazon Web Services, Inc. or its Affiliates, All rights reserved.
Microservice Challenge #1 – Resource Management
Managing a large fleet by hand is impossible:
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
AZ 1 AZ 2
AZ 3

11. © 2017, Amazon Web Services, Inc. or its Affiliates, All rights reserved.
Microservices Challenge #2 – Monitoring
A microservices architecture will have 10s, 100s, 1000s,
maybe even 10,000s of individual services:
• How do you know if an individual service is healthy?
• How do you measure the performance of an individual
service?
• How do you troubleshoot and debug an individual
service?

12. © 2017, Amazon Web Services, Inc. or its Affiliates, All rights reserved.
Microservices Challenge #3: Service Discovery
Each microservice scales up and down independently of
one another:
• How does Service A know the URLs for all instances of
Service B?
• How do you allow services to scale independently while
still using load balancers?
• How does a new instance of a service announce itself to
other services?

13. © 2017, Amazon Web Services, Inc. or its Affiliates, All rights reserved.
Microservices Challenge #4: Deployment
A microservices architecture will have 10s, 100s, 1000s,
maybe even 10,000s of individual services:
• Each service will be developed, tested, and deployed on
its own timeline – How do you manage this across large
numbers of services?
• Services are polyglot – different languages, frameworks
– how do you efficiently deploy them?
• How do you decide which hosts to deploy a service on?

14. © 2017, Amazon Web Services, Inc. or its Affiliates, All rights reserved.
Introducing Amazon ECS
• Fully managed elastic service – You don’t need
to run anything, and the service scales as your
microservices architecture grows
• Shared state optimistic scheduling
• Fully ACID compliant resource and state
management
• Integration with CloudWatch service for
monitoring and logging
• Integration with Code* services for continuous
integration and delivery (CI/CD)

15. © 2017, Amazon Web Services, Inc. or its Affiliates, All rights reserved.
Amazon ECS Architecture

16. © 2017, Amazon Web Services, Inc. or its Affiliates, All rights reserved.
Amazon ECS Architecture
Amazon VPC Container
Amazon ECS

17. © 2017, Amazon Web Services, Inc. or its Affiliates, All rights reserved.
Example Microservice Architecture on ECS
Amazon ECS Amazon ECR
Amazon API Gateway
Amazon RDS
DB Instance

18. © 2017, Amazon Web Services, Inc. or its Affiliates, All rights reserved.
Monitoring with Amazon CloudWatch
Metric data sent to CloudWatch in 1-minute periods and
recorded for a period of two weeks
Available metrics: CPUReservation, MemoryReservation,
CPUUtilization, MemoryUtilization
Available dimensions: ClusterName, ServiceName

19. © 2017, Amazon Web Services, Inc. or its Affiliates, All rights reserved.
Monitoring with Amazon CloudWatch

20. © 2017, Amazon Web Services, Inc. or its Affiliates, All rights reserved.
Service
Discovery
with Route
53 and
Application
Load
Balancers

21. © 2017, Amazon Web Services, Inc. or its Affiliates, All rights reserved.
Deploying Containers on Amazon ECS – Choose a
Scheduler
Batch Jobs
ECS task scheduler
Run tasks once
Batch jobs
RunTask (random)
StartTask (placed)
Long-Running Apps
ECS service scheduler
Health management
Scale-up and scale-down
AZ aware
Grouped containers

22. © 2017, Amazon Web Services, Inc. or its Affiliates, All rights reserved.
Continuous Delivery to Amazon ECS with Jenkins
4. Push image to
Docker registry
2. Build image
from sources
3. Run test on image
1. Code push
triggers build
5. Update service
6. Pull image

23. © 2017, Amazon Web Services, Inc. or its Affiliates, All rights reserved.
Continuous Delivery to Amazon ECS with Jenkins
Easy Deployment
Developers – Merge into master, done!
Jenkins Build Steps
Trigger via webhooks, monitoring, Lambda
Build Docker image via build and publish plugin
Push Docker image into registry
Register updated job with ECS API

24. © 2017, Amazon Web Services, Inc. or its Affiliates, All rights reserved.
But….
many of these tools and
innovations are still coupled to
a shared dependency…

25. © 2017, Amazon Web Services, Inc. or its Affiliates, All rights reserved.
Servers (AAHHHHHHHHH!!)
What size servers are right for
my budget?
How many users create too
much load for my servers?
How much remaining capacity
do my servers have?
How can I detect if a server has
been compromised?
How many servers
should I budget for?
Which OS should my
servers run?
Which users should have
access to my servers?
How can I control access from
my servers?
How will I keep my server
OS patched?
How will new code be
deployed to my servers?
How can I increase
utilization of my servers?
When should I decide to
scale out my servers?
What size server is
right for my performance?
Should I tune OS settings
to optimize my application?
Which packages should be
baked into my server images?
When should I decide to
scale up my servers?
How should I handle server
configuration changes?
How will the application handle
server hardware failure?

26. © 2017, Amazon Web Services, Inc. or its Affiliates, All rights reserved.
Architect to be Serverless
Fully managed
• No provisioning
• Zero administration
• High availability
Developer productivity
• Focus on the code that
matters
• Innovate rapidly
• Reduce time to market
Continuous scaling
• Automatically
• Scale up and scale down

27. © 2017, Amazon Web Services, Inc. or its Affiliates, All rights reserved.
AWS Lambda

28. Serverless, event-driven compute service
Lambda = microservice without servers

29. © 2017, Amazon Web Services, Inc. or its Affiliates, All rights reserved.
Components of Lambda
• A Lambda function (that you write)
• An event source
• The AWS Lambda service
• The function networking environment

30. © 2017, Amazon Web Services, Inc. or its Affiliates, All rights reserved.
Lambda Function
• Your code
(Java, NodeJS, Python, C#)
• The IAM role that code
assumes during execution
• The amount of memory
allocated to your code
(affects CPU and network
as well)
A valid, complete
Lambda function

31. © 2017, Amazon Web Services, Inc. or its Affiliates, All rights reserved.
Event Sources
• When should your function
execute?
• Many AWS services can be
an event source today:
• Amazon S3
• Amazon Kinesis
• Amazon SNS
• Amazon DynamoDB
• Amazon CloudWatch
• AWS Config Rules
• Amazon Echo
• Etc.
• …and Amazon API
Gateway (more later)

32. © 2017, Amazon Web Services, Inc. or its Affiliates, All rights reserved.
AWS Lambda
• Runs your function code without you managing or
scaling servers.
• Provides an API to trigger the execution of your function.
• Ensures function is executed when triggered, in parallel,
regardless of scale.
• Provides additional capabilities for your function
(logging, monitoring).

33. © 2017, Amazon Web Services, Inc. or its Affiliates, All rights reserved.
“But…
in order to utilize Lambda, do I really
need to architect event-driven
applications?” – you (maybe)

34. © 2017, Amazon Web Services, Inc. or its Affiliates, All rights reserved.
SOA still works.

35. © 2017, Amazon Web Services, Inc. or its Affiliates, All rights reserved.
Amazon API Gateway

36. A Fully Managed Service for Your APIs
Create Configure Publish
Maintain Monitor Secure
© 2017, Amazon Web Services, Inc. or its Affiliates, All rights reserved.

37. Serverless Image Handler
© 2017, Amazon Web Services, Inc. or its Affiliates, All rights reserved.

38. © 2017, Amazon Web Services, Inc. or its Affiliates, All rights reserved.
Demo

39. © 2017, Amazon Web Services, Inc. or its Affiliates, All rights reserved.
Microservices Architecture (Serverless)

40. © 2017, Amazon Web Services, Inc. or its Affiliates, All rights reserved.
Go build something!
Amazon API
Gateway
AWS Lambda Amazon
DynamoDB

41. © 2017, Amazon Web Services, Inc. or its Affiliates, All rights reserved.