This webinar by Bohdan Yurov (Senior Solution Architect, Consultant, GlobalLogic, Ukraine, Kharkiv) was delivered at GlobalLogic Ukraine On Air Webinar on June 19, 2020. Effective container orchestration requires good balance between infrastructure cost and operations effort. New AWS features can potentially provide us with better flexibility in container orchestration topology and balance EC2/serverless. Bohdan shared his experience and demo overview of serverless and mixed container orchestration and request routing on AWS using ECS & EKS on FarGate & EC2. More details and presentation: https://www.globallogic.com/ua/about/events/webinar-serverless-and-mixed-container-orchestration/

1. Confidential
AWS EKS & ECS on Fargate & EC2
Basics, comparison and cloud routing
Apr 2020

2. Confidential
Effective container orchestration requires good balance between infrastructure cost and operations
effort. New AWS features can potentially provide us with better flexibility in container orchestration
topology and balance EC2/serverless.
Amazon cloud has shipped a new solution - serverless containers with EKS, that leads to 6 different
scheduling approaches:
• EKS on Fargate
• EKS on EC2
• EKS on mixed Fargate/EC2
• ECS on Fargate
• ECS on EC2
• ECS on mixed Fargate/EC2
Business problem

3. Confidential
Making choice can be difficult when you have multiple intersecting options, so we need to evaluate
all scenarios to make a list of pros/cons.
Choosing specific approach also leads to the routing problem: ingress traffic is handled differently in
EKS/ECS and Fargate also introduces additional limitations. In the scope of this PoC we need to find
and compare possible routing scenarios having these requirements:
• Path-based routing
• DNS-based routing
• Header(content)-based routing
Final requirement is having everything fully automated (infrastructure-as-code) to be able to quickly
replicate PoC deployment.
Business problem

4. Confidential
To evaluate potential benefits, we will build the following setups and will deploy sample application:
1. EKS on Fargate
2. EKS on EC2
3. ECS on Fargate
4. ECS on EC2
While there can also be mixed heterogeneous configs, we assume that they should be fully covered
by "clean" EC2/Fargate solutions.
Abstract

5. Confidential
The following services are going to be used for routing:
• ALB for DNS- and header-based routing: ECS on EC2, ECS on Fargate
• ELB with nginx ingress for DNS- and header-based routing: EKS on EC2
• ALB with ALB ingress controller DNS- and header-based routing: EKS on Fargate
• API-Gateway for DNS- and path-based routing: all four scenarios
Routing

6. Confidential
These are current PoC goals:
• Evaluate EKS and Fargate integration
• Compare EKS with ECS on Fargate
• Compare Fargate and EC2 for ECS and EKS
• Implement three routing schemes:
- Path-based routing
- DNS-based routing
- Header(content)-based routing
• Create automated deployment pipeline (for infrastructure and demo application)
• Deploy resource group to find and managed all tagged resources
Goals

7. Confidential
We are going to use multiple AWS services as well as complex technologies and tooling.
AWS services:
Fargate, ECS, EKS, EC2, Route53, ECR, API Gateway, VPC, IAM, DynamoDB
Technologies:
Kubernetes, Nginx, Terraform 0.12.x, Ansible, Helm 2, Docker, golang
CLI tools:
eksctl, awscli, kubectl, make
Technologies

8. Confidential
Terminology Description
Cluster A combination of Nodes.
Service The unit of deployment used to make a container available for consumption.
Container The runtime of a Docker image.
EKS Amazon Elastic Kubernetes Service.
ECS Amazon Elastic Container Service.
Task Definition This is a declarative configuration of a task and container in ECS.
Task The smallest unit of deployment in ECS.
Pod The smallest unit of deployment in Kubernetes which runs one or more containers.
Serverless technology It is a way to build and run applications without having to manage infrastructure.
Ingress Controller This is a Kubernetes abstraction that allows simple host- or URL-based HTTP routing, etc.

9. Confidential
Amazon ECS
Amazon ECS is a fully managed container orchestration service which developed and support by AWS.
This managed service supports Docker and enables us to run and manage Docker containers. We have the
ability to launch multiple containers in a batch.
In case of failure, ECS auto-recover failed containers and they will be re-launched automatically.
Amazon ECS allows you to define tasks through a declarative JSON template called a Task Definition. Within
a Task Definition, we can specify one or more containers that are required for our task, including the Docker
repository and image, memory and CPU requirements, shared data volumes, and how the containers are
linked to each other.
The containers can be easily updated to new versions. Once we uploaded a new version of our application
task definition, Amazon ECS scheduler automatically starts new containers using the updated image and
stop containers running the previous version. Amazon ECS automatically registers and deregisters containers
from the associated Application Load Balancer.
With Amazon ECS you pay only for underlying resources provisioned for the application purposes.

10. Confidential
ECS service structure example

11. Confidential
Amazon EKS
Amazon Elastic Kubernetes Service (Amazon EKS) is a fully managed Kubernetes service.
The Kubernetes API servers and the etcd persistence layer is managed by AWS. The availability and
scalability for each cluster spread across multiple AWS availability zones.
We can use the IAM roles for control access to other containerized services, AWS resources external to the
cluster such as databases and secrets, or third party services and applications running outside of AWS. This
gives us fine-grained, pod level access control when running clusters with multiple co-located services.
Amazon EKS supports using Elastic Load Balancing including Application Load Balancer (ALB), Network
Load Balancer (NLB), and Classic Load Balancer. We can run standard Kubernetes cluster load balancing or
any Kubernetes supported ingress controller.
You pay $0.10 per hour for each Amazon EKS cluster. It is possible to use a single Amazon EKS cluster to
run multiple applications by taking advantage of Kubernetes namespaces and IAM security policies. Also,
you pay for underlying resources provisioned for the application purposes.

12. Confidential
Kubernetes architecture diagram

13. Confidential
Amazon Fargate
AWS Fargate is a serverless compute engine for containers that works with both Amazon Elastic Container
Service (ECS) and Amazon Elastic Kubernetes Service (EKS). Fargate removes the need to provision and
manage servers.
AWS Fargate pricing is calculated based on the vCPU and memory resources used from the time we start to
download our container image until the Amazon ECS Task or Amazon EKS Pod terminates, rounded up to the
nearest second.

14. Confidential
Amazon Fargate & EKS
Pricing:
Resources the pod requests in addition to a small amount of memory needed to run Kubernetes components
alongside the pod. Pods running on Fargate follow the existing pricing model. vCPU and memory resources are
calculated from the time your pod’s container images are pulled until the pod terminates, rounded up to the
nearest second. A minimum charge for 1 minute applies. Additionally, you pay the standard cost for each EKS
cluster you run, $0.10 per hour.
Limitations:
• There is a maximum of 4 vCPU and 30Gb memory per pod.
• Currently there is no support for stateful workloads that require persistent volumes or file systems.
• On EKS you cannot run Daemonsets, Privileged pods, or pods that use HostNetwork or HostPort.
• The only load balancer you can use is an Application Load Balancer.
• No GPU pod configurations available.

15. Confidential
EC2 instances vs Amazon Fargate

16. Confidential
Amazon EKS vs Amazon ECS
Items Amazon EKS Amazon ECS
Technology Open source Kubernetes + EKS master AWS-native
Container type Kubernetes pod ECS task
Price ~ 72$ per month for each cluster * Free *
Compatibility Supports multi-cloud deployments Amazon-specific
Configuration description type YAML JSON/YAML
Complexity High Low
Extension support Kubernetes compatible AWS-native
Service expose type AWS Load Balancers, Ingress AWS ALB
* you pay for underlying resources provisioned for the application purposes like EC2-instances or AWS Fargate.

17. Confidential
What is best for you, flow for choice
Amazon
ECS
● When you’re looking for a solution that combines simplicity and availability, and you want to have
advanced control over your infrastructure, then ECS is the right choice for you.
You can easily create highly available and scalable applications.
● The learning curve in ECS is much lower. Organizations with limited DevOps resources, or that are
not prepared to re-architect applications around concepts like Pods, may find ECS easier to adopt.
Amazon
EKS
● If you already have your containers running on Kubernetes or want an advanced orchestration
solution with more compatibility, you should use Amazon EKS.
● Amazon EKS is a good option if you don’t want to manage the control plane.
With Amazon EKS you still have to manage the agent nodes.
● Since Amazon EKS is a Kubernetes-as-a-service offering for AWS, it can be run on any
infrastructure, meaning that it’s much easier to run on-premises or with a different service provider.

18. Confidential
Items AWS EC2 with autoscaling AWS Fargate *
Serverless No Yes
Operations effort level High Low
Price m4.large - 8GB RAM/2 CPU ~ $72/month 8GB RAM/2 CPU ~ $84/month
Complexity Medium Low
Availability Medium High
Scalability Medium High
Comparison AWS EC2 instances and AWS Fargate
* - In case your application needs something simple without the need to manage local persistent storage and you don’t want
to worry about scalability and availability, we suggest choosing AWS Fargate. Otherwise, EC2 instances or hybrid
configuration with EC2 and Fargate might be the best option.

19. Confidential
Demo overview
With EC2 worker nodes With Fargate
EKS Ingress type: Nginx
Load balancer: AWS Classic
Single LB provisioned by Kubernetes
Routing: host-based and content-based
Nginx Ingress could be replaced with complex Istio routing
Ingress type: Amazon ALB
Load balancer: Amazon ALB
Multiple LBs provisioned by ALB ingress
Routing: host-based and content-based
Fargate does not support multiple features including Classic LB
and extended security features required for Nginx ingress
ECS Load balancer: Amazon ALB
Multiple LBs provisioned by terraform
Routing: content-based
Load balancer: Amazon ALB
Multiple LBs provisioned by terraform
Routing: content-based
For demo simplicity we focus on the most straightforward solutions.
Hybrid EC2+Fargate configurations are skipped

20. Confidential
Amazon ECS and ALB
Task definition Servicetask
ALB
ECS cluster
tasktask
example.com

21. Confidential
Amazon EKS with Load Balancer.

22. Confidential
Advanced routing with Amazon ALB
An Application Load Balancer functions at the
application layer, the seventh layer of the Open
Systems Interconnection (OSI) model. After the
load balancer receives a request, it evaluates the
listener rules in priority order to determine which
rule to apply, and then selects a target from the
target group for the rule action.

23. Confidential
k8s nginx ingress controller
Kubernetes supports a high-level abstraction called Ingress, which allows simple host- or URL-based HTTP
routing. Ingress exposes HTTP and HTTPS routes from outside the cluster to services within the cluster.
Traffic routing is controlled by rules defined on the Ingress resource.

24. Confidential
Kubernetes path-based routing with ingress controller
Once ingress resource is created, the
ingress controller will watch changes(1)
and do the next steps:
● Create an Amazon ALB and
Listeners(80/443) if they do not exist
yet. (2,4)
● Create a target group on the ALB for
each K8S service. Add only the VMs
where the backend pods are running
(instead of all VMs) to the target
group. (3)
● Update Path and Host ingress configs
on each target group. (5)

25. Confidential
Advanced routing with Istio
Istio is a service mesh for Kubernetes for
advanced traffic routing, network monitoring and,
security features.
In the example shown, you can see
content-based routing based on a custom
end-user header added to the request. All of the
traffic is sent to the v1 version of each of the
services, while "jason"'s requests are routed to
version v2 of the reviews service.

26. Confidential
Advanced routing with Amazon API-gateway
Amazon API Gateway is a fully
managed service that makes it easy to
create, publish, maintain, monitor, and
secure APIs.
API Gateway supports containerized
and serverless workloads, as well as
web applications.
You pay for the API calls you receive
and the amount of data transferred out.