The fourth in our series of webinars, 'Journey Through the AWS Cloud'. This complimentary presentation discusses the use of services offered by AWS that alleviate the need for you to install and manage software on EC2 instances. We introduce the key services customers employ to keep them focused on developing their applications, whilst AWS takes care of running the scalable and reliable building blocks upon which they are built.
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
Journey Through the AWS Cloud; Application Services
1. Journey through the Cloud:
Application Services
Ryan Shuttleworth – Technical Evangelist
@ryanAWS
2. Journey through the cloud
Common use cases & stepping stones into the AWS cloud
Learning from customer journeys
Best practices to bootstrap your projects
3. Application Services
Build upon services built for the cloud
Address common pain points in application architectures
Reduce operational management of software components
Focus on application function, not undifferentiated heavy lifting
4. Agenda
Why AWS for application services
Services overview
Dive into select services
Where to go next
9. Application Services
Scalability Reliability Availability
Do you focus on Does core Is middleware
scaling frameworks component ‘glue’ difficult to
rather than reliability affect make as
optimizing your your application available as you
code? track record? need it?
11. Application Services
Operational Management
Do you spend more time managing
application services than you do
building and managing assets core
to your business?
12. 30% 70%
On-Premise Your Managing All of the
Infrastructure Business “Undifferentiated Heavy Lifting”
13. 30% 70%
On-Premise Your Managing All of the
Infrastructure Business “Undifferentiated Heavy Lifting”
AWS
More Time to Focus on Configuring Your
Cloud-Based
Your Business Cloud Assets
Infrastructure
70% 30%
14. Infrastructure & Compute
Scaling
Application Services Security
CDN Backup
DNS Database
Building blocks for applications
Storage Load Balancing
Designed and built for the cloud
Workflow Monitoring
Available at end of a web service
call Networking
Messaging
15. Storage & Archive
AWS is used in a variety of ways…
By managing time-consuming Brought on massive DynamoDB
database administration tasks, capacity in an operationally short
RDS allows SEGA to focus on period of time to handle demand
business critical applications peaks during SuperBowl
Saved months of development &
Relies on Simple Workflow to
architecture time and focused on
orchestrate complex, heterogeneous
application development instead
scientific workflows
by using Cloud Search
17. Business & technical drivers
You might be able to:
Reduce costs
Running software on EC2 can be
more expensive than consuming
functionality as a service
18. Business & technical drivers
You might be able to:
Reduce costs Improve reliability
Running software on EC2 can be Services built with inherent multi-
more expensive than consuming AZ functionality improve
functionality as a service application reliability
19. Business & technical drivers
You might be able to:
Reduce costs Improve reliability
Running software on EC2 can be Services built with inherent multi-
more expensive than consuming AZ functionality improve
functionality as a service application reliability
Scale easily
Services scale as you grow and
need them, without large
investments in infrastructure
20. Business & technical drivers
You might be able to:
Reduce costs Improve reliability
Running software on EC2 can be Services built with inherent multi-
more expensive than consuming AZ functionality improve
functionality as a service application reliability
Scale easily Re-focus energies
Services scale as you grow and Spend less time doing
need them, without large undifferentiated heavy lifting and
investments in infrastructure more time on your business
21. Q
Instance
Availability Zone
Decoupled Reliable
applications on message
EC2 instances queue on
instance
Region
22. Q Q Q
Instance Instance Instance
Availability Zone Availability Zone Availability Zone
Multiple
Technical instances
implementation for
reliability
Region
23. Q Q Q
Instance Instance Instance
Availability Zone Availability Zone Availability Zone
Remove
software
Region
running on
EC2
24. Simplified
operations
Availability Zone Availability Zone Availability Zone
Reduced costs
Amazon Simple
Queue Service (SQS)
Region
Replace with a regional
service
25. Queuing on AWS
Setup & manage instances
Install & configure queuing
middleware
Set up persistent message store
Implement clustering across AZs
for HA
Implement queue monitoring
systems
Implement queuing in
applications
26. Queuing on AWS Amazon SQS
Setup & manage instances Create a queue
Install & configure queuing HTTP PUT to place messages
middleware HTTP GET (long polling) to pull
Set up persistent message store messages
Implement clustering across AZs
for HA
Implement queue monitoring
systems
Implement queuing in
applications
31. Source: Forrester
Security planning
License training
Backup, recovery
load and unload Script automation
Performance Installation, upgrade,
and tuning patching, migration
32. Migration
Backup and
Schema design recovery
Patching Query construction
Configuration
Software upgrades
Frequent server upgrades Storage upgrades
Query optimization Hardware crash
33. Migration Focus on
Backup and
these
Schema design recovery
things
Patching Query construction
Configuration
Software upgrades
Frequent server upgrades Storage upgrades
Query optimization Hardware crash
34. Migration
Backup and
Schema design recovery
Patching Instead of
Query construction
these
Configuration
Software upgrades
Frequent server upgrades Storage upgrades
Query optimization Hardware crash
35. Near zero administration
Painless patching, automatic upgrades
Cloudwatch monitoring, metric alarms
One click. High Availability.
36.
37. One click.
High availability with Multi-AZ
Automated deployment across multiple AZs
Synchronous replication from master to replica
Automatic fail-over; replica promoted to master
Test fail-over
38. Push-button scale, high performance
Scale storage from 5Gb to 1Tb of storage
Scale instance from small to 4XL (better I/O)
Add Read Replicas with asynchronous replication
Add ElastiCache for performance
39. Relational Database Service
Database-as-a-Service
No need to install or manage database instances
Scalable and fault tolerant configurations
Feature Details
Platform support Create MySQL, SQL Server and Oracle RDBMS
Preconfigured Get started instantly with sensible default settings
Automated patching Keep your database platform up to date automatically
Backups Automatic backups and point in time recovery and full
DB backups
Backups Volumes can be snapshotted for point in time restore
Failover Automated failover to slave hosts in event of a failure
Replication Easily create read-replicas of your data and seamlessly
replicate data across availability zones
40. RDBMS on AWS
Setup & manage instances
Install & configure database
platform
Configure backups
Implement master-slave for HA
Implement read-replicas for
performance
Manage maintenance updates
41. RDBMS on AWS RDS
Setup & manage instances Create RDS instance
Install & configure database Select Multi-AZ
platform Choose backup period
Configure backups Choose maintenance windows
Implement master-slave for HA
Implement read-replicas for
performance
Manage maintenance updates
42. Amazon Relational Database Service (Amazon RDS) databases
stores forum threads, site content, and project configuration
data.
High availability Multi-AZ database deployment to handle live
game metadata and user-generated content.
Enterprise-grade fault tolerance for protecting customer data.
By managing time-consuming database administration tasks,
Amazon RDS allows SEGA to focus on business critical
applications.
49. Low provisioned throughput
Replica Table Replica
Partition Partition Partition
SSD SSD SSD
Availability Zone Availability Zone Availability Zone
Region
Illustrative diagram only
51. Increased provisioned throughput
Table Table Table Table Table
Partition Partition Partition Partition Partition
SS SS SS SS SS
D D D D D
Table Table Table Table Table
Partition Partition Partition Partition Partition
SS SS SS SS SS
D D D D D
Region
Illustrative diagram only
53. DynamoDB
Feature Details
Provisioned throughput NoSQL
Provisioned Dial up or down provisioned read/write
database throughput capacity
Fast, predictable performance Predictable Average single digit millisecond latencies
Fully distributed, fault tolerant performance from SSD backed infrastructure
architecture
Strong consistency Be sure you are reading the most up to
date values
Fault tolerant Data replicated across availability zones
Monitoring Integrated to Cloud Watch
Secure Integrates with AWS Identity and Access
Management (IAM)
Elastic Integrates with Elastic MapReduce for
MapReduce complex analytics on large datasets
57. “AWS gave us the flexibility to bring a massive
amount of capacity online in a short period of
DynamoDB: time and allowed us to do so in an operationally
over 500,000 writes per straightforward way.
second
AWS is now Shazam’s cloud provider of choice,”
Amazon EMR:
more than 1 million writes Jason Titus,
per second CTO
59. Simple Workflow
1
2
Task A
3
Task B
(Auto-scaling)
Flow framework for simplification of
cross system task coordination
Task C
Long running transaction state and task
distribution
61. A typical
business
workflow…
System A System C System System E
Multiple steps Update Update €
D
Ship Send
Multiple decision Inventory $ or €? account order Email
points
Heterogeneous New Order
systems
Update $
account
System B
62. A typical
business Process logic Middleware
workflow…
State managed in System A System C System
D
System E
end systems Update Update € Ship Send
Inventory $ or €? account order Email
Process logic
embedded in
New Order
applications
Complex queuing,
message ordering,
de-duplication and
dependencies
State
Update $
Scaling and failover account
of tasks troublesome System B
63. Implemented in
Workflow metadata & state
Simple
Workflow… Decider
Implement a
‘Decider’ with simple, Update Update € Ship Send
Inventory $ or €? account
linear decision logic order Email
Processes/tasks in a
New Order
workflow become
‘Workers’
All state & metadata
is handled by highly Worker
available and durable
Update $
AWS ‘Workflow’
account
66. Decider
Get Decision Task
Return Result
Amazon Simple
Workflow Service
(SWF)
Get Activity Task
Return Result
Worker
67. Workflows on AWS
Setup & manage instances
Install & configure workflow
middleware
Architect for high availability
Implement workflows in
proprietary/complicated
languages
Implement process audit
68. Workflows on AWS Simple Workflow
Setup & manage instances Implement ‘decider’ in language
Install & configure workflow of choice
middleware Implement task ‘workers’ to
Architect for high availability consume work
Implement workflows in
proprietary/complicated
languages
Implement process audit
94. Data
Document quantity & size
Cloud
Search
Search Instance Search Instance Search Instance
Partition 1 Partition 2 Partition n
Copy 1 Copy 1 Copy 1
Traffic Search Instance Search Instance Search Instance
Partition 1 Partition 2 Partition n
Request Copy 2 Copy 2 Copy 2
volume &
complexity
Search Instance Search Instance Search Instance
Partition 1 Partition 2 Partition n
Copy n Copy n Copy n
95. Search on AWS
Perform capacity planning on size
of indexes
Setup & manage required
number of instances
Install & configure search
software such as Solr across
cluster
Manage cluster partitioning and
size over time
Implement monitoring
96. Search on AWS Cloud Search
Perform capacity planning on size Create search domain in console
of indexes Upload documents
Setup & manage required Retrieve results
number of instances
Install & configure search
software such as Solr across
cluster
Manage cluster partitioning and
size over time
Implement monitoring
103. Elastic MapReduce
Feature Details
Scalable Use as many or as few compute
instances running Hadoop as you
want. Modify the number of
instances while your job flow is
running
Integrated with Works seamlessly with S3 as origin
other services and output. Integrates with
DynamoDB
Managed, elastic Hadoop cluster
Comprehensive Supports languages such as Hive and
Pig for defining analytics, and allows Integrates with S3 & DynamoDB
complex definitions in Cascading, Leverage Hive & Pig analytics scripts
Java, Ruby, Perl, Python, PHP, R, or
Integrates with instance types such as spot
C++
Cost effective Works with Spot instance types
Monitoring Monitor job flows from with the
management console
107. Lots of
instances of ‘X’
Very large
dataset
(e.g TBs)
108. Lots of
instances of ‘X’
Very large
dataset
(e.g TBs) Split the
log into
many small
pieces
109. Process in an
Lots of EMR cluster
instances of ‘X’
Very large
dataset
(e.g TBs) Split the
log into
many small
pieces
110. Process in an
Lots of EMR cluster
instances of ‘X’
Very large
dataset
(e.g TBs) Split the Aggregate
log into the results
many small from all
pieces the nodes
111. Process in an
Lots of EMR cluster
instances of ‘X’
Very large
dataset Aggregate
(e.g TBs) Split the Aggregate view of ‘X’
log into the results
many small from all
pieces the nodes
112. Very large
dataset Aggregate
(e.g TBs) Insight in a fraction of the time view of ‘X’
127. Features powered by Amazon Elastic
MapReduce:
People Who Viewed this Also Viewed
Review highlights
Auto complete as you type on search
Search spelling suggestions
Top searches
Ads
200 Elastic MapReduce jobs per day
Processing 3TB of data
128. Elastic MapReduce
"With Amazon Elastic MapReduce, there
was no upfront investment in hardware,
no hardware procurement delay, and no
need to hire additional operations staff.
Because of the flexibility of the platform,
our first new online advertising campaign
experienced a 500% increase in return on
ad spend from a similar campaign a year
before.”
Mark Taylor, Program Director
131. Powerof60.com
$100 of credits
That’s enough to run an
Amazon Elastic Compute
Cloud (Amazon EC2)
Standard Small Instance
for approximately 600
hours
Or ~600 EC2 Standard
Small Instances for 1 hour
133. AWS is more than compute & storage
Databases, middleware and complex frameworks as services
Remove the operational burden from running common software services
Application services gives access scalable, sophisticated software without the overheads