5. Setup
• aws.amazon.com
• Be sure you have permissions and are
active
• Firefox or Chrome
• Windows download Putty: www.putty.org
6. Watch out for unexpected Costs
When the Technical Workshop comes to an end, to avoid
unwanted costs:
• Delete your S3 objects
• Destroy your CloudFront distributions
• Stop or Shut Down your EC2 and RDS instances
The customer is responsible for the resources he’s using.
AWS declines any responsibility if the customer forgets
to shut down resources.
7. Agenda
• House Keeping & Setup Availability
• Introduction to Amazon Web – Snapshots
Services – Load Balancing
• Lab: Building a Web Property – Auto Scaling
on AWS – Security
– Storage: S3, EBS • Log Processing Scenario
– Compute – Logging to AWS
– Content Delivery – Elastic Map Reduce
– Relational Database – Dynamo DB
• Lab: Scalability and
10. About Amazon
How did Amazon…
Web Services
Deep experience in
building and operating
global web scale
?
systems
…get into cloud computing?
11. – Reliable, scalable, low-cost infrastructure
– Every service offers APIs
– You can use only the components you need
– All the important “building blocks” are provided
[ What is Amazon Web Services? ]
12. Region GovCloud Region
[ Global AWS Infrastructure ]
15. Seattle South Bend New York (3) London (2)Amsterdam (2)
Newark Dublin Stockholm
Palo Alto
Tokyo (2)
SanHayward
Jose
Paris
Frankfurt (2)
Ashburn (2)
Madrid Milan
(2)
Los Angeles (2) Jacksonville Osaka
Hong Kong
Dallas (2) (2)
St.Louis
Miami Singapore
(2)
Sao Paulo
Edge Locations Sydney
[ Global AWS Infrastructure ]
16. amazon
web services
[ The Amazon Web Services universe ]
17. Command Line
API
Web Console SDK
[ The Amazon Web Services universe ]
18. Cross
Service
Features
CloudWatch
Simple Notification Service IAM
Simple WorkFlow CloudFormation
Management +
Elastic BeanStalk Interface
[ The Amazon Web Services universe ]
(some services are omitted here)
19. Cross
Service
Features
Platform
Building
blocks
EMR
Redshift
SQS CloudFront Management +
DynamoDB Interface
RDS
SES CloudSearch
[ The Amazon Web Services universe ]
(some services are omitted here)
20. Cross
Service
Features
Platform
Building
blocks
Infrastructure
Building Management +
Blocks Interface
EC2
Virtual Private Cloud S3
Glacier EBS
[ The Amazon Web Services universe ]
(some services are omitted here)
21. Elastic Beanstalk
Cross Simple WorkFlow
Service CloudFormation
CloudWatch
Features IAM Command Line
Web Console
Platform APIs
RDS CloudSearch SDK
Building
SES CloudFront SQS
blocks EMR DynamoDB
Infrastructure
Building Management +
VPC EC2
Blocks EBS S3
Interface
[ The Amazon Web Services universe ]
(some services are omitted here)
22. Agenda
• House Keeping & Setup • Lab: Scalability and
• Introduction to Amazon Web Availability
Services – Snapshots
• Lab: Building a Web Property – Load Balancing
on AWS – Auto Scaling
– Storage: S3, EBS – Security
– Compute • Log Processing Scenario
– Content Delivery – Logging to AWS
– Relational Database – Elastic Map Reduce
– Dynamo DB
23. Labs
During this workshop, we will build from
scratch a highly available, redundant,
scalable web property on AWS.
25. Our Building Blocks
Compute
Amazon Elastic Compute Cloud (EC2)
Amazon EC2 Instance Instances AMI DB on Instance with Elastic IP
Instance CloudWatch
26. Our Building Blocks
Content Delivery & Database
Amazon Cloudfront Amazon Database Services RDS/DDB
Amazon Download Streaming Edge Location Amazon RDS MySQL DB Instance DynamoDB
Cloudfront Distribution Distribution
28. Amazon S3
Simple Storage Service
• Object-based storage • Scalable
(no Filesystem) • Redundancy is managed
• Easily store/retrieve data transparently
• Durability of 99.999999999% or • File (Object): up to 5 TB each
99.99% • HTTP, HTTPS, BitTorrent
• Integrated with other AWS protocols
Services
28
29. Amazon S3 Redundancy
Your Data
Data is replicated Any Amazon S3 Region
multiple times Data
Data
In case of failure,
data is replicated again, Data
transparently
Data
Data
34. Regions, Availability Zones,
Edge Locations
Amsterdam
Seattle South Bend New York (2)
London
Newark Stockholm
Palo Alto Dublin
Tokyo
San Jose
Frankfurt
Paris
Los Angeles (2) Ashburn
Jacksonville Hong Kong
Dallas
St.Louis Singapore Sydney
Miami
Sao Paulo
38 Edge Locations in total (as of Dec 2012)
35. Let’s simplify a bit:
we consider only a few of them
Stockholm
San Jose
Hong Kong
Sao Paulo
Your web servers
in Singapore
36. Content Delivery Network:
How it works
Stockholm
San Jose
Hong Kong
Sao Paulo
Dynamic pages (PHP, Java) (from web servers) Your web servers
Static content or streaming (with CloudFront) in Singapore
37. Amazon CloudFront Use cases
• Accelerated web content delivery
• Off-load traffic from web servers
• Big spikes in traffic
• Event streaming
• Marketing campaigns
37
42. EC2 Use cases
• Media • Internal Applications
• Hosting • Gaming
• High Performance • ... Everything that
Computing needs computing!
• Dev & Test
42
43. Lab Exercise
• Create a new Security Group
• Launch an Amazon EC2 instance (Linux)
• Log in with SSH as ec2-user@
• Install a web server
• Create a simple web page
• Test it on a browser: it works!
• Create and attach an Elastic IP
• Create an AMI from an EC2 Instance
44. Demo Windows
• Create a new Security Group for Windows
• Launch a new EC2 instance (Windows)
• Log in with RDP
From Windows: log in with RDP
46. EBS
Elastic Block Storage
• Block Level Storage for use • Redundant
with EC2 • Persistent
• Volume: 1 GB to 1 TB • Point-in-time snapshots to
• Raw unformatted block Amazon S3
device • Integration with CloudWatch
• Local to an Availability Zone
46
47. Demo EBS
• Create a new EBS volume
• Attach it to an EC2 Instance
• Partition it and create a File System
• Mount it; create a file in it
• Create an EBS Snapshot
• Create a new volume from the Snapshot
49. RDS
Relational Database Service
• Relational Database “as a • Optional: automatic Standby
Service” Replica
• Simple to Deploy • Optional: multiple Read-Only
• Managed by the AWS team copies
• MySQL, SQLServer or • Easy DB Snapshots and
Oracle (as of 2012) automated backup
• Scalable
49
50. Demo RDS
• Create a DB Instance on RDS (MySQL)
• Enable Multi-AZ Deployment
• Enable one Read Replica
• Optional: connect to the DB Instance
54. EBS
Elastic Load Balancer
• Automatically distribute incoming traffic to
multiple Amazon EC2 instances (in the same
Region).
• Automatic Health check
• IPv6 support
• Can be integrated with AutoScaling
56. Snapshots & AMIs
• Copies of EBS Volumes
• Essential to Reusability
• Copy between Regions
• Durability in S3
57. Lab Exercise
• Duplicate your entire architecture by
making an AMI
• Increase your availability by spreading
your application across availability zones
• Bring up an ELB in front of your website
• Optional – Create a CNAME to the ELB
62. AutoScaling
• Auto Up and Auto Down
• Runs on CloudWatch metrics
• Notifications via SNS
• Spot or On-demand
• No additional Fees
63. AutoScaling
• Launch config: AMI to be used
• Autoscaling group: where/how to launch
• Autoscaling policy: what should AS do
• Autoscaling trigger: what will activate AS
66. Security
• Security Groups
• Granular tiered secure architecture
• Roles for services
• Best Practices - Bastions
68. Before we get started on the controls…
• AWS Reports, Certifications & Accreditations
• SOC 1, Type 2 report
• SOC 2 report
• ISO 27001
• PCI DSS Level 1 service provider
• FISMA Moderate
• MPAA
• Look at http://aws.amazon.com/security
73. Agenda
• House Keeping & Setup • Lab: Scalability and
• Introduction to Amazon Web Availability
Services – Snapshots
• Lab: Building a Web Property – Load Balancing
on AWS – Auto Scaling
– Storage: S3, EBS – Security
– Compute • Log Processing Scenario
– Content Delivery – Logging to AWS
– Relational Database – Elastic Map Reduce
– Dynamo DB
74. Getting your Data into S3
Console Upload S3 3rd Party Commercial
Applications
FTP Tsunami UDP
AWS Import / Export
Storage Gateway
Flume
S3 API CloudFront
Direct Connect
AWS Data Pipeline
75. S3 and Big Data
• Why S3?
• Hadoop Overview
• Hadoop on the Cloud
• Hadoop File System
77. Introducing Apache Hadoop
• Apache Hadoop
• Software for distributed data analysis
• Map/Reduce framework
• Focus on data
78. • But
• Complex
• Hard to setup
• Cap-ex intensive
• Difficult to manage
79. What is Amazon Elastic MapReduce (EMR)?
EMR is Hadoop in the Cloud
Hadoop is an open-source framework for
parallel processing huge amounts of data on
a cluster of machines
80. How does it work?
Choose: Hadoop distribution, #
of nodes, types of nodes, custom
configs, Hive/Pig/etc.
Put the data
into S3
S3 EMR Cluster
011001101 EMR
Launch the cluster using
the EMR console, CLI, SDK,
or APIs
Get the output
from S3 You can also store
everything in HDFS
81. DynamoDB and Big Data
• What is Dynamo?
• Dynamo in Big Data – Volume & Velocity
83. Watch out for unexpected Costs
When the Technical Workshop comes to an end, to avoid
unwanted costs:
• Delete your S3 objects
• Destroy your CloudFront distributions
• Stop or Shut Down your EC2 and RDS instances
The customer is responsible for the resources he’s using.
AWS declines any responsibility if the customer forgets
to shut down resources.
84. amazon
web services
Thank You
James Saull
Solution Architect
jsaull@amazon.com