3. House Keeping
Please Silence your phones
Your instructor:
Joe Ziegler, zieglerj@amazon.com
Technical Evangelist
@jiyosub
4. Audience
System Administrators
Solutions Architects / Cloud Architects
IT Professionals
Software Developers
...With enough experience using AWS
5. Agenda
1. Interacting with AWS using scripts
2. AMI: Amazon Machine Image
3. Route53 and DNS
4. Security on AWS
5. Advanced EC2, ELB and Autoscaling
6. Advanced EBS
7. Advanced RDS (MySQL only)
8. CloudFormation, IAM
6. Content…
Some of the content might have been adapted to
this specific audience.
Feel free to ask questions or clarification if you
want to dive deeper into a specific topic.
Slides are only for support, they don’t rule the
world
7. Attention…
When the Technical Workshop comes to an end,
to avoid unwanted costs:
Delete your S3 objects
Destroy your CloudFront distributions
Stop or Shut Down your EC2 and RDS instances
The customer is responsible for the resources
he’s using. AWS declines any responsibility if the
customer forgets to shut down resources.
10. Several Ways to Interact with AWS
Web Console
Command Line Tools
Software Development Kits (SDK)
Application Program Interfaces (API)
11. Learning Objectives
After this discussion you should be able to:
Interact with AWS using a shell and an SDK
EC2 resources via reusable scripts
Use AWS Management Console and scripts
12. AWS SDKS
.Net, Java, Ruby, Python, PHP
We will be using Python and Boto
Why?
Boto is a well maintain repository for AWS
Python is machine independent
You can using Ipython as a shell
Reusbale
13. Global Infrastructure for Global Enterprises
GovCloud US West US West US East South America EU Asia Asia Asia
(US ITAR (Northern (Oregon) (Northern (Sao Paulo) (Ireland) Pacific Pacific Pacific
Region) California) Virginia) (Singapore) (Tokyo) (Australia)
AWS Regions
AWS Edge Locations
14. Python and Boto
1) Download and Install Python (little different
for Windows, Mac and Linux)
2) Easy_install PIP
3) PIP install Boto
4) PIP install IPython
5) Go to Account -> Security Credentials
6) Create Boto security credentials
7) Create .boto_cfg
8) Test if it works
15. Download and Install Required Files
Python
http://www.python.org/getit/
Python 2.7.3 (32bit) Windows Installer
Add c:Python2.7Scripts to your path
16. Download and Install Required Files
Easy_install
http://pypi.python.org/pypi/setuptools#files
Pip
easy_install pip
Boto
Pip pip boto
Ipython
sudo pip install ipython
22. AWS Script
Lab Exercise
1. Make the Python and Boto work on your laptop or
EC2 instance.
2. Find a Linux or Windows AMI (on the Management
Console)
3. Using the iPython, launch a medium EC2 instance
with that AMI, then describe its status.
4. Login with SSH or RDP. Then exit.
5. Open the Management Console, check if the
instance shows up.
6. Terminate the instance.
22
26. Amazon Machine Images
An AMI is a special type of virtual appliance
which is used to instantiate (create) a virtual
machine within Amazon EC2.
The main component of an AMI is a read-only
filesystem image which includes an operating
system (Linux, Windows, *BSD).
A Manifest XML file describes it.
27. Learning Objectives – AMIs
After this discussion you should be able to:
Instantiate instances using the AWS AMI Catalog
Understand things to check with community AMIs
Create your own AMIs based on custom instances
Register/share your custom AMIs with the community
29. AMIs: Lab Exercise
• For most of the following exercises, we’re going
to use this specific AMI:
ami-3ed4916c (Singapore region)
or search for “Bitnami-SugarCRM-6.3”
• Create a new instance in Singapore region using
this AMI
Note: AMIs, Security Groups, Keypairs, are visible
only within the same Region.
30. AMIs: Shared/Community AMIs
Sooner or later, you will use a shared AMI: an Amazon
Machine Image created by a third party.
Can you blindly trust them?
31. AMIs: Lab Exercise
1) Check the ssh authorized keys file. The only key
in the file should be the key you used to launch
the AMI.
2) Check open ports and running services
3) Change the root password if it is not randomized
on startup.
4) Check if SSH allows root password logins. (you
can disable it on sshd_config)
5) Check whether there are any other user
accounts that might allow backdoor entry to
your instance. Accounts with super user
privileges are dangerous.
6) Verify that all cron jobs are legitimate.
34. AMIs: Lab Exercise
1. Log in to your Linux instance.
2. Create a password for root (passwd)
3. Log out
4. Log in as root. It works.
5. Locate the sshd_config file (e.g. /etc/ssh/)
6. Change the PermitRootLogin to without-password
7. Log out.
8. Log in as root. It shouldn’t work anymore.
35. AMIs: Optional Lab Exercise
1. Via the AWS Management Console create a new AMI
based on the changes you made to your instance
2. Make the AMI available to your Cloud Buddy
3. See http://aws.amazon.com/articles/9001172542712674
for extra information
36. Amazon Machine Images: Resources
Amazon Machine Images Catalog
https://aws.amazon.com/amis
How to share and use public AMIs in a secure manner
http://aws.amazon.com/articles/0155828273219400
Public AMI Publishing: Hardening and Clean-Up reqs
http://aws.amazon.com/articles/9001172542712674
39. CloudFront + Route53
Content Delivery
Edge Locations DNS Service
South New York Amsterdam
Seattle (2) London
Bend
Newark Stockholm
Palo Alto Dublin
Tokyo
San Jose
Frankfurt
Paris
Los Angeles
Ashburn
(2) Hong
Jacksonville Kong
Dallas Sydney
St.Louis Miami Singapore
Sao
Paulo
38 Edge Locations in total (as of Dec 2012)
40. Route53
Domain Name System (DNS)
Global DNS service
Highly available, redundant, scalable
Management Console support
CLI support
Integrated with other AWS services
40
41. Route53
Step by step
1. Create a hosted zone on Route53
2. Update client with the new DNS
names provided by Route53
3. Set up hosting (on Amazon EC2)
4. Create an A Record
5. Test the website, it should work
41
49. Security on AWS
Physical Security
Network Security
VM Security
OS Security
Data Security (in transit and at rest)
Identity and Access Management – MFA
Further Resources
50. Learning Objectives – Securing Your App
After this discussion you should be able to:
Understand the Shared Responsibility Security Model
Use Security Groups and/or VPC Network ACLs
to secure relevant parts of your application
Identify and secure important components of your
application and architecture
51. Shared Responsibility Security Model
To secure your application, we both have responsibilities
AWS Customer (You)
• Facilities • Operating System
• Physical Security • Application
• Physical Infrastructure • Security Groups
• Network Infrastructure • OS Firewalls
• Virtualization Infrastructure • Network Configuration
• Certifications for the above • Account Management
• Certifying your applications
52. AWS Certifications and Accreditations
Based on the Shared Responsibility model
AWS Environment
SAS70 Type II Audit
ISO 27001 Certification
Payment Card Industry Data Security Standard (PCI DSS)
Level 1 Service Provider
FedRAMP (FISMA)
Customers have deployed various compliant
applications:
Sarbanes-Oxley (SOX)
HIPAA (healthcare)
FISMA (US Federal Government)
DIACAP MAC III Sensitive IATO
53. Physical Security of Data Centers
Amazon has been building large-scale data centers for
many years
Important attributes:
Non-descript facilities
Robust perimeter controls
Strictly controlled physical access
2 or more levels of two-factor auth
Controlled, need-based access
All access is logged and reviewed
Separation of Duties
employees with physical access don’t have logical privileges
Maps to an Availability Zone
54. Fault Separation and Geographic Diversity
US East (Northern Virginia) EU (Ireland) Asia Pacific (Tokyo) US West (Oregon)
Availability Availability Availability
Zone A Zone A Zone B
Availability Availability Availability Availability Availability Availability
Zone D Zone B Zone A Zone B Zone A Zone B
Availability Availability
Zone C Zone C
US West (Northern California) Asia Pacific (Singapore) AWS GovCloud (US) South America (Sao Paulo)
Availability Availability Availability Availability Availability Availability Availability Availability
Zone A Zone B Zone A Zone B Zone A Zone B Zone A Zone B
Asia Pacific (Australia)
Availability Availability
Zone A Zone B
55. Physical Security - Lab Exercise
Access http://aws.amazon.com/about-
aws/globalinfrastructure/
Mouse-over on a few of the regions and edge
locations to identify the services available in each
location, the number of AZs available in each region
Access http://cloudping.info
Run a test to identify which region endpoint is
closest to your current location based on latency
56. Network Security Considerations
DDoS (Distributed Denial of Port Scanning:
Service): Violation of AWS TOS.
Standard mitigation Detected, stopped, and blocked.
techniques in effect. Ineffective anyway since inbound
ports blocked by default.
MITM (Man in the Middle)
All endpoints protected by Packet Sniffing:
SSL. Promiscuous mode is ineffective.
Fresh EC2 host keys Protection at hypervisor level
generated at boot time.
IP Spoofing:
Prohibited at host OS level
57. Network Security: Use Multiple Layers of
Defense
Feature Public EC2 Virtual Private Cloud
Security Groups inbound only inbound and outbound
Network ACLs n/a inbound and outbound
Operating System Works Works
firewalls
Border firewall Manual* NAT Instance
VPN Manual* VPN Gateway
Bastion Host Enforce via Security Enforce via Security
Groups Groups or Network
ACLs
IDS HIDS* HIDS* & NAT
Instance
* Third-party tools / solutions
58. Network Security
Inbound Traffic
Amazon Security Groups
Amazon EC2
Instances
iptables
Encrypted
File System Amazon EC2
Instance
Encrypted Swap
File
• Inbound traffic must be explicitly specified by protocol, port, and security group
• iptables may be implemented as a completely user controlled security layer for granular
access control of discrete hosts, including other Amazon Web Services (Amazon
S3/SimpleDB, etc.)
59. Network Security
AWS employs a private network with
Web Tier
ssh support for secure access
between tiers and is configurable to
limit access between tiers
Application Tier
Database Tier
EBS Volume
Ports 80 and 443 only
open to the Internet
Engineering staff have ssh
access to the App Tier,
which acts as Bastion
Authorized 3rd parties can be Amazon EC2
granted ssh access to select Security Group
AWS resources, such as the Firewall
All other Internet ports
Database Tier
blocked by default
60. EC2 Instance Security
Host operating system
Individual SSH keyed logins via bastion host for AWS admins
All accesses logged and audited
Guest (a.k.a. Instance) operating system
Customer controlled (customer owns root/admin)
AWS admins cannot log in
Customer-generated keypairs
Stateful firewall
Mandatory inbound firewall, default deny mode
Customer controls configuration via Security Groups
Signed API calls
Require X.509 certificate or customer’s secret AWS key
61. Amazon EC2 Instance Isolation
Customer 1 Customer 2 … Customer n
Hypervisor
Virtual Interfaces
Customer 1
Security Groups
Customer 2
Security Groups
… Customer n
Security Groups
Firewall
Physical Interfaces
62.
63. Network Security: EC2 Security Groups
• Control inbound traffic
• EC2 instances associated with 1 or more Security
Groups
• Default group: nothing open
When defining inbound rules, specify source by:
CIDR address (e.g. 0.0.0.0/0 for Internet, 10.0.0.0/16
for EC2 private, etc)
Security Group Name
• Restrict access to other EC2 instances in the specified
security group
64. Network Security: Lab Exercise
Security Groups
Example: Web Server Instance
Design a security group for Apache web servers in your
application’s web tier
65. Network Security: Lab Exercise
Security Groups
Example: Web Server Instance
Create security group named Web Tier
Web Tier
security group
• Specify group name and description
66. Network Security: Lab Exercise
Security Groups
Example: Web Server Instance
Specify inbound rules for group
Web Tier
security group
• Protocol, port and source
67. Network Security: Lab Exercise
Security Groups
Example: Web Server Instance
Launch EC2 instances into security group
Web Tier
security group
• Choose the Web Tier group when
launching a new instance
• An instance can belong to more than one
EC2 security group
EC2 Instance
68. Network Security: Lab Exercise
Security Groups
Example: Web Server Instance
Instance subject to inbound traffic
Web Tier
security group specified in Web Tier group
Rules can be added, modified or deleted
“on the fly”
EC2 ELB Source Security Group (e.g., amazon-
EC2 Instance
elb-sg) can be used to limit access to be
thru load balancer
EC2
EC2
Web Server
69. Network Security: Lab Exercise
Bastion Host
• To simplify security management, create a
Bastion host.
• This is an EC2 machine that allows external ssh
connectivity.
• From the bastion host you can ssh into your
other instances.
(Note: EC2 instances in the same Security Group
can connect to each other)
70. Network Security: Lab Exercise
Bastion Host
1) Create a security group “Bastion”, open on TCP
port 22.
2) Create a security group “Web”, open on TCP port
80 and 443.
3) Create an EC2 instance on “Default” + “Bastion”.
4) Create an EC2 instance on “Default” + “Web”.
5) You can SSH into EC2 Bastion, and then to EC2
Web. You can turn off EC2 Bastion.
71. Operating System Security
Guest (i.e., Instance) operating system
Customer controlled (customer owns root/admin)
AWS admins cannot log in
You still need to patch!
Most traditional tools will work
Emerging options
• Puppet (www.puppetlabs.com)
• Chef (www.opscode.com/chef)
• Fabric/Cuisine (www.fabfile.org)
• Capistrano (https://github.com/capistrano/capistrano/wiki)
72. Data Security
Protect privacy and enforce your policies
Encrypt data “in-transit” (SSL/TLS)
Encrypt data “at-rest”
Encrypt records before writing in database
Encrypt objects before storing them
Consider encrypted file systems for sensitive data
• Windows Bitlocker
• Truecrypt
• dm-crypt
• SafeNet
73. AWS Identity and Access Management
(IAM)
• Users and Groups within Accounts
• Unique security credentials
• Access keys
• Login/Password
• optional MFA device
• Policies control access to AWS APIs
• API calls must be signed by either:
• X.509 certificate
• secret key
• Deep integration into some Services
• S3: policies on objects and buckets
• Simple DB: domains
• AWS Management Console supports User log on
• Not for Operating Systems or Applications
• use LDAP, Active Directory/ADFS, etc...
74. How Can You Use AWS Account Isolation?
Environment
development, test, integration, performance, production
Major system
Line of business / function
Customer
Risk level
75. AWS Multi-Factor Authentication
Helps prevent anyone with unauthorized knowledge
of your e-mail address and password from
impersonating you
Additional protection for account information
Works with
Master Account
IAM Users
Integrated into
AWS Management Console
Key pages on the AWS Portal
S3 (Secure Delete)
A recommended opt-in security feature!
76. IAM Security: Optional Lab Exercise
AWS MFA Account Setup
1) Install Virtual MFA application to your phone:
http://aws.amazon.com/mfa/virtual_mfa_applications
/
2) Enable Virtual MFA application
https://console.aws.amazon.com/iam/home
• Enable MFA Device
3) Sign into AWS Management Console with MFA
http://aws.amazon.com/console
77. Security on AWS: Further Resources
Security and Compliance Center
http://aws.amazon.com/security/
AWS Overview of Security Processes whitepaper
http://d36cz9buwru1tt.cloudfront.net/pdf/AWS_Security_W
hitepaper.pdf
Security Best Practices whitepaper
http://d36cz9buwru1tt.cloudfront.net/Whitepaper_Security
_Best_Practices_2010.pdf
AWS Risk and Compliance whitepaper
http://d36cz9buwru1tt.cloudfront.net/pdf/aws-risk-and-
compliance-whitepaper.pdf
80. Amazon EC2
Availability Zones (AZ)
21 Availability Zones in total (as of Dec 2012)
81. ENI
Elastic Network Interfaces
ENIs are virtual network interfaces that
can be attached to EC2 instances in your
Virtual Private Cloud (VPC), up to two per
instance.
Each ENI has a Private IP, Elastic IP,
MAC, Security Group membership.
ENIs improve flexibility on VPC.
81
83. EC2 instance status
+ You can monitor the status of your
instances by viewing status checks and
scheduled events for your instances.
+ Performed every 5 minutes
+ Status is either OK or IMPAIRED
+ System status (network, power, SW,
HW)
+ Instance status (memory, FS, kernel)
83
85. Instance status with Python
Filter impaired EC2 instances only
ec2-describe-instance-status --filter "instance-status.status=impaired"
Send status feedback to the AWS team
ec2-report-instance-status [instance_id ...] [--status ...] [--reason] ..]
85
86. User Data Scripts on EC2
at boot time
It is possible to launch an Amazon EC2
Instance and have it perform a script at boot
time.
Not all AMIs support this. Try this one:
ami-58cc890a (Linux Ubuntu 11.10)
Use this to enable your AMIs:
http://ec2-run-user-data.notlong.com
86
88. Example of a User Data Script:
install a LAMP server
#!/bin/bashset -e -xexport DEBIAN_FRONTEND=noninteractiveapt-get update &&
apt-get upgrade -ytasksel install lamp-serverecho "Remember to set the MySQL
root password."
88
91. Elastic Load Balancing
Automatically distribute incoming traffic to
multiple Amazon EC2 instances (in the
same Region).
Automatic Health check
IPv6 support
Can be integrated with AutoScaling
91
93. Elastic Load Balancing
Lab Exercise
Create an ELB, and add two EC2 Instances to it, with two
different index.html pages.
Test if it works.
Temporarily remove index.html (rename it).
See what happens.
Restore index.html.
See what happens.
93
96. AutoScaling (AS):
Four main components
Get the AS Command Line Tools:
aws.amazon.com/developertools/2535
1) Launch config: AMI to be used
2) Autoscaling group: where/how to
launch
3) Autoscaling policy: what should AS
do?
4) Autoscaling trigger: what will activate
96
106. Script you can use
within each EC2 instance, at boot
#!/bin/bash
aws_instance=$(wget -q -O- http://169.254.169.254/latest/meta-data/instance-id)
aws_region=$(wget -q -O- http://169.254.169.254/latest/meta-data/hostname)
aws_region=${aws_region#*.}aws_region=${aws_region%%.*}aws_zone=`ec2-
describe-instances $aws_instance --region $aws_region`aws_zone=`expr match
"$aws_zone" ".*($aws_region[a-z])"`print "Instance: $aws_instance<br>Region:
$aws_region<br>Availability Zone: $aws_zone<br><br>END" > index.html
106
107. AutoScaling + ELB
Lab Exercise
Create your own Linux AMI, install a web server in it,
create a web page.
Configure AutoScaling + Elastic Load Balancing.
Test it with your Cloud Buddy.
107
110. Fixing Files on the Root EBS
Volume of an EC2 Instance
Complex problem:
1) You lost your SSH keys or password,
but you need to access that EC2 instance.
2) /etc/sudoers edited by mistake
3) The instance is hung, fails to boot
properly, cannot be accessed
110
111. Fixing the root EBS Volume
(1)
Identify the original EC2 instance and
volume
instance_a=i-XXXXXXXXvolume=$(ec2-describe-instances $instance_a | egrep
'^BLOCKDEVICE./dev/sda1' | cut -f3)
Identify the EC2 instance you’re going to
use to recover files or fix the volume
instance_b=i-YYYYYYYY
111
112. Fixing the root EBS Volume
(2)
Stop Instance A and detach the volume
ec2-stop-instances $instance_aec2-detach-volume $volume
Attach the volume to Instance B
ec2-attach-volume --instance $instance_b --device /dev/sdj $volume
Mount the volume on Instance B
$ sudo mkdir -p 000 /vol-a$ sudo mount /dev/sdj /vol-a
112
113. Fixing the root EBS Volume
(3)
Unmount the volume from Instance B
$ sudo umount /vol-a$ sudo rmdir /vol-a
Detach it from Instance B, attach it to A
ec2-detach-volume $volumeec2-attach-volume --instance $instance_a --device
/dev/sda1 $volumeec2-start-instances $instance_a
113
114. Fixing the root EBS Volume
Lab Exercise
Launch an Instance A.
Delete the /vol-a/home/ubuntu/.ssh/authorized_keys on
the Instance A, then log out.
Try to fix the root EBS Volume by following the steps
shown earlier, using the EC2 API Tools.
114
117. Advanced RDS
We are going to move the MySQL
database from our Bitnami EC2 instance
to Amazon RDS.
Make sure that both of them are in the
same Availability Zone.
117
118. Advanced RDS
Step by step
1) Create a DB Security Group for your DB
Instance.
2) Launch a DB Instance with RDS.
(DB name: bitnami_sugarcrm, User: root,
Password: bitnami)
3) Upload your DB from your Bitnami EC2
Instance to your DB Instance.
4) Edit the Connection String.
118
119. 119
Relational Database Service: Security Groups
121. Advanced RDS
(1)
Create a backup of your Bitnami database
mysqldump -u root -pbitnami bitnami_sugarcrm > backup.sql
Upload your Bitnami database to RDS
mysql -u root -pbitnami bitnami_sugarcrm --host= aws201.cfd6wday4byt.ap-
southeast-1.rds.amazonaws.com < backup.sql
Edit the connection string
cd /opt/bitnami/apps/sugarcrm/htdocssudo vi config.php
121
123. Advanced RDS
Lab Exercise
Complete the same operation that I’ve just shown you.
1) Create a DB Security Group for your DB Instance. It
should allow your Bitnami EC2 instance to connect to it.2)
Launch a DB Instance with RDS.(DB name:
bitnami_sugarcrm, User: root, Password: bitnami)3)
Upload your DB from your Bitnami EC2 Instance to your
DB Instance.
4) Edit the connection string.
123
128. IMPORTANT!
When the Technical Workshop comes to an end, to
avoid unwanted costs:
+ Delete your S3 objects
+ Destroy your CloudFront distributions
+ Stop or Shut Down your EC2 and RDS instances
The customer is responsible for the resources he’s
using. AWS declines any responsibility if the
customer forgets to shut down resources.
130. Thank you!
Joe Ziegler
zieglerj@amazon.com
Technical Evangelist
@jiyosub
Notas do Editor
Amazon Web Services is steadily expanding its global infrastructure to help customers achieve lower latency and higher throughput. As our customers grow their businesses, AWS will continue to provide infrastructure that meets their global requirements.
Customer instances have no access to raw disk devices, but instead are presented with virtualized disks. The AWS proprietary disk virtualization layer automatically resets every block of storage used by the customer, so that one customer’s data are never unintentionally exposed to another. AWS recommends customers further protect their data using appropriate means. One common solution is to run an encrypted file system on top of the virtualized disk device.
Just as you would do in your own datacenters, you should encrypt data in motion or at rest.
Mention consolidated billing option.
Amazon Account Security FeaturesAWS provides a number of ways for customers to identify themselves and securely access their AWS Account. A complete list of credentials supported by AWS can be found on the Security Credentials page under Your Account. AWS also provides additional security options that enable customers to further protect their AWS Account and control access: AWS Identity and Access Management (AWS IAM), Multi-Factor Authentication (MFA) and Key Rotation.AWS Multi-Factor Authentication (AWS MFA)AWS Multi-Factor Authentication (AWS MFA) is an additional layer of security that offers enhanced control over AWS Account settings and the management of the AWS Services and resources for which the account is subscribed. When customers enable this opt-in feature, they will need to provide a six-digit single-use code in addition to their standard username and password credentials before access is granted to their AWS Account settings or AWS Services and resources. Customers get this single use code from an authentication device that they keep in their physical possession. This is called Multi-Factor Authentication because two factors are checked before access is granted: customers need to provide both their username (Amazon e-mail in the case of the AWS Account) and password (the first “factor”: something you know) and the precise code from their authentication device (the second “factor”: something you have). Customers can enable MFA devices for their AWS Account as well as for the users they have created under their AWS Account with AWS IAM. It is easy to obtain an authentication device from a participating third party provider and to set it up for use via the AWS website. More information about Multi-Factor Authentication is available on the AWS website: http://aws.amazon.com/mfa/ Key RotationFor the same reasons as it is important to change passwords frequently, AWS recommends that customers rotate their access keys and certificates on a regular basis. To let customers do this without potential impact to their applications’ availability, AWS supports multiple concurrent access keys and certificates. With this feature, customers can rotate keys and certificates into and out of operation on a regular basis without any downtime to their application. This can help to mitigate risk from lost or compromised access keys or certificates. The AWS IAM APIs enables a customer to rotate the access keys of their AWS Account as well as for users created under their AWS Account using AWS IAM.