Amazon RDS allows you to launch an optimally configured, secure and highly available database with just a few clicks. It provides cost-efficient and resizable capacity, automates time-consuming database administration tasks, and provides you with six familiar database engines to choose from: Amazon Aurora, Oracle, Microsoft SQL Server, PostgreSQL, MySQL and MariaDB. In this session, we will take a close look at the capabilities of Amazon RDS and explain how it works. We’ll also discuss the AWS Database Migration Service and AWS Schema Conversion Tool, which help you migrate databases and data warehouses with minimal downtime from on-premises and cloud environments to Amazon RDS and other Amazon services. Gain your freedom from expensive, proprietary databases while providing your applications with the fast performance, scalability, high availability, and compatibility they need.
4. Where do DBAs spend their time?
Application
Optmization
Installation,
Upgrades, and
Patching
Backup and Recovery,
Data Import and Export,
Licensing, Training,
Security
8
5. • Lower TCO because we manage the muck
• Get more leverage from your teams
• Focus on the things that differentiate you
• Built-in high availability and cross-region replication
across multiple data centers
• Available on all engines, including base/standard
editions, not just for enterprise editions
• Now even a small startup can leverage multiple data
centers to design highly available apps with over
99.95% availability
We made things cheaper, easier, and better
6. Automated Backups
MySQL, PostgreSQL, MariaDB, Oracle, SQL Server
• Scheduled daily volume backup of entire instance
• Archive database change logs
• 35-day retention
• Multiple copies in each AZ when running multi-AZ
• Taken from standby when running multi-AZ
Aurora
• Automatic, continuous, incremental backups
• No impact on database performance
• 35-day retention
Every day during your
backup window, RDS
creates a storage
volume snapshot of
your database
Every five minutes,
RDS backs up the
transaction logs of
your database
If database is Multi-AZ, the snapshot is taken from the standby
• Single-AZ deployment = multiple backup copies in one AZ
• Multi-AZ deployment = multiple backup copies in multiple AZs
7. Database Snapshots
bucket
volume
Snapshot 2Snapshot 1 Snapshot 3
Amazon S3
Amazon EBS
BA C C1 D B1 E
BA A C1 D
Always incremental
Amazon S3 99.999999999% durability
Inherit encryption
Copy across accounts, across regions
8. Provisioning and Effortless Scaling
• Scale up/down for higher load or lower usage
• Naturally grow over time
• Control costs – pay as you go
9. Scalability with Read Replicas
Bring data close to your
customer’s applications
in different regions
Relieve pressure on
your master node for
supporting reads and
writes
Promote a Read Replica
to a master for faster
recovery in the event of
disaster
10. High Availability Multi-AZ Deployments
Enterprise-grade fault tolerance
solution for production
databases
Automatic failover
Synchronous replication
Inexpensive & enabled with one click
11. Security and Compliance
• Network Isolation
• Database instance IP firewall protection
• AWS IAM-based resource-level
permission controls
• Encryption at rest using AWS KMS or
Oracle/Microsoft TDE
• SSL protection for data in transit
• Assurance programs for finance,
healthcare, government and more
12. Amazon Virtual Private Cloud (Amazon VPC)
Securely control network configuration
Availability Zone
AWS Region
10.1.0.0/16
10.1.1.0/24
Manage connectivity
VPN
connection
VPC
peering
Internet
gateway
MAWS Direct
Connect
Routing
rules
13. Security Groups
Database IP firewall protection
Protocol Port Range Source
TCP 3306 172.31.0.0/16
TCP 3306 “Application
security group”
Corporate address admins
Application tier
M
14. Identity and Access Management (IAM)
Governed access: use IAM to control who can perform
actions on RDS (on some database engines)
Users and DBAApplications DBA and Ops
Your database RDS
Controlled with IAMControlled with database grants
M
15. At Rest Encryption for all RDS Engines
AWS Key Management Service (KMS)
Two-tiered key hierarchy using envelope encryption:
• Unique data key encrypts customer data
• AWS KMS master keys encrypt data keys
• Available for all RDS engines
Benefits:
• Limits risk of compromised data key
• Better performance for encrypting large data
• Easier to manage small number of master
keys than millions of data keys
• Centralized access and audit of key activity
Data key 1 Data key 2 Data key 3 Data key 4
Custom
application
Customer master
key(s)
Amazon
RDS
instance 3
Amazon
RDS
instance 2
Amazon
RDS
instance 1
17. Compliance
Aurora
SOC 1, 2, 3
ISO 20001/9001
ISO 27107/27018
PCI
HIPAA BAA
MySQL
SOC 1, 2, 3
ISO 20001/9001
ISO 27107/27018
PCI
FedRamp
HIPAA BAA
UK Gov. Programs
Singapore MTCS
Oracle
SOC 1, 2, 3
ISO 20001/9001
ISO 27107/27018
PCI
FedRamp
HIPAA BAA
UK Gov. Programs
Singapore MTCS
MariaDB
SOC 1, 2, 3
ISO 20001/9001
ISO 27107/27018
PCI
PostgreSQL
SOC 1, 2, 3
ISO 20001/9001
ISO 27107/27018
PCI
FedRamp
HIPAA BAA
UK Gov. Programs
Singapore MTCS
SQL Server
SOC 1, 2, 3
ISO 20001/9001
ISO 27107/27018
PCI
UK Gov.
Programs
Singapore MTCS
18. Standard Monitoring
Amazon CloudWatch
metrics for Amazon RDS
CPU utilization
Storage
Memory
Swap usage
DB connections
I/O (read and write)
Latency (read and write)
Throughput (read and write)
Replica lag
Many more
Amazon CloudWatch Alarms
Similar to on-premises custom
monitoring tools
(Virtual Machine Level)
19. Enhanced Monitoring
Access to over 50
new CPU, memory,
file system, and disk
I/O metrics as low as
1 second intervals
(Operating System Level)
20. Simplify monitoring
from the AWS
Management Console
Database load: Identifies
database bottlenecks
Easy
Powerful
Identifies bottleneck source
Top SQL
Adjustable time frame
Hour, day, week, and
longer
Max CPU
Amazon RDS Performance Insights
(Database Level)
22. DBaaS Report
“AWS not only has the
largest adoption of DBaaS, it also
offers the widest range of offerings to
support analytical, operational, and
transactional workloads.”
“AWS’s key strengths lay in its
dynamic scale, automated
administration, flexibility of database
offerings, strong security,
and high-availability capabilities,
which make it a preferred choice for
customers.”
The Forrester Wave™ is copyrighted by Forrester Research, Inc. Forrester and Forrester Wave™ are trademarks of Forrester Research, Inc. The Forrester Wave™ is a graphical
representation of Forrester's call on a market and is plotted using a detailed spreadsheet with exposed scores, weightings, and comments. Forrester does not endorse any vendor,
product, or service depicted in the Forrester Wave. Information is based on best available resources. Opinions reflect judgment at the time and are subject to change.
24. Key questions we asked
• What if we started from a clean sheet of paper with only constraint being that the
database was a relational database?
• Could we offer much better performance by leveraging the massive scale of our
cloud?
• Could we give you a database with designed durability indistinguishable from 100%
and availability of 99.99%?
• …And could we be better and cheaper than the 30-year old commercial databases in
use today?
25. Amazon RDS for Aurora
• MySQL compatible with up to 5x better performance on the
same hardware: 100,000 writes/sec & 500,000 reads/sec
• Scalable with up to 64 TB in single database, up to 15 read
replicas
• Highly available, durable, and fault-tolerant custom SSD
storage layer: 6-way replicated across 3 Availability Zones
• Transparent encryption for data at rest using AWS KMS
• Stored procedures in Amazon Aurora can invoke AWS
Lambda functions
Fastest-growing service
in AWS history
A new relational DB engine, built from the ground up to leverage AWS
26. Amazon Aurora – DB reimagined for the cloud
Speed and availability of high-end commercial databases
Simplicity and cost-effectiveness of open source databases
Drop-in compatibility with MySQL (2015) & PostgreSQL (in
open preview)
Simple pay as you go pricing
Delivered as a managed service
27. Amazon Aurora
Purpose-built log-structured
distributed storage system
designed for databases
Storage volume is striped across
hundreds of storage nodes
distributed over 3 different
availability zones
Six copies of data, two copies in
each availability zone to protect
against AZ+1 failures
Continuous backup to Amazon
S3 (built for 11 9s durability)
Master Replica Replica Replica
Availability
Zone 1
Shared storage volume
Availability
Zone 2
Availability
Zone 3
Storage nodes with SSDs
SQL
Transactions
Caching
SQL
Transactions
Caching
SQL
Transactions
Caching
Scale-out, distributed, multi-tenant architecture
29. Amazon Aurora is now PostgreSQL-compatible
• PostgreSQL 9.6 compatibility with support for PostGIS
• All the features you expect from Amazon Aurora including
15 read replicas with <10ms lag, shared storage, failover
without data loss, 6-way replication across 3 Availability
Zones, encryption with AWS KMS
• Available now in preview
30. In case you missed it…
Cross-region encrypted
replicas
Aurora on T2.Small – $1/day
Fast DDL Operations (Aurora)
IAM integration
USAspending.gov data on RDS
Stop/start database instances
Export into S3 (Aurora)
Database cloning (Aurora)
What’s new and exciting in 2017
31. 32
Craig A. Fonseca - Director, Database and Storage Services
Dow Jones employee for 17 years (prior United States Navy)
My path… operations, Sys Admin, Storage Architect, Database
Manager -- Cloud!
A real story, our experiences, not a marketing pitch
32. 33
• Dow Jones’ AWS journey began in 2012
• You can’t go it alone, partnership is key to success
• Executive sponsorship and support, beyond the bottom line
• Know your AWS account team
• When are budget and man-hour savings “realized”?
It’s a Journey
33. 34
• “Prioritize people over tasks” – Cathy Engelbert, CEO Deloitte
• Embrace and encourage change, it’s a good thing
• Set the journey’s expectation up & down the org chart
• Upskill and retain existing talent; AWS can be a natural transition
Your People
34. 35
• Abstracts the busy work
• Man-hour savings “realized” for focus on architectural and solutions work
• Budget savings “realized”, operating expenses well spent
• Freedom from expensive, rigid and proprietary database solutions
• Service risk mitigation for antiquated and unsupported platforms
RDS Benefits
35. 36
• Loss of traditional database features and functionality
• Whose database is that?? Standardizing & operationalizing AWS
assets
• Defining technical roles and responsibilities
• Selecting the right RDS instance
RDS Challenges
36. 37
• On-premises reduction: Wall Street Journal (.com) Email service to Aurora
• Time to Market: Wall Street Journal (.com) Context application to Aurora
• Security (PCI): Dow Jones Corporate Credit Card services to Aurora
• Break the mold: Proprietary application migrations to RDS Oracle & RDS
SQL Server
Use Case Themes
37. 38
• Dow Jones / AWS RDS workshop and focus group
• Own your best practices, decomm’ the “legacy cloud” accounts!?
• Automation, packer, git, jenkins, terraform
• Ongoing migrations and greenfield deployments
• Leverage existing technology partners, where/when it makes
sense
Dow Jones Roadmap
38. What are DMS and SCT?
AWS Database Migration Service (DMS) easily and securely
migrates and/or replicates your databases and data
warehouses to AWS
AWS Schema Conversion Tool (SCT) converts your commercial
database and data warehouse schemas to open-source engines or
AWS-native services, such as Amazon Aurora and Amazon Redshift
We have migrated over 34,000 unique databases. And counting…
39. Database Conversion Capabilities in SCT
Source Database Target Database
Microsoft SQL Server Amazon Aurora, MySQL, PostgreSQL
MySQL, MariaDB Amazon Aurora, PostgreSQL
Oracle Amazon Aurora, MySQL, PostgreSQL
Oracle Data Warehouse Amazon Redshift
PostgreSQL Amazon Aurora, MySQL
Teradata, Netezza, Greenplum Amazon Redshift
HP Vertica, SQL Server DW Amazon Redshift
MongoDB Amazon DynamoDB
41. When to use DMS and SCT?
Modernize Migrate Replicate
Modernize your database tier –
• Commercial to open source
• Commercial to Amazon Aurora
Modernize your Data Warehouse –
• Commercial to Amazon Redshift
• Migrate business-critical
applications
• Migrate from Classic to VPC
• Migrate data warehouse to
Amazon Redshift
• Upgrade to a minor version
• Consolidate shards into Aurora
• Create cross-regions Read Replicas
• Run your analytics in the cloud
• Keep your dev/test and production
environment sync