Migrating databases to the cloud is a critical part of organizations cloud journey and requires careful planning and architecture considerations including migration methods. This session will provide you with best practices and guidelines in migrating and/or architecting hybrid database architecture on AWS with focus on Microsoft SQL server databases. We will review current SQL on RDS, SQL on EC2 capabilities, compare and contrast various migration methods including SQL Export, Backup and Restore, and using AWS Database Migration Service (DMS). We will also look at how Expedia is migrating monolith SQL server databases to AWS using a hybrid approach leveraging SQL Server Distributed Availability Architecture. Expedia will share lessons learned during initial test and deployment phase followed by a demo of their existing architecture and deployment.

1. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Migrating Microsoft SQL Server
Databases to AWS—Best Practices
and Patterns
J e r r y M a n i o c i , P r i n c i p a l S y s t e m s E n g i n e e r E x p e d i a I n c .
P r a h l a d R a o , S o l u t i o n s A r c h i t e c t A W S
N o v e m b e r 2 8 , 2 0 1 7
AWS re:INVENT

2. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
What to Expect from the Session
• SQL on Amazon Web Services (AWS) – design options
• SQL on Amazon Relational Database Service (Amazon RDS) or Amazon Elastic
Compute Cloud (Amazon EC2) – compare and contrast
• SQL authentication
• SQL migration options to AWS
• AWS Database Migration Service (AWS DMS)
• Hybrid architecture – SQL Server distributed availability groups
• Expedia case study
• Expedia SQL Server hybrid architecture
• Risks/remediation
• Implementation guidelines/best practices
• Demo

3. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Lift and shift
• Self-manage on Amazon EC2
and Amazon Elastic Block Store
(Amazon EBS)
• Greatest flexibility
• Full control over DB instances
• Operating system access
• Hybrid architecture
Optimize
• Amazon Relational Database
Service
• Fully managed database service
by AWS
• Focus on application
optimization
• Speed of provisioning, HA, and
scaling
Transform
• Take advantage of cloud
architectures with Amazon
Aurora MySQL, Amazon Aurora
PostgreSQL, Amazon
DynamoDB
• Transform for scale and volume
• Reduce cost
SQL on AWS—Stages

4. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS Pace of Feature Release
• SQL Server 2017 available on Amazon EC2
(Windows and Linux) within 2 days of
Microsoft announcement
• Available in all AWS Regions
• Including support for x1e.32xlarge instance
type (128 vCPUs, 4 TB memory)
• Flexible licensing—pay as you go, BYOL,
License Mobility through Software Assurance

5. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
No infrastructure
management
Scale up/downCost-effective
Instant provisioning
Application
compatibility
Amazon Relational Database Service (Amazon RDS)

6. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
SQL Server on Amazon RDS or Amazon EC2
Versions supported: 2008 R2, 2012, 2014, 2016 2005, 2008, 2008 R2, 2012, 2014, 2016, 2017
Editions supported: Express, Web, Standard, Enterprise
High availability and
scaling:
Self-managed; AlwaysOn, mirror, log shipMulti-AZ deployment, instance scaling
Encrypted storage using AWS KMS (all editions); TDE supportEncryption:
Authentication: Windows and SQL Server authentication
Maintenance plans and third-party toolsManaged automated backupsBackups:
Self-managedAutomatic software patching, instance replacementMaintenance:
Amazon RDS for SQL Server SQL Server on Amazon EC2
Self-managed, AWS DMS, third-partyAWS DMS, third-partyRead replicas:
BYOL, license included AMIBYOL, license includedLicensing:

7. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Choosing the Right Solution for Your Needs
Amazon RDS for SQL
 Consider Amazon RDS first for
reduced administration and
improved manageability
 Focus on tasks that bring value to
your business
 Focus on high-level database
tuning and schema optimization
 Lack of in-house expertise
managing databases
SQL on Amazon EC2
 You need full control over the DB
instances and underlying OS
 Control over backups, replication, and
clustering
 Use features and options not available
in Amazon RDS (SSRS, SSIS, AlwaysOn
availability groups)
 Hybrid architectures
 Size and performance needs exceed the
Amazon RDS offering

8. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Windows Integrated Authentication
Auth/
LDAP
Auth/
LDAP
Availability Zone
Private Subnet
10.0.2.0/24
AC2EC2
SharePoint.NET Application
Availability Zone
Private Subnet
10.0.3.0/24
EC2EC2
SharePoint.NET Application
AWS Managed Services
AWS Managed Services
AWS
Microsoft AD
D
C
AWS
Microsoft AD
Trust
VPN
Direct
Connect
AD
RDS SQL
Server
RDS SQL
Server
Remote
Users/Admins
Windows Server
Domain
Controllers
Corporate data center
Auth/
LDAP
AD
AD
Use Active Directory (AD) identities
with Amazon RDS SQL Server
• Managed in AWS Microsoft
Active Directory
• Managed on premises and used
via Active Directory one-way
trust
Important consideration for SQL Server
migration

9. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
To Amazon EC2
• Backup/restore
• Import/export
• Bulk copy
• AWS Database Migration Service
• Third-party (for example, CloudBasic)
• Log shipping and replication
• Hybrid architecture
To Amazon RDS
• Backup/restore
• Import/export
• Bulk copy
• AWS Database Migration Service
• Third-party (for example, CloudBasic)
SQL Migration Options to AWS

10. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
SQL Migration Options to AWS
Migration
method
Amazon
RDS
target
Amazon
EC2 target
Downtime DB Objects Cross-
Engine
Backup/Restore Yes Yes Yes
(hrs)
Data, Schemas, Stored
Procedures, Triggers, Indexes
No
Import/Export
Bulk Copy
Yes Yes Yes
(mins-hrs)
Data, Schemas, Stored
Procedures, Triggers, Indexes
No
SQL Log
Shipping*
No Yes Minimal
(secs-mins)
Pre-create the DB, Sync No
Hybrid
Architecture*
No Yes Minimal
(secs-mins)
Pre-create the DB, Sync No
AWS DMS* Yes Yes Minimal
(secs-mins)
with SCT (Data, Schemas,
Stored Procedures, Triggers,
Indexes)
Yes (SCT)
Application
Multicast
Yes Yes Minimal
(secs-mins)
Pre-create the DB,
Simultaneous updates to
source and target DB
Yes (App
specific)
* Network bandwidth and latency considerations

11. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Migration—Additional Best Practices
• Larger instance size for increased throughput during migration, resize for steady
state (C.x, R.x instance types)
• Use GP2 SSD/provisioned IOPS EBS volumes and size based on anticipated
storage growth
• For large data imports and write-activity during migration, consider single-AZ,
switch to multi-AZ for steady state
• Bak file import limitations – 4 TB size and single file (Amazon RDS)
• If migration to Amazon RDS is a limitation, consider phased migration approach
with Amazon EC2 phase 1 and Amazon RDS phase 2

12. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
When to Use AWS DMS and AWS Schema Conversion
Tool
Modernize Migrate Replicate
Heterogeneous database
migrations
Modernize your database tier:
• Commercial to open source
• Commercial to Amazon Aurora
Modernize your data warehouse:
• Commercial to Amazon Redshift
• Migrate business-critical
applications
• Migrate from EC2-Classic to
Amazon VPC
• Migrate data warehouse to
Amazon Redshift
• Upgrade to a minor version
• Consolidate shards into Aurora
• Create cross-region read replicas
• Run your analytics in the cloud
• Keep your dev/test and production
environments in sync

13. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Customer
premises
Application users
AWS
Internet
VPN
Start a replication instance
Connect to source and target
databases
Select tables, schemas, or
databases
 Let AWS DMS create tables,
load data, and keep them in
sync
 Switch applications over to
the target at your convenience
Keep Your Apps Running During the Migration
AWS
DMS

14. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Load is Table by Table
Replication instance
Source Target

15. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Change Data Capture (CDC) and Apply
Replication instanceSource Target
Update
t1 t2
t1
t2
Transactions Change
apply
after bulk
load

16. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Source
Target
What else can I do?
Source
Source
Replication instance

17. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
What else can I do? Target
Source Target
Target
Replication instance

18. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Source
Take It All—or Some
Target
Replication instance

19. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS DMS Best Practices
• Size replication instance appropriately (C.x for large datasets) with additional
storage; account for transactions that are cached to disk during load
• Use multiple tasks (eight tables per instance-task) and multiple replication
instances for parallel loads
• Multiple tasks will increase load on source database; balance source database
performance with migration
• Remove bottlenecks on the target to reduce errors—disable unnecessary triggers,
foreign key check, validation, and secondary indexes

20. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Hybrid Architecture—SQL Server
Distributed Availability Groups
On-Premises AWS
Direct Connect

21. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Expedia Case Study

22. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

23. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Motivation for Hybrid Architecture
• The presentation tier/app tier is being moved to the cloud
• Hosting the database tier (powered by SQL Server) closer
to the presentation/app tier is a top priority
• Part of the lift-and-shift (move-and-tune) strategy
Business justification:
• Reduce latency (direct impact for the customer)
• Data transfer costs

24. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Architecture and Phases
• Deploy distributed AlwaysOn availability group with SQL Server
2016
Three phases (in control of our own destiny):
Phase 1: On-premises DB nodes (read/write) + EC2 DB nodes (read only)
Phase 2: EC2 DB nodes (read/write) + on-premises DB nodes (read only)
Phase 3: AWS-only EC2 DB nodes [us-west2 (read/write), us-east1 (read
only)]

25. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Architecture Diagram
Phase 1: On-premises DB nodes (RW) + AWS DB nodes (RO)
Synchronous
Asynchronous
AWS Primary

26. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Architecture Diagram
Phase 2: AWS DB nodes (RW) + on-premises DB nodes (RO)

27. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Architecture Diagram
Phase 3: AWS-only DB nodes [us-west1 (RW), us-east1 (RO)]

28. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Risks/Remediation
Risk1:
• Read-only EC2 DB replicas can stay stale if there is a network
bottleneck
(not able to transfer data as per the change rate)
Remediation:
• Extensive stress testing to ensure that data can be transferred to
AWS efficiently (per workload demands)

29. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Risks/Remediation
Stress testing—key perfmon counters
• Log bytes flushed/sec
Will help you understand the amount of change rate that is happening (data to be
transferred to AWS)
• Log send queue
Amount of log records in the log files of the primary database, in kilobytes, that
have not yet been sent to the secondary replica
• Recovery queue (redo)
Amount of log records in the log files of the secondary replica that have not yet
been redone

30. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Risks/Remediation
Stress testing—key perfmon counters
Redo queue Send queue

31. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Risks/Remediation
Risk2:
• Hybrid strategy will come to a halt if there is an AWS Direct
Connect failure
“Everything fails, all the time.” – Werner Vogels
Remediation:
• Build redundancy at the Direct Connect/network level

32. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Risks/Remediation
Risk3:
• Automation/scaling requirements
Again, “Everything fails, all the time.”
Amazon EC2 nodes will fail. The ability to quickly detect and add
new nodes is key.
Remediation:
• Invest a good amount of time in developing and testing
automation to add new nodes/clone AG nodes and bring the
cluster back to a healthy state

33. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Implementation Guidelines/Best Practices
Sizing:
• Rightsize Amazon EC2 instances (based on stress test results)
• Amazon EC2 instance type with better network performance is
key
• Use Dedicated Hosts for better control, and you can license the
host (physical cores)
• Use provisioned IOPS (PIOPS) if workload is I/O bound

34. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Implementation Guidelines/Best Practices
Windows settings:
TCP Auto-tuning level – Receive Window Auto-Tuning Level should
be set as normal

35. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Implementation Guidelines/Best Practices
Network settings:
• We found that Windows 2012 R2 instances provisioned on instance types
supporting 10 Gbps networking are capped at ~2.4 Gbps
• The following settings on the Virtual Network Adapter driver were changed to
achieve a network utilization of ~9.5 Gbps:
• Interrupt moderation = disabled
• Jumbo packet = 9014 bytes
• Packet priority and VLan = packet priority and VLan disabled
• Receive buffer = 2048
• Transmit buffer = 2048

36. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Current State and Next steps
• At phase 1 of hybrid setup
Phase 1: on-premises DB nodes (RW) + AWS DB nodes (RO)
• Working closely with Microsoft Product Team to fix bugs as we
explore news ones in Production
• Refine automation and build more resiliency
Next steps:
Early 2018 – phase 2: AWS DB nodes (RW) + on-premises DB nodes
(RO)
End of 2018–early 2019: AWS-only architecture

37. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Demo

38. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Links and Related Sessions
• https://aws.amazon.com/sql/
• https://aws.amazon.com/rds/sqlserver/
• https://d0.awsstatic.com/whitepapers/RDS/Deploying_SQLServer_on_AWS.pdf
• GPSWKS407 – Workshop – Strategies for Migrating Microsoft SQL Server
Databases to AWS
• WIN306 – Breakout Session – Design, Deploy, and Optimize SQL Server on AWS
• WIN405 – Chalk Talk – Best of Both Worlds: Deploying Multi-Region SQL Server
Clusters on AWS for HA and DR
• WIN403 – AWS Directory Service for Microsoft Active Directory Deep Dive

39. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Thank you!