Cloudy with a Chance of Databases

© 2017 Delphix. All Rights Reserved. Private and Confidential.© 2017 Delphix. All Rights Reserved. Private and Confidential.
Kellyn Pot’Vin-Gorman | Technical Intelligence Manager| November, 2017
Cloudy with a Chance of
Databases

© 2017 Delphix. All Rights Reserved. Private and Confidential.
Kellyn Pot’Vin-Gorman
Technical Intelligence Manager, Delphix
• Multi-platform DBA, (Oracle, MSSQL, MySQL, Sybase,
Postgres…..)
• Oracle ACE Director, (Alumni)
• Oak Table Network
• Microsoft Idera ACE
• APEX Women in Technology Award, CTA 2014
• STEM education with Raspberry Pi and Python
• President, Rocky Mtn Oracle User Group
• Liaison, Denver SQL Server User Group
• DevOps author, instructor and presenter.
• Author, blogger, (http://dbakevlar.com)

Cloud Trends
• 85% of Enterprises have a multi-cloud strategy
• 77% are hybrid cloud, (different than the 2017 cloud survey)
• Workloads are being run in the cloud-
• 41% in public clouds
• 38% in private clouds
• Enterprise Companies are choosing cloud
• 65% want public cloud
• 63% want private cloud solutions
• 93% will be hybrid
• State of the Cloud Survey, RightScale

Source: Right Scale State of the Cloud 2016
0%
10%
20%
30%
40%
50%
60%
Physical Machine Virtual Machine Private Cloud Public Cloud
2017
2018
2019
Enterprise Company Cloud Migrations, 2017-2019

2017 Cloud Focus
0% 10% 20% 30% 40% 50% 60%
Private Clouds
Public Clouds
Hybrid Clouds
Cloud Investment

Cloud Adoption and How It’s Changing
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Public Cloud Private Cloud Hybrid Cloud Any Cloud
2016
2017
2018
http://www.rightscale.com/blog/cloud-industry-insights/cloud-computing-trends-2017-state-cloud-survey

Who’s Who
 Azure’s market penetration, (Up from 20% to 34%)
 AWS has remained consistent, (data before S3 outage)
 Google was up 10-15%
 Oracle is making headway in the SaaS arena, (Software as a
Service) and with Oracle Cloud Infrastructure, (Bare Metal)
 IBM? Archaic cloud of the Future?

Cloud by Provider
Cloud LTD, 2017
Trivia: Which Cloud
Vendor benefitted most
from the Amazon S3
outage?
AWS
Azure
IBM
Google
Oracle
Other

The Cloud
 AWS is now a $14 billion business - THIS YEAR.
 Not just Oracle’s investment, but Microsoft, IBM and Google are
all opening up new data centers are impressive rates.
 Consider how much of the cloud is outside of our focus area,
(desktop, applications and mobile)

The Limitations?
 Questions on Security
 Network woes
 Data Gravity
 Which Cloud and…
 Complex Cloud Management

History of Databases

Percentage of Database Types in 2017

When you consider…
An estimate of a 10% increase in data
accessibility will result in more than $65
million in additional net income, you can see
why data gravity is a big deal.

How much data are we talking about?
• More data has been created in just the last two
years than the previous history of humanity
• 1.7MB of new information created every second
per human on the planet.
• Introduction of big data often has same
development pain points.

Database Trend Impact

Biggest Gains?

What Has Lead to PostgreSQL Latest Success?
• Lower cost of ownership
• Advanced feature additions in PostgreSQL 10
• AmazonAurora support with PostgreSQL

Oracle May Be Losing Ground, but It Has a Lot of Ground to
Begin With…

Savings!!
• Stop with the savings
conversation.
• Become Educated on Tools that can
Move the Education Needle
• Learn Encryption and Masking
• Become more fluent in both platform and
DevOps
So How, as DBAs, Do We Make it Easier?

Sticker Shock
 Estimates on Wasted Cloud Expenditures
 30% of cloud is wasted – Perform Audits
 When measured, it was closer to 45%
 Best Tool to Impact Waste?
 Containerize
 Virtualize
 Cloud vendors profit from vendor lock-in.

How Are Companies Migrating to the Cloud?
1. Just copy data and applications into the cloud…
• Straightforward approach
• Inefficient, non-incremental for large environments
• Insecure, physical file transfers
• A truck, (yes, I’m looking at you, “snowmachine”)
• Archaic processes recommended by vendors
2. Start with backups to IaaS storage, then populate re-hosted
applications by restoring from those backups
• Cloud backups are easy, known technology

Standard Cloud Database Migration
Database server
Cloud Storage
Database server

Migration Complete…Not so Much…
• Even if only development or test has been migrated to
the cloud…
• Data is migrated, but this doesn’t count for ongoing data
loads, application connectivity across the network.
• To refresh will take considerable time to perform.
• Rarely a consideration for the difference in cost
structure for processing large amounts of data from on-
premise to the cloud.

Network, the New Bottleneck
• Few tools identify network latency in terms of
percentage or cause.
• Explore potential latency between cloud services and
data transfer conditions on contracts.
• Negotiate requirements before the agreement is
signed and data is handed over.
• Many organization are investing in a dedicated WAN
for cloud environments and operations.

AWS Demonstrates

Jeremiah Wilton, https://www.slideshare.net/AmazonWebServices/dat205-new-launch-introduction-to-aws-database-
migration-service
Amazon Web Services

• https://www.slideshare.net/AmazonWebServices/dat205-new-launch-introduction-to-aws-database-migration-service
Cross Platform Migration from AWS

• https://www.slideshare.net/AmazonWebServices/dat205-new-launch-introduction-to-aws-database-migration-service
• Replicate data in on-premises databases to AWS
• Replicate OLTP data to Amazon Redshift
• Integrate tables from third-party software into your reporting or core OLTP
systems
• Hybrid cloud is a stepping stone in migration to AWS
Amazon Cloud and AWS Choices

• T2 pricing starts at $0.018 per Hour for T2.micro
• C4 pricing starts at $0.154 per Hour for C4.large
• 50GB GP2 storage included with T2 instances
• 100GB GP2 storage included with C4 instances
• Data transfer inbound and within AZ is free
• Data transfer across AZs starts at $0.01 per GB
https://aws.amazon.com/ec2/pricing/
AWS Pricing

Azure SQL Database Migration

Azure Pricing

Oracle Cloud Pricing

There’s a Reason for these…

ProjectTimeline
Optimize
Migrate
Initial Testing
Requirements
Cloud Projects

What’s Missing?
Over 90% of cloud projects don’t
perform a preliminary identification of
latency or data issues that will impact
the project.
No strategy to identify or optimize BEFORE migrating to the cloud.

Secondary Considerations
• For Non-Production Systems…
• Change the way you’ve always performed tasks.
• Performing common tasks the same way as
previously might end up costing more.
• Secure Data
• All IaaS alternatives promote encryption for data in-
flight and for data at-rest, but encryption may not be
the right answer…

Why is there a Sticker Shock with the Cloud?
• If you don’t optimize for the cloud before you migrate..
Different cost structures will come back to bite you-
• Much smaller storage footprint
• Update data-transfers between systems/applications
• Virtualize environments that have high percentage of
duplicate data

Cost Structures?
• Traditional copy data management techniques
• Developed without concerns about infrastructure chargeback
• Nightly batchload ETLs
• Traditional data pulls
• Inefficient applications with thick database design
• IaaS vendors monitor storage and data transfers
• Help meet SLAs, garner profits
• It’s not just the data that exists in the end, so transformations can equal big
money for cloud vendors.

Why Less Focus on Thin Provisioning
Storage costs
Thin-provisioning storage avoidance = $$$
Data transfer costs
Far less data transferred during provisioning/refresh
operations
• As opposed to the volume of data transferred using
traditional cloning techniques
Network-attached storage data transfers offset somewhat
• Most non-production usage modifies or accesses only a
tiny proportion of the total database during lifetime

Cloud and Performance
KNOW THY ENEMY…
• Tune SQL and Apps to perform efficiently as possible- natural life of
database is growth, (in processes, resources, etc.) before migrating.
• The less network latency, the better- network tracing to eliminate
database blame is important.
• Many of the same tools and data provides value- The AWR,
(Automatic Workload Repository) is your friend!
• Take advantage of cloud monitoring tools that can ease the learning
curve as you migrate to the cloud.

• https://www.channele2e.com/2016/01/21/top-50-cloud-monitoring-and-management-tools-which-are-best/
1. Amazon CloudWatch
2. AppDynamics
3. Cloudmonix
4. Datadog
5. Dynatrace
6. Idera
7. Microsoft Cloud Monitoring
8. Nagios
9. New Relic
10.Solarwinds
The Best Cloud Monitoring and Management Tools,
(Alphabetically)

Amazon Cloudwatch
• This is a monitoring service for AWS
cloud Collect and track metrics
• Collect and monitor log files
• Set alarms, and automatically react
to changes

AppDynamics
https://docs.appdynamics.com/display/PRO14S/Architecture

Cloudmonix, (formerly AzureWatch)
• Supports Azure
• Supports both
Windows and
secondary Linux with
Azure
• Agent is required on
all hosts.

Datadog
• Strong DevOps Deployment
• Docker, Jenkins compatible
• Support AWS, Google, IBM
• GUI can be overwhelming

Dynatrace
• Manages both Linux and Windows hosts
• AWS, Azure, Cloud Foundry, OpenStack, OpenShift and more.
• Hybrid management console

• Designed for service
providers, database
professionals and IT
professionals
• Monitor and manage data
center environments.
• Acquired CopperEgg platform
— now known as Uptime
Cloud Monitor.
Idera

Microsoft Cloud Monitoring
• Microsoft Centric cloud
monitoring tools.
• Migration assistant and IIS
assistant

Nagios
• Has a network analyzer!
• Cloud support for Amazon, EC2/S3
• Offers log and application
performance monitoring
• Still has open source features

New Relic
• Multi-tenant sourced
• Monitoring
• AWS, Azure and Google
• Requires agent installation

Solarwinds

Confidential Data
http://ec.europa.eu/justice/data-protection/article-29/documentation/opinion-recommendation/files/2014/wp216_en.pdf
• GDPR, (General Data Protection Regulation States:
• Must have lawful basis to post process data
• Subject must have consented
• Must have contract that needs data
• Necessary and in order:
• Compliance
• Protect vital interests
• In public interest
• In legitimate interest of the owner/other party and doesn’t
violate the freedom and rights of the subject.

Confidential data
Exposure
Production
Non-production

Confidential data
• Encryption is reversible data obfuscation, which is essential for
production access and MUST be DONE.
• Data masking is non-reversible.
• It solves the issue at the data level.
• Is authentication and authorization in non-production in
compliance with security goals?
• All organizations will soon need to review if critical data in
non-production environments be accessible to developers,
testers and users.

Why Masking is Important-
• Masking personally-identifiable, (PII, HIPPA, PCI, etc.)
information renders it useless from a security
standpoint
• Resolves both the technical and personal responsibility
issue.
• The data can be masked before it moves to non-
production, removing unnecessary risk.
• As we discussed, on average, 80% of data is non-
production.

Confidential Data with Masking
Exposure
Production
Non-production
Encryption
Mask
Solution

Target Complete Solutions to Ease Management and
Vulnerability
• Virtualize and tune on-prem
• Mask data before it’s in the cloud
• Smaller footprint before its in flight.
• It’s transformed by securely masking on-premises...
• Then...
• securely virtualized data can be safely replicated to
IaaS

Summary
• Know Your Cloud
• Educate on more than one Database Platform
• Use a cloud monitoring and heterogeneous tools to
simplify management.
• Confidential Data
• Encrypt production data
• Data masking is often integrated with products and
protects PCI, PII, HIPPA and other confidential data.

© 2017 Delphix. All Rights Reserved. Private and Confidential.© 2017 Delphix. All Rights Reserved. Private and Confidential.
Kellyn Pot’Vin-Gorman
Technical Intelligence Manager
kellyn@delphix.com
http://dbakevlar.com

Cloudy with a Chance of Databases

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (20)

Semelhante a Cloudy with a Chance of Databases

Semelhante a Cloudy with a Chance of Databases (20)

Mais de Kellyn Pot'Vin-Gorman

Mais de Kellyn Pot'Vin-Gorman (20)

Último

Último (20)

Cloudy with a Chance of Databases

Notas do Editor