Building with Purpose-Built Databases: Match Your Workload to the Right Database

P U B L I C S E C T O R
S U M M I T
B RUSSE LS

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C TO R
S U M M I T
Building with Purpose-Built Databases
MatchYourWorkloadtotheRightDatabase
Márcio Santos
Solutions Architect
samarcio@amazon.com

S U M M I T
Richsetof servicesfor everchanging workloads
Over 40 data services in a rich
ecosystem of over 160 web services, all
designed to work together using open
standards.
Rapidly improving selection
of services, driven by
customer demand
Maximum agility and
maximum cost savings on a
constantly improving
foundation
For any workload
Analytics
NewApplications
Moving Legacy
Application toCloud

S U M M I T
Trends thatimpactthewaywethinkabout data
Data grows 10x every 5
years driven by network
connected smart devices
Transition from IT to
DevOps increases rate
of change
Micro-services architecture
decreases need for one-size fits all
databases and increases need for
real-time monitoring and analytics
Explosion of data
Micro-services changes
data and analytics
requirements
Rapid rate of change
driven by DevOps
Dev Ops

S U M M I T
Our strategyand our beliefs backin 2010
1. There is going to be an explosion in data.
2. Cloud will enable different architectures.
3. One size does not fit all - databases should be purpose-built.

S U M M I T
Our approach
Architect services ground-up for the cloud and
for the explosion of data
Offer a portfolio of purpose-built services,
optimized for your workloads
Help you innovate faster through
managed services
Provide services that help you migrate
existing apps and databases to the cloud

S U M M I T
Broad and deep portfolio, purpose-built for builders
Data Movement
Analytics
Data Lake Infrastructure & Management Ledger
Visualization & Machine Learning
Databases
+ 10 more
QuickSight SageMaker Comprehen
d
Le
x
Polly Rekognition Translate Transcribe
Deep Learning
AMIs
Aurora
MySQL,
PostgreSQL
RDS and RDS on VMWARE
MySQL, PostgreSQL,
Oracle, SQL Server
MariaDB
DynamoDB
Key-value
ElastiCache
Redis,
Memcached
Timestream
Time Series
Neptune
Graph
DocumentDB
Documents
Redshift
EMR
Spark,
Hadoop
Athena
Elasticsearch
Service
Kinesis Data
Analytics
Glue
Scala, Python
S3/Glacier Lake
Formation
Glue
Crawl, ETL
Managed Blockchain Quantum Ledger
Database (QLDB)
Database Migration Service | Snowball | Snowmobile | Kinesis Data Firehose | Kinesis Data Streams | Managed Streaming for Kafka

S U M M I T
A one sizefitsalldatabasedoesn’t fitanyone
Modernapplicationsneedpurpose-builtdatabases
Users: 1M+
Data volume: TB–PB–EB
Locality: Global
Performance: Milliseconds–microseconds
Request Rate: Millions
Access: Mobile, IoT, devices
Scale: Up-out-in
Economics: Pay-as-you-go
Developer access: Instant API access
Relational Key-value Document In-memory

S U M M I T
Purpose-built
Purpose-built
databases and
analytic engines.
Right tool for the right
job

S U M M I T
Therearedifferent types ofpurpose-built databases
Relational Key-value Document In-memory Graph Time series Ledger
Referential
integrity, ACID
transactions,
schema-
on-write
Low-latency,
key look-ups
with high
throughput and
fast ingestion
of data
Indexing and
storing
documents with
support for
query on any
attribute
Microseconds
latency, key-
based queries,
and specialized
data structures
Creating and
navigating
relations
between data
easily and
quickly
Collecting,
storing, and
processing time
series data
Complete
immutable,
verifiable history
of all changes to
application data
Lift and shift,
EMR, CRM,
finance
Real-time bidding,
shopping cart,
social
Content
management,
personalization,
mobile
Leaderboards,
real-time
analytics, caching
Fraud detection,
social networking,
recommendation
engine
IoT applications,
event tracking
Systems
of record,
registrations,
supply chain,
financial

S U M M I T
AWS: Purpose-built databases
Relational Key-value Document In-memory Graph Search
Amazon
DynamoDB
Amazon
Neptune
Amazon
RDS
Aurora CommercialCommunity
Amazon
ElastiCache
Amazon
Elasticsearch
Service
Amazon
DocumentDB
Time-series Ledger
Amazon
Timestream
Amazon
Quantum
Ledger
DatabaseMemcachedRedi
s

S U M M I T
Amazon RDS
Setuprelationaldatabaseinthecloudwithjustafewclicks
Available & durable
Automatic Multi-AZ data
replication; automated backup,
snapshots, failover
Easy to administer
No need for infrastructure
provisioning, installing and
maintaining DB software
Highly scalable
Scale database compute
and storage with a few
clicks with no application
downtime
Fast & secure
SSD storage and guaranteed
provisioned I/O; data
encryption at rest and
in transit

S U M M I T
AmazonAurora
MySQLandPostgreSQL-compatiblerelationaldatabasebuiltforthecloud
Availability
and durability
Fault-tolerant, self-healing
storage; six copies of data
across three AZs; continuous
backup to S3
Fully managed
Managed by RDS:
no hardware provisioning,
software patching, setup,
configuration, or backups
Highly secure
Network isolation,
encryption at rest/transit
Performance
and scalability
5x throughput of standard MySQL
and 3x of standard PostgreSQL;
scale-out up to
15 read replicas
Performance and availability of commercial-grade databases at 1/10th the cost

S U M M I T
Scale-out,distributed,multi-tenantarchitecture
• Purpose-built log-structured
distributed storage system
designed for databases
• Storage volume is striped across
hundreds of storage nodes
distributed over 3 different
Availability Zones
• Six copies of data, two copies in
each Availability Zone to protect
against AZ+1 failures
• Master and replicas all point to the
same storage
Availability
Zone 1
Availability
Zone 2
Availability
Zone 3
Shared storage volume
Storage nodes with SSDs
Master
SQL
Transactions
Caching
Replica Replica
SQL
Transactions
Caching
SQL
Transactions
Caching

S U M M I T
AuroraGlobal Database
High-performancedatabaseforglobally-distributedapplications
Single Global Database with cross region replication
Replication typically completes in less than a second
No impact on database performance
Write master in one region and read replicas in other regions
Cross-region disaster recovery
Local read latency for applications with global users
Primary Region Secondary Region
Application
Storage Storage
Replication <1s

S U M M I T
Databasesin privatedatacenters
Stilldifficultandexpensivetosetupandmanage
Difficult to set up and
manage databases for
high availability across
multiple nodes
Personnel needed to create
the database image, install
operating system,
packages, and setup
Burdensome to support
multiple versions and
applying patching
?

S U M M I T
Amazon RDSonVMware
Managedserviceforon-premisesdatabases
RDS deployed as a service in on-premises VMware private data centers (vSphere)
Automates management of on-premises databases and hybrid backup and scaling
Available and
durable
Enable hybrid features
and tap into AWS for high
availability, backup, and
restore
Secure and
compliant
Automate management of
databases for workloads that must
remain on-premises to adhere to
strict data policies
Fully managed
Easy to provision, monitor, and
operate relational databases in
your private data center
Scalability and
performance
Scale storage, compute, and
memory of on-premises
databases from a single,
simple interface

S U M M I T
Amazon DynamoDB
Fastandflexiblekeyvaluedatabaseserviceforanyscale
Comprehensive
security
Encrypts all data by default
and fully integrates with AWS
Identity and Access
Management for robust
security
Performance at scale
Consistent, single-digit
millisecond response times at any
scale; build applications with
virtually unlimited throughput
Global database for
global users and apps
Build global applications with fast
access to local data by easily
replicating tables across multiple
AWS Regions
Serverless
No hardware provisioning,
software patching, or upgrades;
scales up or down
automatically; continuously
backs up your data

S U M M I T
DynamoDB –GlobalTables
Fullymanagedactive-activeglobally-distributeddatabase
Build high-performance, globally
distributed applications
Low latency reads and writes
to locally available tables
Multiregional redundancy
and resiliency
Easy to set up and no application
rewrites required

S U M M I T
Amazon DocumentDB (withMongoDB compatibility)
Fast,scalable,highlyavailableMongoDB-compatibledatabase
Fast Scalable Fully managed MongoDB compatible
Millions of requests per second
with millisecond latency; twice
the throughput of MongoDB
Separation of compute and
storage enables both layers
to scale independently; scale
out to 15 read replicas in
minutes
Managed by AWS:
no hardware provisioning;
auto patching, quick setup,
secure, and automatic
backups
Compatible with MongoDB 3.6;
use the same SDKs, tools, and
applications with Amazon
DocumentDB

S U M M I T
Amazon ElastiCachefor Redisand Memcached
Fullymanagedin-memorydatastoreinthecloud
Extreme Performance
In-memory data store and
cache for sub-millisecond
response times
Fully Managed
AWS manages all hardware
and software setup,
configuration, monitoring
Easily Scalable
Read scaling with replicas
Write and memory
scaling with sharding
Non-disruptive scaling
Open Source
Compatible
Redis 5 support
Redis clients compatible
Memcached 1.5 support
Reliable
Multi-AZ
Deep monitoring
Automatic failover
Secure & Compliant
Amazon VPC
HIPAA, PCI, FedRAMP
Encryption at-rest and in-transit
Authentication

S U M M I T
Amazon Neptune
Fullymanagedgraphdatabase
Open Fast Reliable Easy
Supports Apache
TinkerPop & W3C RDF
graph models
Query billions of
relationships with millisecond
latency
6 replicas of your data
across 3 AZs with full backup
and restore
Build powerful queries easily
with Gremlin and SPARQL

S U M M I T
Amazon ElasticsearchService
Fullymanaged,scalable,andsecure
Fully managed;
Deploy production-ready clusters
in minutes
Secure access with VPC to
keep all traffic within AWS
network
Zone awareness replicates
data between two AZs;
automatically monitors &
replaces failed nodes
Direct access to
Elasticsearch open-source
APIs; supports Logstash
and Kibana
Easy to Use Secure AvailableOpen

S U M M I T
AmazonTimestream
Fast,scalable,fullymanagedtimeseriesdatabase
1,000x faster and 1/10th the
cost of relational databases
Collect data at the rate of
millions of inserts per
second (10M/second)
Trillions of
daily events
Adaptive query processing
engine maintains steady,
predictable performance
Analytics optimized
for time series data
Built-in functions for
interpolation, smoothing, and
approximation
Serverless
Automated setup, configuration,
server provisioning, software
patching

S U M M I T
AmazonQuantumLedger Database(QLDB)
Fullymanagedledgerdatabase
Immutable
Maintains a sequenced record of all
changes to your data, which cannot
be deleted or modified; you have
the ability to query and analyze the
full history
Cryptographically
verifiable
Uses cryptography to
generate a secure output
file of your data’s history
Easy to use
Easy to use, letting you
use familiar database
capabilities like SQL APIs
for querying the data
Highly scalable
Executes 2–3x as many
transactions than ledgers
in common blockchain
frameworks

S U M M I T
Amazon Managed Blockchain
Easilycreateandmanagescalableblockchainnetworks
Choice of Hyperledger Fabric or
Ethereum
Hyperledger Fabric available
today; Ethereum coming soon
Fully managed
Create blockchain networks
with a few clicks; Manage them
with simple API calls
Easily analyze
blockchain activity
Easy to move data into
QLDB for further analysis
Scalable and secure
Support thousands of client
applications running millions
of transaction; integrates
with AWS KMS

S U M M I T
Whatifyou wantto liftand shiftto thecloud
Relational
databases
Non-relational
databases
Data
warehouses
Hadoop
and Spark
Redshif
t
EM
R
Operational
analytics
Elasticsearch
ServiceAuror
a
DynamoD
B
Business
Intelligence
QuickSightRDS

S U M M I T
AWS DatabaseMigrationService
M I G R A T I N G
D A T A B A S E S
T O A W S
Migrate between on-premises and AWS
Migrate between databases
Automated schema conversion
Data replication for zero
downtime migration

S U M M I T
Traditionally,analyticslooked likethis
Relational data
GBs-TBs scale [not designed for PB/EBs]
Expensive: Large initial capex + $10K-$50K/TB/year
90% of data was thrown away because of cost
OLTP ERP CRM LOB
Data Warehouse
Business Intelligence

S U M M I T
Exabyte scale
Store and analyze relational and non-relational data
Purpose-built analytics tools
Cost effective
• Store at 2.3 cents per GB-month in Amazon S3
• Query with Amazon Athena at ½ cent per GB scanned
• DW with Amazon Redshift for $1,000/TB/year
Give access to everyone
• Amazon QuickSight: $0.30 for 30 minutes of use
DatalakesonAWS
Snowball
Snowmobile Kinesis
Data Firehose
Kinesis
Data Streams
S3
Redshift
EMR
Athena Kinesis
Elasticsearch Service
Kinesis
Video Streams
AI Services
QuickSight

S U M M I T
Amazon LakeFormation
Buildasecuredatalakeindays
Move, store, catalog, and
clean your data faster
Move, store, catalog,
and clean your data faster
with machine learning
Enforce security policies
across multiple services
Enforce security policies across
multiple services
Gain and manage new insights
Empower analyst and data
scientist to gain and manage new
insights

S U M M I T
Amazon Redshift
Highlyscalableclouddatawarehouse
Virtually unlimited
concurrency
Extends your
data lake
Dynamically scales to support
virtually unlimited number of
concurrent users and growing
data volumes
Analyze exabytes of data in the
Amazon S3 data lake together
with petabytes of data loaded
into Amazon Redshift’s high
performance SSDs
10x performance 1/10th the cost
Get faster time-to-insight for all
types of analytics workloads;
powered by machine learning,
columnar storage and MPP
Start at $0.25 per hour,
scale out as low as $1,000
per terabyte per year

S U M M I T
AmazonRedshiftConcurrencyScaling
Consistently fast performance at virtually unlimited concurrency
Automatically spins up additional clusters on-demand
Handles virtually unlimited number of concurrent users
Accrued minutes make it free for most customers
Redshift Managed
S3
Cluster Leader
Node
Data Data
Caching Layer
Cluster Leader
Node

S U M M I T
AmazonRedshiftSpectrum
Extend the data warehouse to exabytes of data in S3 data lake
S3 data lakeRedshift data
Amazon Redshift Spectrum
query engine
Redshift SQL queries using Amazon S3 at any scale
Join data across Amazon Redshift and Amazon S3

S U M M I T
Amazon RedshiftML based auto-tuning
Clustersalwaysoptimizedforbestperformanceandlowestcost
Tailored recommendations to increase performance & reduce cost
Redshift’s machine learning engine uncovers optimizations
Operations such as vacuum and analyze run behind the scenes
Available today
AUT O
AUT O
AUT O
ADVISE
ADVISE

S U M M I T
Amazon EMR
AnalyticsandMLatscalewithEnterprise-gradeSecurity
Updated with the latest open
source frameworks within 30 days
of release
Process data directly in the
S3 data lake securely with
high performance using the
EMRFS connector
Launch fully managed
Hadoop & Spark in minutes;
no cluster setup, node
provisioning, cluster tuning
Flexible billing with per-
second billing, EC2 spot,
reserved instances and
auto-scaling to reduce costs
50–80%
Latest versions Use S3 storage EasyLow cost

S U M M I T
In summary
1. We have ~40 services in this portfolio today
2. Customers are getting more insights and impact from IT because of the
cloud than ever before
3. Purpose-built databases are powering tens of thousands of our
customer applications

S U M M I T
Thank you!
S U M M I T
Márcio Santos
Solutions Architect
samarcio@amazon.com

S U M M I T
S U M M I T

Building with Purpose-Built Databases: Match Your Workload to the Right Database

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (20)

Semelhante a Building with Purpose-Built Databases: Match Your Workload to the Right Database

Semelhante a Building with Purpose-Built Databases: Match Your Workload to the Right Database (20)

Mais de Amazon Web Services

Mais de Amazon Web Services (20)

Building with Purpose-Built Databases: Match Your Workload to the Right Database