Amazon Elastic Block Store (Amazon EBS) provides persistent block level storage volumes for use with Amazon EC2 instances. In this technical session, we conduct a detailed analysis of the differences among the three types of Amazon EBS block storage: General Purpose (SSD), Provisioned IOPS (SSD), and Magnetic. We discuss how to maximize Amazon EBS performance, with a special eye towards low-latency, high-throughput applications like databases. We discuss Amazon EBS encryption and share best practices for Amazon EBS snapshot management. Throughout, we share tips for success.
9. A few definitions…
•IOPS: Input/output operations per second (#)
•Throughput: Read/write rate to storage (MB/s)
•Latency: Delay between request and completion (ms)
•Capacity: Volume of data that can be stored (GB)
•Block size: Size of each I/O (k)
12. •New default volume type for Amazon EBS
•Every volume can burst up to 3,000 IOPS
-Larger volumes can burst for longer periods
•3 IOPS per GB baseline performance
-Maximum of 3,000 on a 1TB volume
•99% performance consistency
13. (2) Max IO credit per bucket is 5.4M
(1) Always accumulating 3 IOPS per GB per second
(3) You can spend up to 3000 IOPS per second 13
17. VolumeType
BootTime
AccessTime
OS
GP2
3:31
4:33
WindowsServer 2012
Standard
4:30
7:16
WindowsServer 2012
GP2
0:36
0:45
CentOS6
Standard
0:57
1:16
CentOS6
40% Reduction in boot times by using General Purpose (SSD)
20. •Best for I/O intensive databases that require highest consistency
•Provision up to 4,000 IOPS per volume
•Supports IOPS to GB ratio of 30
•Designed to offer 99.9% consistency
21. •Best for cold workloads
•Rarely accessed data that needs always on access
•IOPS: ~100 IOPS steady-state, with best-effort bursts
•Throughput: variable by workload, best effort to 10s of MB/s
•Latency: Varies, reads typically ~20-40 ms, writes typically ~2-10ms
22. General Purpose (SSD)
Provisioned IOPS (SSD)
Magnetic
Recommend use cases
Boot volumes
Small to med DBs Dev and test
I/O intensive
Large DBs
Cold storage
Storage media
SSD-backed
SSD-backed
Magnetic-backed
Volume size
1GB-1TB
1GB-1TB
1GB-1TB
Max IOPS per volume
3 IOPS/GB Burst up to 3000 IOPS
4,000 IOPS
~100 IOPS
Read and write peak throughput
128 MB/s
128 MB/s
~50-90 MBps
Max I/O per node(16k)
48,000
48,000
48,000
Peak throughput node
800 MB/s
800 MB/s
800MB/s
Latency (random read)
1-2ms
1-2 ms
20-40 ms
API name
gp2
io1
standard
Price*
$.10/GB-Month
$.125/GB-Month $.065/provisioned IOPS
$.05/GB-Month $.05/ 1M I/O
32. 1.EC2 instance:Network bandwidth(Mbps)
2.EBS-optimized:EC2 instance option (On/Off)
3.Workload:Block size, read/write ratio, serialization
4.Queue depth:The number of outstanding I/Os
5.RAID:Stripe volumes to maximize performance
6.Pre-warm:Eliminates first touch penalty
33. 1. Amazon EC2 instance
• Compute optimized– C3
• Memory optimized- R3
• General Purpose – M3
Amazon
EBS
Amazon
EC2
Select Amazon EC2 instance that has the right network, RAM, and
CPU resources for your applications.
36. 3. Workload
EBS SSD-backed volumes measure I/O size up to 256KB
EBS SSD-backed delivers same performance for read and write
NEW
37. 4,000 IOPS
PIOPS volume
4,000 IOPS
128MB/s throughput
You can achieve 4,000 IOPS
when driving smaller I/O
You can achieve up to 128MB/s
when driving larger I/O
38. 4,000 IOPS
PIOPS volume
4,000 IOPS
128MB/s throughput
4,000 X 4KB = 16MB/s
4,000 X 32KB =128MB/s
8,000 x 8KB = 64MB/s
4,000 x 8KB = 32MB/s
2,000 x 128KB=256MB/s
1,000 x 128KB =128MB/s
39. Block (I/O) size determines whether your application is IOPS bound or throughput bound
40. 4. Queue depth
An I/O
Amazon
EBS
After it’s gone, it’s gone
Amazon
EC2
Queue depth is the pending I/O in flight
47. 0.08
7.71
845
4,152
0
500
1,000
1,500
2,000
2,500
3,000
3,500
4,000
4,500
0
1
2
3
4
5
6
7
8
9
10
1
4
8
12
16
20
24
28
32
Latency TP90 ( ms)
Queue Depth
16 KB random WRITE IOPS, Latency across various QDs
Latency (TP90)
AvgIOPS
IOPS
Write latency, queue depth and IOPS interaction is similar to that of read latency
48. Optimal queue depth to achieve lower latency and highest IOPS is typically between 4-8; ~1 QD per 500 IOPS
Amazon EBS-optimized offers consistent latency experience
49. 5. RAID
• Increase performance or capacity or both
• Stripe multiple volumes to achieve 10’s of TB and up-to
48,000 IOPS per Instance
• Don’t mix volume types
• Typically RAID 0 or LVM stripe
• Avoid RAID for redundancy Amazon
EBS
Amazon
EC2
50. Maximum performance per instance
12×400 GB PIOPS, pre-warmed, RAID 0 LVM, Stripe size 128 KB, attached to CR1 instance
53. Use large block size to speed up your pre-warming.
ex: sudoddif=/dev/xvdfof=/dev/xvdfconv=notruncbs=1M
54. Final tips
•Ext4 or XFS (understand journal impact!)
•Amazon EBS encryption has no visible IOPS performance impact for many Amazon EC2 instances
•Alignment can matter, check tools use 4k
55. Workload/ Software
Typical block size
Random/ Seq?
Max EBS @ 500 Mb/s
Max EBS @ 1 Gb/s
MaxEBS @ 10Gb/s instances
Oracle DB
Configurable:2 KB–16 KB
Default 8 KB
random
~7800 IOPS
~15,600 IOPS
~96,000 IOPS
Microsoft SQL Server
8 KB w/ 64 KB extents
random
~7800 IOPS
~15,600 IOPS
~80,000 IOPS
MySQL
16 KB
random
~4000 IOPS
~7,800 IOPS
~48,000 IOPS
PostgreSQL
8 KB
random
~7,800 IOPS
~15,600 IOPS
~96,000 IOPS
MongoDB
4 KB
serialized
~15,600 IOPS
~31,000 IOPS
~96,000 IOPS
Cassandra
4 KB
random
~15,600 IOPS
~31,000 IOPS
~96,000 IOPS
GlusterFS
128 KB
sequential
~500 IOPS
~1000 IOPS
~6,000 IOPS
62. Amazon EBS larger and faster volume types
General Purpose (SSD)
Provisioned IOPS (SSD)
Magnetic
Recommend use cases
Boot volumes
Small to med DBs Dev and test
I/O intensive
Large DBs
Cold storage
Storage media
SSD-backed
SSD-backed
Magnetic-backed
Volume size
1GB-16TB
4GB-16TB
1GB-1TB
Max IOPS per volume
10,000 IOPS
20,000 IOPS
~100 IOPS
burst
< 1TB to3000 IOPS
baseline
baseline
Read and write peak throughput
160 MB/s
320 MB/s
~50-90 MBps
Max IOPS per node (16k)
48,000
48,000
48,000
Peak throughput node
800 MB/s
800 MB/s
800MB/s
Latency (random read)
1-2ms
1-2 ms
20-40 ms
API name
gp2
io1
standard
Price*
$.10/GB-Month
$.125/GB-Month
$.065/provisioned IOPS
$.05/GB-Month
$.05/ 1M I/O
67. • 2-tiered key hierarchy using envelope
encryption
• Unique data key encrypt customer data
• AWS KMS master keys encrypt data keys
• Benefits of envelope encryption:
• Limits risk of a compromised data key
• Better performance for encrypting large data
• Easier to manage a small number of master
keys than millions of data keys
Master Key(s)
Data Key 1
S3 object EBS
volume
Redshift
cluster
Data Key 2 Data Key 3 Data Key 4
Custom
application
AWS KMS
68. Select the right volume for your workload
Select the right instance for your workload
Take snapshots
Use encryption if you need it
69. Please give us your feedback on this session.
Complete session evaluations and earn re:Invent swag.
http://bit.ly/awsevals