Frank Ober of Intel’s Solutions Group will review how he achieved 1+ million transactions per second on a single dual socket Xeon Server with SSDs using the open source tools of Aerospike for benchmarking. The presentation will include a live demo showing the performance of a sample system. We will cover: The state of Key-value Stores on modern SSDs. What choices you make in your selection process of hardware that will most benefit a consistent deployment of Aerospike. How to run an Aerospike mesh on a single machine. How to work replication of that mesh, and what values allow for maximum threading and scale. We will also focus on some key learnings and the Total Cost of Ownership choices that will make your deployment more effective long term.

1. Frank Ober
Solutions ArchitectureTeam – Non-Volatile Memory Solutions Group
@fxober

3. http://www.spec.org
http://www.intel.com/info/hyperthreading
http://www.intel.com/go/turbo

6. Type Speed Usage Players
Key value databases Fastest Operational MemcacheDB, Redis, Aerospike
DynamoDB(Amazon). LevelDB (Google),
Rocksdb (Facebook)
BigTable , Column-
based.
Faster Analytics BigTable, Cassandra, Hbase (*Hadoop)
Document Databases Faster Web Sites MongoDB (WiredTiger v3.0 is released)
Couchbase (ForestDB releases June 2015)
Graph databases Fast Social Graphs Neo4J

7. Level DB and how SSTable works - https://www.igvita.com/2012/02/06/sstable-and-log-structured-storage-leveldb/
Flash and Storage Filesystems - http://radar.oreilly.com/2014/08/how-flash-changes-the-design-of-database-storage-
engines.html (sponsored by Aerospike)
NVM KV – Store (Read Side is Direct):
NVM KV Store is asynchronously written through a
memory buffer.
AerospikeConf file setting:
write_block_size
128k is default
Post-write-queue
256 is default

9. 3 Clients
You need to spread the load
Here Dell 620 dual sockets are used
DUAL 10Gbit
networks Dell R730xd Server System
One primary (dual system with replication testing)
DualCPU socket, rack mountable server system
DellA03 Board, Product Name: 0599V5
CPU Model used
2 each - Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz max frequency: 4Ghz
18 cores, 36 logical processors per CPU
36 cores, 72 logical processors total
DDR4 DRAM Memory
128GB installed
BIOSVersion
Dell* 1.0.4 , 8/28/2014
Network Adapters
Intel® EthernetConverged 10G X520 – DA2 (dual port PCIe add-in card)
1 – embedded 1G network adapter for management
2 – 10GB port for workload
Storage Adapters
None
Internal Drives and Volumes
/ (root) OS system – Intel SSD for DataCenter Family S3500 – 480GB Capacity
/dev/nvme0n1 Intel SSD for Data Center Family P3700 – 1.6TBCapacity, x4 PCIe AIC
/dev/nvme1n1 Intel SSD for DataCenter Family P3700 - 1.6TBCapacity, x4 PCIe AIC
/dev/nvme2n1 Intel SSD for Data Center Family P3700 - 1.6TB Capacity, x4 PCIe AIC
/dev/nvme3n1 Intel SSD for DataCenter Family P3700 - 1.6TBCapacity, x4 PCIe AIC
6.4TB of raw capacity for Aerospike database namespaces
Aerospike
Community
Version 3.5.8

10. https://communities.intel.com/community/itpeernetwork/blog/2015/02/17/reaching-one-
million-database-transactions-per-second-aerospike-intel-ssd
Record Size
Aerospike
Number
of clients
threads
Total
TPS
Percent
below
1ms
(Reads)
Percent
below
1ms
(Writes)
Std Dev
of Read
Latency
(ms)
Std Dev
of Write
Latency
(ms)
Approx.
Database
size
1k 576 1,124,875 97.16 99.9 0.79 0.35 100G
2k 448 875,446 97.33 99.57 0.63 0.18 200G
4k 384 581,272 97.22 99.85 0.63 0.05 400G
1k
(replication)
512 1,003,471 96.11 98.98 0.87 0.30 200G
Record Size
iostat
Read MB/sec Write MB/sec Avg queue
size on SSD
Average drive
latency
CPU Busy %
1k 418 29 31 0.11 93
2k 547 43 27 0.13 81
4k 653 52 20 0.16 52
1k
(replication)
396 51 30 0.13 94

11. StorageTypes Cost per GB 1k transaction/socket Memory Capacity
DRAM only $10-15 + (DDR4) ~1.6 million tps (1
socket)
192GB – 768 GB
SSD Configuration $1-3 + (PCIe SSD –
retail channel)
~600k per node (1
socket)
4 x 2TB = 8TB
10# SFF NVMe servers
3x lower transactions per second, yet 5x lower price per GB with NVM.
Capacity is higher, cost is much lower allowing you to do more per unit of rack.

12. Areas of improvement exist:
1. Iowait percentage is pretty high here: 13%, and because all SSD’s degrade with writes, you really want
to achieve the most that you can with good SSD’s. Endurance is not the issue here, what is the issue is
the NAND characteristics related to writes and overall (R+W) latency with write characteristics.
2. Software IRQ (14%)– this is an opportunity to help balance the network and storage drivers more
effectively over time.
1. There are several innovation projects going on between Intel – Aerospike related to optimizations…

14. asadm (asmonitor)
top, htop (mpstat)
iostat
iotop –o
iftop, tcpdump/Wireshark
http://www.tecmint.com/command-line-tools-to-monitor-linux-performance/
Intel CPU monitor (PCM)
vtune
mlc-

16. @IntelSSD
https://twitter.com/intelssd
http://communities.intel.com
https://communities.intel.com/community/itpeernetwork
https://communities.intel.com/people/FrankOber/content
https://communities.intel.com/community/tech/solidstate
http://www.intel.com/support
https://www.linkedin.com/company/it-center
@aerospikedb
http://www.aerospike.com/get-started
http://aerospike.com/docs
http://www.aerospike.com/act/
https://www.aerospike.com/docs/operations/p
lan/ssd/ssd_certification.html