As we move into the world of Big Data and the Internet of Things, the systems architectures and data models we've relied on for decades are becoming a hindrance. At the core of the problem is the read-modify-write cycle. In this session, Al will talk about how to build systems that don't rely on RMW, with a focus on Cassandra. Finally, for those times when RMW is unavoidable, he will cover how and when to use Cassandra's lightweight transactions and collections.
7. Rack & Stack
•Blades & 1U for high CPU with low storage density
•2U for plenty of CPU & storage & air flow
•3U-4U for high-latency / high-density storage
•“racks” don’t have to be literal
•blade chassis
•separate network/power is key
13. Memory
•always get ECC!
•~5 single bit errors in 8 GB RAM per hour (top-end error rate)
•unexplainable crashes
•data corruption
•8GB DIMMs are still the sweet spot
!
•Registered Memory: match to your CPU/motherboard
•Pretty much all server memory is ECC and Registered
!
•Speed: match to fastest rating of CPU/motherboard
14. Motherboards
•Largely out of your control
•Dell / HP / etc. you’re looking at server model, e.g. DL380
•Supermicro: be very careful when picking your VAR
•Features to watch for:
•Socket count (NUMA)
•IPMI
•onboard SAS or SATA port speed/count
•PCIe speed & layout
•RAM capacity
15. Storage Adapters
•Serial Attached SCSI
•Bit Error Rate: 1 in 10^16 bits or 1bit in 1,250TiB
•Supports SATA drives over STP
•Near-line SAS drives are SATA chassis with SAS boards
•Always use SAS if you need an expander
•Check out enclosure services in Linux
•Serial ATA
•Bit Error Rate: 1 in 10^15 or 1 bit in 125 TiB
•Avoid expanders
16. Storage Adapters
•JBOD
•cheap
•OS manages drives
•drivers usually shipped with OS
•CPU overhead is negligible
•HW RAID is sometimes faster, usually comes with cache
•writethrough v.s. writeback
•writeback + BBU provides interesting performance options
•driver + utilities management
21. RAID
•JBOD
•mount every drive with individual filesystems
•cheap
•RAID0
•single drive failure means node rebuild
•cheap
•RAID10
•fast, protects against single disk failure
•expensive
22. RAID
•RAID 5 / 6 (and beyond)
•parity data protection
•performance heavily dependent on implementation
•cheapest option for drive failure protection
•RAID 50 / 60
•stripe across multiple RAID[56] volumes
•mostly useful with large number of drives
•can provide decent performance esp. on HW RAID
23.
24. Hard Drives
•SATA HDD
•there’s only one head carriage
•seeks kill
•decent performance on sequential IO
•bit errors
•cheap!
25.
26. Hard Drives
•SAS HDD
•there’s only one head carriage
•seeks kill
•bit errors
•expensive!
•faster RPMs may help a little with seek latency
27.
28. Hard Drives
•SATA SSD
•very low latency seeks
•slightly lower sequential IO throughput
•more expensive than SATA HDD
•vendors might not want to sell them to you!
•sometimes called “value series” or similar
•Cassandra runs fine on consumer-grade SSDs
•make sure your SATA/SAS bus and HBA are up to the task
31. Hard Drives
•PCIe SSD
•e.g. FusionIO, ioSwitch
•highest performance potential
•not as expensive as you think
•lots of new products entering the market
•generally not hot-swappable
32.
33. Networking
•you don’t need 10gig
•but it’s awesome
•Broadcom cards are common and commonly buggy
•Intel cards are expensive but a good bet
•Consider lesser-known add-in cards, e.g. Myricom
34. To the Cloud!
•Amazon, Google, etc. all use similar gear under the VM
•same constraints apply, but you only get a fraction of the box
•pass-through PCIe devices for the best performance
•Avoid EBS in EC2, go with ephemerals
•GCE PD’s may need additional read/write threads