O slideshow foi denunciado.
Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.

Cassandra Core Concepts

1.071 visualizações

Publicada em

.

Publicada em: Tecnologia
  • Seja o primeiro a comentar

Cassandra Core Concepts

  1. 1. ©2013 DataStax Confidential. Do not distribute without consent. Jon Haddad, Luke Tillman Technical Evangelists, DataStax @rustyrazorblade, @LukeTillman Cassandra Core Concepts 1
  2. 2. Small Data • 100's of MB to low GB, single user • sed, awk, grep are great • sqlite • Limitations: • bad for multiple concurrent users (file sharing!)
  3. 3. Medium Data • Fits on 1 machine • RDBMS is fine • postgres • mysql • Supports hundreds of concurrent users • ACID makes us feel good • Scales vertically
  4. 4. Can RDBMS work for big data?
  5. 5. Replication: ACID is a lie Client Master Slave replication lag Consistent results? Nope!
  6. 6. Third Normal Form Doesn't Scale • Queries are unpredictable • Users are impatient • Data must be denormalized • If data > memory, you = history • Disk seeks are the worst (SELECT CONCAT(city_name,', ',region) value, latitude, longitude, id, population, ( 3959 * acos( cos( radians($latitude) ) * cos( radians( latitude ) ) * cos( radians( longitude ) - radians($longitude) ) + sin( radians($latitude) ) * sin( radians( latitude ) ) ) ) AS distance, CASE region WHEN '$region' THEN 1 ELSE 0 END AS region_match FROM `cities` $where and foo_count > 5 ORDER BY region_match desc, foo_count desc limit 0, 11) UNION (SELECT CONCAT(city_name,', ',region) value, latitude, longitude, id, population, ( 3959 * acos( cos( radians($latitude) ) * cos( radians( latitude ) ) * cos( radians( longitude ) - radians($longitude) ) + sin( radians($latitude) ) * sin( radians( latitude ) ) ) )
  7. 7. Sharding is a Nightmare • Data is all over the place • No more joins • No more aggregations • Denormalize all the things • Querying secondary indexes requires hitting every shard • Adding shards requires manually moving data • Schema changes
  8. 8. High Availability.. not really • Master failover… who's responsible? • Another moving part… • Bolted on hack • Multi-DC is a mess • Downtime is frequent • Change database settings (innodb buffer pool, etc) • Drive, power supply failures • OS updates
  9. 9. Summary of Failure • Scaling is a pain • ACID is naive at best • You aren't consistent • Re-sharding is a manual process • We're going to denormalize for performance • High availability is complicated, requires additional operational overhead
  10. 10. Lessons Learned • Consistency is not practical • So we give it up • Manual sharding & rebalancing is hard • So let's build in • Every moving part makes systems more complex • So let's simplify our architecture - no more master / slave • Scaling up is expensive • We want commodity hardware • Scatter / gather no good • We denormalize for real time query performance • Goal is to always hit 1 machine
  11. 11. What is Apache Cassandra? • Fast Distributed Database • High Availability • Linear Scalability • Predictable Performance • No SPOF • Multi-DC • Commodity Hardware • Easy to manage operationally • Not a drop in replacement for RDBMS
  12. 12. Hash Ring • No master / slave / replica sets • No config servers, zookeeper • Data is partitioned around the ring • Data is replicated to RF=N servers • All nodes hold data and can answer queries (both reads & writes) • Location of data on ring is determined by partition key
  13. 13. CAP Tradeoffs • Impossible to be both consistent and highly available during a network partition • Latency between data centers also makes consistency impractical • Cassandra chooses Availability & Partition Tolerance over Consistency
  14. 14. Replication • Data is replicated automatically • You pick number of servers • Called “replication factor” or RF • Data is ALWAYS replicated to each replica • If a machine is down, missing data is replayed via hinted handoff
  15. 15. Consistency Levels • Per query consistency • ALL, QUORUM, ONE • How many replicas for query to respond OK
  16. 16. Multi DC • Typical usage: clients write to local DC, replicates async to other DCs • Replication factor per keyspace per datacenter • Datacenters can be physical or logical
  17. 17. Reads & Writes
  18. 18. The Write Path • Writes are written to any node in the cluster (coordinator) • Writes are written to commit log, then to memtable • Every write includes a timestamp • Memtable flushed to disk periodically (sstable) • New memtable is created in memory • Deletes are a special write case, called a “tombstone”
  19. 19. Compaction • sstables are immutable • updates are written to new sstables • eventually we have too many files on disk • Merged through compaction, only latest data is kept based on timestamp sstable sstable sstable sstable
  20. 20. The Read Path • Any server may be queried, it acts as the coordinator • Contacts nodes with the requested key • On each node, data is pulled from SSTables and merged • Consistency< ALL performs read repair in background (read_repair_chance)
  21. 21. What are your options?
  22. 22. Open Source • Latest, bleeding edge features • Monthly releases (tick tock) • File JIRAs • Support via mailing list & IRC • Fix bugs • cassandra.apache.org • Perfect for hacking
  23. 23. DataStax Enterprise • Integrated Multi-DC Search • Integrated Spark for Analytics • Free Startup Program • <3MM rev & <$30M funding • Extended support • Additional QA • Focused on stable releases for enterprise
  24. 24. ©2013 DataStax Confidential. Do not distribute without consent. 25

×