O slideshow foi denunciado.
Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.

Couchbase Live Europe 2015: Introducing Multi Dimensional Scale: A New Architecture for Workload Optimization

769 visualizações

Publicada em

Publicada em: Software
  • Seja o primeiro a comentar

  • Seja a primeira pessoa a gostar disto

Couchbase Live Europe 2015: Introducing Multi Dimensional Scale: A New Architecture for Workload Optimization

  1. 1. Multi-Dimensional Scaling A new Architecture for Workload Optimization in Couchbase Server 4.0 Cihan Biyikoglu Dir. Product Management 1
  2. 2. ©2015 Couchbase Inc. 2 Agenda  Brief History of Scaling  Scaling up and out  NoSQL Workloads and Scalability Model  Core Data operations, Indexing and Query  Introducing Multi-Dimensional Scaling  Services and Independent Scaling  Demo  Q&A
  3. 3. ©2015 Couchbase Inc. 3 Disclaimer  Couchbase Server 4.0 is still in development. Detail presented in this presentation may change based on customer feedback and other factors by the time the final version of the product is released.
  4. 4. History of Scaling
  5. 5. ©2015 Couchbase Inc. 5 Question Few million people are looking for a setup to efficiently live and interact. What is the most efficient way to build this infra? A) Build one giant high-rise? B) Build some mid-rises? C) Build many single-family homes
  6. 6. ©2015 Couchbase Inc. 6 Scaling Up Build one big high-rise  Vertical Scaling  Cluster processors – hyper-threading to cores  Locally partition workload among processors  Communicate over memory Great for fast processing but limited in scalability and elasticity
  7. 7. ©2015 Couchbase Inc. 7 Scaling out Build a large community of single-family houses  Horizontal Scaling  Cluster commodity HW  Partition workload among nodes  Communicate over network Great for scaling and elasticity but slower communication
  8. 8. ©2015 Couchbase Inc. 8 So what is the right model ?
  9. 9. NoSQLWorkloads & Scalability Model
  10. 10. ©2015 Couchbase Inc. 10 NoSQL Workloads  One Database ManyWorkloads  Core Data Processing:GETs & SETs for given key  Indexing: Index maintenance and lookups  Querying: Combine index and data with complex just-in-time data re- shaping, ordering, grouping, aggregations and more Varying resource requirements - CPU, RAM, I/O, Network Varying methods to optimize latency & throughput for each ©2014 Couchbase, Inc. 10
  11. 11. ©2015 Couchbase Inc. 11 Scalability ModelToday Homogenous Scaling  Each node get a slice of the workload  Simple to do… But...  Workloads compete and interfere with each other  Cant fine tune each workload - Core Data operation are partition-able so great with wider fan-out - Indexing and Query not always partition-able so worse with wider fan-out Index Service Couchbase Cluster Query Service Data Service node1 node8
  12. 12. Introducing Multi-Dimensional Scaling
  13. 13. ©2015 Couchbase Inc. 13 Modern Architecture What is Multi-Dimensional Scaling? MDS is the architecture that enables independent scaling of data, query and indexing workloads. Index Service Couchbase Cluster Query Service Data Service node1 node8
  14. 14. ©2015 Couchbase Inc. 14 Modern Architecture  Isolated Service for minimized interference  Independent “zones” for Query, Index and Data Services Minimize indexing and query overhead on core KV operations Index Service Global Secondary Indexes Couchbase Cluster Query Service Data Service Views and GeoViews node1 node8
  15. 15. ©2015 Couchbase Inc. 15 Modern Architecture  Independent Scalability for Best Computational Capacity per Service Heavier indexing (index more fields) : scale up index service nodes More RAM for query processing: scale up query service nodes Couchbase Cluster node1 node8 node9 Data Service Index ServiceQuery Service
  16. 16. Services Architecture Data, Index & Query
  17. 17. ©2014 Couchbase Inc. Full Cluster Architecture 17 STORAGE Couchbase Server 1 SHARD 7 SHARD 9 SHARD 5 SHARDSHARDSHARD Managed Cache Cluster Manager Cluster Manager Managed Cache Storage Data Service Index Service Query Service STORAGE Couchbase Server 2 SHARD 7 SHARD 9 SHARD 5 SHARDSHARDSHARD Managed Cache Cluster Manager Cluster Manager Managed Cache Storage Data Service Index Service Query Service STORAGE Couchbase Server 3 SHARD 7 SHARD 9 SHARD 5 SHARDSHARDSHARD Managed Cache Cluster Manager Cluster Manager Managed Cache Storage Data Service Index Service Query Service STORAGE Couchbase Server 4 SHARD 7 SHARD 9 SHARD 5 SHARDSHARDSHARD Managed Cache Cluster Manager Cluster Manager Managed Cache Storage Data Service Index Service Query Service STORAGE Couchbase Server 5 SHARD 7 SHARD 9 SHARD 5 SHARDSHARDSHARD Managed Cache Cluster Manager Cluster Manager Managed Cache Storage Data Service Index Service Query Service STORAGE Couchbase Server 6 SHARD 7 SHARD 9 SHARD 5 SHARDSHARDSHARD Managed Cache Cluster Manager Cluster Manager Managed Cache Storage Data Service Index Service Query Service
  18. 18. ©2015 Couchbase Inc. 18 Data Service Projector & Router New in Couchbase Server 4.0  Data Service: GET & SET operations and Map/ReduceViews  Arch mostly the same as 3.0  Index Service: new Global Secondary Indexes (GSI)  Query Service: new Query processor (N1QL) Query Service Query Processor cbq-engine Index Service Supervisor Index maintenance & Scan coordinator Index#2Index#1 Query Processor cbq-engine Bucket#1 Bucket#2 DCP Stream Index#4Index#3 ... B u c k e t # 2 B u c k e t # 1
  19. 19. DEMO
  20. 20. Recap
  21. 21. ©2015 Couchbase Inc. 21 Recap  MSD enables unprecedented control on scalability with Couchbase Server  Separate out competing workloads to independent services  Independently scale each service “zone” within the cluster  Couchbase Server with MDS maximizes scalability and performance  Improves scale and performance to degrees not possible with other NoSQL or Big Data engines on premise or in the cloud  Improved price/performance and squeezes more performance and throughput for mission critical systems
  22. 22. ©2015 Couchbase Inc. 22 Couchbase Server 4.0 Couchbase Server 4.0 Download the Developer Preview in a few weeks… Couchbase.com/
  23. 23. Q&A Cihan Biyikoglu cihan@couchbase.com @cihangirb

×