SlideShare uma empresa Scribd logo
1 de 18
Sharding and More
    weng.wei@zalora.sg
• Sharding is about scaling
• scaling up VS scaling out
Scaling Out


• Add more nodes
• Each node shared nothing
Key / Value


• server = serverlist[ hash(key) %
  len(serverlist) ]
Relational DB


• On Primary Key
• On Index
Middle layer


• MySQL Proxy
Client Side


• Memcached client
• ORM
Limitation

• No joining
 • Join on the same shard is OK
• Data maintenance
Pre-Sharding


• Move database is easier than move data
Problem of sharding


• serverlist[ hash(key) % len(serverlist)
Consistent Hash


• ketama from last.fm
 • De facto Standard for memcached
Do we use sharding?


• Memcached Client
• MySQL for different ventures
SQL vs NoSQL


• NoSQL is “Not Only SQL”
• NoSQL completes SQL, but not replaces it
NoSQL is about

• Performance?
• Scaling (easier to shard)?
• Flexibility (schema-less)?
Performance

• Memcacahed is faster than MySQL?
• SQL faster than K/V?
• Memory faster than disk?
MySQL for K/V


• handler socket
Schema - less


• Friendfeed’s solution
 • Table as secondary index, then shard
• No silver bullet
• Trade off

Mais conteúdo relacionado

Mais procurados

(SPRING)KAFKA - ONE MORE ARSENAL IN A DISTRIBUTED TOOLBOX
(SPRING)KAFKA - ONE MORE ARSENAL IN A DISTRIBUTED TOOLBOX(SPRING)KAFKA - ONE MORE ARSENAL IN A DISTRIBUTED TOOLBOX
(SPRING)KAFKA - ONE MORE ARSENAL IN A DISTRIBUTED TOOLBOXnklmish
 
Aiki Framework in 32 Slides
Aiki Framework in 32 SlidesAiki Framework in 32 Slides
Aiki Framework in 32 SlidesJon Phillips
 
Building an E-commerce website in MEAN stack
Building an E-commerce website in MEAN stackBuilding an E-commerce website in MEAN stack
Building an E-commerce website in MEAN stackdivyapisces
 
SharePoint on Imaginary Hardware - IndyTechFest 2010
SharePoint on Imaginary Hardware - IndyTechFest 2010SharePoint on Imaginary Hardware - IndyTechFest 2010
SharePoint on Imaginary Hardware - IndyTechFest 2010John Ferringer
 
Random thoughts on sql server performance
Random thoughts on sql server performanceRandom thoughts on sql server performance
Random thoughts on sql server performanceNigel Foulkes-Nock
 
Tis the Season to Scale
Tis the Season to ScaleTis the Season to Scale
Tis the Season to ScaleJames Cryer
 
Simple Db & Dynamo Db
Simple Db & Dynamo DbSimple Db & Dynamo Db
Simple Db & Dynamo DbUchit Vyas ☁
 
Part One: Building Web Apps with the MERN Stack
Part One: Building Web Apps with the MERN StackPart One: Building Web Apps with the MERN Stack
Part One: Building Web Apps with the MERN StackMongoDB
 
SQL Azure for ISUG(SQL Server Israeli User Group)
SQL Azure for ISUG(SQL Server Israeli User Group)SQL Azure for ISUG(SQL Server Israeli User Group)
SQL Azure for ISUG(SQL Server Israeli User Group)Pini Krisher
 
Short introduction to Redis
Short introduction to RedisShort introduction to Redis
Short introduction to RedisJimmyZoger
 
Managing users and aws accounts
Managing users and aws accountsManaging users and aws accounts
Managing users and aws accountsAleksandr Maklakov
 
Multi-tenant Database Design for SaaS
Multi-tenant Database Design for SaaSMulti-tenant Database Design for SaaS
Multi-tenant Database Design for SaaSVõ Duy Tuấn
 

Mais procurados (20)

Javascript for Wep Apps
Javascript for Wep AppsJavascript for Wep Apps
Javascript for Wep Apps
 
Functional Css
Functional CssFunctional Css
Functional Css
 
Introducing project spartan
Introducing project spartanIntroducing project spartan
Introducing project spartan
 
(SPRING)KAFKA - ONE MORE ARSENAL IN A DISTRIBUTED TOOLBOX
(SPRING)KAFKA - ONE MORE ARSENAL IN A DISTRIBUTED TOOLBOX(SPRING)KAFKA - ONE MORE ARSENAL IN A DISTRIBUTED TOOLBOX
(SPRING)KAFKA - ONE MORE ARSENAL IN A DISTRIBUTED TOOLBOX
 
Aiki Framework in 32 Slides
Aiki Framework in 32 SlidesAiki Framework in 32 Slides
Aiki Framework in 32 Slides
 
Building an E-commerce website in MEAN stack
Building an E-commerce website in MEAN stackBuilding an E-commerce website in MEAN stack
Building an E-commerce website in MEAN stack
 
SharePoint on Imaginary Hardware - IndyTechFest 2010
SharePoint on Imaginary Hardware - IndyTechFest 2010SharePoint on Imaginary Hardware - IndyTechFest 2010
SharePoint on Imaginary Hardware - IndyTechFest 2010
 
Random thoughts on sql server performance
Random thoughts on sql server performanceRandom thoughts on sql server performance
Random thoughts on sql server performance
 
Cassandra on EPAM Cloud
Cassandra on EPAM CloudCassandra on EPAM Cloud
Cassandra on EPAM Cloud
 
Html5
Html5Html5
Html5
 
Tis the Season to Scale
Tis the Season to ScaleTis the Season to Scale
Tis the Season to Scale
 
Simple Db & Dynamo Db
Simple Db & Dynamo DbSimple Db & Dynamo Db
Simple Db & Dynamo Db
 
MEAN Stack
MEAN StackMEAN Stack
MEAN Stack
 
Part One: Building Web Apps with the MERN Stack
Part One: Building Web Apps with the MERN StackPart One: Building Web Apps with the MERN Stack
Part One: Building Web Apps with the MERN Stack
 
SQL Azure for ISUG(SQL Server Israeli User Group)
SQL Azure for ISUG(SQL Server Israeli User Group)SQL Azure for ISUG(SQL Server Israeli User Group)
SQL Azure for ISUG(SQL Server Israeli User Group)
 
Short introduction to Redis
Short introduction to RedisShort introduction to Redis
Short introduction to Redis
 
Managing users and aws accounts
Managing users and aws accountsManaging users and aws accounts
Managing users and aws accounts
 
Node ts1
Node ts1Node ts1
Node ts1
 
Multi-tenant Database Design for SaaS
Multi-tenant Database Design for SaaSMulti-tenant Database Design for SaaS
Multi-tenant Database Design for SaaS
 
.Less - CSS done right
.Less - CSS done right.Less - CSS done right
.Less - CSS done right
 

Destaque

我对ORM的思考
我对ORM的思考我对ORM的思考
我对ORM的思考Wei Weng
 
用PY实现"Go元编程"
用PY实现"Go元编程"用PY实现"Go元编程"
用PY实现"Go元编程"Weng Wei
 
About Caching
About CachingAbout Caching
About CachingWeng Wei
 
谈谈缓存
谈谈缓存谈谈缓存
谈谈缓存Weng Wei
 
Why use Go for web development?
Why use Go for web development?Why use Go for web development?
Why use Go for web development?Weng Wei
 
Metaprogramming Go
Metaprogramming GoMetaprogramming Go
Metaprogramming GoWeng Wei
 
Migrate PHP E-Commerce Site to Go
Migrate PHP E-Commerce Site to GoMigrate PHP E-Commerce Site to Go
Migrate PHP E-Commerce Site to GoWeng Wei
 

Destaque (7)

我对ORM的思考
我对ORM的思考我对ORM的思考
我对ORM的思考
 
用PY实现"Go元编程"
用PY实现"Go元编程"用PY实现"Go元编程"
用PY实现"Go元编程"
 
About Caching
About CachingAbout Caching
About Caching
 
谈谈缓存
谈谈缓存谈谈缓存
谈谈缓存
 
Why use Go for web development?
Why use Go for web development?Why use Go for web development?
Why use Go for web development?
 
Metaprogramming Go
Metaprogramming GoMetaprogramming Go
Metaprogramming Go
 
Migrate PHP E-Commerce Site to Go
Migrate PHP E-Commerce Site to GoMigrate PHP E-Commerce Site to Go
Migrate PHP E-Commerce Site to Go
 

Semelhante a Sharding

Non-Relational Databases at ACCU2011
Non-Relational Databases at ACCU2011Non-Relational Databases at ACCU2011
Non-Relational Databases at ACCU2011Gavin Heavyside
 
Optimize MySQL performance for developers
Optimize MySQL performance for developersOptimize MySQL performance for developers
Optimize MySQL performance for developersjackbillow
 
Optimize MySQL For Developers-Qcon2011
Optimize MySQL For Developers-Qcon2011Optimize MySQL For Developers-Qcon2011
Optimize MySQL For Developers-Qcon2011Yiwei Ma
 
Big Data (NJ SQL Server User Group)
Big Data (NJ SQL Server User Group)Big Data (NJ SQL Server User Group)
Big Data (NJ SQL Server User Group)Don Demcsak
 
Big Data Platforms: An Overview
Big Data Platforms: An OverviewBig Data Platforms: An Overview
Big Data Platforms: An OverviewC. Scyphers
 
Cassandra an overview
Cassandra an overviewCassandra an overview
Cassandra an overviewPritamKathar
 
Cassandra Core Concepts
Cassandra Core ConceptsCassandra Core Concepts
Cassandra Core ConceptsJon Haddad
 
MySQL HA Sharding-Fabric
MySQL HA Sharding-FabricMySQL HA Sharding-Fabric
MySQL HA Sharding-FabricAbdul Manaf
 
MariaDB Server Compatibility with MySQL
MariaDB Server Compatibility with MySQLMariaDB Server Compatibility with MySQL
MariaDB Server Compatibility with MySQLColin Charles
 
Using MySQL Fabric for High Availability and Scaling Out
Using MySQL Fabric for High Availability and Scaling OutUsing MySQL Fabric for High Availability and Scaling Out
Using MySQL Fabric for High Availability and Scaling OutOSSCube
 
Cassandra Core Concepts - Cassandra Day Toronto
Cassandra Core Concepts - Cassandra Day TorontoCassandra Core Concepts - Cassandra Day Toronto
Cassandra Core Concepts - Cassandra Day TorontoJon Haddad
 
HBase Advanced - Lars George
HBase Advanced - Lars GeorgeHBase Advanced - Lars George
HBase Advanced - Lars GeorgeJAX London
 
HBase in Practice
HBase in PracticeHBase in Practice
HBase in Practicelarsgeorge
 

Semelhante a Sharding (20)

Voldemort Nosql
Voldemort NosqlVoldemort Nosql
Voldemort Nosql
 
Non-Relational Databases at ACCU2011
Non-Relational Databases at ACCU2011Non-Relational Databases at ACCU2011
Non-Relational Databases at ACCU2011
 
NoSQL
NoSQLNoSQL
NoSQL
 
Optimize MySQL performance for developers
Optimize MySQL performance for developersOptimize MySQL performance for developers
Optimize MySQL performance for developers
 
Optimize MySQL For Developers-Qcon2011
Optimize MySQL For Developers-Qcon2011Optimize MySQL For Developers-Qcon2011
Optimize MySQL For Developers-Qcon2011
 
Big Data (NJ SQL Server User Group)
Big Data (NJ SQL Server User Group)Big Data (NJ SQL Server User Group)
Big Data (NJ SQL Server User Group)
 
Big Data Platforms: An Overview
Big Data Platforms: An OverviewBig Data Platforms: An Overview
Big Data Platforms: An Overview
 
Cassandra an overview
Cassandra an overviewCassandra an overview
Cassandra an overview
 
How and when to use NoSQL
How and when to use NoSQLHow and when to use NoSQL
How and when to use NoSQL
 
Cassandra Core Concepts
Cassandra Core ConceptsCassandra Core Concepts
Cassandra Core Concepts
 
No SQL
No SQLNo SQL
No SQL
 
NoSql
NoSqlNoSql
NoSql
 
MySQL HA Sharding-Fabric
MySQL HA Sharding-FabricMySQL HA Sharding-Fabric
MySQL HA Sharding-Fabric
 
MariaDB Server Compatibility with MySQL
MariaDB Server Compatibility with MySQLMariaDB Server Compatibility with MySQL
MariaDB Server Compatibility with MySQL
 
Using MySQL Fabric for High Availability and Scaling Out
Using MySQL Fabric for High Availability and Scaling OutUsing MySQL Fabric for High Availability and Scaling Out
Using MySQL Fabric for High Availability and Scaling Out
 
Cassandra Core Concepts - Cassandra Day Toronto
Cassandra Core Concepts - Cassandra Day TorontoCassandra Core Concepts - Cassandra Day Toronto
Cassandra Core Concepts - Cassandra Day Toronto
 
Revision
RevisionRevision
Revision
 
HBase in Practice
HBase in Practice HBase in Practice
HBase in Practice
 
HBase Advanced - Lars George
HBase Advanced - Lars GeorgeHBase Advanced - Lars George
HBase Advanced - Lars George
 
HBase in Practice
HBase in PracticeHBase in Practice
HBase in Practice
 

Mais de Weng Wei

Go for web
Go for webGo for web
Go for webWeng Wei
 
Go for web
Go for webGo for web
Go for webWeng Wei
 
Python to go
Python to goPython to go
Python to goWeng Wei
 
论使用Python开发推荐引擎的优越性
论使用Python开发推荐引擎的优越性论使用Python开发推荐引擎的优越性
论使用Python开发推荐引擎的优越性Weng Wei
 
Zalora php to_go
Zalora php to_goZalora php to_go
Zalora php to_goWeng Wei
 
A byte of git
A byte of gitA byte of git
A byte of gitWeng Wei
 
My understanding of Git
My understanding of GitMy understanding of Git
My understanding of GitWeng Wei
 
Inside Zend Framework
Inside Zend FrameworkInside Zend Framework
Inside Zend FrameworkWeng Wei
 

Mais de Weng Wei (8)

Go for web
Go for webGo for web
Go for web
 
Go for web
Go for webGo for web
Go for web
 
Python to go
Python to goPython to go
Python to go
 
论使用Python开发推荐引擎的优越性
论使用Python开发推荐引擎的优越性论使用Python开发推荐引擎的优越性
论使用Python开发推荐引擎的优越性
 
Zalora php to_go
Zalora php to_goZalora php to_go
Zalora php to_go
 
A byte of git
A byte of gitA byte of git
A byte of git
 
My understanding of Git
My understanding of GitMy understanding of Git
My understanding of Git
 
Inside Zend Framework
Inside Zend FrameworkInside Zend Framework
Inside Zend Framework
 

Sharding

Notas do Editor

  1. \n
  2. \n
  3. \n
  4. \n
  5. \n
  6. \n
  7. \n
  8. \n
  9. \n
  10. \n
  11. \n
  12. \n
  13. \n
  14. \n
  15. \n
  16. \n
  17. \n
  18. \n