高度に進化した分散データストアについて。Incline と Pacific の背景にある考え方を説明

1. Cybozu Labs, Inc.
Kazuho Oku

2.  Oracle (RAC)
 force.com (Salesforce)
 GAE DataStore
 Kumofs, hBase, etc.
Q. Oracle ?
Nov 30 2009 2

3. PHP RDBMS
 RDBMS
 OSS RDBMS ?
 MySQL Cluster?
 Spider Storage Engine?
 Oracle OSS ?
 KVS RDBMS
 Salesforce GAE DataStore SQL-like
Nov 30 2009 3

4. MySQL (InnoDB)
 SELECT
 max. 40k QPS
 memcached 1/5

 OS
 KVS
 SQL ACID


Nov 30 2009 4

5.  ACID








Nov 30 2009 5

6. (SNS )




 :
 (JOIN )


Nov 30 2009 6

7. Incline & Pacific



 (eventual) consistency

 SPOF Proxy-less Daemon-less
OSS RDBMS
Nov 30 2009 7

8. Incline –
 RDB Sharding
 shard
Incline
Nov 30 2009 8

9. Incline –
 DDL

 DDL
 shard
 ACID

Nov 30 2009 9

10. Incline – (1)
 JOIN
uid:1-2000
uid:2001-4000
uid:4001-6000
tweet
tweet
tweet
following
following
following
followed_by
followed_by
followed_by
...
timeline
timeline
timeline
queue
queue
queue
when uid:123 tweets, write only to his tweet table. Incline updates
other tables automatically
Nov 30 2009 10

11. Incline – (2)

uid:1-2000
uid:2001-4000
uid:4001-6000
tweet
tweet
tweet
following
following
following
followed_by
followed_by
followed_by
...
timeline
timeline
timeline
queue
queue
queue
when uid:2431 starts following uid:940 only write to his following table
Nov 30 2009 11

12. Pacific –

 RDBMS



 < 10
 SPOF

Nov 30 2009 12

13. Incline & Pacific
 Incline

 MySQL PostgreSQL
 Windows
 Pacific

 see developer.cybozu.co.jp/kazuho
Nov 30 2009 13

14. 
 Incline Pacific
 DBIx::ShardManager

Nov 30 2009 14