O slideshow foi denunciado.
Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.
Consistency without consensus:

CRDTs in production at SoundCloud
InfoQ.com: News & Community Site
• 750,000 unique visitors/month
• Published in 4 languages (English, Chinese, Japanese an...
Purpose of QCon
- to empower software development by facilitating the spread of
knowledge and innovation
Strategy
- practi...
Consistency without consensus:

CRDTs in production at SoundCloud
Me
Some guy
Embedded, sensor networks
Distributed systems
SoundCloud infrastructure
Theory
Distributed programming
“The art of solving the same problem that 

you can solve on a single computer

using multiple com...
Distributed programming
“Generally a bad idea,

best avoided.”

	 	 	 	 	 	 	 	 —me
>>> x = 1
>>> print x
1
$ curl -XPOST -d'{"val": 1}' http://db/vars/x
HTTP 502 Bad Gateway
$ curl -XGET http://db/vars/x
HTTP 503 Service Unavaila...
Idioms
1980s — RPC
1990s — CORBA
2000 — CAP
Partition tolerance
“The system continues to operate
despite message loss due to network
and/or node failure.”

	 	 	 	 	 ...
Partition

tolerance
Consistency Availability
✗
CP AP
CP
Chubby, Doozer — Paxos
ZooKeeper — Zab
Consul, etcd — Raft
? — Viewstamped Replication
AP
Cassandra
Riak
Mongo
Couch
Message failure
Delayed
Dropped
Delivered out-of-order
Duplicated
CALM principle
Consistency
As
Logical
Monotonicity
ACID 2.0
Associative
Commutative
Idempotent
Distributed, sure, whatever
CRDT
Conflict-free
Replicated
Data
Type
Increment-only

counter
A'
A → A'
+
(1 + 2) + 3 = 1 + (2 + 3)
1 + 2 = 2 + 1
1 + 1 ≠ 1
∪
( 1 ∪ 2 ) ∪ 3 = 1 ∪ ( 2 ∪ 3 )
1 ∪ 2 = 2 ∪ 1
1 ∪ 1 = 1
{ }
{ }
{ }
{ }
{ }
{ }
123
{ }
{ }
{ }
123
123
{ }
{ }
{ }
123
123
{ }
{ }
{ }
123
123
123
{ }
{ }
{ }
123
123
123
{ }
{ }
{ }
123
123
123
456
{ }
{ }
{ }
123
123
123, 456
456
{ }
{ }
{ }
123
123
123, 456
✘
{ }
{ }
{ }
123
123, 456
123, 456
{ }
{ }
{ }
123
123, 456
123, 456
Read
{123, 456} ∪ {123} ∪ {123, 456} = {123, 456}

{123, 456} ∆ {123} ∆ {123, 456} = {456}
{ }
{ }
{ }
123
123, 456
123, 456
456
{ }
{ }
{ }
123, 456
123, 456
123, 456
{ }
{ }
{ }
123, 456
123, 456
123, 456
{ }
{ }
{ }
123, 456
123, 456
123, 456
Interlude —

Bending the problem
CRDTs in production
Event
Timestamp
Actor
Verb
Thing
Event
2014-04-01T15:16:17.187Z
snoopdogg
reposted
theeconomist/election-day
Fan out on write
༼⍢༽ ●
●●●● (•ི̛ᴗ•̛)ྀ
●●● ༼•͟ ͜•༽
●●●●● (ಠ_ಠ)
●
●
●
●
Fan in on read
༼⍢༽ •••••
(•ི̛ᴗ•̛)ྀ
༼•͟ ͜•༽
(ಠ_ಠ)
[¬º-°] •••
••••••
Unique events — use a set
G-set — can’t delete
2P-set — add, remove once
OR-set — storage overhead
A wild set appears
Roshi set
S+ { A/1 B/2 C/3 }
S– { D/4 }
S { A B C }
Roshi set
S = actor’s outbox key

snoopdogg·outbox
A/B/C/D = actor+verb+thing

snoopdogg·repost·theeconomist/election-day
...
Reading is easy
Writing is interesting
Insert
• If either key+ or key– already contains element,

and the existing score >= score,

no-op and exit.
• Insert (ele...
Delete
• If either key+ or key– already contains element,

and the existing score >= score,

no-op and exit.
• Insert (ele...
Example
S+ { A/1 B/2 }
S– { C/3 }
S+ { A/1 B/2 }
S– { C/3 }
Insert D/4
S+ { A/1 B/2 D/4 }
S– { C/3 }
Insert D/4
S+ { A/1 B/2 D/4 }
S– { C/3 }
S+ { A/1 B/2 D/4 }
S– { C/3 }
Insert D/4
S+ { A/1 B/2 D/4 }
S– { C/3 }
Insert D/4
S+ { A/1 B/2 D/4 }
S– { C/3 }
S+ { A/1 B/2 D/4 }
S– { C/3 }
Delete D/3
S+ { A/1 B/2 D/4 }
S– { C/3 }
Delete D/3
S+ { A/1 B/2 D/4 }
S– { C/3 }
S+ { A/1 B/2 D/4 }
S– { C/3 }
Delete D/5
S+ { A/1 B/2 D/4 }
S– { C/3 D/5 }
Delete D/5
S+ { A/1 B/2 }
S– { C/3 D/5 }
S+ { A/1 B/2 }
S– { C/3 D/5 }
Delete D/6
S+ { A/1 B/2 }
S– { C/3 D/6 }
Delete D/6
S+ { A/1 B/2 }
S– { C/3 D/6 }
Making it real
Pool
Cluster
Pool
Cluster
Pool
Cluster
Pool
Cluster
Farm
Writing is easy
Reading is interesting
Pool
Cluster
Pool
Cluster
Pool
Cluster
Farm
Pool
Cluster
Pool
Cluster
Pool
Cluster
Farm
ClusterCluster Cluster
{A B C} {A C} {A B C}
∪ = {A B C}
∆ = {B}
Pool
Cluster
Pool
Cluster
Pool
Cluster
Farm
github.com/soundcloud/roshi
In conclusion,
Consistency without consensus = CRDT.

Embrace your invariants.

Maybe bend your problem, not your solution.
Thanks!
soundcloud.com/jobs
@peterbourgon
☞ ☜
Watch the video with slide synchronization on
InfoQ.com!
http://www.infoq.com/presentations/crdt-
soundcloud
Consistency without Consensus: CRDTs in Production at SoundCloud
Consistency without Consensus: CRDTs in Production at SoundCloud
Próximos SlideShares
Carregando em…5
×

Consistency without Consensus: CRDTs in Production at SoundCloud

1.771 visualizações

Publicada em

Video and slides synchronized, mp3 and slide download available at URL http://bit.ly/1DKnwXr.

Peter Bourgon provides a practical introduction to Conflict-free Replicated Data Types (CRDTs) and describes a production CRDT system built at SoundCloud to serve several product features. Filmed at qconsf.com.

Peter Bourgon is a distributed systems engineer who has seen things. He works at SoundCloud, building and improving the infrastructure that powers the world's largest audio platform.

Publicada em: Tecnologia
  • Seja o primeiro a comentar

Consistency without Consensus: CRDTs in Production at SoundCloud

  1. 1. Consistency without consensus:
 CRDTs in production at SoundCloud
  2. 2. InfoQ.com: News & Community Site • 750,000 unique visitors/month • Published in 4 languages (English, Chinese, Japanese and Brazilian Portuguese) • Post content from our QCon conferences • News 15-20 / week • Articles 3-4 / week • Presentations (videos) 12-15 / week • Interviews 2-3 / week • Books 1 / month Watch the video with slide synchronization on InfoQ.com! http://www.infoq.com/presentations /crdt-soundcloud
  3. 3. Purpose of QCon - to empower software development by facilitating the spread of knowledge and innovation Strategy - practitioner-driven conference designed for YOU: influencers of change and innovation in your teams - speakers and topics driving the evolution and innovation - connecting and catalyzing the influencers and innovators Highlights - attended by more than 12,000 delegates since 2007 - held in 9 cities worldwide Presented at QCon San Francisco www.qconsf.com
  4. 4. Consistency without consensus:
 CRDTs in production at SoundCloud
  5. 5. Me Some guy Embedded, sensor networks Distributed systems SoundCloud infrastructure
  6. 6. Theory
  7. 7. Distributed programming “The art of solving the same problem that 
 you can solve on a single computer
 using multiple computers.”
 —book.mixu.net
  8. 8. Distributed programming “Generally a bad idea,
 best avoided.”
 —me
  9. 9. >>> x = 1 >>> print x 1
  10. 10. $ curl -XPOST -d'{"val": 1}' http://db/vars/x HTTP 502 Bad Gateway $ curl -XGET http://db/vars/x HTTP 503 Service Unavailable
  11. 11. Idioms
  12. 12. 1980s — RPC
  13. 13. 1990s — CORBA
  14. 14. 2000 — CAP
  15. 15. Partition tolerance “The system continues to operate despite message loss due to network and/or node failure.”
 —book.mixu.net
  16. 16. Partition
 tolerance Consistency Availability ✗ CP AP
  17. 17. CP Chubby, Doozer — Paxos ZooKeeper — Zab Consul, etcd — Raft ? — Viewstamped Replication
  18. 18. AP Cassandra Riak Mongo Couch
  19. 19. Message failure Delayed Dropped Delivered out-of-order Duplicated
  20. 20. CALM principle Consistency As Logical Monotonicity
  21. 21. ACID 2.0 Associative Commutative Idempotent Distributed, sure, whatever
  22. 22. CRDT Conflict-free Replicated Data Type
  23. 23. Increment-only
 counter
  24. 24. A' A → A'
  25. 25. + (1 + 2) + 3 = 1 + (2 + 3) 1 + 2 = 2 + 1 1 + 1 ≠ 1
  26. 26. ∪ ( 1 ∪ 2 ) ∪ 3 = 1 ∪ ( 2 ∪ 3 ) 1 ∪ 2 = 2 ∪ 1 1 ∪ 1 = 1
  27. 27. { } { } { }
  28. 28. { } { } { } 123
  29. 29. { } { } { } 123 123
  30. 30. { } { } { } 123 123
  31. 31. { } { } { } 123 123 123
  32. 32. { } { } { } 123 123 123
  33. 33. { } { } { } 123 123 123 456
  34. 34. { } { } { } 123 123 123, 456 456
  35. 35. { } { } { } 123 123 123, 456 ✘
  36. 36. { } { } { } 123 123, 456 123, 456
  37. 37. { } { } { } 123 123, 456 123, 456
  38. 38. Read {123, 456} ∪ {123} ∪ {123, 456} = {123, 456}
 {123, 456} ∆ {123} ∆ {123, 456} = {456}
  39. 39. { } { } { } 123 123, 456 123, 456 456
  40. 40. { } { } { } 123, 456 123, 456 123, 456
  41. 41. { } { } { } 123, 456 123, 456 123, 456
  42. 42. { } { } { } 123, 456 123, 456 123, 456
  43. 43. Interlude —
 Bending the problem
  44. 44. CRDTs in production
  45. 45. Event Timestamp Actor Verb Thing
  46. 46. Event 2014-04-01T15:16:17.187Z snoopdogg reposted theeconomist/election-day
  47. 47. Fan out on write ༼⍢༽ ● ●●●● (•ི̛ᴗ•̛)ྀ ●●● ༼•͟ ͜•༽ ●●●●● (ಠ_ಠ) ● ● ● ●
  48. 48. Fan in on read ༼⍢༽ ••••• (•ི̛ᴗ•̛)ྀ ༼•͟ ͜•༽ (ಠ_ಠ) [¬º-°] ••• ••••••
  49. 49. Unique events — use a set G-set — can’t delete 2P-set — add, remove once OR-set — storage overhead
  50. 50. A wild set appears
  51. 51. Roshi set S+ { A/1 B/2 C/3 } S– { D/4 } S { A B C }
  52. 52. Roshi set S = actor’s outbox key
 snoopdogg·outbox A/B/C/D = actor+verb+thing
 snoopdogg·repost·theeconomist/election-day 1/2/3 = timestamp
 2014-04-01T15:16:17.187Z
  53. 53. Reading is easy
  54. 54. Writing is interesting
  55. 55. Insert • If either key+ or key– already contains element,
 and the existing score >= score,
 no-op and exit. • Insert (element, score) into add set key+. • Delete (element) from remove set key–.
  56. 56. Delete • If either key+ or key– already contains element,
 and the existing score >= score,
 no-op and exit. • Insert (element, score) into add set key–. • Delete (element) from remove set key+.
  57. 57. Example
  58. 58. S+ { A/1 B/2 } S– { C/3 }
  59. 59. S+ { A/1 B/2 } S– { C/3 } Insert D/4
  60. 60. S+ { A/1 B/2 D/4 } S– { C/3 } Insert D/4
  61. 61. S+ { A/1 B/2 D/4 } S– { C/3 }
  62. 62. S+ { A/1 B/2 D/4 } S– { C/3 } Insert D/4
  63. 63. S+ { A/1 B/2 D/4 } S– { C/3 } Insert D/4
  64. 64. S+ { A/1 B/2 D/4 } S– { C/3 }
  65. 65. S+ { A/1 B/2 D/4 } S– { C/3 } Delete D/3
  66. 66. S+ { A/1 B/2 D/4 } S– { C/3 } Delete D/3
  67. 67. S+ { A/1 B/2 D/4 } S– { C/3 }
  68. 68. S+ { A/1 B/2 D/4 } S– { C/3 } Delete D/5
  69. 69. S+ { A/1 B/2 D/4 } S– { C/3 D/5 } Delete D/5
  70. 70. S+ { A/1 B/2 } S– { C/3 D/5 }
  71. 71. S+ { A/1 B/2 } S– { C/3 D/5 } Delete D/6
  72. 72. S+ { A/1 B/2 } S– { C/3 D/6 } Delete D/6
  73. 73. S+ { A/1 B/2 } S– { C/3 D/6 }
  74. 74. Making it real
  75. 75. Pool Cluster
  76. 76. Pool Cluster Pool Cluster Pool Cluster Farm
  77. 77. Writing is easy
  78. 78. Reading is interesting
  79. 79. Pool Cluster Pool Cluster Pool Cluster Farm
  80. 80. Pool Cluster Pool Cluster Pool Cluster Farm
  81. 81. ClusterCluster Cluster {A B C} {A C} {A B C} ∪ = {A B C} ∆ = {B}
  82. 82. Pool Cluster Pool Cluster Pool Cluster Farm
  83. 83. github.com/soundcloud/roshi
  84. 84. In conclusion,
  85. 85. Consistency without consensus = CRDT.
 Embrace your invariants.
 Maybe bend your problem, not your solution.
  86. 86. Thanks! soundcloud.com/jobs @peterbourgon ☞ ☜
  87. 87. Watch the video with slide synchronization on InfoQ.com! http://www.infoq.com/presentations/crdt- soundcloud

×