SlideShare uma empresa Scribd logo
1 de 15
Timothy Perrett
Bath Camp 2010
What is Riak?
• Documented orientated database
• Written in Erlang
• Based on Dynamo[1] and CAP Theorem[2]
• Highly fault tolerant
• HTTP and ProtoBuff interface
• Write MapReduce in Erlang or JavaScript
1. http://goo.gl/r8Np
2. http://www.julianbrowne.com/article/viewer/brewers-cap-theorem
Same, Same but different
• Riak solves similar problems to MongoDB
• Semi-structured data modeled as "documents”
• Storage of non-document data in the database
• High write-availability
• Riak is intrinsically multi-node scalable
• Mongo in comparison is single system (+ sharding)
• Riak achieves availability via quorum writes
• Mongo uses performant in-place writes
• Riak uses “masterless” replication
N/R/W – Dynamo
N = Number of replicas to store
R = Number of replicas needed to read
W = Number of replicas needed to read
• These principals first appeared in an Amazon
research paper known as Dynamo
• 160bit integer key
space. Each node that
joins is assigned part
of that space for
consistent hashing
• Hashing means any
node can service any
request making the
cluster masterless and
eventually consistant
Number of replicas
• Number of replies
before Riak gives
the client a
successful reply.
• Tries to access all
nodes, but as soon
as the N/R is
satisfied a response
is given
Reads
• Same as reads; W
implies the number
of successful nodes
that must reply
before the write
is considered
consistent by
the client
Writes
Extreme example
• Given N=10, R=W=2 we
could have 8 nodes
down and the cluster
would still be fully
available to all clients
What does this all mean?
• N/R/W specified at request time, so each
client can specify its own tolerance for
outages dynamically
• Despite any outages within the cluster, the whole
cluster can still appear available based on N/R/W
• Given N=3 and R=W=2, we can have 3-2=1 node
down/unreachable/laggy in the cluster
• Stupidly high availability complete with eventual
consistency controlled by dynamic clients
Brewer’s CAP Theorem
• Consistency
• Availability
• Partition Tolerance
• You cant have all things, all the time…
• …but you can have some of each, all the time!
• Riak is about choosing your own levels of
each according to your use case
Consistency
• Start with document
version zero
• Things get redistributed
and n0 and n2 are
sitting in NYC and n1
and n3 are in London
• What if stuff changes??
Consistency
• Uh oh: inconsistency
• Both parts of the cluster
are still fully available
• NYC serves v1 whilst
London serves v0
• The network resumes
and Riak determines
the latest version by
using vector clocks
Consistency
• What if both sides of
the Atlantic changed?
• Riak is unable to
determine which is the
right document, both
are returned to the
client with an indication
of the inconsistency
• Distributed, fault-tolerant full-text searching
• Lucene syntax for queries
• No need for index sharding
• Linier scaling
• Double the number of nodes to get double the
search capacity (awesome!)
• Search via:
• Fields, wildcards, fuzzy text or token proximity
Riak Search
Questions?
basho.com/riak.html
github.com/basho/riak
twitter.com/timperrett
github.com/timperrett
blog.getintheloop.eu

Mais conteúdo relacionado

Mais procurados

Flink Forward San Francisco 2019: Scaling a real-time streaming warehouse wit...
Flink Forward San Francisco 2019: Scaling a real-time streaming warehouse wit...Flink Forward San Francisco 2019: Scaling a real-time streaming warehouse wit...
Flink Forward San Francisco 2019: Scaling a real-time streaming warehouse wit...Flink Forward
 
Flink Forward Berlin 2017: Dominik Bruhn - Deploying Flink Jobs as Docker Con...
Flink Forward Berlin 2017: Dominik Bruhn - Deploying Flink Jobs as Docker Con...Flink Forward Berlin 2017: Dominik Bruhn - Deploying Flink Jobs as Docker Con...
Flink Forward Berlin 2017: Dominik Bruhn - Deploying Flink Jobs as Docker Con...Flink Forward
 
Aljoscha Krettek - Portable stateful big data processing in Apache Beam
Aljoscha Krettek - Portable stateful big data processing in Apache BeamAljoscha Krettek - Portable stateful big data processing in Apache Beam
Aljoscha Krettek - Portable stateful big data processing in Apache BeamVerverica
 
Building your own Distributed System The easy way - Cassandra Summit EU 2014
Building your own Distributed System The easy way - Cassandra Summit EU 2014Building your own Distributed System The easy way - Cassandra Summit EU 2014
Building your own Distributed System The easy way - Cassandra Summit EU 2014Kévin LOVATO
 
How to manage large amounts of data with akka streams
How to manage large amounts of data with akka streamsHow to manage large amounts of data with akka streams
How to manage large amounts of data with akka streamsIgor Mielientiev
 
ChronoLogic Tools Demo: 6/12/18
ChronoLogic Tools Demo: 6/12/18ChronoLogic Tools Demo: 6/12/18
ChronoLogic Tools Demo: 6/12/18ChronoLogic
 
Kafka At Scale in the Cloud
Kafka At Scale in the CloudKafka At Scale in the Cloud
Kafka At Scale in the Cloudconfluent
 
Thoughts on consistency models
Thoughts on consistency modelsThoughts on consistency models
Thoughts on consistency modelsrogerbodamer
 
Virtual Flink Forward 2020: Autoscaling Flink at Netflix - Timothy Farkas
Virtual Flink Forward 2020: Autoscaling Flink at Netflix - Timothy FarkasVirtual Flink Forward 2020: Autoscaling Flink at Netflix - Timothy Farkas
Virtual Flink Forward 2020: Autoscaling Flink at Netflix - Timothy FarkasFlink Forward
 
Parallel processing for splitter in mule esb
Parallel processing for splitter in mule esbParallel processing for splitter in mule esb
Parallel processing for splitter in mule esbSunil Kumar
 
Alexander Kolb – Flink. Yet another Streaming Framework?
Alexander Kolb – Flink. Yet another Streaming Framework?Alexander Kolb – Flink. Yet another Streaming Framework?
Alexander Kolb – Flink. Yet another Streaming Framework?Flink Forward
 
Fabian Hueske_Till Rohrmann - Declarative stream processing with StreamSQL an...
Fabian Hueske_Till Rohrmann - Declarative stream processing with StreamSQL an...Fabian Hueske_Till Rohrmann - Declarative stream processing with StreamSQL an...
Fabian Hueske_Till Rohrmann - Declarative stream processing with StreamSQL an...Flink Forward
 
Flink Forward SF 2017: Till Rohrmann - Redesigning Apache Flink’s Distributed...
Flink Forward SF 2017: Till Rohrmann - Redesigning Apache Flink’s Distributed...Flink Forward SF 2017: Till Rohrmann - Redesigning Apache Flink’s Distributed...
Flink Forward SF 2017: Till Rohrmann - Redesigning Apache Flink’s Distributed...Flink Forward
 
Flink Forward Berlin 2017: Andreas Kunft - Efficiently executing R Dataframes...
Flink Forward Berlin 2017: Andreas Kunft - Efficiently executing R Dataframes...Flink Forward Berlin 2017: Andreas Kunft - Efficiently executing R Dataframes...
Flink Forward Berlin 2017: Andreas Kunft - Efficiently executing R Dataframes...Flink Forward
 
Introduction to Structured streaming
Introduction to Structured streamingIntroduction to Structured streaming
Introduction to Structured streamingdatamantra
 
Matthias J. Sax – A Tale of Squirrels and Storms
Matthias J. Sax – A Tale of Squirrels and StormsMatthias J. Sax – A Tale of Squirrels and Storms
Matthias J. Sax – A Tale of Squirrels and StormsFlink Forward
 
Flink Forward SF 2017: Stephan Ewen - Convergence of real-time analytics and ...
Flink Forward SF 2017: Stephan Ewen - Convergence of real-time analytics and ...Flink Forward SF 2017: Stephan Ewen - Convergence of real-time analytics and ...
Flink Forward SF 2017: Stephan Ewen - Convergence of real-time analytics and ...Flink Forward
 
Apache Software Foundation: How To Contribute, with Apache Flink as Example (...
Apache Software Foundation: How To Contribute, with Apache Flink as Example (...Apache Software Foundation: How To Contribute, with Apache Flink as Example (...
Apache Software Foundation: How To Contribute, with Apache Flink as Example (...Apache Flink Taiwan User Group
 

Mais procurados (20)

Project Reactor By Example
Project Reactor By ExampleProject Reactor By Example
Project Reactor By Example
 
Flink Forward San Francisco 2019: Scaling a real-time streaming warehouse wit...
Flink Forward San Francisco 2019: Scaling a real-time streaming warehouse wit...Flink Forward San Francisco 2019: Scaling a real-time streaming warehouse wit...
Flink Forward San Francisco 2019: Scaling a real-time streaming warehouse wit...
 
Flink Forward Berlin 2017: Dominik Bruhn - Deploying Flink Jobs as Docker Con...
Flink Forward Berlin 2017: Dominik Bruhn - Deploying Flink Jobs as Docker Con...Flink Forward Berlin 2017: Dominik Bruhn - Deploying Flink Jobs as Docker Con...
Flink Forward Berlin 2017: Dominik Bruhn - Deploying Flink Jobs as Docker Con...
 
Aljoscha Krettek - Portable stateful big data processing in Apache Beam
Aljoscha Krettek - Portable stateful big data processing in Apache BeamAljoscha Krettek - Portable stateful big data processing in Apache Beam
Aljoscha Krettek - Portable stateful big data processing in Apache Beam
 
Building your own Distributed System The easy way - Cassandra Summit EU 2014
Building your own Distributed System The easy way - Cassandra Summit EU 2014Building your own Distributed System The easy way - Cassandra Summit EU 2014
Building your own Distributed System The easy way - Cassandra Summit EU 2014
 
How to manage large amounts of data with akka streams
How to manage large amounts of data with akka streamsHow to manage large amounts of data with akka streams
How to manage large amounts of data with akka streams
 
ChronoLogic Tools Demo: 6/12/18
ChronoLogic Tools Demo: 6/12/18ChronoLogic Tools Demo: 6/12/18
ChronoLogic Tools Demo: 6/12/18
 
Kafka At Scale in the Cloud
Kafka At Scale in the CloudKafka At Scale in the Cloud
Kafka At Scale in the Cloud
 
Thoughts on consistency models
Thoughts on consistency modelsThoughts on consistency models
Thoughts on consistency models
 
Virtual Flink Forward 2020: Autoscaling Flink at Netflix - Timothy Farkas
Virtual Flink Forward 2020: Autoscaling Flink at Netflix - Timothy FarkasVirtual Flink Forward 2020: Autoscaling Flink at Netflix - Timothy Farkas
Virtual Flink Forward 2020: Autoscaling Flink at Netflix - Timothy Farkas
 
Parallel processing for splitter in mule esb
Parallel processing for splitter in mule esbParallel processing for splitter in mule esb
Parallel processing for splitter in mule esb
 
Alexander Kolb – Flink. Yet another Streaming Framework?
Alexander Kolb – Flink. Yet another Streaming Framework?Alexander Kolb – Flink. Yet another Streaming Framework?
Alexander Kolb – Flink. Yet another Streaming Framework?
 
Fabian Hueske_Till Rohrmann - Declarative stream processing with StreamSQL an...
Fabian Hueske_Till Rohrmann - Declarative stream processing with StreamSQL an...Fabian Hueske_Till Rohrmann - Declarative stream processing with StreamSQL an...
Fabian Hueske_Till Rohrmann - Declarative stream processing with StreamSQL an...
 
Flink Forward SF 2017: Till Rohrmann - Redesigning Apache Flink’s Distributed...
Flink Forward SF 2017: Till Rohrmann - Redesigning Apache Flink’s Distributed...Flink Forward SF 2017: Till Rohrmann - Redesigning Apache Flink’s Distributed...
Flink Forward SF 2017: Till Rohrmann - Redesigning Apache Flink’s Distributed...
 
Flink Forward Berlin 2017: Andreas Kunft - Efficiently executing R Dataframes...
Flink Forward Berlin 2017: Andreas Kunft - Efficiently executing R Dataframes...Flink Forward Berlin 2017: Andreas Kunft - Efficiently executing R Dataframes...
Flink Forward Berlin 2017: Andreas Kunft - Efficiently executing R Dataframes...
 
Introduction to Structured streaming
Introduction to Structured streamingIntroduction to Structured streaming
Introduction to Structured streaming
 
Matthias J. Sax – A Tale of Squirrels and Storms
Matthias J. Sax – A Tale of Squirrels and StormsMatthias J. Sax – A Tale of Squirrels and Storms
Matthias J. Sax – A Tale of Squirrels and Storms
 
Notes on Netty baics
Notes on Netty baicsNotes on Netty baics
Notes on Netty baics
 
Flink Forward SF 2017: Stephan Ewen - Convergence of real-time analytics and ...
Flink Forward SF 2017: Stephan Ewen - Convergence of real-time analytics and ...Flink Forward SF 2017: Stephan Ewen - Convergence of real-time analytics and ...
Flink Forward SF 2017: Stephan Ewen - Convergence of real-time analytics and ...
 
Apache Software Foundation: How To Contribute, with Apache Flink as Example (...
Apache Software Foundation: How To Contribute, with Apache Flink as Example (...Apache Software Foundation: How To Contribute, with Apache Flink as Example (...
Apache Software Foundation: How To Contribute, with Apache Flink as Example (...
 

Destaque

Actividad1 curriculum
Actividad1 curriculumActividad1 curriculum
Actividad1 curriculumLuzfrida
 
Lectura ser digital
Lectura ser digitalLectura ser digital
Lectura ser digitalLUISFDAVILA
 
Initial presentation Tesla management project (Swinburne University)
Initial presentation Tesla management project (Swinburne University)Initial presentation Tesla management project (Swinburne University)
Initial presentation Tesla management project (Swinburne University)Anthony Campana
 
Gas licuado de petróleo GLP
Gas licuado de petróleo GLPGas licuado de petróleo GLP
Gas licuado de petróleo GLPHéctor Chire
 
Condylar fractures /certified fixed orthodontic courses by Indian dental acad...
Condylar fractures /certified fixed orthodontic courses by Indian dental acad...Condylar fractures /certified fixed orthodontic courses by Indian dental acad...
Condylar fractures /certified fixed orthodontic courses by Indian dental acad...Indian dental academy
 
Mandibular fracture 2 / fixed orthodontic courses
Mandibular fracture 2 / fixed orthodontic coursesMandibular fracture 2 / fixed orthodontic courses
Mandibular fracture 2 / fixed orthodontic coursesIndian dental academy
 
Mod morphology of deciduous dentition
Mod morphology of deciduous dentitionMod morphology of deciduous dentition
Mod morphology of deciduous dentitionJamil Kifayatullah
 
Phần 1 công ty kết cấu sx thép thái nguyên
Phần 1   công ty kết cấu sx thép thái nguyênPhần 1   công ty kết cấu sx thép thái nguyên
Phần 1 công ty kết cấu sx thép thái nguyêntranthihoaivan
 
A2 Media Studies Preproduction Development
A2 Media Studies Preproduction DevelopmentA2 Media Studies Preproduction Development
A2 Media Studies Preproduction DevelopmentGeorginaMediaStudies
 
Feuillet memento Degremont - n°1 Ultragreen
Feuillet memento Degremont - n°1 UltragreenFeuillet memento Degremont - n°1 Ultragreen
Feuillet memento Degremont - n°1 UltragreenDegrémont
 

Destaque (19)

Electronica jj
Electronica jjElectronica jj
Electronica jj
 
Actividad1 curriculum
Actividad1 curriculumActividad1 curriculum
Actividad1 curriculum
 
ville du mont-dore
ville du mont-doreville du mont-dore
ville du mont-dore
 
Que es slideshare
Que es slideshareQue es slideshare
Que es slideshare
 
CSR Events at Technika 15
CSR Events at Technika 15CSR Events at Technika 15
CSR Events at Technika 15
 
Lectura ser digital
Lectura ser digitalLectura ser digital
Lectura ser digital
 
Exposicion equipo 1
Exposicion equipo 1Exposicion equipo 1
Exposicion equipo 1
 
P
PP
P
 
Initial presentation Tesla management project (Swinburne University)
Initial presentation Tesla management project (Swinburne University)Initial presentation Tesla management project (Swinburne University)
Initial presentation Tesla management project (Swinburne University)
 
Wiring of-mandible
Wiring of-mandibleWiring of-mandible
Wiring of-mandible
 
Gas licuado de petróleo GLP
Gas licuado de petróleo GLPGas licuado de petróleo GLP
Gas licuado de petróleo GLP
 
Condylar fractures /certified fixed orthodontic courses by Indian dental acad...
Condylar fractures /certified fixed orthodontic courses by Indian dental acad...Condylar fractures /certified fixed orthodontic courses by Indian dental acad...
Condylar fractures /certified fixed orthodontic courses by Indian dental acad...
 
Favolare
FavolareFavolare
Favolare
 
Mandibular fracture 2 / fixed orthodontic courses
Mandibular fracture 2 / fixed orthodontic coursesMandibular fracture 2 / fixed orthodontic courses
Mandibular fracture 2 / fixed orthodontic courses
 
Mod morphology of deciduous dentition
Mod morphology of deciduous dentitionMod morphology of deciduous dentition
Mod morphology of deciduous dentition
 
Phần 1 công ty kết cấu sx thép thái nguyên
Phần 1   công ty kết cấu sx thép thái nguyênPhần 1   công ty kết cấu sx thép thái nguyên
Phần 1 công ty kết cấu sx thép thái nguyên
 
A2 Media Studies Preproduction Development
A2 Media Studies Preproduction DevelopmentA2 Media Studies Preproduction Development
A2 Media Studies Preproduction Development
 
Human Dentition
Human DentitionHuman Dentition
Human Dentition
 
Feuillet memento Degremont - n°1 Ultragreen
Feuillet memento Degremont - n°1 UltragreenFeuillet memento Degremont - n°1 Ultragreen
Feuillet memento Degremont - n°1 Ultragreen
 

Semelhante a Bathcamp 2010-riak

Scalable Persistent Storage for Erlang: Theory and Practice
Scalable Persistent Storage for Erlang: Theory and PracticeScalable Persistent Storage for Erlang: Theory and Practice
Scalable Persistent Storage for Erlang: Theory and PracticeAmir Ghaffari
 
HPC Controls Future
HPC Controls FutureHPC Controls Future
HPC Controls Futurercastain
 
Getting started with Riak in the Cloud
Getting started with Riak in the CloudGetting started with Riak in the Cloud
Getting started with Riak in the CloudInes Sombra
 
Running a distributed system across kubernetes clusters - Kubecon North Ameri...
Running a distributed system across kubernetes clusters - Kubecon North Ameri...Running a distributed system across kubernetes clusters - Kubecon North Ameri...
Running a distributed system across kubernetes clusters - Kubecon North Ameri...Alex Robinson
 
Multi-Datacenter Kafka - Strata San Jose 2017
Multi-Datacenter Kafka - Strata San Jose 2017Multi-Datacenter Kafka - Strata San Jose 2017
Multi-Datacenter Kafka - Strata San Jose 2017Gwen (Chen) Shapira
 
Highly available, scalable and secure data with Cassandra and DataStax Enterp...
Highly available, scalable and secure data with Cassandra and DataStax Enterp...Highly available, scalable and secure data with Cassandra and DataStax Enterp...
Highly available, scalable and secure data with Cassandra and DataStax Enterp...Johnny Miller
 
Lessons learned from scaling YARN to 40K machines in a multi tenancy environment
Lessons learned from scaling YARN to 40K machines in a multi tenancy environmentLessons learned from scaling YARN to 40K machines in a multi tenancy environment
Lessons learned from scaling YARN to 40K machines in a multi tenancy environmentDataWorks Summit
 
[EUC2016] DockerCap: a software-level power capping orchestrator for Docker c...
[EUC2016] DockerCap: a software-level power capping orchestrator for Docker c...[EUC2016] DockerCap: a software-level power capping orchestrator for Docker c...
[EUC2016] DockerCap: a software-level power capping orchestrator for Docker c...Matteo Ferroni
 
Jay Kreps on Project Voldemort Scaling Simple Storage At LinkedIn
Jay Kreps on Project Voldemort Scaling Simple Storage At LinkedInJay Kreps on Project Voldemort Scaling Simple Storage At LinkedIn
Jay Kreps on Project Voldemort Scaling Simple Storage At LinkedInLinkedIn
 
Tuning kafka pipelines
Tuning kafka pipelinesTuning kafka pipelines
Tuning kafka pipelinesSumant Tambe
 
End-to-End Reactive Data Access Using R2DBC with RSocket and Proteus
End-to-End Reactive Data Access Using R2DBC with RSocket and ProteusEnd-to-End Reactive Data Access Using R2DBC with RSocket and Proteus
End-to-End Reactive Data Access Using R2DBC with RSocket and ProteusVMware Tanzu
 
TDC2017 | São Paulo - Trilha Containers How we figured out we had a SRE team ...
TDC2017 | São Paulo - Trilha Containers How we figured out we had a SRE team ...TDC2017 | São Paulo - Trilha Containers How we figured out we had a SRE team ...
TDC2017 | São Paulo - Trilha Containers How we figured out we had a SRE team ...tdc-globalcode
 
High performace network of Cloud Native Taiwan User Group
High performace network of Cloud Native Taiwan User GroupHigh performace network of Cloud Native Taiwan User Group
High performace network of Cloud Native Taiwan User GroupHungWei Chiu
 
Building Distributed Systems With Riak and Riak Core
Building Distributed Systems With Riak and Riak CoreBuilding Distributed Systems With Riak and Riak Core
Building Distributed Systems With Riak and Riak CoreAndy Gross
 
Incremental Export of Relational Database Contents into RDF Graphs
Incremental Export of Relational Database Contents into RDF GraphsIncremental Export of Relational Database Contents into RDF Graphs
Incremental Export of Relational Database Contents into RDF GraphsNikolaos Konstantinou
 
Building Large-Scale Stream Infrastructures Across Multiple Data Centers with...
Building Large-Scale Stream Infrastructures Across Multiple Data Centers with...Building Large-Scale Stream Infrastructures Across Multiple Data Centers with...
Building Large-Scale Stream Infrastructures Across Multiple Data Centers with...confluent
 

Semelhante a Bathcamp 2010-riak (20)

Scalable Persistent Storage for Erlang: Theory and Practice
Scalable Persistent Storage for Erlang: Theory and PracticeScalable Persistent Storage for Erlang: Theory and Practice
Scalable Persistent Storage for Erlang: Theory and Practice
 
HPC Controls Future
HPC Controls FutureHPC Controls Future
HPC Controls Future
 
Getting started with Riak in the Cloud
Getting started with Riak in the CloudGetting started with Riak in the Cloud
Getting started with Riak in the Cloud
 
Running a distributed system across kubernetes clusters - Kubecon North Ameri...
Running a distributed system across kubernetes clusters - Kubecon North Ameri...Running a distributed system across kubernetes clusters - Kubecon North Ameri...
Running a distributed system across kubernetes clusters - Kubecon North Ameri...
 
Multi-Datacenter Kafka - Strata San Jose 2017
Multi-Datacenter Kafka - Strata San Jose 2017Multi-Datacenter Kafka - Strata San Jose 2017
Multi-Datacenter Kafka - Strata San Jose 2017
 
MySQL on Ceph
MySQL on CephMySQL on Ceph
MySQL on Ceph
 
My SQL on Ceph
My SQL on CephMy SQL on Ceph
My SQL on Ceph
 
Aurora_session.pdf
Aurora_session.pdfAurora_session.pdf
Aurora_session.pdf
 
Highly available, scalable and secure data with Cassandra and DataStax Enterp...
Highly available, scalable and secure data with Cassandra and DataStax Enterp...Highly available, scalable and secure data with Cassandra and DataStax Enterp...
Highly available, scalable and secure data with Cassandra and DataStax Enterp...
 
Lessons learned from scaling YARN to 40K machines in a multi tenancy environment
Lessons learned from scaling YARN to 40K machines in a multi tenancy environmentLessons learned from scaling YARN to 40K machines in a multi tenancy environment
Lessons learned from scaling YARN to 40K machines in a multi tenancy environment
 
[EUC2016] DockerCap: a software-level power capping orchestrator for Docker c...
[EUC2016] DockerCap: a software-level power capping orchestrator for Docker c...[EUC2016] DockerCap: a software-level power capping orchestrator for Docker c...
[EUC2016] DockerCap: a software-level power capping orchestrator for Docker c...
 
Jay Kreps on Project Voldemort Scaling Simple Storage At LinkedIn
Jay Kreps on Project Voldemort Scaling Simple Storage At LinkedInJay Kreps on Project Voldemort Scaling Simple Storage At LinkedIn
Jay Kreps on Project Voldemort Scaling Simple Storage At LinkedIn
 
Tuning kafka pipelines
Tuning kafka pipelinesTuning kafka pipelines
Tuning kafka pipelines
 
End-to-End Reactive Data Access Using R2DBC with RSocket and Proteus
End-to-End Reactive Data Access Using R2DBC with RSocket and ProteusEnd-to-End Reactive Data Access Using R2DBC with RSocket and Proteus
End-to-End Reactive Data Access Using R2DBC with RSocket and Proteus
 
TDC2017 | São Paulo - Trilha Containers How we figured out we had a SRE team ...
TDC2017 | São Paulo - Trilha Containers How we figured out we had a SRE team ...TDC2017 | São Paulo - Trilha Containers How we figured out we had a SRE team ...
TDC2017 | São Paulo - Trilha Containers How we figured out we had a SRE team ...
 
High performace network of Cloud Native Taiwan User Group
High performace network of Cloud Native Taiwan User GroupHigh performace network of Cloud Native Taiwan User Group
High performace network of Cloud Native Taiwan User Group
 
Building Distributed Systems With Riak and Riak Core
Building Distributed Systems With Riak and Riak CoreBuilding Distributed Systems With Riak and Riak Core
Building Distributed Systems With Riak and Riak Core
 
Incremental Export of Relational Database Contents into RDF Graphs
Incremental Export of Relational Database Contents into RDF GraphsIncremental Export of Relational Database Contents into RDF Graphs
Incremental Export of Relational Database Contents into RDF Graphs
 
Building Large-Scale Stream Infrastructures Across Multiple Data Centers with...
Building Large-Scale Stream Infrastructures Across Multiple Data Centers with...Building Large-Scale Stream Infrastructures Across Multiple Data Centers with...
Building Large-Scale Stream Infrastructures Across Multiple Data Centers with...
 
Scalable Web Apps
Scalable Web AppsScalable Web Apps
Scalable Web Apps
 

Mais de Timothy Perrett

Nelson: Rigorous Deployment for a Functional World
Nelson: Rigorous Deployment for a Functional WorldNelson: Rigorous Deployment for a Functional World
Nelson: Rigorous Deployment for a Functional WorldTimothy Perrett
 
Online Experimentation with Immutable Infrastructure
Online Experimentation with Immutable InfrastructureOnline Experimentation with Immutable Infrastructure
Online Experimentation with Immutable InfrastructureTimothy Perrett
 
Enterprise Algebras, Scala World 2016
Enterprise Algebras, Scala World 2016Enterprise Algebras, Scala World 2016
Enterprise Algebras, Scala World 2016Timothy Perrett
 
Large-scale Infrastructure Automation at Verizon
Large-scale Infrastructure Automation at VerizonLarge-scale Infrastructure Automation at Verizon
Large-scale Infrastructure Automation at VerizonTimothy Perrett
 
Reasonable RPC with Remotely
Reasonable RPC with RemotelyReasonable RPC with Remotely
Reasonable RPC with RemotelyTimothy Perrett
 
Building Enigma with State Monad & Lens
Building Enigma with State Monad & LensBuilding Enigma with State Monad & Lens
Building Enigma with State Monad & LensTimothy Perrett
 
Functional Programming at Verizon
Functional Programming at VerizonFunctional Programming at Verizon
Functional Programming at VerizonTimothy Perrett
 
Scalalable Language for a Scalable Web
Scalalable Language for a Scalable WebScalalable Language for a Scalable Web
Scalalable Language for a Scalable WebTimothy Perrett
 
Javazone 2011: Goal Directed Web Applications
Javazone 2011: Goal Directed Web ApplicationsJavazone 2011: Goal Directed Web Applications
Javazone 2011: Goal Directed Web ApplicationsTimothy Perrett
 
Concurrency and Parallelism with Scala
Concurrency and Parallelism with ScalaConcurrency and Parallelism with Scala
Concurrency and Parallelism with ScalaTimothy Perrett
 
Scaladays 2011: Task Driven Scala Web Applications
Scaladays 2011: Task Driven Scala Web ApplicationsScaladays 2011: Task Driven Scala Web Applications
Scaladays 2011: Task Driven Scala Web ApplicationsTimothy Perrett
 
Javazone 2010-lift-framework-public
Javazone 2010-lift-framework-publicJavazone 2010-lift-framework-public
Javazone 2010-lift-framework-publicTimothy Perrett
 
Devoxx 2009: The Lift Framework
Devoxx 2009: The Lift FrameworkDevoxx 2009: The Lift Framework
Devoxx 2009: The Lift FrameworkTimothy Perrett
 

Mais de Timothy Perrett (15)

Nelson: Rigorous Deployment for a Functional World
Nelson: Rigorous Deployment for a Functional WorldNelson: Rigorous Deployment for a Functional World
Nelson: Rigorous Deployment for a Functional World
 
Online Experimentation with Immutable Infrastructure
Online Experimentation with Immutable InfrastructureOnline Experimentation with Immutable Infrastructure
Online Experimentation with Immutable Infrastructure
 
Enterprise Algebras, Scala World 2016
Enterprise Algebras, Scala World 2016Enterprise Algebras, Scala World 2016
Enterprise Algebras, Scala World 2016
 
Large-scale Infrastructure Automation at Verizon
Large-scale Infrastructure Automation at VerizonLarge-scale Infrastructure Automation at Verizon
Large-scale Infrastructure Automation at Verizon
 
Reasonable RPC with Remotely
Reasonable RPC with RemotelyReasonable RPC with Remotely
Reasonable RPC with Remotely
 
Building Enigma with State Monad & Lens
Building Enigma with State Monad & LensBuilding Enigma with State Monad & Lens
Building Enigma with State Monad & Lens
 
Functional Programming at Verizon
Functional Programming at VerizonFunctional Programming at Verizon
Functional Programming at Verizon
 
Scalalable Language for a Scalable Web
Scalalable Language for a Scalable WebScalalable Language for a Scalable Web
Scalalable Language for a Scalable Web
 
BRUG - Hello, Scala
BRUG - Hello, ScalaBRUG - Hello, Scala
BRUG - Hello, Scala
 
Scala Helix
Scala HelixScala Helix
Scala Helix
 
Javazone 2011: Goal Directed Web Applications
Javazone 2011: Goal Directed Web ApplicationsJavazone 2011: Goal Directed Web Applications
Javazone 2011: Goal Directed Web Applications
 
Concurrency and Parallelism with Scala
Concurrency and Parallelism with ScalaConcurrency and Parallelism with Scala
Concurrency and Parallelism with Scala
 
Scaladays 2011: Task Driven Scala Web Applications
Scaladays 2011: Task Driven Scala Web ApplicationsScaladays 2011: Task Driven Scala Web Applications
Scaladays 2011: Task Driven Scala Web Applications
 
Javazone 2010-lift-framework-public
Javazone 2010-lift-framework-publicJavazone 2010-lift-framework-public
Javazone 2010-lift-framework-public
 
Devoxx 2009: The Lift Framework
Devoxx 2009: The Lift FrameworkDevoxx 2009: The Lift Framework
Devoxx 2009: The Lift Framework
 

Último

Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024The Digital Insurer
 
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure serviceWhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure servicePooja Nehwal
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)Gabriella Davis
 
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptxHampshireHUG
 
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking MenDelhi Call girls
 
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slidevu2urc
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Drew Madelung
 
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 3652toLead Limited
 
[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdfhans926745
 
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking MenDelhi Call girls
 
Kalyanpur ) Call Girls in Lucknow Finest Escorts Service 🍸 8923113531 🎰 Avail...
Kalyanpur ) Call Girls in Lucknow Finest Escorts Service 🍸 8923113531 🎰 Avail...Kalyanpur ) Call Girls in Lucknow Finest Escorts Service 🍸 8923113531 🎰 Avail...
Kalyanpur ) Call Girls in Lucknow Finest Escorts Service 🍸 8923113531 🎰 Avail...gurkirankumar98700
 
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...HostedbyConfluent
 
Swan(sea) Song – personal research during my six years at Swansea ... and bey...
Swan(sea) Song – personal research during my six years at Swansea ... and bey...Swan(sea) Song – personal research during my six years at Swansea ... and bey...
Swan(sea) Song – personal research during my six years at Swansea ... and bey...Alan Dix
 
GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationGenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationMichael W. Hawkins
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationSafe Software
 
FULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | Delhi
FULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | DelhiFULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | Delhi
FULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | Delhisoniya singh
 
The Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxThe Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxMalak Abu Hammad
 
08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking MenDelhi Call girls
 
08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking MenDelhi Call girls
 
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024Rafal Los
 

Último (20)

Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024
 
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure serviceWhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)
 
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
 
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
 
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slide
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
 
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
 
[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf
 
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
 
Kalyanpur ) Call Girls in Lucknow Finest Escorts Service 🍸 8923113531 🎰 Avail...
Kalyanpur ) Call Girls in Lucknow Finest Escorts Service 🍸 8923113531 🎰 Avail...Kalyanpur ) Call Girls in Lucknow Finest Escorts Service 🍸 8923113531 🎰 Avail...
Kalyanpur ) Call Girls in Lucknow Finest Escorts Service 🍸 8923113531 🎰 Avail...
 
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
 
Swan(sea) Song – personal research during my six years at Swansea ... and bey...
Swan(sea) Song – personal research during my six years at Swansea ... and bey...Swan(sea) Song – personal research during my six years at Swansea ... and bey...
Swan(sea) Song – personal research during my six years at Swansea ... and bey...
 
GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationGenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day Presentation
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
 
FULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | Delhi
FULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | DelhiFULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | Delhi
FULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | Delhi
 
The Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxThe Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptx
 
08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men
 
08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men
 
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024
 

Bathcamp 2010-riak

  • 2. What is Riak? • Documented orientated database • Written in Erlang • Based on Dynamo[1] and CAP Theorem[2] • Highly fault tolerant • HTTP and ProtoBuff interface • Write MapReduce in Erlang or JavaScript 1. http://goo.gl/r8Np 2. http://www.julianbrowne.com/article/viewer/brewers-cap-theorem
  • 3. Same, Same but different • Riak solves similar problems to MongoDB • Semi-structured data modeled as "documents” • Storage of non-document data in the database • High write-availability • Riak is intrinsically multi-node scalable • Mongo in comparison is single system (+ sharding) • Riak achieves availability via quorum writes • Mongo uses performant in-place writes • Riak uses “masterless” replication
  • 4. N/R/W – Dynamo N = Number of replicas to store R = Number of replicas needed to read W = Number of replicas needed to read • These principals first appeared in an Amazon research paper known as Dynamo
  • 5. • 160bit integer key space. Each node that joins is assigned part of that space for consistent hashing • Hashing means any node can service any request making the cluster masterless and eventually consistant Number of replicas
  • 6. • Number of replies before Riak gives the client a successful reply. • Tries to access all nodes, but as soon as the N/R is satisfied a response is given Reads
  • 7. • Same as reads; W implies the number of successful nodes that must reply before the write is considered consistent by the client Writes
  • 8. Extreme example • Given N=10, R=W=2 we could have 8 nodes down and the cluster would still be fully available to all clients
  • 9. What does this all mean? • N/R/W specified at request time, so each client can specify its own tolerance for outages dynamically • Despite any outages within the cluster, the whole cluster can still appear available based on N/R/W • Given N=3 and R=W=2, we can have 3-2=1 node down/unreachable/laggy in the cluster • Stupidly high availability complete with eventual consistency controlled by dynamic clients
  • 10. Brewer’s CAP Theorem • Consistency • Availability • Partition Tolerance • You cant have all things, all the time… • …but you can have some of each, all the time! • Riak is about choosing your own levels of each according to your use case
  • 11. Consistency • Start with document version zero • Things get redistributed and n0 and n2 are sitting in NYC and n1 and n3 are in London • What if stuff changes??
  • 12. Consistency • Uh oh: inconsistency • Both parts of the cluster are still fully available • NYC serves v1 whilst London serves v0 • The network resumes and Riak determines the latest version by using vector clocks
  • 13. Consistency • What if both sides of the Atlantic changed? • Riak is unable to determine which is the right document, both are returned to the client with an indication of the inconsistency
  • 14. • Distributed, fault-tolerant full-text searching • Lucene syntax for queries • No need for index sharding • Linier scaling • Double the number of nodes to get double the search capacity (awesome!) • Search via: • Fields, wildcards, fuzzy text or token proximity Riak Search