Tom Santero, Technical Evangelist, Basho Technologies delivered this presentation at a recent Big Data Warehousing Meetup that was held with Caserta Concepts.
Full-text search capabilities have existed in Riak since 2010. Known simply as Riak Search, the implementation is a homegrown adaptation inspired by Lucene. While Riak Search has been used in production for many years now, Basho has been developing a replacement over the course of the past 12 months in a project codenamed 'Yokozuna'. In this talk, Tom discussed the motivation behind Yokozuna, how it works and why you want to run it in production.
For more information, visit http://basho.com/ or http://www.casertaconcepts.com/.
11. NODE 0 NODE 1 NODE 2
Ka Kb KcKd Ke KfKg Kh Ki
Kj KkKm KlKp Kn KoKq Kr
Naive Hashing
Tuesday, September 17, 13
12. NODE 0 NODE 1 NODE 2
Ka Kb Kc KdKgKi
NODE 3
Ke Kf KhKj Kk Kl
Km Kn Ko KpKq Kr
Naive Hashing
Tuesday, September 17, 13
13. K * (NN - 1) / NN => K
• K = # OF KEYS
• NN = # OF NODES
• AS NN GROWS FACTOR ESSENTIALLY BECOMES 1, THUS
ALL KEYS MOVE
Naive Hashing
Tuesday, September 17, 13
14. PARTITION # = HASH(KEY) % PARTITIONS
• # PARTITIONS REMAINS CONSTANT
• KEY ALWAYS MAPS TO SAME PARTITION
• NODES OWN PARTITIONS
• PARTITIONS CONTAIN KEYS
• EXTRA LEVEL OF INDIRECTION
Consistent Hashing
Tuesday, September 17, 13
15. P9P6P3P8P5P2P7P4P1
NODE 0 NODE 1 NODE 2
Ka Kb KcKd Ke KfKg Kh Ki
Kj KkKm KlKp Kn KoKq Kr
Consistent Hashing
Tuesday, September 17, 13
17. NN * K/Q => K/Q
• K = # OF KEYS
• NN = # OF NODES
• Q = # OF PARTITIONS
• AS K GROWS NN BECOMES CONSTANT,THUS K/Q KEYS
MOVE
Consistent Hashing
Tuesday, September 17, 13