a 8tracks ama o Redis
•
1 gostou•828 visualizações
Na 8tracks a gente acredita na ferramenta certa pra resolver os problemas que a gente encontra pela frente! Passamos por vários desafios, mas encontramos uma ferramenta excelente pra nos ajudar a servir mais de 100 mil request por minuto e tornar nossos problemas de persistência bem mais simples. Redis salvou a nossa pele mais de uma vez!
Recomendados
Recomendados
Mais conteúdo relacionado
Destaque
Destaque (20)
Semelhante a a 8tracks ama o Redis
Semelhante a a 8tracks ama o Redis (20)
Mais de Ivayr Farah Netto
Último
Último (20)
a 8tracks ama o Redis
- 1. redis love at 8tracks @nettofarah
- 2. @nettofarah full stack developer na 8tracks.com 8tracks.com/nettofarah netto@8tracks.com
- 3. 8tracks
- 4. 8tracks
- 7. stats • users: 13.3mi! • public playlists: 1.2mi (4,2 mi total)! • uploaded tracks: 31mi
- 8. not a sysadmin talk
- 10. a melhor ferramenta para o problema http://peopletakingpictureswithipads.tumblr.com/
- 12. para o problema! http://peopletakingpictureswithipads.tumblr.com/
- 13. esforço humano esforço da máquina
- 14. F.C.P.P. = dev <3 + machine <3
- 16. key-value store • open source • in memory (most of the times) • data-structure server
- 17. data types • strings • lists • sets • sorted sets • hashes
- 18. just like that data structures class
- 19. except it is COOL!
- 20. you can turn this
- 21. into this head next next next next tail
- 22. or this next next next
- 23. things to keep in mind • all in memory (but not restricted to) • no joins or iterating over data (but not restricted to) • you should be able to rebuild data
- 24. why? • sometimes basic data structures is all you need • some problems are hard to fit in a relational way
- 25. redis @ 8tracks = <3 sidekiq feature toggles caching player (moved to couchdb, then to mysql) autocomplete (moved to SOLR, then to ElasticSearch) tag browsing! news feed listening sessions rate limiter
- 28. sidekiq
- 30. what’s indie? tag artist playlist
- 31. 1 SELECT name ! ! ! FROM tags ! ! ! WHERE name ! ! ! LIKE 'ind%' LIMIT 10;! ! 2 SELECT name ! ! ! FROM mixes ! ! ! WHERE name ! ! ! LIKE 'ind%' LIMIT 10;! ! 3 SELECT login ! ! ! FROM users ! ! ! WHERE login ! ! ! LIKE 'ind%' LIMIT 10;! ! 4 SELECT name ! ! FROM artists ! ! WHERE name ! ! LIKE 'ind%' LIMIT 10;!
- 32. you could do 1 (SELECT name FROM tags WHERE name LIKE 'ind%' LIMIT 10)! 2 UNION ALL! 3 (SELECT name FROM mixes WHERE name LIKE 'ind%' LIMIT 10);!
- 33. the problems • cross-type search • I don’t want 10 of each, I want the 10 most relevant ones • can’t select different column counts
- 34. not a fair comparison people don’t use sql databases for search. they use SOLR (or elasticsearch)
- 35. sometimes basic data structures! is all you need
- 36. sorted sets • score • add, remove, or update log(n) • elements in order, fast existence test, fast access to elements in the middle
- 37. breaking it down indie => probably irrelevant relevant i ! ! ! ! ! ! ! ! in ! ! ! ! ! ! ! ! ! ind! ! ! ! ! ! ! ! indi! ! ! ! ! ! ! ! indie
- 38. our collection The indie summer Indiana Industrial
- 39. one zset per n-gram ind [ ] [ ] [ ] indi indie
- 40. which is the same as ZADD ind, <score>, ZADD indi, <score>, ZADD indie, <score>,
- 41. searching 1 ind = $r.zevrange('ind', 0, 10)! ! 2 indie = $r.zevrange('indie', 0, 10)!
- 42. tag browsing
- 43. how?
- 45. a single tag 1 SELECT m.* FROM mixes m! 2 INNER JOIN taggings tg! 3 ON tg.mix_id = m.id! 4 WHERE tg.tag_id = 28! 5 LIMIT 10;! chill
- 46. two tags 1 SELECT m.* FROM mixes m! 2 INNER JOIN taggings tg! 3 ON tg.mix_id = m.id! 4 INNER JOIN taggings tg2! 5 ON tg2.mix_id = m.id! 6 WHERE tg.tag_id = 28! 7 AND tg2.tag_id = 12! 8 LIMIT 10;! chill acoustic
- 47. adding sort 1 SELECT m.* FROM mixes m! 2 INNER JOIN taggings tg! 3 ON tg.mix_id = m.id! 4 INNER JOIN taggings tg2! 5 ON tg2.mix_id = m.id! 6 WHERE tg.tag_id = 28! 7 AND tg2.tag_id = 12! 8 ORDER BY m.first_published_at! 9 LIMIT 10;!
- 50. zinterstore intersects two sets, then store them in a new different set Time complexity: O(N*K)+O(M*log(M)) we’ll get to this
- 51. chill
- 52. acoustic
- 53. chill acoustic +
- 54. 1 sets = ['chill', 'acoustic']! 2 cache_key = 'chill:acoustic:cache'! 3 $r.zinterstore(cache_key, sets, :weights => [1, 1])! chill acoustic
- 55. 1 sets = ['popular', 'chill', 'acoustic']! 2 cache_key = 'popular:chill:acoustic:cache'! 3 $r.zinterstore(cache_key, sets, :weights => [1, 0, 0])! popular chill acoustic chill
- 56. back to the BIG O thing O(N*K)+O(M*log(M)) - N: smallest input - K: number of sorted sets - M: number of elements of the resulting set
- 58. cool stuff worth checking out • pipelining • lua scripting • replication + persistence
- 59. lessons • it consumes a LOT of memory (duuuh) • sentinel for failure recovery • DO NOT USE “keys” in production • duplication is tricky
- 60. gracias! thank you! obrigado!