Redis is an in-memory database that provides fast performance for powering lightning fast apps. It supports many data structures like strings, hashes, lists, sets and sorted sets. Redis is efficient due to its support for many data structures and commands, as well as its complexity-aware design. Redis Labs provides fully-managed cloud services for Redis and Memcached, and helps customers with challenges around scalability, high availability, performance and monitoring for large-scale Redis deployments.
2. 2
The newest NoSQL
The fastest data store available today (served entirely from
RAM)
Among the top 3 databases chosen by developers
Much more than a simple key/value - Strings, Hashes, Lists,
Sets, Sorted Set, LUA, transactions, Bits operations
Strong use cases, dynamic community, large eco-system
Redis
3. 3
Leading the commercial Redis market
Founded in 2011; GA in 02/2013
2,400+ paying customers; 52,000+ DBs; 100+ new
DBs/day
2nd largest contributor to open source Redis
Raised $13M - Bain/Carmel/Strategic/Angels
Offices in Santa Clara and Tel-Aviv
Redis Labs
5. 5
100msec =
Fast apps requirements
max E2E response time, under any
load
50msec = average Internet latency
50msec = required app response time (includes
processing & multi DB accesses)
1msec = required DB response time
The only database to meet requirement=
7. 7
Why is Redis efficient ?
Many data-structures
Many cool commands (atomicity maintained)
Complexity aware
8. 8
Real world use case:
• 500+GB
• 400K writes/sec
• 1500 reads/sec
• 37.5KB average object size
Efficiency
No extra
work at
app level
1.5Gbps 120Gbps
Tones of
work at app
level
NoSQL
6 Nodes
cluster
150+ Nodes
cluster
10. 10
• Multi-TB in memory
• ~ 300,000 reads/sec
• ~ 5,000*N writes/sec
N - # of followers
Twitter
Every Timeline
(800 tweets per user)
is on Redis
11. 11
• 20TB+ in memory
• ~ 6,000,000 reads/sec
• ~ 600,000 writes/sec
Weibo (Chinese Twitter)
• Counting
• Reverse cache
• Top 10 lists
• Last Index
• Relational list/Message Queue
• Fast transactions w/ LUA
12. 12
Pinterest
Object graph:
• Per user (Sorted Set w/ timestamp as score)
store the users followed (explicit+ implicit)
store the user’s followers (explicit+implicit)
• Per board
Redis Hash for storing explicit followers
Redis Set for storing explicit unfollowers
13. 13
Stack Overflow
Three levels of cache:
• Local cache (no persistence)
sessions, and pending view count updates
• Site cache
hot question id lists, users acceptance rates..
• Global cache
Inboxes, API usage quotas, …
14. 14
Github
• Redis is used for routing info
• Matching user repositories to server names
16. 16
Youporn
Most data is found in Hashes with ordered Sets used to know
what data to show
(1) ZinterStore on:
{videos:filters:release}{videos:filters:orientation:straight}
{videos:filters:categories(id)}{videos:ordering:rating}
(2) Perform a ZRANGE to get the pages we want and get the list
of video_ids back
(3) Start pipelining to get all the videos from Hashes
28. 28
Various instance types in the same cluster
• High load scenarios
• High memory usage scenarios
• New generation of instances
Dedicated instances
As cheap as possible
Cluster’s node requirements
29. 29
Adrian Cockcroft's Blog - Understanding and using Amazon
EBS - Elastic Block Store
• use large instances and get dedicated instances for free
The tip
30. 30
What we use today
C3 & R3 A4/5/6/7
n1-standard
n1-highmem
n1-highcpu
BM+VM
31. 31
Challenge #3
How to mange data-persistence
with high volumes of ‘writes’ and
slow cloud storage ?
32. 32
Ephemeral vs. Persistence storage
Ephemeral
EBS/Cloud Drive/Persistent
Disk/SAN
Network attached
Persistent
Slow
Direct attached
Ephemeral
“Fast”
33. 33
Adrian’ s Blog use the larger EBSes if you want speed
Google (GCP) “Larger volumes can achieve higher I/O
levels than smaller volumes”
The tips
34. 34
We use large volumes (1TB+)
We use both ephemeral and persistent storage
We improved/tuned/optimized the Redis persistent
storage interface
If replication is enabled, slave writes to disk
We don’t use PIOPS
What we do
39. 39
Team /Method/Spirit
Tiny devops team
Core dev. team knows ops (very well)
Baby steps, especially in production
The practical approach always wins
Review your plans every 3 months
44. 44
Cool commands
• SET if it doesn’t exist – O(1)
• Blocking POP (with timeout) – O(1)
• (blocking) POP from one list, PUSH to another – O(1)
• Get/Set string ranges (and bit operation) – O(N)
• Union/Intersect/Ranges of SETs – O(N)+O(Mxlog(M))
• Pub/Sub – O(1)/O(M)/O(M+N)
• LUA / Transactions / Pipelining
Notas do Editor
Choozen
20 13
Cont ributer
At bottom
An at the top is Redis
Let’s run through it quickly
What are the benefits of using us
You performance can vary quite a bit
Maa ner
Multi-tenant technology
As many as you want
Without further ado let’s go to the demo