After looking at a variety of NoSQL alternatives like MongoDB, Riak, and LevelDB, Hulu selected Redis. Describing the process, Andres Rangel, Senior Software Engineer, stated, “We chose Redis for several key reasons – it was simple to set up, had great documentation, offered replication, and allowed us to use data structures. Data structures are extremely powerful and allow us to architect solutions to many use cases very efficiently. For example, depending on the operation, we have the need to query either a specific video a user watched, or all of them. With Redis, this was easy using hashes.”
To learn more, visit pivotal.io/big-data/redis.
Axa Assurance Maroc - Insurer Innovation Award 2024
Hulu Case Study
1. Hulu is an online video service that offers a selection of hit
TV shows, clips, movies and more on the free, ad-supported
Hulu.com service, and the subscription service Hulu Plus. One
of the top video streaming sites in the U.S., today the service
has over four million subscribers and approximately 30 million
unique viewers per month.
CHALLENGE
Inability to Scale MySQL and Memcached
In 2012, Hulu’s subscriber base passed the two million mark,
and the back-end systems that tracked viewer history started
to breakdown. When a video is played, the system records
information from the player to keep track of both the video and
the viewing position or timeframe. When the video application
is closed, the stored information allows the user to resume
the video where they left off. The system also provides
recommendations for what videos to watch next based on
user history.
Originally designed as a Python application, Hulu’s viewed history
tracking system relied on Memcached for reads on top of a
sharded MySQL database for writes. When the Hulu engineering
team started to see that MySQL couldn’t handle the volume
of writes, the only way to scale was to add more shards. Reads
were done in Memcached to preserve I/O on the database, but
Memcached could not be replicated. So, user history was served
out of one shard in one datacenter.
With the occurrence of peak time failures and an understanding
of root causes, the core Hulu engineering team began to design
a solution with four overarching requirements:
1. Faster reads and writes
2. The capacity to scale to 10,000 queries/second with low
latency
3. Replication of data across datacenters
4. High availability of cached data with no single point of failure
AT-A-GLANCE
Challenges
• MySQL overwhelmed by the volume of writes
• Memcached could not be replicated across
datacenters to distribute load
• Latency on queries with degrading performance
• No high-availability strategy
Solution
• Redis
Key Benefits
• Accelerated writing and retrieval of information
with 800% performance improvement for queries
• Replication across datacenters
• Capacity to handle at least 10,000 queries per second
with low latency
• Open management APIs allow for high availability
• Ability to use data structures for flexible and efficient
queries
CASE STUDY
Hulu
LEADING VIDEO COMPANY SCALES TO SERVE 4 BILLION VIDEOS
WITH 800% QUERY PERFORMANCE IMPROVEMENT
OVERVIEW
“ We chose Redis because it was simple to
set up, had great documentation, offered
replication, and allowed us to use data structures.
Data structures are extremely powerful and
allow us to architect solutions to many use
cases very efficiently.”
— Andres Rangel, Senior Software Engineer, Hulu
pivotal.io