1. The document discusses the evolution of caching strategies at Build.com as their systems and traffic grew rapidly over time. They initially used a Java-based distributed cache and later switched to Redis which proved more effective. 2. As Build.com moved to a continuous delivery model with multiple environments, they needed a "shared" cache that both environments could use. They implemented a unified caching model where each version of code has its own bucket in the cache but objects can be promoted from older versions if they are compatible. 3. The key aspects of the unified caching model are using a serialization checksum to detect changes between versions, using a build number as the cache key so each version is separate, and attempting to promote

1. Evolving Your Distributed Cache In a
Continuous Delivery World
Tyler Van Gorder
@tkvangorder
Build.com, Principal Software Engineer

2. PRESENTED BY
1 The Early Days
Who we are and how our systems have evolved
2 Today’s Architecture
A peek into our current system architecture
3 Unified Caching Model (Live Coding Example)
Solving Caching Problems in a Continuous Delivery Environment
Agenda:
4 Questions

3. A personalized, online, home
improvement shopping experience
Tyler Van Gorder
Principal Software Engineer
https://github.com/tkvangorder
@tkvangorder

4. BUILD.COM AT A GLANCE
109K
Professional Tradespeople
91M
Annual Visits
>569K
Annual Customer Chats
>1.3M
Customer Calls Annually
#72
Top 100 Internet
Retailers in the US
710+
Employees

5. PRESENTED BY
The Early Days

6. Started as a single web site
The site and company grew rapidly
Even the CEO helped with coding (oh dear)
Deployment were done at night
Performance issues

7. There were several distributed
caching options
We tried a Java-based distributed
cache first
We decided to use Redis
Caching?
Round 1 - Fight!
https://pixabay.com/users/12019-12019/?utm_source=link-attribution&utm_medium=referral&utm_campaign=image&utm_content=115800

8. It’s hard to argue with the results
Why Redis?
Redis can be clustered
Redis is blazingly fast
Easy to maintain/upgrade
Built-in support in for Spring.

9. Introducing a cache proved very effective.
We did deployments at night
Deployment meant purging the cache
It worked for a while
And we continued to grow
Caching To Rescue
https://pixabay.com/vectors/superhero-super-hero-hero-figure-304713
Distributed
Cache!

10. Multiple web storefronts
We put more data into cache
Sites were really slow after deployment
We introduced cache priming
Held our breath as morning arrived
We need to fix this!
We Are Killing Ourselves
Story Time
https://pixabay.com/vectors/superhero-super-hero-hero-figure-304713

11. Two identical environments (prod./dark)
Both environments use the same data stores
Deploy to the dark servers in the day
Still had to purge and prime the cache
Blue / Green Deployments

12. https://pixabay.com/vectors/superhero-super-hero-hero-figure-304713
Production traffic continued to grow
We still had two different redis clusters
Internal canary
Difficult to move to continuous delivery
Story Time
We Need a “Shared” Cache

13. PRESENTED BY
Today’s Architecture

14. Consumer Store Nodes
Mobile API
Internal Tools
Platform API Services
System
Architecture

15. Data Access
Common Business Logic
Security
Caching!
Platform API
Image by PIRO4D from Pixabay

16. Load Balancing
System Metrics
Log Aggregation
Twelve-Factor Applications
Circuit breakers
Rolling deployments
https://12factor.net/

17. Push code to production often
Changes are smaller/less risk
Immediate user feedback
Continuous Delivery
Automate the process
Monitor for issues
Need the ability to rollback
https://harness.io/

18. Rolling Deployments

19. The Trouble Spots In Rolling Deployment
System must be resilient to having two versions of your application running concurrently.
Both versions of the application need to play nice when they are sharing a cache.

20. PRESENTED BY
Unified Caching Model

21. The application is caching customer data
Version 1 has basic customer object
Version 2 adds address information to the
customer
Demo Setup
Round 2 - Fight
(and a demo?)
https://pixabay.com/users/12019-12019/?utm_source=link-attribution&utm_medium=referral&utm_campaign=image&utm_content=115800
https://github.com/tkvangorder/redis-shared-cache-sample

22. What can go wrong?
https://github.com/tkvangorder/redis-shared-cache-sample

23. Cache Serialization “Checksum”
https://github.com/tkvangorder/redis-shared-cache-sample

24. V1.0 : Cache Get, Cache Hit, Checksum Failure, Retrieve, Cache Put
V1.1 : Cache Get, Cache Hit, Checksum Failure, Retrieve, Cache Put
V1.0 : Cache Get, Cache Hit, Checksum Failure, Retrieve, Cache Put
V1.1 : Cache Get, Cache Hit, Checksum Failure, Retrieve, Cache Put
V1.0 : Cache Get, Cache Hit, Checksum Failure, Retrieve, Cache Put
V1.1 : Cache Get, Cache Hit, Checksum Failure, Retrieve, Cache Put
V1.0 : Cache Get, Cache Hit, Checksum Failure, Retrieve, Cache Put
V1.1 : Cache Get, Cache Hit, Checksum Failure, Retrieve, Cache Put
Cache Thrashing
Insanity:
Doing the same thing
over and over again
and expecting
different results.
- Albert Einstein
https://github.com/tkvangorder/redis-shared-cache-sample

25. Cache Thrashing In The Wild
https://github.com/tkvangorder/redis-shared-cache-sample

26. Caching Version Buckets
https://github.com/tkvangorder/redis-shared-cache-sample

27. Cache Promotion
Version cache miss, look for another version’s copy
If the checksum is successful, copy is compatible
Promote that copy into cache for the new version
https://github.com/tkvangorder/redis-shared-cache-sample

28. PRESENTED BY
1 Use a serialization checksum
The use of an incrementing uid for each complex object insures changes to nested objects are also
detected. This requires a change in the development process!
2 Use a build number as your nested hash key
Each active build will have it’s own version of a cached object. Automate the injection of this build
number into your software artifacts!
3 Attempt a cache promotion on a cache miss.
If a new build does not have a copy of the cached value, attempt to deserialize the previous version. If
there are no structural changes, promote!
Summary:

29. PRESENTED BY
1 What about cache eviction?
The root key is still pinned to the record, eviction in one version will evict ALL copies.
2 Time to live helps manage the overall resource requirements
Multiple copies of the same object requires more resources for you Redis cluster. In practice, we have
found this to be a non-issue in our environment.
3 We have written the unified cache implementation twice
We originally extended the default Redis cache implementation in Spring Boot 1.5.x, now we have our
own cache implementation.
Final Notes:
4 Open Source?
All of the sample code, including the unified cache library demonstrated here are available for your
use. If there is enough interest, we may make an official open source offering.

30. PRESENTED BY
Questions?
Tyler Van Gorder
Principal Software Engineer
https://github.com/tkvangorder
@tkvangorder

31. Thank you!