This document discusses caching patterns for microservices. It begins by introducing the speaker and their background at Wix. It then provides examples of when and how to cache from Wix's 2000 microservices, including caching external critical configuration data, improving data layer latency, and caching before services with very high external traffic. The document outlines three caching patterns in detail - S3 backed static caching, DynamoDB caching with change data capture, and HTTP reverse proxy caching with Varnish. It provides guidance on choosing the appropriate caching pattern based on factors like data criticality, dynamism, and invalidation needs.
10. @NSilnitsky
When & how to Cache
Examples from 2000 microservices of various use cases
Caching Patterns
#1
high risk/cost
of network
failure
Like caching
external critical
configuration
data.
* S3 backed
11. @NSilnitsky
When & how to Cache
Examples from 2000 microservices of various use cases
Caching Patterns
#1
high risk/cost
of network
failure
Like caching
external critical
configuration
data.
#2
To improve
average latency
for data layer
access
DynamoDB +
CDC +
LRU cache.
12. @NSilnitsky
When & how to Cache
Examples from 2000 microservices of various use cases
Caching Patterns
#1
high risk/cost
of network
failure
Like caching
external critical
configuration
data.
#2
To improve
average latency
for data layer
access
DynamoDB +
CDC +
LRU cache.
#3
Some very high
external traffic
cases
Caches are very
important -
cache before
the service.
13. @NSilnitsky
When & how to Cache
Examples from 2000 microservices of various use cases
Caching Patterns
#1
high risk/cost
of network
failure
Like caching
external critical
configuration
data.
#2
To improve
average latency
for data layer
access
DynamoDB +
CDC +
LRU cache.
#3
Some very high
external traffic
cases
Caches are very
important -
cache before
the service.
#4
When NOT to -
young products
Don’t cache
prematurely
20. @NSilnitsky
@NSilnitsky
Caching Patterns
Cache Type Cache Size Best For Not For
Read-through cache
Before the 3rd party service
Data has to
fit into memory
External source
of static configuration
Dynamically
updating data
31. @NSilnitsky
@NSilnitsky
Caching Patterns
Cache Type Cache Size Best For Not For
Cache-Aside
Before the DB
No limitation on DB data size
LRU cache size is configurable
Make DB read access faster
for “popular” values
Highly critical application
startup information
32. @NSilnitsky
Next 3 Caching Patterns:
S3 backed static cache
DynamoDB+CDC based cache
HTTP Reverse Proxy cache
Caching Patterns
37. @NSilnitsky
@NSilnitsky
Varnish
cache
Size limit depends on the configured
Storage backends:
1. malloc is memory based — so the size is limited by
(more) expensive memory cost
2. file is memory-backed-by-disk based — which is
limited by less expensive disk cost, but can cause
performance penalty due to fragmentation…
Caching Patterns
38. @NSilnitsky
@NSilnitsky
Varnish
cache
Size limit depends on the configured
Storage backends:
1. malloc is memory based — so the size is limited by
(more) expensive memory cost
2. file is memory-backed-by-disk based — which is
limited by less expensive disk cost, but can cause
performance penalty due to fragmentation…
3. Massive Storage Engine (MSE) — also backed by file,
but has a “fragmentation proof algorithm.” part of
Varnish Cache Plus (paying customers only).
Caching Patterns
39. @NSilnitsky
@NSilnitsky
Caching Patterns
Cache Type Cache Size Best For Not For
Read-through
before the app
Depends on storage
backend
Improving latency for “stable”
HTTP responses
Hard to invalidate
aggregated data
42. @NSilnitsky
@NSilnitsky
Caching Patterns
Choose
Cache Pattern
Is your data crucial
for startup?
Is your data highly
dynamic?
S3 Persisted data cache
No
Yes
Is your data highly
dynamic?
Yes
No
External clients &
easy invalidation?
No
DynamoDB backed
cache / Redis
Varnish cache /
CDN
Yes
No
Yes