How HasOffers uses DynamoDB

1. DynamoDB at HasOffers

2. • Attribution for web & mobile marketing
• Tons of data
• Ad-hoc analysis
• Near real-time

3. DynamoDB at HasOffers
18 months in production
207 billion items
87 TB of data

4. Use Cases
Tracking sessions
Deduplication
Attribution

5. Tracking
Sessions

6. Use Case: Tracking Sessions
Session
Data
671c6a9a-bed0-4cec-a465-613cca8c9832
• Impression
• Click

7. Use Case: Tracking Sessions
• Ideal for NoSQL
• KV lookups
• Scalability
• Availability

8. Deduplication

9. Use Case: Event Deduplication
671c6a9a-bed0-4cec-a465-613cca8c9832
ebe0efef-aa71-471c-b3f8-7b66b458bd79
ebe0efef-aa71-471c-b3f8-7b66b458bd79
d149e52c-d2b3-49bc-b6af-f304b5fab57f
DEDUPE!
(Idempotency)

10. Use Case: Event Deduplication
Hash uuid
Range <none>
Write capacity = X, Read capacity = 1
Conditional puts

11. Use Case: Event Deduplication
def is_duplicate( dynamo_tbl, uuid ):
row_data = {‘host’: socket.gethostname(),
‘created’: int(time.time())}
row = dynamo_tbl.new_item( hash_key=uuid, attrs=row_data )
DEDUPE!
try:
row.put( expected_value={‘host’: False} )
except DynamoDBConditionalCheckFailedError:
return True
# not a duplicate
return False

12. Use Case: Event Deduplication
• Real-time dedupe
• Conditional put
• Writes only; no read
• Extra debug info
{uuid: d149e52c-d2b3-49bc-b6af-f304b5fab57f,
host: server03.hasoffers.com,
created: 1398884990}
DEDUPE!

13. Why NoSQL
SaaS?

14. Cluster Resizing
• Upsize and downsize trivial in DynamoDB
• Infrequent on own cluster = limits operational experience

15. Cluster Software Updates
0.7 1.0
• Production depends on NoSQL solution
being available.
• Software version updates somewhere
between nail-biting and terrifying

16. Dynamo
Operations

17. DynamoDB Capacity
Per table R/W capacity
Capacity units
Exceed = throttling

18. DynamoDB Throttling
CloudWatch = 5 minutes
DynamoDB = 1 second

19. DynamoDB Throttling
• Double capacity on table
• Root cause from log analysis
– capacity usage
– key hot-spotting
• Fix root cause
• Reduce capacity on table

20. Closing
Thoughts

21. Closing Thoughts
• Compelling offering
• Simple operations
• Avoid hammer syndrome
• Very happy

22. Thank you!
niek@hasoffers.com