AWS Summit Berlin 2013 - Tadaa - HD Camera and Photo Community
•
1 like•3,112 views
tadaa is the camera app replacing your DSLR, coupled with a massive community of pro and amateur photographers. Let's talk about cost-efficient living, scaling and growing in the cloud. Speaker: Friedemann Wachsmuth, CTO, menschmaschine Publishing GmbH
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (9)
Similar to AWS Summit Berlin 2013 - Tadaa - HD Camera and Photo Community
Similar to AWS Summit Berlin 2013 - Tadaa - HD Camera and Photo Community (20)
More from AWS Germany
More from AWS Germany (20)
Recently uploaded
Recently uploaded (20)
AWS Summit Berlin 2013 - Tadaa - HD Camera and Photo Community
- 2. Scaling tadaa Photo Broadcasting in the Cloud
- 3. What is tadaa? •HD Camera & Photo Editor for iOS •Photosharing Community for Prosumers •3 Million App Users
- 4. tadaa is a Realtime Application... •Constant Relationship Changes •Reactions, Interactions & Notifications •Fan-outs, Deletions •Sync Notification Counts
- 5. ...with a Lot of Data Underneath •250M Contact Hashes •Hundreds of Millions of Images on S3 •~2500 Messages / Second
- 6. Architecture
- 7. Architecture •100% Hosted on AWS •Mostly Using AWS Services: EC2, EMR, CloudFront, S3, DynamoDB, ElastiCache, CloudWatch, IAM...
- 9. Load can be Bursty •If a Single User has 10k Followers ➡ One Photo can cause 10k Push Notifs! •With 25% Open Rate, they create ➡ 2500 API calls within few Seconds •leading to Thousands of Likes and Reactions... Within a few Seconds.
- 10. When Instagram Changed T&Cs... ALL YOUR BASE ARE BELONG TO US
- 11. Using CouchDB... Pros: •Simple REST API •Replicates •Schemaless •Map/Reduce •Everything versioned Cons: •Sharding is Hard •JSON = Data Inflation •Very Disk-I/O Dependent •Everything versioned
- 12. Juggling 8TB Database Files on EBS isn‘t Fun.
- 13. Hello, DynamoDB!
- 14. Moving to DynamoDB •Predictable Performance •Infinite Table Size •Full Redundancy •8TB Became 150GB!
- 15. DynamoDB Scales •We Migrated the Live System with no Downtime within a Few Days •Query Latency is Really just 2-3ms •Worst Value ever seen was 8ms •Now You get Some Free Burst Allowance!
- 16. Cost Scales Up, too... „How Can we Handle Bursts Better?“
- 17. Optimizing Access Patterns •Level Your Reads and Writes •Vary Hashkeys •Throttle Non-Realtime Tasks •Mirror in ElastiCache Where Possible, Persist Lazily ➡Reduce Required CUs by 75%
- 18. Leveling Reads/Writes •Avoid Bursts and Scans •Queue Your I/O through a Messaging System •If You Can, Separate Hot and Cold Data
- 19. Vary Your Hashkeys •Your Table is Partitioned •# of Partitions Grows with Size of Table and Provisioned CUs •You don‘t know the # of Actual Partitions •You only get your Provisioned CUs when using all Partitions Equally!
- 20. Throttle Non- Realtime Tasks •Purge old Data through a Throttled Message Queue •Spool Data with expected Redundancies and drain only the uniqued Changes (e.g. Facebook LiveAPI) •Scale your Worker Throughput based on CloudWatch Data
- 21. Use ElastiCache for hot Data •Write hot Data to ElastiCache and your DAL •Alter Hot Data in the Cache (e.g. Like- Counts) •Benefit e.g. from Increments •Read latest Data from Cache once you persist to DynamoDB
- 22. It is all well worth it.
- 23. Thank You!