1. #MongoDBDays
Capacity Planning:
Deploying MongoDB
Asya Kamsky
Solutions Architect, 10gen

2. #MongoDBDays
Capacity Planning:
Deploying MongoDB
Asya Kamsky
Solutions Architect, 10gen

3. Capacity Planning: Why, What, When
• Don't be the "goat" – spent too much $ or caused
failure of site, etc.
• Users frequently ask about HW they need for their
application. What does 10gen "recommend"?
• No right answer in a vacuum.
• Why we need to plan to meet expectations, etc.
– future planning
• data increases, don't want performance drop-off
Capacity Planning, Asya Kamsky, 10gen

4. Capacity Planning: Why, What, When
Why?
What are the consequences of not planning?
Capacity Planning, Asya Kamsky, 10gen

5. Why
• Once we launch, we don't want to have avoidable
down time due to poorly selected HW
• As our success grows we want to stay in front of
the demand curve
• We want to meet business' and users' expectations
• We want to keep our jobs J
• and get big raises! ;)
Capacity Planning, Asya Kamsky, 10gen

6. Capacity Planning: Why, What, When
Why?
What are the consequences of not planning?
Capacity Planning, Asya Kamsky, 10gen

7. Why
• We want to keep our jobs J
• and get big raises! ;)
• so we should stay within reasonable budget
Capacity Planning, Asya Kamsky, 10gen

8. Capacity Planning: Why, What, When
What?
Why?
Requirements
Capacity Planning, Asya Kamsky, 10gen

9. What
• There is one thing that is absolutely mandatory to
have in order to succeed in capacity planning
• Without it, you will not be successful
• We must have REQUIREMENTS from business
– without requirements, we're building a roadmap without
knowing the desired destination
Imagine building a car without knowing what its top speed
should be, acceleration, MPH, and cost?
Capacity Planning, Asya Kamsky, 10gen

10. Capacity Planning: Why, What, When
What?
• Availability
• Throughput
• Responsiveness
Capacity Planning, Asya Kamsky, 10gen

11. What
• Availability: what is uptime requirement?
• Throughput
– average read/write/users
– peak throughput?
– OPS (operations per second)? per hour? per day?
• Responsiveness
– what is acceptable latency?
– is higher during peak times acceptable?
Capacity Planning, Asya Kamsky, 10gen

12. Capacity Planning: Why, What, When
What?
• Availability
• Throughput
• Responsiveness
Capacity Planning, Asya Kamsky, 10gen

13. Capacity Planning: Why, What, When
When?
Before it's too late!
Start Launch Version 2
Capacity Planning, Asya Kamsky, 10gen

14. Capacity Planning: Why?
• Capacity
– Under
– Over
– Just right?
• Prediction Models
– User/Load
– System(s) Behavior
• Change Velocity (reaction time)
– Data/Resource-Allocation/Provisioning
Capacity Planning, Asya Kamsky, 10gen

15. Capacity Planning: What?
• Understand Resources
– Storage
– Memory
– CPU
– Network
• Understand Your Application
– Monitor and Collect Metrics
– Model to Predict Change
– Allocate and Deploy
– (repeat process)
Capacity Planning, Asya Kamsky, 10gen

16. Resource Usage
• Storage • CPU
– IOPS – Speed
– Size – Cores
– Data & Loading Patterns
• Memory • Network
– Latency
– Working Set – Throughput
Capacity Planning, Asya Kamsky, 10gen

17. Storage
• Active
• Archival
• Loading Patterns
• Integration (BI/DW)
Capacity Planning, Asya Kamsky, 10gen

18. Storage
Example IOPS
• Active
• Archival
• Loading Patterns
• Integration (BI/DW)
Capacity Planning, Asya Kamsky, 10gen

19. Storage Capability
Example IOPS
7,200 rpm SATA ~ 75-100 IOPS
15,000 rpm SAS ~ 175-210 IOPS
Amazon EBS/Provisioned ~ 100 IOPS "up to" 2,000 IOPS
Amazon SSD 9,000 – 120,000 IOPS
Capacity Planning, Asya Kamsky, 10gen

20. Storage Capability
Example IOPS
7,200 rpm SATA ~ 75-100 IOPS
15,000 rpm SAS ~ 175-210 IOPS
Amazon EBS/Provisioned ~ 100 IOPS "up to" 2,000 IOPS
Amazon SSD 9,000 – 120,000 IOPS
Intel X25-E (SLC) ~ 5,000 IOPS
Fusion IO ~ 135,000 IOPS
Violin Memory 6000 ~ 1,000,000 IOPS
Capacity Planning, Asya Kamsky, 10gen

21. Storage Costs
Cost of IOPS
7,200 rpm SATA ~ 75-100 IOPS
15,000 rpm SAS ~ 175-210 IOPS
Amazon EBS/Provisioned ~ 100 IOPS "up to" 2,000 IOPS
Amazon SSD 9,000 – 120,000 IOPS
Intel X25-E (SLC) ~ 5,000 IOPS
Fusion IO ~ 135,000 IOPS
Violin Memory 6000 ~ 1,000,000 IOPS
Capacity Planning, Asya Kamsky, 10gen

22. Storage
Measuring and Monitoring
Capacity Planning, Asya Kamsky, 10gen

23. Storage
Measuring and Monitoring
Capacity Planning, Asya Kamsky, 10gen

24. Storage
Measuring and Monitoring
Capacity Planning, Asya Kamsky, 10gen

25. Memory
• Working Set
– Active Data in Memory
– Measured Over Periods
Capacity Planning, Asya Kamsky, 10gen

26. Memory
• Work: SORTS
– Sorting Connections
– Aggregation
– Connections Aggregations
Capacity Planning, Asya Kamsky, 10gen

27. Memory
Measuring and Monitoring
• New in 2.4 (just out)
– workingSet option on db.serverStatus()
db.serverStatus( { workingSet: 1 } )
Capacity Planning, Asya Kamsky, 10gen

28. Memory & Storage
?
>
<
Capacity Planning, Asya Kamsky, 10gen

29. Memory & Storage
MOPS: MongoDB Ops/sec
Capacity Planning, Asya Kamsky, 10gen

30. Memory & Storage
MOPs
MOPS: MongoDB Ops/sec
PFs
Capacity Planning, Asya Kamsky, 10gen

31. Memory & Storage
% Disk Util
MOPS
Capacity Planning, Asya Kamsky, 10gen

32. CPU
• Non-indexed Data
• Sorting
• Aggregation
– Map/Reduce
– Framework
• Data
– Fields
– Nesting
– Arrays/Embedded-Docs
Capacity Planning, Asya Kamsky, 10gen

33. CPU
MOPs
Capacity Planning, Asya Kamsky, 10gen

34. CPU
MOPs
CPU %
Capacity Planning, Asya Kamsky, 10gen

35. Memory & Storage
% Disk Util
MOPS
Capacity Planning, Asya Kamsky, 10gen

36. Network
• Latency
– WriteConcern
– ReadPreference
– Batching
– Documents (and Collections)
• Throughput
– Update/Write Patterns
– Reads/Queries
Capacity Planning, Asya Kamsky, 10gen

37. Starter Questions
• What is the working set?
– How does that equate to memory
– How much disk access will that require
• How efficient are the queries?
• What is the rate of data change?
• How big are the highs and lows?
Capacity Planning, Asya Kamsky, 10gen

38. Deployment Types
All of these use the same resources:
• Single Instance
• Multiple Instances (Replica Set)
• Cluster (Sharding)
• Data Centers
Capacity Planning, Asya Kamsky, 10gen

39. Capacity Planning: When?
Monitoring
§ Storage
§ Memory
§ CPU
§ Network
§ Application Metrics
Capacity Planning, Asya Kamsky, 10gen

40. Tools
• MMS (MongoDB Monitoring Service)
• MongoDB: mongotop, mongostat
• Linux: iostat, vmstat, sar, etc
• Windows: Perfmon
Measure realistic loads (generated by Load testing)
Capacity Planning, Asya Kamsky, 10gen

41. Models
• Load/Users
– Response Time/TTFB
• System Performance
– Peak Usage
– Min/avg Usage
Capacity Planning, Asya Kamsky, 10gen

42. Velocity of Change
• Limitations -> takes time
– Data Movement
– Allocation/Provisioning (servers/mem/disk)
• Improvement
– Limit Size of Change (if you can)
– Increase Frequency
– MEASURE its effect
– Practice
Capacity Planning, Asya Kamsky, 10gen

43. Repeat (continuously)
• Repeat Testing
• Repeat Evaluations
• Repeat Deployment
Capacity Planning, Asya Kamsky, 10gen

44. Capacity Planning: What If...
What if I skip capacity planning?
You will be featured ...
Capacity Planning, Asya Kamsky, 10gen

45. #MongoDBDays
Thank You
Asya Kamsky
Senior Solutions Architect, 10gen