To operate MongoDB efficiently, you need to have insight into database performance. And with that in mind, we’ll dive into monitoring in this second webinar in the ‘Become a MongoDB DBA’ series. MongoDB offers many metrics through various status overviews and commands, but which ones really matter to you? How do you trend and alert on them? What is the meaning behind the metrics? We’ll discuss the most important ones and describe them in ordinary plain MySQL DBA language. And we’ll have a look at the open source tools available for MongoDB monitoring and trending. Finally, we’ll show you how to leverage ClusterControl’s MongoDB metrics, dashboards, custom alerting and other features to track and optimize the performance of your system. AGENDA How does MongoDB monitoring compare to MySQL Key MongoDB metrics to know about Trending or alerting? Available open source MongoDB monitoring tools How to monitor MongoDB using ClusterControl Demo SPEAKER Art van Scheppingen is a Senior Support Engineer at Severalnines. He’s a pragmatic MySQL and Database expert with over 15 years experience in web development. He previously worked at Spil Games as Head of Database Engineering, where he kept a broad vision upon the whole database environment: from MySQL to Couchbase, Vertica to Hadoop and from Sphinx Search to SOLR. He regularly presents his work and projects at various conferences (Percona Live, FOSDEM) and related meetups.

1. Confidential
MongoDB Monitoring
Become a MongoDB DBA - What to Monitor (If you’re really a MySQLer)
Art van Scheppingen
Senior Support Engineer, Severalnines

2. Confidential
Logistics
☐ Webinar is recorded
☐ Replay available soon
☐ Feel free to ask questions at any time
☐ Use your control panel to contact us
☐ Or email us as well: info@severalnines.com

3. Confidential
Agenda
☐ Trending and alerting
☐ Why do we collect data?
☐ How does MongoDB monitoring compare to MySQL
☐ Key MongoDB metrics to know about
☐ Available open source MongoDB monitoring tools
☐ How to monitor MongoDB using ClusterControl
☐ Demo

4. Confidential
Trending and alerting

5. Confidential
Why do you need a good trending system?

6. Confidential

7. Confidential
Monitoring vs trending
☐ Monitoring system (i.e. Nagios)
☐ Checks if services are healthy
☐ Sends pages
☐ Trending system (i.e. Cacti, Graphite)
☐ Collects metrics
☐ Generate graphs

8. Confidential
Monitoring: Availability
☐ Do more than just opening a connection
☐ Measure true status of nodes and cluster
☐ Test read/write
☐ Open essential databases and collections
☐ Keep an eye on the replication lag
☐ Increase oplog size?
☐ Check the full topology

9. Confidential
Trending: Why do we need trends?
☐ Trending
☐ Plot trends of key (performance) metrics
☐ Find problems before they arise
☐ Pre-emptive problem management
☐ Trending tools
☐ Granularity of sampling
☐ More datapoints = better

10. Confidential
Why do we collect data?

11. Confidential
Why do we collect data?
☐ Periodical (daily/weekly) healthchecks
☐ Insight into all aspects of the database operations
☐ Post mortem and proactive monitoring
☐ Capacity planning

12. Confidential
Healthchecks
☐ Healthchecks are a pain
☐ You want to see aggregated
data
☐ You want to be able to drill down
to a particular host
☐ You want to see the most
important data first and dig in
later on

13. Confidential
Post mortem and proactive monitoring
☐ Ability to dig into past data
☐ Even less than 5s of data
granularity (hardware-
dependent)
☐ Low granularity allows you to
catch the issue as it evolves -
no need to wait 5 minutes for a
graph to refresh

14. Confidential
Insight into internals, capacity planning
☐ Graphs based on MySQL
status counters
☐ Overall status and per-node
graphs
☐ Ability to get a timeshifted
graphs - useful for comparing
workload changes across the
time

15. Confidential
How does MongoDB monitoring compare to MySQL

16. Confidential
Type of metrics to collect
☐ Quite similar to MySQL
☐ Host metrics
☐ Operational metrics
☐ Storage engine metrics
☐ Replication metrics

17. Confidential
Host Metrics: what for?
☐ Similar to most other databases
☐ Understand the utilization of the hardware
☐ Capacity planning
☐ Determine the type of an issue
☐ I/O related?
☐ CPU related?
☐ Network related?

18. Confidential
Host Metrics: what to look for?
☐ CPU utilization (should I add more nodes to the cluster?)
☐ Network utilization (am I running out of bandwidth?)
☐ Ping (how badly latency affects my MongoDB cluster?)
☐ Disk throughput and IOPS (am I within my hardware limits?)
☐ Disk space (do I have to plan for larger disks?)
☐ Memory utilization (do I suffer from a memory leak?)

19. Confidential
Operational metrics
☐ Similar to most other databases
☐ Throughput of the cluster
☐ Relate throughput to cluster performance
☐ Determine the type of an issue
☐ Request spikes?
☐ Write amplification related?
☐ Queueing?

20. Confidential
Storage engine metrics
☐ Storage engine specific
☐ MMAP
☐ Wired Tiger
☐ MongoRocks
☐ Insight in how the engine performs
☐ Internal congestion

21. Confidential
Replication Metrics
☐ Throughput of the replication
☐ Durability of the oplog
☐ Replication lag
☐ Comparable to Galera replication
☐ Quorum based
☐ At least one secondary needs to acknowledge

22. Confidential
Eventual consistency

23. Confidential
Key MongoDB metrics to know about

24. Confidential
Oplog
☐ Similar to MySQL binary logs
☐ Oplog: a special collection
☐ Limited size
☐ Eviction of transactions (FIFO)
☐ Replication window
☐ Time between first and last transaction in the oplog

25. Confidential
Oplog
mongo_replica_0:PRIMARY> db.getReplicationInfo()
{
"logSizeMB" : 1895.7751951217651,
"usedMB" : 419.86,
"timeDiff" : 281419,
"timeDiffHours" : 78.17,
"tFirst" : "Fri Jul 08 2016 10:56:01 GMT+0000 (UTC)",
"tLast" : "Mon Jul 11 2016 17:06:20 GMT+0000 (UTC)",
"now" : "Mon Jul 11 2016 17:15:06 GMT+0000 (UTC)"
}

26. Confidential
Replication lag
☐ CPU, IO or lock related
☐ Outcome: similar to Galera
☐ Lagging behind could cause a full sync

27. Confidential
Replication lag
my_mongodb_0:PRIMARY> db.runCommand( { replSetGetStatus: 1 } ) {
…
"members" : [
{
"_id" : 0,
"name" : "10.10.32.11:27017",
"stateStr" : "PRIMARY",
"optime" : {
"ts" : Timestamp(1466247801, 5),
"t" : NumberLong(1)
},
},
{
"_id" : 1,
"name" : "10.10.32.12:27017",
"stateStr" : "SECONDARY",
"optime" : {
"ts" : Timestamp(1466247801, 5),
"t" : NumberLong(1)
},
},
…
],
"ok" : 1
}

28. Confidential
Connections
☐ Similar to MySQL when handling connections
☐ Client drivers may support connection pooling
☐ Multiple non-blocking queries can use the same connection
☐ Spawns new connections when low on threshold
☐ Increase of connections
☐ Locking issues
☐ Application request bursts

29. Confidential
Connections
mongo_replica_0:PRIMARY> db.serverStatus().connections
{ "current" : 25, "available" : 794, "totalCreated" : NumberLong(122418) }

30. Confidential
Transactions
☐ Atomicity on document level
☐ No “real” transactions
☐ Write data with the $isolated operator
☐ Similar to READ UNCOMMITTED
☐ No rollback
☐ Does not work on shards

31. Confidential
Transactions
mongo_replica_0:PRIMARY> db.serverStatus().opcounters
{
"insert" : 1355272,
"query" : 20712,
"update" : 8995,
"delete" : 0,
"getmore" : 400791,
"command" : 2405749
}

32. Confidential
Locks (generic)
☐ Three levels of (generic) locking
☐ Global
☐ Database
☐ Collection

33. Confidential
Locks (generic)
mongo_replica_0:PRIMARY> db.serverStatus().locks
{
"Global" : {
"acquireCount" : {
"r" : NumberLong(6050583),
"w" : NumberLong(2416551),
"R" : NumberLong(1),
"W" : NumberLong(7)
},
"acquireWaitCount" : {
"r" : NumberLong(1),
"w" : NumberLong(1),
"W" : NumberLong(1)
},
…
}

34. Confidential
Locks (WiredTiger)
☐ Document level locking
☐ Tickets (threads)
☐ Read
☐ Write

35. Confidential
Locks (WiredTiger)
mongo_replica_0:PRIMARY> db.serverStatus().wiredTiger.concurrentTransactions
{
"write" : {
"out" : 0,
"available" : 128,
"totalTickets" : 128
},
"read" : {
"out" : 0,
"available" : 128,
"totalTickets" : 128
}
}

36. Confidential
Cache
☐ MongoDB uses three tiers of cache
☐ Filesystem
☐ Active memory
☐ Storage engine (WiredTiger / MongoRocks)
☐ Page faults
☐ Evictions

37. Confidential
Page faults and cache usage
mongo_replica_0:PRIMARY> db.serverStatus().extra_info.page_faults
37912924
mongo_replica_0:PRIMARY> db.serverStatus().wiredTiger.cache
{
"bytes currently in the cache" : 887889617,
"modified pages evicted" : 561514,
"tracked dirty pages in the cache" : 626,
"unmodified pages evicted" : 15823118
}

38. Confidential
Available open source MongoDB monitoring tools

39. Confidential
Alerting Solutions
☐ Open Source
☐ Nagios
☐ Zabbix
☐ Subscription based
☐ MongoDB Cloud Manager
☐ VividCortex
☐ ClusterControl

40. Confidential
Nagios
☐ Nagios-MongoDB
☐ https://github.com/mzupan/nagios-plugin-mongodb/
☐ Performs some very important checks
☐ Replication lag
☐ Lock time percentage
☐ Index miss ratio

41. Confidential
Zabbix
☐ MongoDB Zabbix monitoring plugin
☐ https://github.com/nightw/mikoomi-zabbix-mongodb-monitoring
☐ All the necessary metrics and more
☐ Entries in oplog
☐ Pre-canned triggers

42. Confidential
Trending Solutions
☐ Trending tools
☐ Statsd/Grafana
☐ Cacti
☐ Zabbix
☐ Subscription based
☐ MongoDB Cloud Manager
☐ VividCortex
☐ ClusterControl

43. Confidential
Cacti
☐ Percona MongoDB Monitoring Templates
☐ https://www.percona.com/doc/percona-monitoring-plugins/1.
1/cacti/mongodb-templates.html

44. Confidential
Cacti

45. Confidential
Orchestration systems: Percona Monitoring & Management
☐ PMM
☐ https://www.percona.com/doc/percona-monitoring-and-management/
☐ Open Source Monitoring & Management framework
☐ Can deploy, manage and monitor MySQL & MongoDB
☐ Uses Prometheus and Grafana
☐ Still in Beta

46. Confidential
How to monitor MongoDB using ClusterControl

47. Confidential
Orchestration Systems: ClusterControl
☐ ClusterControl
☐ http://www.severalnines.com
☐ Deploy Mongo shards & replicasets
☐ Monitor and trend
☐ Manage configuration and backups
☐ Scale
☐ Community edition

48. Confidential
Deploy Mongo replicaSets and Shards

49. Confidential
Monitor and Trend

50. Confidential
Management

51. Confidential
Scale

52. Confidential
Demo

53. Confidential
Q & A

54. Confidential
Additional Resources
☐ Blog series: Become a MongoDB DBA
☐ http://severalnines.com/blog-categories/mongodb
☐ Webinar series: Become a MongoDB DBA
☐ http://severalnines.com/upcoming-webinars
☐ Visit our website for more resources!
☐ http://www.severalnines.com