Alexis Le Quoc's presentation on Diving into Nagios Analytics The presentation was given during the Nagios World Conference North America held Sept 25-28th, 2012 in Saint Paul, MN. For more information on the conference (including photos and videos), visit: http://go.nagios.com/nwcna

1. A Deep Dive into
Nagios Analytics
Alexis Lê-Quôc (@alq)
http://datadoghq.com

2. @alq
Dev & Ops
Nagios user since
2008
Datadog co-founder

3. A little survey

4. Top 3 failed checks

5. That woke me up
That I responded to
That I responded to 5 weeks ago
last week
Top 3 failed checks
That impacts our business
That most of my team the most?
responded to at least once

6. That woke me up
That I responded to
last week
That I responded to
5 weeks ago
Top 3 failed checks
That most of my team That impacts our business
responded to at least once the most?

7. At best, finding local optimums
Using memory to
prioritize
remediation...
At worst, brownian motion

8. Analytics

9. Performance Metrics Nagios Traffic Other Sources
In the “Cloud”

10. Nagios a “chatty” source
out of 40+ Datadog supports

11. One example

13. Almost 13000 Nagios “events”
over past week

14. Constant stream

15. 86 notifications!

16. Pattern

17. Pattern

18. More data?
More questions.

19. A dialog with data
Not a scientific study

20. Population
25% 50% 75% 100%
20 93 322 904

21. Does size matter?

22. Weekly Count per host split by quartile

23. Weekly count per host split by quartile
Outliers
Sick hosts,
silenced checks

24. Notifications

25. Notifications
1-3% of alerts notify
Little difference per quartile

26. Does time of day
matter?

28. Mean about the same
across quartiles
Time-based deviation?

29. Does the day of week
matter?

31. Not really

32. Squeaky wheels?
(checks)

33. Outlier

34. Outlier in more detail

35. Long Tail

36. Squeaky wheel?
(hosts)

37. Same outlier

38. Similar pattern as checks

39. Long Tail

40. Recurring alerts

41. Happen
s
Often
Seldom
happens
Young Old

42. Occur often, for a long time Tolerated
Happen once in a while

43. More data?
More questions.

44. HOWTO?

45. Awk
R
Find out tomorrow!
d3
Postgres

46. Presentation matters

48. Take-away?

49. Take-aways
• Don’t rely on your memory
• Your Nagios logs are a treasure trove
• Have a dialog with your data
• Presentation matters