This document discusses anomaly detection and root cause analysis in distributed application transactions. It introduces challenges in monitoring complex enterprise applications with massive and heterogeneous data. The solution presented uses machine learning and engineering to collect business transaction data, identify relevant fields that are abnormal, and determine the root cause. An example is given of how the system would detect and analyze unexpected slow transactions from one airline using a travel booking site.

1. Anomaly Detection and Root Cause Analysis
in Distributed Application Transactions
Yuchen Zhao @

2. Software is Eating the World

5. it’s critical to make sure
the software
is running properly

6. How?
Through monitoring!

7. Monitoring shouldn’t be very hard… right?

8. Well, it can become a bit more complex...

9. Or… really complex...

12. Keep applications running
is hard.

14. Challenge 1:
Enterprise applications are complex

15. Challenge 2:
Data is heterogeneous.
Its volume is massive and growing

16. Challenge 3:
Too many signals.
Finding anomalies & root causes
are non-trival.

17. Our solution: Relevant Fields
Machine Learning + Engineering

18. Q1: How to get & organize data?

19. Collect data in the form of
Business Transactions

21. Q2: Can you give a real use case?

22. A hypothetical travel booking site with data in BT

23. An unexpected incident:

25. Step 1: filtering

26. Step 2: find relevant fields

27. the relevancy score
“airline:AA” related transactions:
● 2% occurrence normally among all
travel bookings
● 82% of the current slow transactions
are from “AA”.
● 41 times more significant than normal.

28. What’s the root cause?

29. Step 3: take actions!

30. Q3: What’s the design of the system?

31. Architecture Overview

32. Data Collection

33. Smart Code Instrumentation
watch every line of code, self-learning, automatic

34. Stream Processing & Storage

35. Relevant Fields Processing

36. all transactions
(baseline)
error/slow
transactions
(query)
Baseline & Query Sets

37. Q4: How to score the field?

38. all transactions
(baseline)
error/slow
transactions
(query)

39. Optimization: Dynamics Baseline

40. Infer baseline context from query automatically
query
transactions
transactions
of Entity 1
query
transactions
transactions
of Entity 2
transactions
of Entity n

41. Baseline entity is auto learned from two dimensions:
● physical (applications, tiers, nodes, etc)
● temporal

42. Score Normalization
Normalize the score using a function derived from
sigmoid:

43. Score Example

44. Fore more details, please check out our demo paper in ICDM 2015:
Discovering Anomalies and Root Causes in Applications via Relevant Fields Analysis,
in Proceedings of the 15th IEEE International Conference on Data Mining

45. Ongoing work...
Support rich data types
● time series
● text
● graphs
● ...

46. We’re selling!

47. We’re hiring too!
Contact Mara
or
me yuchen.zhao@appdynamics.com

48. Thank you!