Kusto (Azure Data Explorer) Training for R&D - January 2019

1. Kusto
Azure Data Explorer
For Taboola LA R&D
Monitoring in Production
Maher Odeh (Taboola Production IT), Adi Eldar (Microsoft), Tal Bar Zvi (Taboola R&D) 1
https://youtu.be/iWay1PeoGhg
Click here to watch
the recorded session

2. Maher Odeh, Taboola
Production IT
2
Adi Eldar, Microsoft
Principal Data Scientist
Tal Bar Zvi, Taboola
R&D, User Data

3. 3

4. 4
Applause

5. Gift Economy
5

6. Goals of This Training
6
1. Kusto Queries
1. Dashboards
1. Alerts
1. Bonus: Data Science

7. Let’s begin
7

8. Kusto is...
8
A new way to look at data / logs
What is it actually?What is it actually?
It’s a new, innovative thing
Developed by Microsoft
We are one of the first to use it
It helps us to get the picture of our service in a few
seconds
What is it actually?
It’s a new, innovative thing
Developed by Microsoft
We are one of the first to use it
It helps us to get the picture of our service in a few
seconds
Wow, sounds cool

9. True Story
9

10. Now Really Let’s begin
10
✓ Big Data
✓ Database
✓ Tables
✓ Functions
✓ Scripting
✓ Join
✓ Union
✓ Fast Search
✓ Graphs
✓ Dashboards
✓ Alerts
✓ HTTP Logs (for now)
✓ Notebooks
✓ Python

11. Why Kusto?
● Kibana-Fastly replacement
● It has a WOW effect
● It is easy to use and learn
11
It is new, for all,
we learn it together.
(This is Rare!)

12. 12
Different payment model.
Kusto is already paid - flat.
Queries do not* cost extra money.
*prod-it are gonna hate me after this slide
✓ Credits: Shaked Zychlinski

13. What’s in?
13

14. Which Data / Logs Are In Kusto?
14

15. 15
Request
URL
Referrer
HTTP Status
Response Time
+
DC
Server IP
more...

16. 16
Javascript files (*.js)
(loader.js, impl, newsroom, userx...)
Image files (*.jpg, *.png...)
Events (available, visible, click, social,
debug, performance…)
Etc.

17. Architecture
17
Log files - from Fastly (CDN)
Kusto
Web interface
● Query
● Graphs
● Dashboards
API
● Alerts (Sensu)
● Scripting
● Jupyter
● Programming

18. What is a CDN (Fastly & Akamai for example)?
18
50 server farms
7 Data Centers
Caching our HTTP responses
HTTP Logs
CDN = Content Delivery Network

19. Kusto Database Sizes (as of Jan 2019)
19
Database Size RETENTION
COLD / HOT (CACHED)
fastly-
backstage
15 GB 60 days (31 days 🔥)
fastly-c3 10 TB 30 days (3 days 🔥)
fastly-trc 250 TB 30 days (3 days 🔥)

20. SLIDE | 20
Take Away
Messages No. 1
20
1. Kusto is BigData database
1. It holds our HTTP requests
1. Hot vs. Cold

21. Query
21

22. 22
Tabs
Select:
Cluster & Database
Docs Settings
Output
Query
Tabs, Statistics, Info
Column
Selection
Pivoting
Deep link sharingExport ImportRun Recall output
Documentation

23. Click on Query Language
23

24. Query - KQL
24
● Query = statement ; statement ; ….. ; statement
● At least one statement is a tabular expression
● Returns result back
source |
operator1 |
[ | operator2 ]
[ | render ]
(Taboolar?!)

25. KQL vs. SQL
25

26. Example No. 1 of 7
26
● trc_access | count
Hot vs. Cold...

27. Example No. 1 of 7 - corrected
27
● trc_access | where timestamp > ago(1d) | count

28. Example No. 2 of 7 - by publisher
28
trc_access |
where timestamp > ago(1d) |
where publisher_name == ‘msn-msn’ |
count

29. Example No. 3 of 7 - take (like “limit”)
29
trc_access |
where timestamp > ago(5m) |
where publisher_name == ‘msn-msn’ |
take 5Geo Referrer Time Action URL

30. Example No. 4 of 7 - summarize & top
30
trc_access | where timestamp > ago(1h) |
summarize count() by geo_country_code |
top 5 by count_ desc;
trc_access | where timestamp > ago(1h) |
summarize count() by action |
top 5 by count_ desc
; Semicolon

31. Example No. 5 of 7 - render
31
trc_access | where timestamp > ago(1h) |
summarize count() by geo_country_code |
top 5 by count_ desc; | render piechart
WOW

32. Alert:
32

33. Example No. 6 of 7 - timechart
33
trc_access | where timestamp > ago(10d) |
summarize count() by bin(timestamp, 30m) |
render timechart

34. Example No. 7 of 7 - extract & extend
34

35. Complex Example
True Story from Last Week
35

36. Exmple - HTTP errors, where? what?
36
● Step 1 - See HTTP error increased
● Step 2 - Summarize by data center
● Step 3 - Summarize by action
● Step 4 - Union with normal traffic
HTTP Error Spike
Step 1
NJ & CH are
suffering
Step 2Step 2`Step 3
Found the actionsUnion
Project
Alias
Low
errors
Normal
Traffic
Both Normal
and Errors rise
Errors
Gone

37. SLIDE | 37
Take Away
Messages No. 2
37
1. Kusto has fast query capacities
1. It can create graphs
1. Can aggregate and create fields on-the-fly
1. Helps in:
a. Find root cause
b. Traffic sampling
c. Insights & trends
d. Integration validations

38. Brain Notebooks
38

39. Want more use cases? Use Brain. Sharing is Caring.
39
Team’s
wisdom
Your personal
wisdom

40. Here
some
40
Click to run on Kusto
(deep link)
Calculates response
time percentiles
Credits:
Taboola News

41. Some
more
41
1. action == ‘json’
2. unkown pub
3. extend data (add column)
4. url_decode(%20 - out)
5. parse_json
6. extend (again)
7. project
8. summarize by pub, json field
9. top 30 by count
Credits:
Taboola Mobile

42. Kusto Community
42

43. #kusto @kusto-mentors
43

44. SLIDE | 44
Take Away
Messages No. 3
44
1. Use Slack and Brain to share
1. Document your usage for others to learn

45. Dashboards
Lens Explorer

46. 46

47. 47

48. 48
Lens Explorer - Rich Data Visualisations

49. Alerts

50. 50
Kusto Sensu Integration
Elastic based check
Same check w/ Kusto

51. 51
Alerts (using Sensu)
Period &
Threshold
Kusto
Query

52. Data Science
&
Notebooks

53. Jupyter Notebooks - Kqlmagic (Azure & Locally)
53
Kqlmagic Connect
Run queries
Output saved
Standardized

54. Use make-series (it’s fast)
To see the HTTP error
spike
Remember the example from 15 min. ago?
54
Use autocluster to find
similar error characteristics
DC is CH
Newsroom
affected
This is
the host
Using diffpatterns to find
clues
DC is CH Newsroom
affected
This is
the host

55. Kusto + Grafana

56. 56

57. Summary
57
1. You know where to find me (tal.b@taboola.com)
1. You know you have accessible Resources
(Brain, WWW, Pluralsight free course, Videos, #kusto, Microsoft)
1. You saw how easy it is to run Kusto queries
1. You saw that there are Dashboards & Alerts
1. You are aware of the existence of built-in Data Science power

58. @kusto-mentors
58

59. Thank You
59

60. FAQ
60
1. Does it cost money? It is prepaid
2. What about Kibana, Grafana, BQ? Here to stay for now
3. What about applicative logs / my data? Currently Fastly logs
4. Will my elastic-fastly alerts be converted to Kusto for me? No
5. When will the other fastly logs be available? Updates in slack #kusto
6. Can we have more Kusto trainings? Dashboard? Workshops? Yes
7. Does Kusto support distinct count? Yes
8. Does Kusto have materialized views? Yes
9. Can we add to the schema our common recommendation fields? Yes
10. What about API 2.0 HTTP POST payload? It is in discussions
11. Can I look in all fields like in Kibana? Yes
12. Do all have access? Many have, or else ticket to prod-it
13. Can I use the alerts? Work in progress
14. Can I automatically derive smaller tables? Yes

61. Demo Time!
61

62. Pics - Atmosphere
62