by Abe Stanway Senior Software Engineer, Planet Labs

1. BUILDING A DATA
DRIVEN ORGANIZATION
@AbeStanway

2. 1. why
2. how

3. 1. why
2. how

4. “DATA IS THE NEW GOLD”

5. Predict the future!

6. Retain Customers!

7. Grow the business!

8. Recommend content!

9. Drive Engagement!

10. unclear paths to $$$

11. IN IT, It’s clear.

12. Data are Dollars

13. .
IT Working = +$$$
IT Not Working = -$$$
.
.

14. How do you know if
your IT is working
right now?

15. How do you know if you
are earning money
right now?

16. KPIs. What are they?

17. Etsy:
Literally a Money per
second Graph

18. Planet Labs:
Literally an Images
per day graph

19. What are the
Kpis for kips?

20. $ per
second
items
bought per
second page
requests
per second
database
queries per
second memcache
hits per
second
fread()
per second

21. LEsson:
If you do not have the data
about your infrastructure,
it is already broken.

23. Test driven development
->
data driven development

24. Without data, you
are flying blind

25. How do you know you’re
hitting your goals?

26. How do you know if You’re
making the right ones in the
first place?

27. How do you know if
you’re still in
business?

28. How do you even
know what planet
you live on?

29. Assumptions are death

30. You need data, yo.

31. 1. why
2. how

32. 1. collect
2. analyze
3. ???
4. Profit!

33. 1. collect
2. analyze
3. ACT
4. Profit!

34. data that cannot be
acted upon should
not be analyzed.

36. You are running a
business, not an
art museum

37. You are Trying to
Win the market, not
a fields medal

38. This can be
disappointing

39. Data SCientist?

40. Data Scientist?
Realist.

41. Find a way to Align your
LEssoN:
employees intellectual curiosity
With your Real business needs.

42. Train your organization

43. you need a data culture.

44. “It’s not shipped
until it’s monitored”

45. “If you are not looking
at Dashboards, you are
not doing your job”

46. Building instrumentation
and watching dashboards
are hard And Time consuming

47. by hand by hand by hand by hand by hand
App code -> statsD -> Graphite -> Dashboards -> Insights

48. Developers just
want to code

49. Let’s
automate

50. Which is easier
to automate?
Insights or data
collection?

51. Insights are
sexy and fun

52. Collection is
hard And unsexy

53. Collection is
hard And Boring

54. Collection is
hard And unsexy

55. We’re on track to
have excellent
automated insights

56. anomaly
detection

57. by hand by hand by hand AUTOMATIC! AUTOMATIC!
App code -> statsD -> Graphite -> Dashboards -> Insights

58. (…if only we had the data)

59. How do we
automate data
collection?

60. currently have
ganglia, New relic,
collectD, etc

61. NOT WHAT
WE NEED

62. they provide data about
your raw machines, not your
CUSTOM DEVELOPED TECHNOLOGY
And Application level logic

63. Healthy servers
don’t make you money.
Healthy services do.

64. enter
LARIMAR

65. Full disclosure:
this is my new PROJECT
!
we’re going to talk about it
because i’m pretty excited and the
beta is opening up soon.

66. LARIMAR uses raw
machine metrics to infer
App level architecture
and inform developers
about problems

67. A service:
cpu resources
disk io
PCAP data
ports Used
syscalls

68. A service:
cpu resources
disk io
PCAP data
ports Used
MACHINE LEARNING
service fingerprint syscalls

69. A service:
cpu resources
disk io
PCAP data
ports Used
MACHINE LEARNING
ABNORMAL BEHAVIOR syscalls

70. a system:
service
service
service
service
service
MACHINE LEARNING
graphical system
fingerprint

71. a system:
service
service
service
service
service
MACHINE LEARNING
Abnormal, holistic
system behavior

72. Larimar automates
both analysis And
Relevant data
collection

73. so your developers
can focus on coding
and acting on insights

74. No configuration!

75. by hand AUTOMATIC! AUTOMATIC! AUTOMATIC! AUTOMATIC!
App code -> statsD -> Graphite -> Dashboards -> Insights

76. 1. collect
2. analyze
3. ACT
4. Profit!

77. organizational
shifts are still
needed to inspire
ACTION on Data

78. but ACTION is easier to
inspire when there is
lots of data and lots
of insight everywhere

79. Create a culture where
your developers create
these kinds of tools

80. When a data driven
mindset is the default,
tools will build
themselves.

81. Thanks!
@abestanway
!
!
larimar.io
@larimarhq