1. Monitoring for the
Public Good
Mike Julian
Friday, May 3, 13

2. Who am I?
• ORNL operations staff
• LOPSA Technical Staff
• Generalist IT engineer for ~10 years
Friday, May 3, 13

3. Who am I?
• ORNL operations staff
• LOPSA Technical Staff
• Generalist IT engineer for ~10 years
• I am obsessed with monitoring.
Friday, May 3, 13

4. Who am I?
• ORNL operations staff
• LOPSA Technical Staff
• Generalist IT engineer for ~10 years
Friday, May 3, 13

5. Who am I?
• ORNL operations staff
• LOPSA Technical Staff
• Generalist IT engineer for ~10 years
• I am obsessed with data.
Friday, May 3, 13

6. A dashboard is a visual display
of the most important
information needed to
achieve one or more
objectives; consolidated and
arranged on a single screen so
the information can be
monitored at a glance.
What is a dashboard?
* Stephen Few, Intelligent Enterprise
Friday, May 3, 13
The highlighted bits are what I think are the most important points to remember.

7. But first...
You need some data!
Friday, May 3, 13
Visualization is a lot of fun, but it’s no use if you don’t have the right data to display. Finding the data is even more fun, and
potentially much more challenging.
So, what data should you get. That depends on what you need to show.

8. Let’s say you wanted
to show...
• Bandwidth usage: border router(s), SNMP
query
• Peak usage hours: databases, network
traffic, VPN usage
• User demographics: lots of database work.
• There’s always the case that the data may
not exist.
Friday, May 3, 13
These are just a couple examples of information you might want to show, and where you might find the data for them.

9. Your data is all over
the place
• ERP, CRM
• Your ticket system
• Product databases
• Google Analytics
• Operational monitoring systems (eg,
Nagios)
Friday, May 3, 13
Your data is stored in lots of places. Google Analytics, your operational monitoring systems, various databases...
This is where the engineer in you shines: use your language of choice, SQL, and knowledge of your infrastructure to find the
data and do the necessary magic on it to turn it into meaningful information.

10. Building information dashboards: an
exercise in not thinking like an ops
engineer.
An exercise in not thinking Ops
Friday, May 3, 13

11. “What gets measured, gets managed.”
- Peter Drucker, management guru
Friday, May 3, 13

12. Key Performance Indicator (KPI)
A way to measure the progress
of a service, program, or initiative
towards the goals.
Friday, May 3, 13
The purpose of the KPI is to drive change.
How do you know you’re doing better or worse without the numbers?
KPIs differ depending on use. Bandwidth utilization is a KPI for a datacenter, but perhaps not for a doctor’s office.
In fact, they differ within companies. New customer acquisitions is a KPI for marketing and sales, but not for IT. Likewise,
Finance doesn’t care about utilization metrics of the ERP system.

13. KPI examples
Friday, May 3, 13

14. You are the CEO of a web hosting
company. What are your KPIs?
KPI examples
Friday, May 3, 13

15. You are the CEO of a web hosting
company. What are your KPIs?
•Total sales (dollar amount)
•Total expenses (dollar amount)
•Profit margin (percentage)
•Number of new customers
•Number of cancellations
KPI examples
Friday, May 3, 13

16. You are the CEO of a web hosting
company. What are your KPIs?
•Total sales (dollar amount)
•Total expenses (dollar amount)
•Profit margin (percentage)
•Number of new customers
•Number of cancellations
You are the customer service
manager for a call center.
What are your KPIs?
KPI examples
Friday, May 3, 13

17. You are the CEO of a web hosting
company. What are your KPIs?
•Total sales (dollar amount)
•Total expenses (dollar amount)
•Profit margin (percentage)
•Number of new customers
•Number of cancellations
You are the customer service
manager for a call center.
What are your KPIs?
•Call volume
•Call length
•Queue wait time
•Number of call abandons
KPI examples
Friday, May 3, 13

18. Let’s build a dashboard!
Friday, May 3, 13

19. Let’s build a dashboard!
LOPSA Board of Directors Dashboard
Friday, May 3, 13

20. Let’s build a dashboard!
LOPSA Board of Directors Dashboard
KPIs
Friday, May 3, 13

21. Let’s build a dashboard!
LOPSA Board of Directors Dashboard
Membership
•New members
•Lapsed members
•Member satisfaction
KPIs
Friday, May 3, 13

22. Let’s build a dashboard!
LOPSA Board of Directors Dashboard
Membership
•New members
•Lapsed members
•Member satisfaction
Financial
•Program income
•Contribution income
•Program expenses
•Operating expenses
KPIs
Friday, May 3, 13

23. Let’s build a dashboard!
LOPSA Board of Directors Dashboard
Membership
•New members
•Lapsed members
•Member satisfaction
Financial
•Program income
•Contribution income
•Program expenses
•Operating expenses
Locals Program
•Chapter attendance
•New chapters
•Conversions
KPIs
Friday, May 3, 13

24. Let’s build a dashboard!
LOPSA Board of Directors Dashboard
Membership
•New members
•Lapsed members
•Member satisfaction
Financial
•Program income
•Contribution income
•Program expenses
•Operating expenses
Locals Program
•Chapter attendance
•New chapters
•Conversions
Mentorship Program
•Pairing requests
•Pairing assignments
•Pair ratings
KPIs
Friday, May 3, 13

25. Dashboard design tips
• Know your audience
• Only display relevant data
• Both to your audience and your topic
• Watch your granularity
• Keep it simple
Friday, May 3, 13
Your audiences care about different things. Even among IT teams, different metrics matter. Keep this in mind.
Granularity: a dashboard is high-level. Look for the balance between showing enough information and too much information.
Keep it simple. Eye-candy is great, but don’t overdo it: the purpose of a dashboard is to answer questions quickly.

26. (numbers used not actual values)
Friday, May 3, 13
Visualization was done with D3.js

27. What does YOUR boss care about?
Friday, May 3, 13

28. Let’s try another...
Friday, May 3, 13
KPIs do not have to be quantitative. Sometimes they’re qualitative.
Think of things this way: what information would your boss wish he had in a staff meeting with other managers? What
information shows how well your department is doing?

29. Let’s try another...
An IT Manager’s dashboard
Friday, May 3, 13
KPIs do not have to be quantitative. Sometimes they’re qualitative.
Think of things this way: what information would your boss wish he had in a staff meeting with other managers? What
information shows how well your department is doing?

30. KPIs:
•Support request volume
•Incident resolution time
•Customer satisfaction (survey)
•Infrastructure availability %
•Significant system events
•Upcoming changes
Let’s try another...
An IT Manager’s dashboard
Friday, May 3, 13
KPIs do not have to be quantitative. Sometimes they’re qualitative.
Think of things this way: what information would your boss wish he had in a staff meeting with other managers? What
information shows how well your department is doing?

31. That’s cool and all, but we’re
engineers...
Friday, May 3, 13
But Mike, we’re engineers! What has all this got to do with us?

32. No problem!
Friday, May 3, 13
It’s got everything to do with you!
This has given you some insight into how your boss and executives think. If you start answering their questions before they can
even ask them, what do you think they’re going to do? You’ve just become more valuable to them, and you’ve developed a
crucial skill.
Remember, IT supports the business.

33. While you’re there...
(or: interesting data)
What else is in your data that no one has
thought to ask about?
Friday, May 3, 13
In the course of working on LOPSA projects, I realized an interesting project could be built.

34. LOPSA Membership Heatmap
* No PII was used in the
creation of this map
Friday, May 3, 13
Don’t worry, no personally identifiable information was used in the creation of this map.
This map is fairly obvious: a heat map of active LOPSA members. It’s done in the Google Maps API. On the live version, you can
zoom in and see more granularity. At this zoom level, LA and San Diego are just one blob, while they are clearly distinct at a
closer zoom.
I just used a SQL query, Python, and Javascript for this.

35. Resources
• Information Dashboard Design by Stephen Few
• Actually, anything by Stephen Few.
• What Your CEO Wants You to Know by Ram Charan
• For learning how to think like management
• A Practical Intro to Data Science - Zipfian Academy blog
• Deep intro into data science--for those who really want to get
into it
• D3.js (d3js.org)
• Awesome visualization library
Friday, May 3, 13

36. That’s all, folks.
mike@mikejulian.com
Friday, May 3, 13

37. Thank You for Attending LOPSA-East '13
Please fill out the Trainer Evaluation
http://lopsa-east.org/2013/training-survey
Rate LOPSA-East '13
http://www.lopsa-east.org/2013/rate-lopsa-east-13
Friday, May 3, 13