The ability to instrument and interrogate data as it moves through a processing pipeline is fundamental to effective machine learning at scale. Applied in this capacity, information visualization technologies drive product innovation, shorten iteration cycles, reduce uncertainty, and ultimately improve the performance of predictive models. It can be challenging, however, to understand where in a workflow to employ data visualization, and, once committed to doing so, developing revealing visualizations that suggest clear next steps can be similarly daunting. In this talk we’ll describe the role that information visualization technologies play in the LinkedIn data science ecosystem, and explore best practices for understanding the structure of large-scale data in a production environment. From hypothesis generation and feature development to model evaluation and tooling, visualization is at the heart of LinkedIn’s machine learning workflows, enabling our data scientists to reason and communicate more effectively. Broken down into clear, structured insights based on proven technology and workflow patterns, this talk will help you understand how to apply information visualization to the analytical challenges you encounter every day. Meet Michael Conover Mike Conover builds machine learning technologies that leverage the behavior and relationships of hundreds of millions of people. A senior data scientist at LinkedIn, Mike has a Ph.D. in complex systems analysis with a focus on information propagation in large-scale social networks. His work has appeared in the New York Times, the Wall Street Journal, Science, MIT Technology Review and on National Public Radio.

1. Information Visualization for
Large-Scale Data Workflows
Michael Conover
Senior Data Scientist, LinkedIn
@vagabondjack

2. Emergent Structure

3. Credit Pedro Cruz, University of Coimbra
David Crandall, Indiana University
John Nelson, IDV Solutions Elegant Complexity

4. Intellectual Dividends
Realistic Mental Models
Verification of Assumptions
Shortened Iteration Cycles
Improved Predictive Performance
Product Insights
Clarity of Communication

5. Hypothesis Generation

7. @whitehouse #RSVP
Color Commentary

8. Flock Together

9. Political Polarization on Twitter

10. Feature Development

11. Anscombe’s Quartet
http://en.wikpedia.org/Anscombe’s_quartet

12. Basic Workflow Structure

13. 0.0
0.1
0.2
0.3
0.4
−5.0 −2.5 0.0 2.5 5.0
Standard Normal
Density
0.0
0.1
0.2
0.3
0.4
−5.0 −2.5 0.0 2.5 5.0
Standard Normal
Density
100,0001,000,000

14. geom_point()
A Lens on the Joint Distribution

15. geom_point(alpha=1/5)
A Lens on the Joint Distribution

16. geom_bin2d(bins=35)
A Lens on the Joint Distribution

17. geom_point(alpha=1/5, aes(color=label))
A Lens on the Joint Distribution

18. geom_density2d(aes(color=label), bins=20)
A Lens on the Joint Distribution

19. Marginal Histograms
A Lens on the Joint Distribution

20. GGally (ggpairs)
A Lens on the Joint Distribution

21. Model Fitting & Evaluation

22. Model A Model B
Training Data I
Training Data II
Evaluation Batteries

23. stanford.edu/~jhuang11/
Homework at Scale

24. github.com/StanfordHCI/termite
Topic Modeling

25. Layercake

26. Workflow Principles
Latent, Pervasive
Modular
Consistent Visual Language

27. Workflow Management

28. data.linkedin.com/opensource/azkaban
LinkedIn Azkaban

29. data.linkedin.com/opensource/white-elephant
LinkedIn White Elephant

30. github.com/Netflix/Lipstick
Netflix Lipstick

31. driven.cascading.io Concurrent Driven

32. bigml.com
BigML Sunburst

33. Information Visualization for
Large-Scale Data Workflows
Michael Conover
Senior Data Scientist, LinkedIn
@vagabondjack

34. Tableau

35. RStudio Shiny
rstudio.com/shiny/

36. RStudio Shiny
rweb.stat.ucla.edu/ggplot2/

37. Data Driven Documents (D3)
d3js.org

38. Superconductor
superconductor.github.io

39. Stamen Tiles
maps.stamen.com