Data can be viewed as the exhaust of online activity. With the rise of cloud-based data platforms, barriers to data storage and transfer have crumbled. The demand for creative applications and learning from those datasets has accelerated. Rapid acceleration can quickly accrue disorder, and disorderly data design can turn the deepest data lake into an impenetrable swamp. In this talk, I will discuss the evolution of the data science workflow at Expedia with a special emphasis on Learning to Rank problems. From the heroic early days of ad-hoc Spark exploration to our first production sort model on the cloud, we will explore the process of industrializing the workflow. Layered over our story, I will share some best practices and suggestions on how to keep your data productive, or even pull your organization out of the data swamp.

1. … and associated Idiosyncratic Operating Principles.
Industrializing
DataScience Workflows
Sean Downes
Sr DataScientist @ Expedia, Inc.

2. The Problem
So you’ve been asked to bring the infrastructure into the cloud.
So your Data Lake is actually a Data Swamp.

4. Login
Impressions
Clicks
Purchases
The Problem
Every Line of Business has its own Structure
Every MicroService has a log
And you want to A|B Test

5. Can you please turn…
The Problem
into
using

6. Preview
Context / Disclaimer
Lightning Review of Data Platforms
(idiosyncratic) Organizing Principles

7. Context / Disclaimer
Academic
Theoretical Physicist
We’ve got some work to do.
So…
I’m implicitly assuming this talk will be one in an ensemble of opinions

8. supercomputers…
Lightning Review of Data Platforms

9. … the Commerical Data Center…
Lightning Review of Data Platforms

10. … Virtualized Everything
Lightning Review of Data Platforms
1. Assign Tasks their own virtual hardware
2. Expend / Contract Resources by demand
3. Real-time HotSwapping
4. Software Updates Built In
5. Etc Etc Etc.

11. idiosyncratic Organizing Principles
iOP1) Clarity
iOP2) Engineers are not Data Scientists
iOP3) PMs are not Data Scientists
iOP4) Data Scientists are not Engineers
iOP5) Close the Data Loop

12. iOP1: Data Clarity
PUBLISH THIS INTERNALLY!
“big data, big noise”
Where is what data?
Who owns what field?
What is this this field?
Where did this field go?
Why is this field NULL?

13. iOP1: Data Clarity
Minibatch streaming into nested JSON?
O(10kB)?
GZip?
O(50-500 MB)
Parquet.
Snappy.
“Expect Data Science”
Spark
Big Thanks to Jason Pohl @ DB!
And Charles Pritchard!

14. iOP2: Engineers are not Data Scientists
“why would you need to do that?”
Scratch Space
Cluster Bootstrap Permissions
Access S3 Buckets
Sandbox Clusters
Share Notebooks Across Accounts
We DO NOT SPEAK IAM Role/Anything

15. iOP2: Engineers are not Data Scientists
“why would you need to do that?”
if possible:
Write your own Pipelines.
else:
Explain Data Science.

16. iOP3: PMs are not Data Scientists
“you don’t need that!”
Once upon a time in the Flight DataLake…
Only 10% of a Search Impression was recorded
Worse: It was only the Cheapest10%
Many of the bookings where not included in this list!

17. iOP4: Data Scientists are Not Engineers
“we need to support models in @#&%? format”
Pick a Robust Standard and Stick to It.
If you’re big enough to worry about this, you can commit code
jPMML
Everybody Use Git. Now. Yes You.
Production Code Matters. Format. Document.
Pipelines Count as Production Code.

18. iOP5: Close that Data Loop
What is your data doing?
New Data? Consider Bandits!
Big Data? Set up a learning problem.
Empower by Design.

19. Empower by Design.

20. Contact information or call to action goes here.
Thank You.