Dynamic talks SF: So you have heard all the hype around how Machine Learning is going to change the world. But within your business context, where do you start? How do you get leadership buy-in for investment? And how and when you start scaling your ML Services? In this session, you will walk away with an actionable framework to bootstrap and scale a machine learning services team. You will see the framework in action through an actual 0 to 1 product journey involving deep learning where we delivered value in record speed in-spite of not having a dataset when we started. You will get practical tips on how to make decisions about when and how to scale your capability to scale ML Services and platform. Some of the tips are pretty counterintuitive and revealed themselves with our experience of productizing ML services over the last 5+ years. (Using a diverse range of technologies - Vision, Language, Graph, Anomaly Detection, Search Relevance, Personalization)

1. ML Services - How to Begin Well and Scale Right
Madhura Dudhgaonkar, Senior Director - Machine Learning

2. This presentation may contain forward-looking statements for which there are risks, uncertainties, and
assumptions. If the risks materialize or assumptions prove incorrect, Workday’s business results and directions
could differ materially from results implied by the forward-looking statements. Forward-looking statements
include any statements regarding strategies or plans for future operations; any statements concerning new
features, enhancements or upgrades to our existing applications or plans for future applications; and any
statements of belief. Further information on risks that could affect Workday’s results is included in our filings
with the Securities and Exchange Commission which are available on the Workday investor relations
webpage: www.workday.com/company/investor_relations.php
Workday assumes no obligation for and does not intend to update any forward-looking statements. Any
unreleased services, features, functionality or enhancements referenced in any Workday document, roadmap,
blog, our website, press release or public statement that are not currently available are subject to change at
Workday’s discretion and may not be delivered as planned or at all.
Customers who purchase Workday, Inc. services should make their purchase decisions upon services,
features, and functions that are currently available.
Safe Harbor Statement

3. Agenda
Workday Confidential
Introduction
A Practical Example of Machine Learning 0 to 1 Product Journey
Framework to Begin Well and Scale Right

4. Workday Inc.
Madhura Dudhgaonkar
Senior Director, Machine Learning

5. Mar 2009 - March 2006

6. Mar 2009 -March 2009 - Outrageous Thought
Mt. Denali | 20,320 ft | 6,194 m

7. Leading provider of
enterprise cloud
applications
Plan
Execute
Analyze
Workday Planning
Workday
Financial Management
Workday
Human Capital
Management
Workday Prism
Analytics and
Benchmarking

8. Business
Process
Framework
Object
Data Model
Reporting and
Analytics
Security
Integration
Cloud
One Source for Data | One Security Model | One Experience | One Community
Machine
Learning
One PlatformPower of One

9. Unique
Data Sets
40%+ of Fortune 500
50%+ of Fortune 50
Clean operational
data
Critical business
workflows

10. Workday Machine Learning Teams
Workday Confidential
Victoria, Canada
Portland, OR
Dublin, Ireland
Boulder, CO
San Francisco
Pleasanton

11. Example 0 - 1 ML Product Journey
Workday Confidential

12. A ML Service to scan receipts & auto-populate expense reports...
The Challenge
Workday Confidential

13. ...in Six Months!
March
Su Mo Tu We Th Fr Sa
1 2 3
4 5 6 7 8 9 10
11 12 13 14 15 16 17
18 19 20 21 22 23 24
25 26 27 28 29 30 31
April
Su Mo Tu We Th Fr Sa
1 2 3 4 5 6 7
8 9 10 11 12 13 14
15 16 17 18 19 20 21
22 23 24 25 26 27 28
29 30
May
Su Mo Tu We Th Fr Sa
1 2 3 4 5
6 7 8 9 10 11 12
13 14 15 16 17 18 19
20 21 22 23 24 25 26
27 28 29 30 31
June
Su Mo Tu We Th Fr Sa
1 2
3 4 5 6 7 8 9
10 11 12 13 14 15 16
17 18 19 20 21 22 23
24 25 26 27 28 29 30
July
Su Mo Tu We Th Fr Sa
1 2 3 4 5 6 7
8 9 10 11 12 13 14
15 16 17 18 19 20 21
22 23 24 25 26 27 28
29 30 31
August
Su Mo Tu We Th Fr Sa
1 2 3 4
5 6 7 8 9 10 11
12 13 14 15 16 17 18
19 20 21 22 23 24 25
26 27 28 29 30 31
Workday Confidential

14. A ML Service that scans receipts with >80% accuracy that is delivered
in private and public cloud for all customers by end of September.
Define the Win
Workday Confidential

15. Optical Character Recognition - Why build?
Workday Confidential

16. First obstacle - data bootstrap
Workday Confidential
Deep Commitment to Ethics.
Trust and Transparency about data with customers since day one.

17. Transferred Learning Data
Workday Confidential

18. Variety of Text
Workday Confidential

19. Step 1 - Bounding Box Detection
A deep learning model based on Residual
Networks
Output: center of the box, height, width and
angle of tilt, confidence
Workday Confidential

20. Step 2 - Text Recognition
A deep learning model based on
Residual Networks
Output: text
Workday Confidential
TOTAL
63.87
Brooks
Brother
01/27/15

21. Step 3 - Mapping
An assortment of models (rule
based + deep learning/ensembles)
Output: a map of value to field
Workday Confidential
63.87
Brooks
Brother
01/27/15
Total
Merchant
Date
Shipping N/A

22. Production: Ensemble
Workday Confidential

23. Human In The Loop
Minimum
Viable
Product
Data
Cleansing
Data
Transformation
Data Labelling
Feature
Engineering
Model
Selection
Training
Validation:
Results and UX
Metrics
Selection
Model & UX
Exploration
Validation
Data cleansing and labelling
Validation:
Results and
User Experience
Workday Confidential

24. Victory!
Productized >90% accuracy

25. Framework to Begin Well and Scale Right
Workday Confidential

26. For 0 to 1 - START!
Workday Confidential

27. S - Select One Win (Unambiguous Value)
Workday Confidential
Be Precise!

28. T - Team (<1 pie, 3-6 months)
Workday Confidential

29. A - Articulate and Align
Workday Confidential
Articulate
(Win & Gameplan)
Align
(Stakeholders)

30. R - Rally & Support

31. T - Take Shortcuts (tech debt - accrue it!)
Workday Confidential

32. For 0 to 1 - START!
Workday Confidential
S - Select the first win
T - Team (<1 pie, 3-6 months)
A - Articulate (win/game plan) Align (stakeholders)
R - Rally & Support
T - Take shortcuts

33. To Scale - GET!
Workday Confidential

34. G - Get Credit and Gather Capital
Workday Confidential
First win in the bag!

35. E - Establish Repeatable Processes & Platform
Workday Confidential

36. T - Transfer and Apply Learnings from the 0 - 1
Workday Confidential

37. For Scale - GET!
Workday Confidential
G - Get credit for the first win, Gather more capital
E - Establish repeatable processes and platform
T - Transfer learnings to scale to 10
G

38. START & go GET ’em
Workday Confidential

39. Hiring - Dev Managers, Engineers, PMs, Data scientists.