Mais conteúdo relacionado Semelhante a Analytics Teams: 5 Things You Need to Know Before You Deploy Your Model (20) Analytics Teams: 5 Things You Need to Know Before You Deploy Your Model1. @jamet123 #decisionmgt © 2016 Decision Management Solutions
James Taylor
CEO
Analytics Teams: 5 Things You Need to
Know Before You Deploy Your Model
2. @jamet123 #decisionmgt © 2016 Decision Management Solutions 2
James Taylor
Decision Management Solutions
▶ We work with clients to
improve their business by
applying business rules &
analytic technology to
automate & improve decisions.
▶ Vendor-neutral
▶ Original DMN submitter
▶ Using decision modeling since 2011
▶ I have spent the last 14 years
focused on the development of
analytic applications using
Decision Management
3. @jamet123 #decisionmgt © 2016 Decision Management Solutions 3
Agenda
▶ Challenges Deploying Analytics
▶ Five Questions To Ensure Deployment
▶ Decision Modeling for Analytics
▶ Decision Models Answer The Five Questions
▶ Wrap Up
5. @jamet123 #decisionmgt © 2016 Decision Management Solutions 5
Data Science
Insights
Business
Value
There Is A Dangerous Gap In Data Science
6. @jamet123 #decisionmgt © 2016 Decision Management Solutions 6
Closing The Gap Delivers Analytic Success
Data Science
Insights
Business
Value
Frame The Problem Build The Model
Plan For
Deployment
7. @jamet123 #decisionmgt © 2016 Decision Management Solutions 7
Challenges Deploying Analytics
Analytic results are
prioritized, not
business resultsIneffective
deployment leaves
analytics on the
shelf
Timeliness and
sustainability are
not considered
8. @jamet123 #decisionmgt © 2016 Decision Management Solutions 8
Operational
Systems
Analytic
Systems
Analytics
need to drive
operations to
add value
40%-60% of analytic
models are not
deployed and used
Models On The Shelf Add No Value
9. @jamet123 #decisionmgt © 2016 Decision Management Solutions 9
Analytics Must be Deployed
?
?
“A presentation ABOUT a model, however
interesting, can’t power the website”
“A 40 page report,
full of math”
10. @jamet123 #decisionmgt © 2016 Decision Management Solutions 10
Business Results
▶ Too many projects focus on analytic accuracy
▶ Need to focus on business value
▶ Teams must connect their analytic to this value
11. @jamet123 #decisionmgt © 2016 Decision Management Solutions 11
Timeliness and Sustainability
▶ Time to deploy matters ▶ Deployed models age
12. @jamet123 #decisionmgt © 2016 Decision Management Solutions 12
Five Questions To Ensure Deployment
1. Does the model evaluate well against business
objectives?
2. Who’s using the model, to decide what, where and
how?
3. What will it take to make the model actionable?
4. What’s the plan for capturing and analyzing data
about model effectiveness?
5. When do we need to revisit the model to make sure
it is still effective?
14. @jamet123 #decisionmgt © 2016 Decision Management Solutions 14
Decision Modeling Frames Analytics
▶ Provides “structure” - who, what, how, when
▶ Standardizes approach to decision making
▶ Provides “transparency” of decision approach
▶ Fosters innovation in the business model
▶ Improves/changes the business model
▶ Promotes “buy in”
15. @jamet123 #decisionmgt © 2016 Decision Management Solutions 15
Decision Models Frame The Problem
Decision
Input Data
Knowledge
Sources
Sub-
decisions
Analytic
Source: DecisionsFirst Modeler
@jamet123 #decisionmgt
16. @jamet123 #decisionmgt © 2016 Decision Management Solutions 16
Decision Models Put Analytics In Context
Process
Decision
Motivation Organization
Department
Team
Role
Role
Team
Department
Organization
Analytic
18. @jamet123 #decisionmgt © 2016 Decision Management Solutions 18
Five Questions To Ensure Deployment
1. Does the model evaluate well against business
objectives?
2. Who’s using the model, to decide what, where and
how?
3. What will it take to make the model actionable?
4. What’s the plan for capturing and analyzing data
about model effectiveness?
5. When do we need to revisit the model to make sure
it is still effective?
19. @jamet123 #decisionmgt © 2016 Decision Management Solutions 19
1. Does the model evaluate well against
business objectives?
▶ Objectives of the
sponsors
▶ Measures of success
▶ For instance
▶ Reduce claims in litigation
▶ Improve the loss ratio
▶ Objectives of the
modelers
▶ Measures of accuracy
▶ For instance
▶ Error rate
▶ Lift
Business Measures Not Analytic Ones
20. @jamet123 #decisionmgt © 2016 Decision Management Solutions 20
Consider Impact On All Business Objectives
Loan Repayment
Insurance Revenue
Customer Satisfaction
21. @jamet123 #decisionmgt © 2016 Decision Management Solutions 21
2. Who’s using the model, to decide what,
where and how?
▶ What is the analytic
used for?
▶ Who has to use the
analytic model?
▶ Where is the analytic
used?
▶ How is the analytic
used?
▶ The analytic is used in
an explicit decision
▶ Those that make the
decision
▶ The decision has a
business process context
▶ The other elements of
the decision are clear
22. @jamet123 #decisionmgt © 2016 Decision Management Solutions 22
3. What will it take to make the model
actionable?
▶ Decisions often involve analytics and business rules
Business
Rules
Analytics
23. @jamet123 #decisionmgt © 2016 Decision Management Solutions 23
Take Action With Business Rules
▶ Adding business rules to analytic models turns
predictions into actions
▶ Managing sets of business rules and analytic models
in a single decision flow, a single execution,
encapsulate decision-making
▶ Business user find business rules approachable,
increasing their comfort with analytic decision-
making
▶ Business rules give them some control over what to
DO about the predictions you give them
24. @jamet123 #decisionmgt © 2016 Decision Management Solutions 24
4. What’s the plan for capturing and
analyzing data about model effectiveness?
Decision
Made
Supporting
Decisions
Scores
Business
Results
▶ Decision Models show what to store
25. @jamet123 #decisionmgt © 2016 Decision Management Solutions 25
5. When do we need to revisit the model to
make sure it is still effective?
Data Changes
Direct
Outcomes
Indirect
Outcomes
Business
Results
▶ Decisions models reveal tracking needs
26. @jamet123 #decisionmgt © 2016 Decision Management Solutions 26
Champion Challengers
Champion/Challenger in Decision Models
27. @jamet123 #decisionmgt © 2016 Decision Management Solutions 27
Five Questions To Manage Deployment
1. Does the model evaluate well
against business objectives?
2. Who’s using the model, to
decide what, where and how?
3. What will it take to make the
model actionable?
4. What’s the plan for capturing
and analyzing data about model
effectiveness?
5. When do we need to revisit the
model to make sure it is still
effective?
▶ Decision models link analytics to
decisions to business metrics
▶ Decision models show who owns,
makes and cares about decisions
▶ Decision models wrap analytics in
rules-based decisions
▶ Decision outcomes can be
captured with input data, business
results and scores
▶ Decision models clarify business
context, pace of change and show
data usage clearly
29. @jamet123 #decisionmgt © 2016 Decision Management Solutions 29
Decision
Most Analytic Projects Leave Decisions
In The Dark
Modeling Decisions
Brings Transparency
29©2015 Decision Management Solutions
30. @jamet123 #decisionmgt © 2016 Decision Management Solutions 30
Data
Business
Understanding
Data
Understanding
Modeling
Data
Preparation
Evaluation
Deployment
CRISP-DM Phases
1. Business Understanding
2. Data Understanding
3. Data Preparation
4. Modeling
5. Evaluation
6. Deployment
31. @jamet123 #decisionmgt © 2016 Decision Management Solutions 31
Specific Analytic
Decision
Analytic Models
Overall decision to be
improved
Business Understanding As a Decision Model
Regulations and
Policies
Data
32. @jamet123 #decisionmgt © 2016 Decision Management Solutions 32
Decision Models Ensure Analytic
Deployment
Tie the analytic to
decisions and decisions
to business results
Specify the
decision(s) that use
the analytic
Use the decision
context to
frame
deployment and
sustainability
needs
33. @jamet123 #decisionmgt © 2016 Decision Management Solutions 33
Decision Modeling For Analytic Projects
▶ Steps
1. Training in decision modeling
2. Establish business value
3. Model effective decision-making
4. Add process, organization, business context
5. Frame and scope the analytic project
▶ Deliverables
▶ Decision Model showing impact of proposed analytic
▶ Analytic, business and IT teams trained & self-sufficient
▶ Best practices and go-forward recommendations
▶ Software, consulting and training
34. @jamet123 #decisionmgt © 2016 Decision Management Solutions 34
Decision Modeling for Analytic Projects
▶ Learn more
▶ Download the five
questions brief
▶ bit.ly/5QAnalytics
▶ Ready to start?
▶ Kick off a decision
modeling project today
▶ bit.ly/DMNAnalytics
36. @jamet123 #decisionmgt © 2016 Decision Management Solutions 36
Decision Modeling for Analytic Projects
▶ Learn more
▶ Download the five
questions brief
▶ bit.ly/5QAnalytics
▶ Ready to start?
▶ Kick off a decision
modeling project today
▶ bit.ly/DMNAnalytics
37. @jamet123 #decisionmgt © 2016 Decision Management Solutions 37
DecisionsFirst Modeler
Collaborative, cloud-based decision modeling software
that puts decisions at the heart of your business
architecture, building better requirements and driving
successful business rules implementations
Key Features
Multi-user, Collaborative
Decision Model and Notation
(DMN) Support
Shared Database, Reusable
components
Automated Consistency and
Completeness Checks
Living Repository
38. Thank You
@jamet123 #decisionmgt © 2016 Decision Management Solutions
For more on
Decision Management, go to:
decisionmanagementsolutions.com
James Taylor
james@decisionmanagementsolutions.com