Predictive Analytics as a Product

1. Intended for Knowledge Sharing only Predictive Analytics as a Product Feb 2017

2. Intended for Knowledge Sharing only Disclaimer: Participation in this summit is purely on personal basis and is not meant to represent VISA’s position on this or any other subject and in any form or matter. The talk is based on learning from work across industries and firms. Care has been taken to ensure no proprietary or work related information of any firm is used in any material.

3. Intended for Knowledge Sharing only Quick recap of what it is Intended for Knowledge Sharing only Data Scientist, eh… 3

4. Intended for Knowledge Sharing only Quick recap of what it is Intended for Knowledge Sharing only FEELS LIKE A ROCKSTAR, DOESN’T IT? 4 http://modernservantleader.com/servant-leadership/narcissism-kills-morale/

5. Intended for Knowledge Sharing only Quick recap of what it is Intended for Knowledge Sharing only ..BUT A KANYE & NOT COLDPLAY 5 https://imgflip.com/memegenerator/7064654/Kanye-West

6. Intended for Knowledge Sharing only Quick recap of what it is Intended for Knowledge Sharing only So what happened? 6

7. Intended for Knowledge Sharing only Intended for Knowledge Sharing only SOME CHALLENGES 7 Unrealistic expectations on RoI. Operates in siloes, not complemented by user research/other internal or external data/experimentation results. Field testing & iterative development still predominantly offline. Deployment, Post Deployment management & monitoring expensive. Not easy to turn on/off, tweak, flip, scale. Predictions driven significantly by historical trends and relationships. Expectations modeled as simulations.

8. Intended for Knowledge Sharing only Quick recap of what it is Intended for Knowledge Sharing only Explain it a bit more... 8

9. COMPLICATION 1: PREDICTIVE ANALYTICS IS INTRICATE & COMPLEX Intended for Knowledge Sharing only 9 Objective Translation to Analytical Framework Data Collection and Preparation Analysis, Validation & Verification Actionable insights and impact sizing A/B Testing Rollouts • Understand need, fit with Strategic needs, actionability, stakeholders buy-in, engineering RoI, project management • Decide on the Analytical methodology based on nature of the problem, dependent variable, frequency, sample, time, required precision, actionability • Hypothesized driver list • Data Collection: Internal & external sourcing • Data Preparation: Blending, aggregations • Data Transformations: Outlier, Missing, math transformation, interactions, redundancy treatments, variable selections • Sampling methodology & split • Model development and validation: In-time, Out-of-time • Stand alone, ensemble • Performance diagnostics & cross check with other sources • Recommendations, impact sizing, cross leverage scores • Field Testing (Champion vs. Challenger) • Iteration plan based on user feedback (VOC), performance • Model deployment, post deployment monitoring & management • Integration with Product Line– New product, 1 2 3 4 5 6 7 Description

10. COMPLICATION 2: MULTIPLE AUDIENCE, PRIORITIES, DEPENDENCIES Intended for Knowledge Sharing only 10 Objective Translation to Analytical Framework Data Collection and Preparation Analysis, Validation & Verification Actionable insights and impact sizing A/B Testing Rollouts • Analyst & Stakeholder • Analyst, Data Instrumentation, Data Manager, Stakeholder • Analyst, Data Instrumentation, Data Manager • Analyst • Analyst, Stakeholder, Cross Functional team, Leadership • Analyst, Experimentation Team, User Researcher, Developer, Stakeholder • Analyst, Developer, Stakeholder, Leadership 1 2 3 4 5 6 7 Who does it? • Agile and may undergo iteration • Changes in Strategic goals, newer initiatives, releases, discoveries, reorgs • Sourcing/Blending challenges: Data handovers between systems, blending challenges • Scalability/automation • Data movements/latencies/ teams/approvals • Evolution of hypotheses, data changes/errors, success criteria • Competing priorities, data movements, Scenario Simulations • Success criteria, integration with research/testing tools, iterations • Integration with host systems, engineering investment, model tweaking, monitoring, customization Key Challenges

11. COMPLICATION 3: OUTPUT OF ONE CAN BE INPUT/ADDITION TO ANOTHER Intended for Knowledge Sharing only 11 Behavioral Merchant Performance Clickstream/ Ops Campaign Performance VOC/Social/ CRM • Probability of Engagement/LTV Growth/Churn/Loyalty • Life event changes • Product/Price Migrations • Probability of Growth/Churn • Next Best Product/Offer • Network partners • Conversion Rate Optimization • Server Response Times • Time to Purchase • Campaign Responses • Next Best Product/Offer • Cross Channel target • Promoter/Detractor & drivers • Brand Appeal • Theme/entity of engagement Data Lake: Enriched with predictions e.g., Uber’s cross sell platform, Google Calendar, VDP

12. COMPLICATION 4: REAL DECISION MAKING NEEDS ADDITIONAL REASONING BEYOND ANALYTICS Intended for Knowledge Sharing only 12 Analytics provides insights into “actions”, Research context on “motivations” & Testing helps verify the “tactics” in the field and everything has to be productized… Strategy Data Tagging Data Platform Reporting Analytics Research Data Products Iterative Loop Why such complexity? Focus on Big Wins Reduced Wastage Quick Fixes Adaptability Assured execution Learning for future initiatives Optimization

13. Intended for Knowledge Sharing only Intended for Knowledge Sharing only COMPLICATION 5: DEMANDS ON PREDICTIVE ANALYTICS HAVE INCREASED 13 Predictive Analytics Behavioral Analytics What are the customers doing? Voice of Customer What are the customers telling you? Platform Performance How are you delivering? Competitive Are the customers buying elsewhere? Social Listening How are customers discussing you? …aaanddd Better, Faster, Cheaper, Monetizable

14. Intended for Knowledge Sharing only Quick recap of what it is Intended for Knowledge Sharing only So, what do we need then? 14

15. Intended for Knowledge Sharing only • Extensible • Scalable • Flexible • Easy to integrate with other techniques Intended for Knowledge Sharing only HIGH LEVEL SUMMARY OF NEEDS: MODULAR, SHAREABLE & MONETIZABLE 15 keywordsuggest.org Iconfinder WebPT • Documentation • Governance • Integration with project management tools (collaboration) • Security/Privacy Management • Value Abstraction • API-able Modular Shareable Monetizable

16. Intended for Knowledge Sharing only Quick recap of what it is Intended for Knowledge Sharing only Potential Solutions 16

17. TWO DEPLOYMENT SOLUTIONS- PMML & PFA Intended for Knowledge Sharing only 17 Data Mining Group an independent Vendor Led Consortium that develops Data Mining Standards has come up with PMML (Predictive Model Mark Up Language) and PFA (Portable Format for Analytics) http://www.kdnuggets.com/2016/01/portable-format-analytics-models-production.html http://dmg.org/ https://www.ibm.com/developerworks/library/ba-predictive-analytics4/ba-predictive-analytics4-pdf.pdf https://www.ibm.com/developerworks/library/ba-ind-PMML1/ http://www.kdnuggets.com/faq/pmml.html https://journal.r-project.org/archive/2009-1/RJournal_2009-1_Guazzelli+et+al.pdf PMML PFA File XML JSON & YAML Maturity Mature but expanding Evolving Nesting/Customization Model Parameters Control Structures (Type System of Model Parameters & data - Callback function allowed) Flexibility Standard across most scoring engines (better than custom code) More flexible than PMML but safer than Custom Code Scope Data prep, Modeling, Scoring, Sharing +Pre/Post processing, enforced memory model

18. PMML PROJECTS Intended for Knowledge Sharing only 18 http://data-informed.com/pmml-puts-big-data-to-work/

19. POSITIONING OF PFA Intended for Knowledge Sharing only 19 http://data-informed.com/pmml-puts-big-data-to-work/

20. Intended for Knowledge Sharing only Quick recap of what it is Intended for Knowledge Sharing only Why this, Why now, why here? 20

21. Intended for Knowledge Sharing only Intended for Knowledge Sharing only BIGGER TRENDS THAT ARE SHAKING UP THE ANALYTICS WORLD FROM INSIDE OUT… 21 Demand Pressures: Complexity and nature of problems and their solutions, type of audience & consumption framework evolving Monetization opportunities- Direct, Indirect, Recurring Artificial Intelligence, IoE and “Smart”ening of devices/systems faster than expected. Evolution of input data sources and integration of multiple insights sources into decision making (A/B Testing, Research, Predictions/Scores from other models) Evolution from Service to Product to Platform (Build Once, Use Everywhere) …APIs are eating up our world

22. Intended for Knowledge Sharing only Quick recap of what it is Intended for Knowledge Sharing only The parting words… 22

23. SUMMARY Intended for Knowledge Sharing only Predictive Analytics has stopped being “one-off competitive edge project exercise” – it’s a necessary survival initiative for organizations Scale, complexity, breadth of needs (including Monetization) demand Platform approach. “Build Once, Use Everywhere” -consumption of predictive analytics outputs need to be easy to use, integrate, re-use/collaborate across multiple initiatives As everything becomes Productized via APIs, together they can become a business problem solving ANI 23 Streaming Analytics is quickly evolving into Streaming Predictive Analytics

24. Intended for Knowledge Sharing only Quick recap of what it is Intended for Knowledge Sharing only Appendix

25. THANK YOU! Intended for Knowledge Sharing only Would love to hear from you on any of the following forums… https://twitter.com/decisions_2_0 http://www.slideshare.net/RamkumarRavichandran https://www.youtube.com/channel/UCODSVC0WQws607clv0k8mQA/videos http://www.odbms.org/2015/01/ramkumar-ravichandran-visa/ https://www.linkedin.com/pub/ramkumar-ravichandran/10/545/67a RAMKUMAR RAVICHANDRAN

26. Intended for Knowledge Sharing only Disclaimer: Participation is purely on a personal basis and does not represent VISA,Inc. in any form or matter. The talk is based on learning from work across industries and firms. Care has been taken to ensure no proprietary or work related info of any firm is used in any material. Director, Insights at Visa, Inc. Enable Decision Making at the Executives/ Product/Marketing level via actionable insights derived from Data. RAMKUMAR RAVICHANDRAN