Recomendados
Recomendados
Mais conteúdo relacionado
Mais procurados
Mais procurados (20)
Semelhante a GIAF USA Spring 2015 - Demystifying data
Semelhante a GIAF USA Spring 2015 - Demystifying data (20)
Mais de Lauren Cormack
Mais de Lauren Cormack (6)
Último
Último (20)
GIAF USA Spring 2015 - Demystifying data
- 1. 1
- 3. Demystifying Data PALLAS HORWITZ SENIOR DATA SCIENTIST BLUE SHELL GAMES
- 4. What is wrong with being overwhelmed by data? • Problem: • Only data team members are comfortable accessing and analyzing the data • Data teams spend too much time pulling data instead of driving ROI positive insights • Executive team does not value data driven decisions • Solution: • Identify the analytic needs of every department • Create infrastructure and culture that empowers departments to perform simple analyses autonomously
- 5. Blue Shell Games, LLC
- 6. Problem 1: “I can’t access the data” Common Causes: • The visual report is broken or out of date • The user doesn’t have access to the database • The user doesn’t know where to find the relevant data
- 7. Problem 2: “We don’t have that data” Common Causes: • Asking the wrong question • Lack of Metrics QA • The requested data is not actionable
- 8. Problem 3: “A/B Testing is too hard” Common Causes: • Business intuition leads to the same conclusions as the A/B test results • Test results are too fuzzy and p-values are statistically insignificant • Even statistically significant results won’t change the product roadmap
- 9. Solution: Empower everyone to be a data consumer • Identify the needs of every department • Anyone can do simple SQL • Data Standardization Document: • Database Logins • Template Queries Data Product Engineering QA Community
- 10. Product Solution Result: • Only the “right” questions are asked • “Is my feature making money?” • High Level Statistics • DAU, Installs, ARPDAU, Payers, Conversion • Trends: DoD, WoW, 2Wo2W, MoM • Read Every Spec!
- 11. Engineering Solution • “What do I need to implement?” • Act as translator • Metrics Spec Result: • Useful data is collected and proxies are available for missing metrics
- 12. QA Solution • “How do I test this?” • A feature is not complete until metrics are implemented and QA’d • Setup user friendly metrics logs Result: • Implemented metrics act as intended
- 13. Community Solution • “How do I keep players engaged?” • Tailor contests to in-game activity • Have contest entries queryable Result: • Marketing decisions are informed by actual player behavior
- 14. Data Solution • “How can I contribute to the bottom line?” • Naïve customer segmentation can greatly impact revenue • Only implement tests that will affect change User Segment Revenue Delta p-value non payer -34% 0.45 payer 137% 0.02 small whale 65% 0.16 whale -43% 0.23 all -6% 0.8 Result: • Data improves the bottom line and drives product change
- 16. JOIN IN THE CONVERSATION PARTICIPATE IN THE NEXT GIAF Analytics for Games events@deltaDNA.com
Notas do Editor
- Asking the wrong question: ? Lack of Metrics QA: no soft launch data Inactionable data: CTR on obscure modals
- Asking the wrong question: ? A/B Test results not relevant to decision being made
- Required: SELECT, FROM, WHERE Optional: GROUP BY Teaching UI artist to SQL
- Is my feature making money? High Level Statistics DAU, Installs, ARPDAU, Payers, Conversion Trends: DoD, WoW, 2Wo2W, MoM Read Every Spec! DAU Interaction Effects on Wallet and Income Story: FB Connection and closing the funnel
- What do I need to implement? Act as translator Know what Product needs Know what is reasonable to implement Metrics Spec Include description, when it should fire, types, pseudo-code Acts as reference for PMs later Enables Metrics QA Story: Timestamps as booleans
- How do I test this? A feature is not complete until metrics are implemented and QA’d Potential one month lag between identifying and fixing logging errors on iOS Setup user friendly Metrics logs Tailing the logs is sufficient short term Story: No soft launch data because game state vs game summary
- How do make sure players feel valued? Set a goal for turn around time for payer tickets Make sure data is recent enough to meet that goal Setup a simple table for wallet transactions Ensure you aren’t giving out too much free currency Easier to identify payers and cheaters Story: Picking random contest winners – could point throw a dart at the fan page or have data actually queryable
- What do I do with all of the free time I have now? Pick a project that is cross product and improves the bottom line CRM, LTV, A/B testing, new tools Story: 3x increase in revenue by identifying quitters