With millions of new data points every moment, how are Product Managers expected to make sense of it all? This talk outlined the steps required to distill and synthesize data to drive actionable product decisions. The most effective Product Managers are those who know their data: they can justify product priority and roadmap changes, calibrate resource asks and manage their own time more effectively. This lecture equipped the audience with the tools necessary to draw insight from unstructured data using Google’s cloud analytics suite.

1. How to Use Data for Product Decisions by
YouTube Product Manager
www.productschool.com

2. FREE INVITE
Join 23,000+ Product Managers on

3. COURSES
Product Management
Learn the skills you need to land a product manager job

4. COURSES
Coding for Managers
Build a website and gain the technical knowledge to lead software engineers

5. COURSES
Data Analytics for Managers
Learn the skills to understand web analytics, SQL and machine learning concepts

6. COURSES
Blockchain for Managers
Learn how to trade cryptocurrencies and build products using the blockchain

7. Joao Fiadeiro
TONIGHT’S SPEAKER

8. Data and Analytics for
Product Managers
Joao Fiadeiro, PM @ Google
June 2018

9. A little about me...
● Product Manager at Google,
where I work in the YouTube
Music team
○ Involved in social/community
efforts for artists and analytics
● I began my career as a data
scientist 5 years ago
● Proud generalist!

10. Today’s content
Part I: Analytics Concepts for PMs
● The key concepts within analytics
Part II: Implementing Analytics
● From vision to metrics
Part III: Experimentation
● Data-driven product roadmaps
Part IV: Reporting
● Let your analysis be heard
High-level overview of
the role of analytics in a
Product Manager’s day-
to-day…
...Followed by a hands-
on demo of some
useful tools

11. Part I: Analytics Concepts
for PMs

12. Deriving value from data: key concepts
Data Points
Segmentation Funnels Cohorts
Analytics Concepts
Implementing Analytics
Experimentation
Reporting

13. Data Points
Data points are the individual points of collected data that are
measurements of particular items within the platform
● Data usually exists somewhere; though it may be hard to find
○ Is the right data being collected?
○ Do you have access to it?
● As a PM, your job is to know what data is being collected and
where it lives
● If you don’t have measurements, you don’t have anything…
Get creative.
Analytics Concepts
Implementing Analytics
Experimentation
Reporting

14. Segmentation
Segmentation is about grouping together people by a
common characteristic and seeing what the usage patterns of
the product are as a group
● Segments must be measurable; typical segments are
behavioral, technical or demographic
● Segmentation slices the analytics, allowing underlying
patterns in behaviour and usage to be observed, rather than
be hidden by noise and averaging
Analytics Concepts
Implementing Analytics
Experimentation
Reporting

15. Gangnam Style: A study in Attention Span
Analytics Concepts
Implementing Analytics
Experimentation
Reporting
Watch Time (%) by Gender and Age Avg. % Completion by Age

16. Funnels
A funnel is made up of the measurement of the key event at
each step of the flow or user journey
● Users don’t just do something in isolation. Instead, they
perform a series of actions to accomplish a task or goal.
These flows or user journeys can be measured using funnels
● Optimize for improving every step of the funnel. A solid
conceptual funnel allows for rich metrics between every layer
Analytics Concepts
Implementing Analytics
Experimentation
Reporting

17. Funnels at YouTube
Each layer expressed a
ratio of the previous
layer… is worth more than
a thousand words.
Find the golden path!
Analytics Concepts
Implementing Analytics
Experimentation
Reporting

18. Cohorts
The primary purpose of cohort analysis is for comparative analysis
to answer the question of how users’ behaviour changes over
time
● Segment users into buckets and explore differences in
behavior.
○ For example: how does the behaviour of users who registered a
week ago differ from that of users who registered a month ago?
● Used to understand retention and churn
Analytics Concepts
Implementing Analytics
Experimentation
Reporting

19. Cohorts
Analytics Concepts
Implementing Analytics
Experimentation
Reporting

20. Part II: Implementing
Analytics in the Real World

21. Implementing Analytics
Product Managers must establish a clear product vision but
should also be able to articulate key performance indicators
The process of planning consists of these steps:
1. Define the product vision
2. Define the KPIs that meet the product vision
3. Define the metrics that allow you to hit your KPIs
4. Define the funnels (via user journeys) that affect your metrics
Analytics Concepts
Implementing Analytics
Experimentation
Reporting

22. The Vision
● By starting with the vision, you ensure that what you measure
will help you achieve the product vision
● Avoid the trap of vanity metrics by tying everything that is
measured to what you are trying to achieve
● It is the filter that allows you to ignore the potential mass of
data you can collect
Analytics Concepts
Implementing Analytics
Experimentation
Reporting

23. Key Performance Indicators (KPIs)
● KPIs are derived from the product vision and tell you how well
your product is meeting the vision.
○ They are product focused and only indicate the performance of
the product.
● KPIs are used to set targets for the performance of the
product
● The KPIs need to reflect the current stage your product is at
Analytics Concepts
Implementing Analytics
Experimentation
Reporting

24. Metrics: what should you be measuring?
Engagement vs Transactional Apps
Analytics Concepts
Implementing Analytics
Experimentation
Reporting

25. Growth Transactions
MonetizationRetention
The four buckets of metrics...
Analytics Concepts
Implementing Analytics
Experimentation
Reporting

26. Growth:
Gaining new users
How many new users do you have
and where do they come from?
● # of daily/weekly/monthly new
user signups
● Metrics by acquisition channel
Analytics Concepts
Implementing Analytics
Experimentation
Reporting
Growth
● DAU/WAU/MAU
● Trial Users/Paying subscribers

27. Transactions & Engagement:
Increasing usage of the app
How much are your users engaging with
the product?
● Engagement: Typically
consumption
● Transactions: Average Order Value
Analytics Concepts
Implementing Analytics
Experimentation
Reporting
Transactions
● Views/Watch Time (per user)
● Likes/Shares/Comments

28. Retention:
Ensuring that existing users come back
Are your users coming back for more?
● How many of your users are
coming back within 24 hrs, 7 days
and 28 days?
Analytics Concepts
Implementing Analytics
Experimentation
Reporting
● Avg. Session length
● DAU/MAU
Retention

29. Monetization:
Converting usage into dollars
How effectively are you converting
usage into revenue?
● Ad-supported or Subscription
m10n
● Revenue share/margins
Analytics Concepts
Implementing Analytics
Experimentation
Reporting
● Revenue per Watch Hr (RPH)
● Avg Revenue Per User (ARPU)
Monetization

30. A word of caution...
Most of these metrics, on their own, might make you feel good,
but they don’t offer clear guidance for what to do…
Every new feature that is being considered should move the
needle on the key metrics in A/B tests otherwise the feature
may be of questionable value
Analytics Concepts
Implementing Analytics
Experimentation
Reporting

31. Experiments (aka. A/B Tests)
A/B experiments produce the most actionable of all metrics,
because they explicitly refute or confirm a specific hypothesis
Analytics → “What is going on with x?”
Experiments → “How do we improve x?”
Experimentation informs:
Planning of (new)
experiments
Product backlog
Development
prioritisation
Analytics Concepts
Implementing Analytics
Experimentation
Reporting

32. Experimentation Cycle
Plan
Monitor Implement
Analytics Concepts
Implementing Analytics
Experimentation
Reporting
Start with a question and
formulate a hypothesis.
Identify (in)dependent
variables
Implement in code (or tool)
and be patient
Do not change the control!
Segment, segment,
segment
Ensure statistical validity

33. Closing the loop
Ask yourself these two questions:
● What do these results mean for development prioritisation?
● Why did I get these results?
The result whether positive or negative is immaterial, what is
material is that you learnt something from the test
Keep experimenting - ensure it’s in the product & engineering
culture
Analytics Concepts
Implementing Analytics
Experimentation
Reporting

34. Part IV: Reporting

35. Reporting:
The danger zone
● Don’t get caught up in the data/analytics! If you don’t report
your findings, you’re wasting your time
● Beautiful visualizations are great, but less is more
● Some tips:
○ Do it in a manner that is easily grasped by everyone in the
company
○ Allow for as much input/interaction from the audience as
possible
Analytics Concepts
Implementing Analytics
Experimentation
Reporting

36. Reporting:
Managing stakeholders
● When presenting to senior stakeholders, you’ll likely get many
“what if?” follow up questions
● Advice: come prepared with multiple scenarios. If possible,
complement with a spreadsheet that allows custom inputs
Projected
MAU
Watch Hours
per MAU RPH
Total
Revenue Probability
Expected
Revenue
Best Case 300M 20 $0.020 $120M 10% $12M
Expected Case 280M 15 $0.015 $63M 70% $44M
Worst Case 200M 12 $0.012 $29M 20% $6M
TOTAL $62M
Analytics Concepts
Implementing Analytics
Experimentation
Reporting

37. Reporting:
What about dashboards?
● Dashboards can be very useful, but also very dangerous to PMs
● Pros:
○ Audience can self-serve data needs
○ PM does not waste time with data requests
○ Can be widely published and shared
● Cons:
○ Audience can make incorrect assumptions about data
○ PM may have to maintain them → Huge time sink
○ Orphaned data is a real problem
Analytics Concepts
Implementing Analytics
Experimentation
Reporting

38. Tying it all together:
Google Big Data

39. Google Big Data Solutions
Google offers an integrated, serverless Big Data
platform for data-driven applications

40. Today we’ll look at three products
A Product Manager can use these products to analyze, explore,
and present data in an effective way:
1. BigQuery: fully managed, low cost analytics data warehouse.
Use SQL to query massive datasets.
2. Dataprep: data service for visually exploring, cleaning, and
preparing structured and unstructured data for analysis
3. Data Studio: turns data into dashboards and reports that are
easy to read, share, and customize.

41. BigQuery Demo
Goals:
● Pick a public dataset
● Explore schema
● Lightweight querying
Codelab
Link

42. Dataprep Demo
Goals:
● Import a CSV file
● Create a ‘Flow’ then a recipe
● Explore data transformations
● Export dataset
Link

43. Data Studio Demo
Goals:
● Import data source
● Create a few charts
● Make and customize a table
● Add filters
● Publish
Link

44. Part-time Product Management Courses in
San Francisco, Silicon Valley, Los Angeles, New York, Austin,
Boston, Seattle, Chicago, Denver, London, Toronto
www.productschool.com