Main takeaways: - How to balance decision making between qualitative and quantitative metrics - Developing your first data strategy - Creating a lean analytic process to build, measure, learn

1. www.productschool.com
Webinar: How to be Data Driven with
Product by Carbon Five Sr PM

2. Join 35,000+Product
Managers on
Free Resources
Discover great job
opportunities
Job Portal
prdct.school/PSJobPortalprdct.school/events-slack

3. C O U R S E S
Product
Management
Learn the skills you need to land
a Product Manager job

4. C O U R S E S
Coding
for Managers
Build a website and gain the
technical knowledge to lead
software engineers

5. C O U R S E S
Data Analytics
for Managers
Learn the skills to understand web
analytics, SQL and machine learning
concepts

6. C O U R S E S
Learn how to acquire more users
and convert them into clients
Digital Marketing
for Managers

7. C O U R S E S
UX Design
for Managers
Gain a deeper understanding of your
users and deliver an exceptional end-to-
end experience

8. C O U R S E S
For experienced Product Managers
looking to gain strategic skills needed
for top leadership roles
Product
Leadership

9. C O U R S E S
Corporate
Training
Level up your team’s Product
Management skills

10. Who am I?
Kayak -> Arts Administrator -> Incubator Director -> Startup
Founder ->
Growth Director -> Product Manager @ Carbon Five

11. Why am I
here?

12. Being data driven is
● Starting with a baseline
● Tracking the vitals
● Defining a North Star
● Defining and Prioritizing Assumptions
● Creating and tracking experiments
● Reviewing and restarting as needed

13. Being data driven is not:
● Using metrics to hide behind because the
team can’t debate and decide together
● Valuing quantitative metrics over
qualitative metrics
● Making decisions based on data but without
considering product/business goals

14. Qualitative vs
Quantitative

15. Qualitative +
Quantitative 🤝

16. Start with a
baseline.

17. Existing Product Baseline

18. Measures of Success
Get a baseline for existing measures of success.
- How does the business measure success for
the business model?
- How does the product team measure success?
- Do these things align?

19. Measures of Success
Toolkit
- Stakeholder Interviews
- Existing use of metrics
- Create an artifact
- Resolving any differences between
business and product

20. User Level Behavioral Data
- Start tracking and record existing user-level
data
- Set the usage metrics as another baseline
- Create dashboards for data transparency

21. User Level Behavioral Data
Toolkit
- Existing Metrics
- Implement a ‘track everything’ tool (like
Heap) quickly 🔥
- Create an artifact
- Dashboards

22. Qualitative Baseline
Talk to users, one-on-one
Surveys and focus groups are not a
substitute for this.

23. Qualitative Baseline
Toolkit
- Personas
- User Interviews
- User Observation
- Screencasting of user sessions, like Full
Story
- Synthesize and create an artifact of
learnings

24. Scaling Product Baseline

25. Scaling Product Baseline
- Growth Focus
- Create funnel to highest user
vale
- Conversion Rate
- Lifetime Value
- Retention

27. Scaling Baseline
Toolkit
- Optimizely for A/B tests
- Path view analysis
- Consider BI tools (Tableau, etc)
- Unit economics

28. New Product
Baseline

29. New Product
Baseline - Do we have
product/market fit?

30. New Product
Tools
- Business Model Canvas / Lean Canvas
- Personas
- User Interviews
- Invision Prototypes
- Paper Prototypes
- Design Sprints

31. New Product
What about quantitative?
- Existing market studies
- Academic research
- Surveys
- User interviews at scale
- Landing page A/B tests

32. Track the
vitals.

33. Monitor vitals
● Active Users (Daily, Weekly,
Monthly…)
● Conversion Rate
● Retention
● Feature Usage
● Session length
● User Paths

34. Monitor vitals
● Note when a change happens to the
product to see how it affects usage.

35. Monitor vitals
● Segment users, understand patterns
● Provide an avenue for users to contact
you

36. Define your north star…
your first ✨at least.

37. North Star
● Focus on Customer
Value
● Adjust North Star over
time

38. North Star
“1. Does improving this signal definitely help us
execute on our product strategy?
2.Does it represent a customer getting some unique
value from our product?
3.Is it a leading indicator of an outcome that my
business’ success is measured on?
4. Can it be broken into actionable input metrics?”
- Amplitude North Star

39. Define and Prioritize
Assumptions

40. Prioritize
Assumptions
● Keep track as you go
● Prioritize
● Align these with
releases

41. Develop a process and cadence to
Build -> Measure -> Learn with
clearly defined experiments.

42. Experiments
● Test New
Ideas AND
● Validate
Assumptions

44. Experiments
● Hypothesis that can be disproven
● What we want to learn
● Method
● Audience
● Metrics
● Results
● Next Steps

45. Existing Product
- Improve UX
- Simplify Offerings
- Change Information
Architecture
- Integrations
- Testing new offerings
- Attract a new market

46. Scaling Product Metrics & Tests
- Growth Focus
- Funnel Optimization
- Conversion Rate optimization
(CRO)
- A/B Testing
- Testing complimentary offerings

47. Startup of MVP
- Core value proposition
- Onboarding flows
- Vet assumptions about user
workflows
- UX
- Messaging

48. Review and Re-Kick when
needed

49. carbonfive.com
Thanks!
Allie O’Connell • @allie_wonders• Product School

50. Resources
https://amplitude.com/blog/2018/03/21/product-north-star-metric
https://blog.carbonfive.com/2015/03/19/delivering-value-making-money/
https://www.amazon.com/Product-Management-Practice-Real-World-
Connective/dp/1491982276

51. www.productschool.com
Part-time Product Management, Coding, Data Analytics, Digital
Marketing, UX Design, Product Leadership courses and
Corporate Training