An introduction to the heart, mind, and soul of Product Management: Customer Obsession, Metrics, and Product Sense. Presented at Product School Bellevue.
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
Intro to Product Management
1. Part-time Product Management Courses in
San Francisco, Silicon Valley, Los Angeles, New York, Austin,
Boston, Seattle, Chicago, Denver, London, Toronto
www.productschool.com
3. +5000
Alumni Graduated
across 14
Campuses
· San Francisco
· Silicon Valley
· New York
· Los Angeles
· Santa Monica
· Orange County
· Austin
· Boston
· Boulder
· Chicago
· Denver
· Seattle
· Toronto (Canada)
· London (UK)
10. What does a Product Manager do?
Source: https://www.onedesk.com/product-manager-meme/
11. “Mini-CEO” responsible for overseeing and leading
teams through product planning, feature definition and
execution
Plan
DefineExecute
User Research: Identify underlying
user needs
Frameworks: Structure and
frame problem spaces
Metrics: Prioritize features,
Validate hypotheses
13
2
14. Categories of User Research
By research source
• Primary
• Secondary
• Third party
By type of insights
• Qualitative
• Quantitative
• Behavioral
By context
• Contextual
• Scripted
15. Landscape of research methods by type of insights
What people do
What people say
Why & How to
improve
How many & how
much
Behavioral
Qualitative
N=10 N=Thousands+
Quantitative
Reference: Christian Rohrer 2014
N=Hundreds
Optimize /
improve existing
product
Build a new
product
16. What people do
What people say
Why & How to
improve
How many & how
much
Legend
Behavioral
Qualitative
Quantitative
N=10
N=Thousands+
N=Hundreds
Ethnographies
SurveysEye-tracking
Observations
Diary Studies
Task History
Longitudinal Studies
Path & Click Analysis
Panel studies
Cohort Analysis
Focus groups
Scripted Remote Studies
Concept-value tests
Message testing
Landscape of research methods by sample size
Reported Behavior
Actual Behavior
Optimize / improve
existing product
Build a new
product
18. Top research methods used @Microsoft
1. Focus Groups
1. Unmoderated studies
1. Surveys
1. Message testing
1. Cohort Analysis
19. • When to use:
• Deep qualitative feedback on complicated experiences
• Directional feedback in early product development
•PM provides:
• Research questions
• Sample experiences – mockups, prototypes, competitors
• Participant screening (i.e. women <34 that use Prime Now app on iPhone
weekly)
• Optional: Pre-work
•Variations:
• Moderated remote studies, diary studies & customer journals
1. Focus groups
20. • When to use:
• Validating results of focus groups
• Comparing design options
• Prioritize features during design, before
development begins
•PM provides:
• List of multiple-choice Q&A (few open-ended Qs)
• Sample experiences
• Participant screening
•Variations:
• Concept value tests
2. Surveys
21. • When to use:
• For more detailed feedback than surveys
• Budget or resource constraints
• Message testing, Usability study of prototypes
•PM provides:
• Research questions
• Sample experiences
• Participant screening
•Variations:
• Diary studies
• Customer journals or Ethnographies
3. Remote Studies
22. • When to use:
• When planning go-to-market activities
• Prioritize among message alternatives to land the best story with
consumers
•PM provides:
• List of canvases: Banner Ads, headers, in-product alerts
• List of channels: social, email, blogs, influencers
• List of messages for each type of canvas & channel
• GTM-ready experiences
• Participant screening
•Variations:
• Conversion rate optimization using Persado or competitors
4. Message testing
23. • When to use:
• Deeply understand user behavior of an existing or newly
launched product
•PM provides:
• Work closely with Data Scientist to define the problem
statement, research questions, and cohort type (gender, location,
start date)
• Research questions
• Metrics to measure: Engagement, Revenue, Returning Usage
• User screening (i.e. users who visited Prime Day deals during
crash window)
•Variations:
• Path or clickstream analysis, Task histories
• Longitudinal studies, A/B Tests
5. Cohort Analysis
24. Takeaways / Things to remember
Pros Cons When to use
Qualitative research –
moderator driven
focus groups
Great for brainstorming, discovering
latent needs
Experienced moderators distill key
insights
Expensive Early product envisioning
Qualitative research –
remote studies
Good for unearthing pain points,
concept value testing
Cheaper than moderator-led focus
groups
Operational aspects of group chat
tool can be challenging
Best for 1:1 observations
Early product envisioning, Design, best
for message testing
Quantitative research
- surveys
Quantitative validation of known
pain points and concepts
Best for feature prioritization
Does not represent actual user
behavior
Post qualitative research for product
development
Behavioral analyses Quantify actual user behavior (how
much, how often)
Needs investments in
instrumentation and data collection
Post launch
27. Why Frameworks?
• Find your niche
• Build consensus with popular
frameworks
Customer
CompetitiveBusiness
Niche
28. Landscape of Product Management Frameworks
Customer
• AAARRR / AARM / REAN /
AIDA
• RFM
• HEART
• Jobs Theory
• Fogg Behavior Model
(Hooked)
• Lean UX
Business & Decision making
• 4 Ps
• SWOT
• Pricing: 5 Cs
• Kano Model
• BCG Matrix
Competitive
• Porter’s 5 forces
• Blue Ocean Strategy
• McKinsey 7 degrees of
Freedom - Growth
29. Why would you hire a milkshake?
https://hbswk.hbs.edu/item/clay-christensens-milkshake-marketing
Image: McDonalds.com
30. Jobs Theory
Seeing tasks from a customer lens vs. product lens
#1
Functional
“Help me wake up
with the best coffee
at consistent
quality”
Social
“Give me a place to
connect with my
friends”
Emotional
“Help me treat
myself
at the end of a long
day”
31. Jobs Theory: Framework
Deeply understand the customer
• What’s their Situation “When”: skills, wealth, time, demographic, spending habits?
• What’s their Motivations and Expected outcomes
1
Identify the high-priority job-to-be-done
• How does the customer do the job today?
• What barriers prevent the customer from doing it (well)?
2
Frame your value proposition
• Relative to competing solutions, where does our solution meet the bar?
• Where to delight?
• Where can our solution be below the bar?
3 Identify related emotional, social and functional jobs
4
32. Jobs Theory guides you to ask the right question and frame the
topmost user need
Framing: WHEN <…> I WANT <…> SO I CAN <…>
https://design.org/wp-content/uploads/2016/11/Jobs-To-Be-Done-Framework.jpg
Jobs
Main Job
Functional
Emotional
Social
Emotional
Related
Jobs
33. Case Study: Immediate Clinic
• Time to appointment
• Time to care
• Cost of treatment
• Effective treatment
• Provider expertise
From: Yelp.com
WHEN my daughter is hurt, I WANT immediate care SO I CAN relax knowing that she is pain-free
34. Jobs Theory: Summary & Takeaways
• An innovative approach to solving customer problems
• Framing the customer’s problem in 3 dimensions:
• Situation “When..”
• Motivation “I want to..”
• Expected Outcome “…so I can …”
• Identify where your product needs to compete vs. delight vs. compromise
• Great framework as you prepare for focus groups
35. Blue Ocean Strategy
Systematic approach to finding uncontested markets or creating new demand
From: cirquedusoleil.com
#2
2 types of blue oceans:
1. Launch completely new industries e.g. Amazon
with online book retailing, ebay with online
auctions
1. Expand boundaries of existing industry e.g.
Cirque du Soleil
Read more: https://www.blueoceanstrategy.com/bos-moves/cirque-du-soleil/
36. Blue Ocean Strategy: Four Actions Framework
•Which factors that
the industry has long
competed on should
be eliminated?
•Which factors should
be created that the
industry has never
offered?
•What factors were a
result of competing
against other
industries and can be
reduced?
•What factors should
be raised well above
the industry's
standard?
Raise Reduce
EliminateCreate
1. Similar to Jobs Theory, considers when a
“job” or a “factor” is good enough or can
be eliminated
1. Modern take on classic Porter’s 5 Forces
1. Focus on strategic decision making than
competitor analysis
37. Case Study: iPhone
• Computing power
• Ease of use
• Easy to carry
• Always on
• Built-in camera
Source:
https://i.ytimg.com/vi/U3i7TTX5SRI/hqdefault.jpg
After years of competing with the PC, Steve Jobs expanded market boundaries to Mobile. Disrupted the camera
industry. Build incredible momentum and market cap bringing together Devices & Services
38. Jobs Theory & Blue Ocean: Recap
• Complementary frameworks
• Use Jobs Theory first to deeply understand what the customers are trying to get done, their
circumstances, desired outcomes, and constraints
• Use Blue Ocean to identify a unique market space that delights customers and is hard for
competitors to replicate
• Goal: Avoid head-to-head competition
39. From: cdn.slidemodel.com
Porter’s Five Forces
Classic theory of competitive analysis to win
#3
1. Threat of new Entrants
2. Threat of Substitutes
3. Bargaining power of Buyers
4. Bargaining power of
Suppliers
5. Rivalry among Competitors
41. Blue Ocean vs. Porter’s Forces
Read more: http://www.richardsona.com/blog/2010/7/7/what-you-can-learn-from-virgin-americas-strategy-canvas.html
42. From: businessnewsdaily.com
BCG Matrix
Useful for strategic planning to decide where to invest your $$
#4
1. Milk the Cash Cows
1. Divest in the Dogs
1. Spend on the Stars
1. Experiment with Question Marks
43. BCG Matrix in the Product Lifecycle
1. Most products start as question
marks
1. If market share is low, they will
absorb lot of cash
1. Some question marks, become
dogs while other grow to be
stars and cash cows
Source: https://www.slideshare.net/9579764806/bcg-matrix-with-example
45. BCG Matrix - Applications
1. Mini-CEO mindset
1. Macro - Managing product features within product, or products
within product area
1. Micro - manage attributes within a feature
1. Understanding and making career choices
46. From: mindtheproduct.com
Kano Model
Famous ASQ methodology to understand and classify customer needs
#5
Customers don’t
know to ask for
this
“Must” be met to
avoid
unhappiness
Customer
“wants” for
happiness
47. Source: foldingburritos.com/kano-model
Feature present
How would you feel if our battery life was longer 8 hours?
Feature NOT present
How would you feel if our battery lasted less 8 hours?
Like This would be very helpful to me
Expect This is a basic requirement for me X
Don’t care This would not affect me
Live with This would be a minor inconvenience
Dislike This would be a major problem for me
Like This would be very helpful to me
Expect This is a basic requirement for me
Don’t care This would not affect me
Live with This would be a minor inconvenience
Dislike This would be a major problem for me X
Like Expect Don’t
care
Live
with
Dislike
Like Q D D D S
Expect R I I I E
Don’t
care
R I I I E
Live
with
R I I I E
Dislike R R R R Q
FEATURE PRESENT
FEATURENOTPRESENT
Classifying customer needs
D - Delighters
S – Satisfiers
E – Expectations
X – Don’t care
Q – Questionable
R – Reverse
48. Like Expect Don’t
care
Live
with
Dislike
Like Q D D D S
Expect R I I I E
Don’t
care
R I I I E
Live
with
R I I I E
Dislike R R R R Q
FEATURE PRESENT
FEATURENOTPRESENT
Classifying customer needs
D - Delighters
S – Satisfiers
E – Expectations
X – Don’t care
Q – Questionable
R – Reverse
Must-haves
Line of indifference
49. Case Study: Disney Cruises
• Kids eat free
• Free room service
• Fireworks
• Special acts of kindness
• Towel Animals!
Source: cruises.priceline.com
Further reading: https://sigmazone.com/kanos_model
50. • Analogies
• Personal, social and emotional moments
• 5 Whys
• Free beer
• Under promise, over deliver
• Value stream mapping
• Fishbones
• Voice of customer analysis
See more: bing.com/explore
Finding Delighters
Read more: http://asq.org/learn-about-quality/six-sigma/tools.html
https://www.google.com/doodles
52. KANO: Takeaways and things to remember
• Rigorous approach to identifying and classifying customer needs
• Popularized by ASQ and used widely in LEAN / Six Sigma methodologies
• Supplemented with rigorous behavioral analyses to be effective
• Identifying delighters is tricky, use analogies
53. Lean UX#6
Source: https://www.agriya.com/blog/
Read more: https://www.interaction-design.org
• Quick and iterative
• Focus on users
• Early feedback
• Hypothesis testing
• Focus on
collaboration
Takeaway: If no data or evidence, walk away from an idea
62. A Typical User Journey
DISCOVER RETURN BUY SHARE=+ENGAGE ++
Determine primary and secondary
goals for your feature / product
Pick the features and metrics that
will move the right goal
63. Measuring Feature Discovery
DISCOVER
1. How many total Users
1. How many total Active Users – Daily, Monthly using
feature, How many Active Days
1. What % of total users see feature X
1. Where are the users coming from – entry points,
marketing channels Product Level
User Level
Feature Level
64. Measuring User Engagement
DISCOVER ENGAGE+
1. How many Users are seeing feature X
1. How many Users are clicking or engaging with feature X
1. How many Users leave after seeing feature X (churn, bounce rate)
1. What is a user’s engagement before / after seeing feature X?User Level
Feature Level
65. Measuring User Retention
DISCOVER RETURNENGAGE ++
1. In a 30 day window, how many Users that saw feature X
returned to use feature X
1. In a 30 day window, how many Users that saw feature X
returned to the product AND used a different feature Y
1. For users that saw Feature X, did overall engagement
with product increase?
1. Corollary: Are engaged users of product using feature
X?
Product Level
User Level
66. Measuring Monetization
DISCOVER RETURN BUY+ENGAGE ++
1. What is the customer acquisition cost?
1. What is the average revenue per user?
1. What is the revenue per user for feature X compared
with rest of product?
Product Level
User Level
67. Measuring Advocacy & Fandom
DISCOVER RETURN BUY SHARE=+ENGAGE ++
1. What is the life-time value for the user?
1. Did user refer friends?
1. What is the user saying about the product?
1. What is the revenue per user for the product?Product Level
User Level
68. If you were the product manager for FB Events, what metrics
would you use to measure success?
69. RECAP
Plan
DefineExecute
User Research: Identify underlying
user needs
Frameworks: Structure and
frame problem spaces
Metrics: Prioritize features,
Validate hypotheses
13
2
Start with
Why
How
What