Presentation on How to Build Winning Products given at Product School Bellevue on 83018

1. www.productschool.com
Part-time Product Management, Coding, Data, Digital
Marketing and Blockchain courses in San Francisco, Silicon
Valley, New York, Santa Monica, Los Angeles, Austin, Boston,
Boulder, Chicago, Denver, Orange County, Seattle, Bellevue,
Toronto, London and Online

2. +5000
Alumni Graduated
across 14
Campuses
· San Francisco
· Silicon Valley
· New York
· Los Angeles
· Santa Monica
· Orange County
· Austin
· Boston
· Boulder
· Chicago
· Denver
· Seattle
· Toronto (Canada)
· London (UK)

3. COURSES
Product Management
Learn the skills you need to land a product manager job

4. COURSES
Coding for Managers
Build a website and gain the technical knowledge to lead software engineers

5. COURSES
Data Analytics for Managers
Learn the skills to understand web analytics, SQL and machine learning concepts

6. COURSES
Digital Marketing for Managers
Learn how to acquire more users and convert them into clients

7. COURSES
Blockchain for Managers
Learn how to trade cryptocurrencies and build products using the blockchain

8. FREE INVITE
Join 23,000+ Product Managers on

9. Ria Sankar
TONIGHT’S SPEAKER

10. Building Winning Products
Decoding Customer Obsession, Product Sense, and
Execution
Ria Sankar
Product Management @Microsoft

11. What does a Product Manager do?
Source: https://www.onedesk.com/product-manager-

12. “Mini-CEO” responsible for overseeing and leading
teams through product planning, feature definition and
execution
Plan
DefineExecute
User Research: Identify underlying
user needs
Frameworks: Structure and
frame problem spaces
Metrics: Prioritize features,
Validate hypotheses
13
2

13. Today’s Takeaways: Top tools for product management
Landscape of top research methods + closer look at 5
For each: When to use, Expectations of PM, Variations
Landscape of top product management frameworks + closer look at 4
For each: Framework, case studies
Time permitting: Picking the right metric to measure success of your
feature
1
3
2
Closing & Q&A

14. User Research
Plan
DefineExecute
User Research: Identify underlying
user needs
1

16. dictionary.com wordcloud generated by

17. Categories of User Research
By research source
• Primary
• Secondary
By type of insights
• Qualitative
• Quantitative
• Behavioral
By context
• Contextual
• Scripted

18. Landscape of research methods by type of insights
What people do
What people say
Why & How to
improve
How many & how
much
Behavioral
Qualitative
N=10 N=Thousands+
Quantitative
Reference: Christian Rohrer 2014
N=Hundreds
Optimize /
improve existing
product
Build a new
product

19. What people do
What people say
Why & How to
improve
How many & how
much
Legend
Behavioral
Qualitative
Quantitative
N=10
N=Thousands+
N=Hundreds
Ethnographies
SurveysEye-tracking
Observations
Diary Studies
Task History
Longitudinal Studies
Path & Click Analysis
Panel studies
Cohort Analysis
Focus groups
Scripted Remote Studies
Concept-value tests
Message testing
Landscape of research methods by sample size
Reported Behavior
Actual Behavior
Optimize / improve
existing product
Build a new
product

20. Top research methods used @Microsoft
1. Focus Groups
1. Unmoderated studies
1. Surveys
1. Message testing
1. Cohort Analysis

21. • When to use:
• Deep qualitative feedback on complicated experiences
• Directional feedback in early product development
•PM provides:
• Research questions
• Sample experiences – mockups, prototypes, competitors
• Participant screening (i.e. women <34 that use Prime Now app on iPhone
weekly)
• Optional: Pre-work
•Variations:
• Moderated remote studies, diary studies & customer journals
1. Focus groups

22. • When to use:
• Validating results of focus groups
• Comparing design options
• Prioritize features during design, before
development begins
•PM provides:
• List of multiple-choice Q&A (few open-ended Qs)
• Sample experiences
• Participant screening
•Variations:
• Concept value tests
2. Surveys

23. • When to use:
• For more detailed feedback than surveys
• Budget or resource constraints
• Message testing, Usability study of prototypes
•PM provides:
• Research questions
• Sample experiences
• Participant screening
•Variations:
• Diary studies
• Customer journals or Ethnographies
3. Remote Studies

24. • When to use:
• When planning go-to-market activities
• Prioritize among message alternatives to land the best story with
consumers
•PM provides:
• List of canvases: Banner Ads, headers, in-product alerts
• List of channels: social, email, blogs, influencers
• List of messages for each type of canvas & channel
• GTM-ready experiences
• Participant screening
•Variations:
• Conversion rate optimization using Persado or competitors
4. Message testing

25. • When to use:
• Deeply understand user behavior of an existing or newly
launched product
•PM provides:
• Work closely with Data Scientist to define the problem
statement, research questions, and cohort type (gender, location,
start date)
• Research questions
• Metrics to measure: Engagement, Revenue, Returning Usage
• User screening (i.e. users who visited Prime Day deals during
crash window)
•Variations:
• Path or clickstream analysis, Task histories
• Longitudinal studies, A/B Tests
5. Cohort Analysis

26. Flowchart of research methods
New
Product?
Purpose?
Major
Rev?
Quantitative
insights
Observations
Focus groups
Actual
Behavior?
Attitudinal
insights
Yes
Yes
Behavioral
insights
NoNo
Actual
Behavior?
Why?
Ethnographies
Diary Studies
Eye-tracking Surveys
CVTs
Remote
Studies
Message Tests
Panel Studies
Clickstream
Task History
Path Analysis
Extended
time
period
GTM
Directional early
feedback (why, how)
Design & feature
prioritization
(How many/ much)
Measurable insights
on actual
interactions
(what/ when)
For deeper whys move to qualitative
Cohort
Analysis
Purpose?
Read more: https://www.linkedin.com/pulse/user-research-
turning-data-delightful-experiences-ria-sankar/

27. Takeaways / Things to remember

28. Frameworks
Plan
DefineExecute
Frameworks: Structure and
frame problem spaces
2

29. Businessdictionary.com

30. Why Frameworks?
• Find your niche
• Build consensus with popular
frameworks
Customer
CompetitiveBusiness
Niche

31. Landscape of Product Management Frameworks
Customer
• AAARRR / AARM / REAN /
AIDA
• RFM
• HEART
• Jobs Theory
• Fogg Behavior Model
(Hooked)
• Lean UX
Business & Decision making
• 4 Ps
• SWOT
• Pricing: 5 Cs
• Kano Model
• BCG Matrix
Competitive
• Porter’s 5 forces
• Blue Ocean Strategy
• McKinsey 7 degrees of
Freedom - Growth

32. Why would you hire a milkshake?
https://hbswk.hbs.edu/item/clay-christensens-milkshake-marketing
Image: McDonalds.com

33. Jobs Theory
Seeing tasks from a customer lens vs. product lens
#1
Functional
“Help me wake up
with the best coffee
at consistent
quality”
Social
“Give me a place to
connect with my
friends”
Emotional
“Help me treat
myself
at the end of a long
day”

34. Jobs Theory: Framework
Deeply understand the customer
• What’s their Situation “When”: skills, wealth, time, demographic, spending habits?
• What’s their Motivations and Expected outcomes
1
Identify the high-priority job-to-be-done
• How does the customer do the job today?
• What barriers prevent the customer from doing it (well)?
2
Frame your value proposition
• Relative to competing solutions, where does our solution meet the bar?
• Where to delight?
• Where can our solution be below the bar?
3 Identify related emotional, social and functional jobs
4

35. Jobs Theory guides you to ask the right question and frame the
topmost user need
Framing: WHEN <…> I WANT <…> SO I CAN <…>
https://design.org/wp-content/uploads/2016/11/Jobs-To-Be-Done-Framework.jpg
Jobs
Main Job
Functional
Emotional
Social
Emotional
Related
Jobs

36. Case Study: Immediate Clinic
• Time to appointment
• Time to care
• Cost of treatment
• Effective treatment
• Provider expertise
• Complexity of care
From: Yelp.com
WHEN my daughter is hurt, I WANT immediate care SO I CAN relax knowing that she is pain-free

37. Jobs Theory: Summary & Takeaways
• An innovative approach to solving customer problems
• Framing the customer’s problem in 3 dimensions:
• Situation “When..”
• Motivation “I want to..”
• Expected Outcome “…so I can …”
• Identify where your product needs to compete vs. delight vs. compromise
• Great framework as you prepare for focus groups

38. Blue Ocean Strategy
Systematic approach to finding uncontested markets or creating new demand
From: cirquedusoleil.com
#2
2 types of blue oceans:
1. Launch completely new industries e.g. Amazon
with online book retailing, ebay with online
auctions
1. Expand boundaries of existing industry e.g.
Cirque du Soleil
Read more: https://www.blueoceanstrategy.com/bos-moves/cirque-du-soleil/

39. Blue Ocean Strategy: Four Actions Framework
•Which factors that
the industry has long
competed on should
be eliminated?
•Which factors should
be created that the
industry has never
offered?
•What factors were a
result of competing
against other
industries and can be
reduced?
•What factors should
be raised well above
the industry's
standard?
Raise Reduce
EliminateCreate
1. Similar to Jobs Theory, considers when a
“job” or a “factor” is good enough or can
be eliminated
1. Modern take on classic Porter’s 5 Forces
1. Focus on strategic decision making than
competitor analysis

40. Case Study: iPhone
• Computing power
• Ease of use
• Easy to carry
• Connectivity
• Built-in camera
• Reliability
Source:
https://i.ytimg.com/vi/U3i7TTX5SRI/hqdefault.jpg
After years of competing with the PC, Steve Jobs expanded market boundaries to Mobile. Disrupted the camera
industry. Build incredible momentum and market cap bringing together Devices & Services.

41. Jobs Theory & Blue Ocean: Recap
• Complementary frameworks
• Use Jobs Theory first to deeply understand what the customers are trying to get done, their
circumstances, desired outcomes, and constraints
• Use Blue Ocean to identify a unique market space that delights customers and is hard for
competitors to replicate
• Goal: Avoid head-to-head competition

42. From: cdn.slidemodel.com
Porter’s Five Forces
Classic theory of competitive analysis to win
#3
1. Threat of new Entrants
2. Threat of Substitutes
3. Bargaining power of Buyers
4. Bargaining power of
Suppliers
5. Rivalry among Competitors

43. Further reading:
https://hbr.org/video/3590615226001/the-explainer-porters-five-forces
https://www.porteranalysis.com/porters-five-forces-analysis-model-of-southwest-airlines/
Case Study: South West Airlines
Southwest Airlines well known for disrupting the airlines industry by identifying new ways to compete on cost (fuel
hedging)
• Price
• Personalized Entertainment
• Immersive Experience
• Easy check-in
• Easy flight changes

44. Blue Ocean vs. Porter’s Forces in the Airlines Industry
Read more: http://www.richardsona.com/blog/2010/7/7/what-you-can-learn-from-virgin-americas-strategy-canvas.html
Southwest Airlines used Porter’s Forces while Virgin Atlantic adopted Blue Ocean to formalize their business strategies

45. From: businessnewsdaily.com
BCG Matrix
Useful for strategic planning to decide where to invest your $$
#4
• Milk the Cash Cows
• Divest in the Dogs
• Spend on the Stars
• Experiment with Question Marks

46. BCG Matrix in the Product Lifecycle
1. Most products start as question
marks
1. If market share is low, they will
absorb lot of cash
1. Some question marks, become
dogs while other grow to be
stars and cash cows
Source: https://www.slideshare.net/9579764806/bcg-matrix-with-example

47. Microsoft through the BCG Lens
Disclaimer: This is one PM’s perspective

48. BCG Matrix - Applications
1. Mini-CEO mindset
1. Macro - Managing product features within product, or products
within product area
1. Micro - manage attributes within a feature
1. Understanding and making career choices

49. From: mindtheproduct.com
Kano Model
Famous ASQ methodology to understand and classify customer needs
#5
Customers don’t
know to ask for
this
“Must” be met to
avoid
unhappiness
Customer
“wants” for
happiness

50. Source: foldingburritos.com/kano-model
Feature present
How would you feel if our battery life was longer 8 hours?
Feature NOT present
How would you feel if our battery lasted less 8 hours?
FEATURE NOT PRESENT
FEATUREPRESENT
Classifying customer needs
D - Delighters
S – Satisfiers
E – Expectations
X – Don’t care
Q – Questionable
R – Reverse

51. Like Expec
t
Don’t
care
Live
with
Dislike
Like Q D D D S
Expec
t
R I I I E
Don’t
care
R I I I E
Live
with
R I I I E
Dislike R R R R Q
FEATURE NOT PRESENT
FEATUREPRESENT
Classifying customer needs
D - Delighters
S – Satisfiers
E – Expectations
X – Don’t care
Q – Questionable
R – Reverse
Must-haves
Line of indifference

52. Case Study: Disney Cruises
• Kids eat free
• Free room service
• Fireworks
• Special acts of kindness
• Towel Animals!
Source: cruises.priceline.com
Further reading: https://sigmazone.com/kanos_model

53. • Analogies
• Personal, social and emotional moments
• 5 Whys
• Free beer
• Under promise, over deliver
• Value stream mapping
• Fishbones
• Voice of customer analysis
See more: bing.com/explore
Finding Delighters
Read more: http://asq.org/learn-about-quality/six-sigma/tools.html
https://www.google.com/doodles

54. Finding Delighters
Review reported needs. SMART questions. 5 Whys.
Read more:
https://norulesjustwords.wordpress.com/2012/03/14/identifying-customer-delighters-with-criticial-to-quality-ctq/

55. KANO: Takeaways and things to remember
• Rigorous approach to identifying and classifying customer needs
• Popularized by ASQ and used widely in LEAN / Six Sigma methodologies
• Supplemented with rigorous behavioral analyses to be effective
• Identifying delighters is tricky, use analogies

56. Perception campaigns, TV Ads
Value prop alignment
Referral partnerships (Disti deals(
Engaging video (welcome to Bing, user tour)
Constant drumbeat
Event sponsorships (salon dinners)
Account based marketing (@Work)
Print media: Magazines/Newsletter
(A)AARRR Pirate Metrics#6
Awareness0
Read more: AARRR
https://www.pierrelechelle.com/aarrr-pirate-metrics

57. Awareness (Case Study: Bing International)
Web
TV Placements
Social
Media

58. Acquisition / Activation / Engagement / Retention
(Case Study: Bing)

59. Referral (Case Study: Bing)
Press release /
Blog
WebsiteVideo / Social
Amplification
Email
In-product

60. Revenue
(Case Study: Amazon)

61. Lean UX#7
Source: https://www.agriya.com/blog/
Read more: https://www.interaction-design.org
• Quick and iterative
• Focus on users
• Early feedback
• Hypothesis testing
• Focus on
collaboration
Takeaway: If no data or evidence, walk away from an idea

62. Lean UX: Canvas & Tools
Source: https://realtimeboard.com/examples/lean-ux-canvas/
Tools per stage
Steps 1 and 2:
• Job Stories (use Jobs to be done)
Steps 3 and 4:
• Personas & Job Stories
Steps 4, 6, 7:
• Affinity Diagrams & 2 x 2s
Step 5:
• Task Flows, Designs, Story boards

63. Personas
Lean UX Tools (Case Study: Fitbit)
Read more: https://uxdesign.cc/fitbit-a-usability-case-study-b23e4c539c3c
Job Stories
Affinity
Diagrams &
2x2s
Task Flows,
Designs, Story
boards
Metrics &
Experimentation

64. Metrics
Plan
DefineExecute
Metrics: Prioritize features,
Validate hypotheses
3

65. A different perspective..

66. A Typical User Journey
DISCOVER RETURN BUY SHARE=+ENGAGE ++
Determine primary and secondary
goals for your feature / product
Pick the features and metrics that
will move the right goal

67. Measuring Feature Discovery
DISCOVER
1. How many total Users
1. How many total Active Users – Daily, Monthly using
feature, How many Active Days
1. What % of total users see feature X
1. Where are the users coming from – entry points,
marketing channels Product Level
User Level
Feature Level

68. Measuring User Engagement
DISCOVER ENGAGE+
1. How many Users are seeing feature X
1. How many Users are clicking or engaging with feature X
1. How many Users leave after seeing feature X (churn, bounce rate)
1. What is a user’s engagement before / after seeing feature X?User Level
Feature Level

69. Measuring User Retention
DISCOVER RETURNENGAGE ++
1. In a 30 day window, how many Users that saw feature X
returned to use feature X
1. In a 30 day window, how many Users that saw feature X
returned to the product AND used a different feature Y
1. For users that saw Feature X, did overall engagement
with product increase?
1. Corollary: Are engaged users of product using feature
X?
Product Level
User Level

70. Measuring Monetization
DISCOVER RETURN BUY+ENGAGE ++
1. What is the customer acquisition cost?
1. What is the average revenue per user?
1. What is the revenue per user for feature X compared
with rest of product?
Product Level
User Level

71. Measuring Advocacy & Fandom
DISCOVER RETURN BUY SHARE=+ENGAGE ++
1. What is the life-time value for the user?
1. Did user refer friends?
1. What is the user saying about the product?
1. What is the revenue per user for the product?Product Level
User Level

72. If you were the product manager for FB Events, what metrics
would you use to measure success?

73. RECAP
Plan
DefineExecute
User Research: Identify underlying
user needs
Frameworks: Structure and
frame problem spaces
Metrics: Prioritize features,
Validate hypotheses
13
2
Start with
Why
How
What

74. Recommended Reading & Viewing
What
How
Why

75. Q&A

76. Thank you

77. UPCOMING EVENTS
Wednesday, September 5
Seattle
How to Know Your
Customers

78. UPCOMING Product Management
Courses
6 Spots left1 Spot left
October 13 - December 8
9:30am - 3:30pm
SaturdaysSaturdays
September 8 - October 27
9:30am - 3:30pm

79. UPCOMING Coding for Managers Courses
Tuesdays & Thursdays
September 4 - October 25
6:30pm - 9:00pm
1 Spot left

80. Mondays & Wednesdays
1 Spot left
UPCOMING Data Analytics for Managers Courses
September 10 - October 31
6:30pm - 9:00pm