How To Build Amazing Products Through Customer Feedback
1. How to build amazing Products through
Customer Feedback
/Productschool @ProdSchool /ProductmanagementNY
Venue Sponsored by
2. Eli Holder
- Product Leader, Designer and
Entrepreneur
- Founder of Notch.me and Unblab
www.productschool.com
How to build amazing Products
through Customer Feedback
3. Good Qs,
Good
Pâs.31 Good Questions Product Managers
can ask to build Good Products.
by Eli Holder
eli.holder@nodd.co
6. unblab. âSpent about 10 minutes training it to
recognize my important emails and it's already
saved me hours and hours of sorting throughacquired by
7. âThe SMS like model of having in and
outbound messages keeping a conversation
timely is the killer feature.â
âI found it very refreshing and energizing to
see creative products such as this one at
8. âUntil I saw your graphics and your âresult
storiesâ, I really wasnât that dissatisïŹed.
Now, I want to view all of my data this
9. âEli has been instrumental in bringing an
organized process to studying users"
10.
11. Nodd helps team
leaders get honest,
constructive feedback
from their teams at
work.
"I loved the emojis and
the survey format was
awesome. I was
engaged the whole time.
Excellent platform.â
Nodd.co
20. 1. Who are we? Why are we here? âš
(strategy & existential crises)
2. Whoâs our audience? What do
they want? (user research)
3. What problem should we solve? âš
(opportunity assessment)
4. How should we solve the
problem? (design sprinting)
5. Are we approaching Product /
Market Fit? (build, measure, learn loop)
Big Product
Qâs
(for each Product Lifecycle
Stage)
21. Related Product Qâs.
Who are we? Whatâs our mission? Whatâs our aligning metric? What are our organizational objectives for the quarter/year (OKRs)? Who are
the stakeholders? What are their goals? Why invest more in the product (e.g. instead of sales?) Whoâs going to be building it? What are their
personal goals and interests? What are their strengths and weaknesses? How much of their time do we have (e.g. are they cross functional? do
they have other projects? are there churn risks?) What are the teamâs strengths and weaknesses? What are peopleâs expected communication
styles? Whatâs the organizationâs culture (e.g. Risk tolerance)? Whatâs the organizationâs perceived brand? What are the expectations for a
product manager? Whatâs the (target) business model? What stage is the company at? Whatâs the prevailing role within the company? Who are
the other companies in the space? How is your company different? Tech advantages / debt? Design advantages / debt? Data advantages? What
are the religious issues that everyoneâs tired of discussing? External requirements? Whats your support team like? How do you (think) you make
money? Whoâs our audience? What do our users think about on the subway? Who are our primary (and secondary) users? What does a
typical day look like for them? What problems are they currently experiencing? (What are the 5 whys for the problems? What are they doing now
to solve them? What do they like about the current solution? What do they dislike about the current solution? What are they feeling?) What are
the most important dimensions to describe them? (e.g. demographics? behavior? company role? values? physical attributes? ) Who are the
extreme users? What do our users value? What are their goals? What are their obligations? What are their frustrations? What are their
stresses? What gets them excited? Who are the people in their lives? How do they typically interact with them? Whatâs the users value to us?
What language do they use to describe their experiences? What other products / tools are they using? What are they reading? Who are they
following? How do they ïŹnd / buy related products? How do they ïŹt into purchasing decisions? What can they spend? When do they spend?
Whatâs their organizationâs culture like? How do they ïŹt in? Whatâs their organizationâs business model? How do competitors perceive them?
What does academia think about them? What problem are we solving? (value prop) What problems are incidental (non-goals)? For
whom do we solve this problem? (target market) How big is the opportunity? (market size) How will we measure success? (metrics)
What alternatives are out there now? (competitors) Why us? Whatâs our differentiator? Why now? (market window) Why might this a
bad idea? How will we market this? Other considerations for success? How should we solve this? What are all the different ways we
could possibly solve this? How are our competitors solving this? How have academics solved this? How have people solved similar problems in
different industries? Which solution is the best for us to pursue? Which solution is most interesting to the team? Which solution has the most
learning potential? Is it technically feasible? (Whatâs the relative complexity?) Is it consistent with the behavior / data weâve collected so far?
22. Who are we?
Why are we
here?Product Lifecycle Stage #1:
Q:
Strategy & Existential Crises
23. Whatâs the mission?Q:
Who are the
stakeholders? What do
they want?
Q:
What are our strengths &
weaknesses relative to
competitors?
Q:
Strategy & Existential Crises
30. 1. Conformity bias
2. Positivity bias
3. They donât know
4. Donât want to look dumb
challenges:
Who are the
stakeholders?âš
What do they want?
Q:
31. The Hopes / Fears
Exercise:
Who are the
stakeholders?âš
What do they want?
Q:
âWhat are your 3 biggest hopes for this
project? (e.g. what do you hope this project
accomplishes for [company]?)
âWhat are your 3 biggest fears about this
project? (e.g. what do you think will go wrong?)
âWhat are your other considerations for this
project? (anything else on your mind?)
Ask:
32. Hopes:
âą That project X opens up a new market for us
âą Iâm optimistic about the chance at a new market
âą That project X opens up a substantial new
revenue stream for us
âą That project X has interesting technical
challenges
Fears:
âą That project X is much more technically difïŹcult
than we expected
âą That we get distracted and donât give project X a
fair shot
âą That weâre more excited about the revenue
opportunity than solving a real customer problem.
Hopes / Fears Exercise Results:
Who are the
stakeholders?âš
What do they want?
Q:
Answers:
34. Whatâs our mission? Whatâs our aligning metric? What are our
organizational objectives for the quarter/year (OKRs)? Who are the
stakeholders? What are their goals? Why invest more in the product
(e.g. instead of sales?)
Whoâs going to be building it? What are their personal goals and
interests? What are their strengths and weaknesses? How much of their
time do we have (e.g. are they cross functional? do they have other
projects? are there churn risks?) What are the teamâs strengths and
weaknesses? What are peopleâs expected communication styles?
Whatâs the organizationâs culture (e.g. Risk tolerance)? Whatâs the
organizationâs perceived brand? What are the expectations for a product
manager? Whatâs the (target) business model? What stage is the company
at? Whatâs the prevailing role within the company?
Who are the other companies in the space? How is your company
different? Tech advantages / debt? Design advantages / debt? Data
advantages?
Strategy: Relevant Qâs.
37. âSet up at least 30 meetings,
or you wonât have a good
Michael Sippey
Former VP Product
Twitter
User Research: Why?
38. Allison Smith*
27, Cleveland
New Mom
Yahoo Mail User
*Not really Allison But
thank you to this person
for CC Flickr photo.
How does Allison
use email?
39.
40. What does a typical day look
like?
Whatâs important to them?
Whatâs holding them back?
What do they think about on the
Qs:
Whoâs our Audience?Q:
43. âIf I had asked people what they
wanted, they would have said:
Faster Horses.â
Henry Ford*
Model-T Inventor
*Not really
44. What do users want?Q:
1. They canât say what they want
2. They wonât know whatâs
possible
3. Itâs not their job to know
challenges:
45. What do users want?Q:
âHow do you currently do [area you
want to explore]?â
âWhat are the top 3 things you love
about how youâre doing it now?â
âWhat are the 3 biggest frustrations
with how youâre doing it now?â
Ask:
46. Happy Event
Frustrating Event
Emotion Timeline
Exercise.
What do users want?Q:
"If you were to draw a chart of your
emotions throughout your experience
[doing X], with peaks when you're most
happy and valleys when you're most
unhappy, what would that look like?"
Ask:
47. A designer describes the 360
performance review experience.
Choose my
reviewers.
Find out itâs perf
review time.
Find out I have to
write 4 reviews
I write the reviews,
feel relieved
A long time goes
by and I forget
about it.
Boss emails to
say itâs time to
have the talk.
Talk day.
Promotion!
Emotion Timeline Exercise
Results:
What do users want?Q:
Answer:
48. Who are your competitors
customers? What do they
value?
Q:
49. â[your industry]â (testimonials|our
clients|our customers)
site:[competitor_site] filetype:pdf
site:twitter.com "apple maps" ("is
aâ|"sucks"|"can't"|"fuck"|"shit")
https://twitter.com/search?q=to
%3A[COMPETITORS_TWITTER]%20%3A)
:
Who are your competitors
customers? What do they
value?
Q:
50. User Research: Relevant Qâs.
What do our users think about on the subway?
Who are our primary (and secondary) users? What does a typical day look like
for them? What problems are they currently experiencing? (What are the 5
whys for the problems? What are they doing now to solve them? What do they
like about the current solution? What do they dislike about the current
solution? What are they feeling?)
What are the most important dimensions to describe them? (e.g.
demographics? behavior? company role? values? physical attributes? ) Who
are the extreme users? What do our users value? What are their goals? What
are their obligations? What are their frustrations? What are their stresses?
What gets them excited? Who are the people in their lives? How do they
typically interact with them? Whatâs the users value to us?
What language do they use to describe their experiences? What other products /
tools are they using? What are they reading? Who are they following? How do they
ïŹnd / buy related products? How do they ïŹt into purchasing decisions? What can
they spend? When do they spend? Whatâs their organizationâs culture like? How do
54. The Most Important Question:
What problem are weQ:
*Independently of how we might solve it.
55. *Independently of how we might solve it.
Advil.
Features are for suckers..
What problem are we
solving?
Q:
310M Americans experience
mild headaches and body
pain due to inïŹammation at
least once per year. Weâll
make that pain go away.
This:
Weâre manufacturing doses of
ibuprofen. Theyâll come in
blue or brown pills. The bottle
is child proof. Weâre
advertising the hell out of
them.
Not This:
56. âShiny feature X.â
CEO: It needs to be social. Add share buttons everywhere!
PM: Sir, itâs a porn site. How many people share porn on Facebook?
CEO: The investors say weâre not growing fast enough. And some site
called âBuzzfeed" is really taking off because of the sharing.
PM: It sounds like youâre saying the real problem is growth? Are
there other ways we could solve that?
*Independently of how we might solve it.
What problem are we
solving?
Q:
Get to the root of the problem. âš
Avoid building things that donât add value.
57. What problem are we solving?*Q: *Independently of how we might solve it.
80M Americans have have a
hard time ïŹnding things on
the internet.
Finding information on the internet.
Problem
Idea #1 Portals and directories (yahoo)
Idea #2 Search by phrase matching
(altavista)
Idea #3 search by phrase matching
and back links signal (google)
Problem framing leaves room for multiple solutions.
58. What problem are we solving?*Q: *Independently of how we might solve it.
PM:
Designer:
*Quits. Goes to Facebook.*
Problem framing helps you avoid irritating âš
your teamsâ designers and engineers.
The signup button must
be big and red so more
people will see it and click
on it.
59. What problem are we solving?Q:
Problem Statements:
1. Focus on value, not features.
2. Avoid building useless shit.
3. Allow for alternate solutions.
4. Arenât prescriptive.
5. Force clearer thinking.
*Independently of how we might solve it.
61. Opportunity Assessment:âš
The 11 most important questions for any product.
1. What problem are we solving? (value prop)
2. What problems are incidental (non-goals)?
3. For whom do we solve this problem? (target market)
4. How big is the opportunity? (market size)
5. How will we measure success? (metrics)
6. What alternatives are out there now? (competitors)
7. Why us? Whatâs our differentiator?
8. Why now? (market window)
9. Why might this a bad idea?
10.How will we market this?
11. Other considerations for success?
From The PM Handbook
(mostly).
62. Big Product
Qâs
(for each Product Lifecycle
Stage)
1. Who are we? Why are we here? âš
(strategy & existential crises)
2. Whoâs our audience? What do
they want? (user research)
3. What problem should we solve? âš
(opportunity assessment)
4. How should we solve the
problem? (design sprinting)
5. Are we approaching Product /
Market Fit? (build, measure, learn loop)
64. 1. Fear of Retribution
2. Timing
3. Management Middleman
challenges:
âHow can I
improve?â
Q:
How to get Honest Team Feedback at work.
65. Problem:
People at work- especially managers - have
difïŹculty getting honest, constructive
feedback from their teams at work, causing
missed opportunities for learning and
development. Root causes include:
Reviewersâ fear of retribution, feedback
infrequency, and concerns about how
criticism can negatively impact coworkersâ
careers.
66. Nodd helps team leaders ask the right questions to get
honest, constructive feedback from their teams at work.
It solves feedbackâs honesty problem with 3 simple
rules. Feedback through Nodd is:
Nodd keeps reviewersâ
identities conïŹdential and
responses are only ever shown
in aggregate from 3+
reviewers.
Nodd helps create a feedback
ritual by prompting teams with
simple, reviewer friendly micro-
surveys every two weeks or at
the end of each month.
Feedback is sent directly from
person-to-person and never
through management. So
reviewers can feel safe
knowing it wonât affect the
personâs pay.
Anonymous. Frequent. Private.
Nodd.co
73. âHow could we solve this problem AND REALLY DO
AN AWFUL JOB & EMBARRASS OURSELVES?
âHow would we solve this problem IN SPACE?!
âHow would we solve this problem IF OUR USERS
WERE EXCLUSIVELY 19th CENTURY PIRATES?
âHow would we solve this problem WITH
BENEVOLENT, AUTONOMOUS NANO-BOTS?
âHow would you solve this problem BY DOING THE
OPPOSITE OF YOUR LEFT-NEIGHBORâS IDEA?
What are all the different ways we could
possibly solve this?
Q:
Ask:
74. To assess team alignment, how many different ways do people answer:âš
âWhat problem are we solving for our users over
the next 2 months?â
When someone shows you work:âš
âWhat were your goals for this?â
To assess teamâs understanding of a story:âš
âOn a scale of 1, 2, 3, 5, 8, 13, 21, how complex
does everyone think this story is?â (planning
poker: okay for planning, great for
communication).
Are we on the same page?
Q:
Ask:
(probably not)
75. How can we improve our process?
Q:
Ask: Ask each team member to write: âš
âWhat are 3 things that we donât currently
do, that we should start doing?â
âWhat are 3 things that we currently do, that
we should stop doing?â
âWhat are 3 things that we currently do, that
we should continue doing?â