When it comes to delivering a solid product/market fit, customer validation is key. After all, you want to provide customers with a product that not only fills the need, but is delightful -- right? To get there, you need to commit to a vital blend of market research, strategy, and user testing.
Join John Little, Head of Product Marketing at Centercode, as he explains a two-part approach to customer validation. First, how do you strategically identify your top product areas that need attention prior to release. And second, how do you turn user data into strategic priorities that help the right teams make the right decisions.
(NEHA) Call Girls Katra Call Now 8617697112 Katra Escorts 24x7
Empowering Product Teams: Build Delightful Products with Customer Validation
1. Build Delightful Products with Customer
Validation
John Little Hannah Flynn
With: Moderated by:
TO USE YOUR COMPUTER'S AUDIO:
When the webinar begins, you will be connected to audio using
your computer's microphone and speakers (VoIP). A headset is
recommended.
Webinar will begin:
11:00 am, PST
TO USE YOUR TELEPHONE:
If you prefer to use your phone, you must select "Use Telephone"
after joining the webinar and call in using the numbers below.
United States: +1 (213) 929-4232
Access Code: 424-277-812
Audio PIN: Shown after joining the webinar
--OR--
Empowering Product Teams
Webinar Series
2. Revulytics gives any software producers deep and actionable insight into
who is using their software products and how they are being used, and the
out-of-box analytics that enable them to grow incremental revenue, convert
and retain customers, and make decisions about licensing and cloud
transformation strategies.
3. Empowering Product Teams
Webinar Series
Click on the Questions panel to
interact with the presenters
www.productmanagementtoday.com/webinar-series/empowering-product-teams
4. About John Little
John Little is the VP of Product Marketing at Centercode and co-creator of the Customer Validation Framework.
His background is in engineering and project management, with an emphasis on leadership starting with skills
gained in the Marine Corps. After 4 years working customer feedback into product development at Western Digital,
he teamed up with Centercode to build a better approach to customer testing. He loves to talk about product
development and customer feedback. His favorite leadership advice (stolen from the Corps) is “know yourself and
seek self-improvement.”
About Hannah Flynn
Hannah went to The University of Chicago, where she majored in Environmental Studies with a concentration in
Economics and Policy. She now works with Aggregage on social media strategy and webinar production for Product
Management Today and B2B Marketing Zone.
Empowering Product Teams
Webinar Series
5. 5
Empowering Product Teams
Webinar Series
5
Agenda
1 Need for Early Product Feedback
2 The Modern Approach Process
3 Analyzing and Presenting Findings
4 Q&A
6. 6
Empowering Product Teams
Webinar Series
Customer Validation…
• First, strategically identify your top product areas that need
attention prior to release.
• Second, turn user data into strategic priorities that help the
right teams make the right decisions.
7. 7
Empowering Product Teams
Webinar Series
Accomplished Through
1 Choosing an Achievable Feedback Goal
2 Organizing Experiences with Topics
3 A Simple Customer Acceptance Survey
4 Categorizing & Prioritizing Results
7
8. 8
Empowering Product Teams
Webinar Series
The Challenging State of Technology Today
Highly
Sophisticated
Software, Hardware, Services
Deeply
Connected
Alexa, Homekit, SSO, APIs...
Continuously
Evolving
OTA, Updates Forever! 30+ Smart Pet Collars
Highly
Competitive
8
9. 9
Empowering Product Teams
Webinar Series
Targeted Users
Market Audience (Demographics)
Actual Products
Post-Prototype
Real Environments
Homes, Offices (Technographics)
Over Time
Weeks or Months
Defining Attributes of Customer Validation
9
10. 10
Empowering Product Teams
Webinar Series
Quality
Release
Customer Validation Lifecycle
10
Launch Confident (110%)
All relevant results and recommendations
have been implemented. The product
is ready to go to market.
Test Ready (60-80%)
The product has been vetted by
professional testing resources
and deemed ready for initial
usage by end-users.
Alpha
Beta
Near Feature Complete (80-90%)
Engineering and Product are confident that
most severe issues have been resolved and the
product is ready for representative customers.
Delta
Ongoing Iteration (X.x)
The product team continues to evolve and
improve the product, focusing testing on new
features and quality enhancements
11. PROFIT
Effective Customer Validation Impact & ROI
QUALITY COST
DEV TIME
INTERNAL BIAS
TIME TO MARKET
RETURNS
SUPPORT COST
QUALITY
CX/UX
REVIEWS
CUST SAT
PLANNING
SALES
11
12. 12
Empowering Product Teams
Webinar Series
Key Customer Validation Challenges
12
Project design and scoping Persistent user engagement
Collecting useful feedback
Deriving actionable recommendations
Allocating qualified resources
Recruiting ideal testers
17. 17
Empowering Product Teams
Webinar Series
Generate and Share Impact Analysis
Customer Validation Process Overview
17
Recruitment and Selection; Project KickoffPrep
Weekly
Planning Test Scoping and Product Mapping
Directed
Engagement
Closure Incentives, Collection, and Wrap-Up
Ongoing
Feedback
Tester
Support
18. 18
Empowering Product Teams
Webinar Series
Test Scoping and Product Topic Mapping
18
Topics represent the individual features, functions, and experiences you want tested
Product
Setup
The process of connecting the
product for first time use.
Find a spot to plug the device
into both the network and wall.
High M
Account
Creation
The process of creating a
personal account to customize
the product.
Log into www.acme.com and
create a new user account.
Low S
Topic
Out of
the Box
Description
The initial experience opening
the product package.
Activity
Receive and unbox the product.
Priority
Med
Size
S
19. 19
Empowering Product Teams
Webinar Series
Tester Recruitment
19
US Resident, Own 2+ Smart Home Products
Android iOS .
Smart Home Interest + Experience
Age 18-25 Age 26-35 Age 36-50 .
Customer Base + Recruited
Core Requirements
Technographic Segmentations
Additional Traits
Demographic Segmentations
Executed via a 5-10 Question "Qualification Survey"
21. 21
Empowering Product Teams
Webinar Series
21
Ongoing Feedback Collection
All Ongoing Feedback should be aligned with Topics
Test quality, interoperability, and real-world performanceIssues1
Measure acceptance, prioritize backlog, generate new ideasIdeas2
Gather delights, strengths, and real-world usage insights
Emulate real-world discussion; drive focused collaborationDiscussion4
Praise3
22. 22
Empowering Product Teams
Webinar Series
Announce
Weekly Topic
Activities
Engage
& Support
Testers
Customer
Acceptance
Survey
1 2 3 4
22
Weekly Engagement Process
23. 23
Empowering Product Teams
Webinar Series
1. Please rate your satisfaction of unboxing the product
23
It sounds like you ran into issues that lead to a negative
experience. If you haven't already done so, please take a
moment to report any of these Issues right now.
24. 24
Empowering Product Teams
Webinar Series
2. Please rate your satisfaction of setting up the product.
24
It looks like you see room for improvement in this area.
If you haven't already done so, please take a moment to
share any Ideas that would lead to a higher score.
25. 25
Empowering Product Teams
Webinar Series
3. Please rate your satisfaction of creating your account.
25
Awesome! Would you please take a moment to
share your Praise regarding whatever specifically
earned the 5th star in your rating. Thanks!
26. 26
Empowering Product Teams
Webinar Series
Announce
Weekly Topic
Activities
Gather
& Present
Results
Engage
& Support
Testers
Customer
Acceptance
Survey
1 2 3 4
26
Weekly Engagement Process
30. 30
Empowering Product Teams
Webinar Series
Impact Scoring is a prioritization system
which considers the relative Weight and
Popularity of each piece of tester feedback,
based specifically your Objectives.
Feedback Impact Scoring
30
i.e. Severity * Frequency
31. 31
Empowering Product Teams
Webinar Series
Severity Frequency
Item Example
Type Issue
Topic Unboxing
Severity Critical
Status Reproduced
Item
Duplicates
Votes
Comments
Views
32. 32
Empowering Product Teams
Webinar Series
Issue Idea Praise
Weight 2 1 1
Low Medium High
Weight 1 2 4
Cosmetic Minor Major Critical
Weight 1 2 4 6
Reproduced
Weight 2
Feedback Type
Topic Area
Issue Severity
Other Status
33. 33
Empowering Product Teams
Webinar Series
Example Use Case
Consider a hardware product, where an antenna is
bending during the unboxing process, which
results in an issue reported by multiple testers.
33
34. 34
Empowering Product Teams
Webinar Series
Issue Idea Praise
2 1 1
Low Medium High
1 2 4
Cosmetic Minor Major Critical
1 2 4 6
Reproduced
2
Weighted Average = 2.5 (3)
Type of Issue
Topic Area
Issue Severity
Other Status
35. 35
Empowering Product Teams
Webinar Series
Frequency Weight
Low 1
Med 2
High 4
Very High 6
Adding Issue Frequency
Duplicates
Votes
Comments
Views
36. 36
Empowering Product Teams
Webinar Series
Frequency Weight
Low 1
Med 2
High 4
Very High 6
Adding Issue Frequency
Duplicates
Votes
Comments
Views
37. 37
Empowering Product Teams
Webinar Series
1 2 3 4
Low (1) 2 3 4 5
Med (2) 3 4 5 6
High (4) 5 6 7 8
Very High (6) 7 8 9 10
Severity
Frequency
38. 38
Empowering Product Teams
Webinar Series
1 2 3 4
Low (1) 2 3 4 5
Med (2) 3 4 5 6
High (4) 5 6 7 8
Very High (6) 7 8 9 10
Severity
Frequency
Issue Impact = 7
40. 40
Empowering Product Teams
Webinar Series
Leveraging Impact Score
Prioritize all tester feedback
Identify product problem areas
Identify areas of improvement
Identify praise-worthy feature sets
Benchmark products against each other
Identify most impactful testers
40
42. 42
Empowering Product Teams
Webinar Series
42
CUSTOMER ACCEPTANCE SCORECARDS
Topic Grading Scale
The comparison of your topic Customer Acceptance score is
based on Centercode's collection of Customer Acceptance
Scores across all Beta Tests. See Appendix for more details.
OUT OF THE BOX D+68%
PRODUCT SETUP A-92%
ACCOUNT CREATION B85%
Greater than 90% A
82% - 90% B
70% - 81% C
62% - 69% D
Less than 62% F
Week 1 Topics
43. 43
Empowering Product Teams
Webinar Series
Topic Overview Scorecard Sample
43
D+
68%
Recommendations to FIX 3
TOPIC: OUT OF THE BOX
Customer Acceptance Score
This score is BELOW
AVERAGE.
Recommendations to IMPROVE 4
Recommendations to
PROMOTE 1
RecommendationsUnboxing Task
TASK
Review the packaging of the product.
DESCRIPTION
Review the product packaging and
information without opening the box.
Get to know the product as if you were
considering it prior to purchase.
44. 44
Fix Recommendation Sample
Tester Quotes
SOURCE FEEDBACK
BUG-0015
Antenna broke pulling
it out of the box
Reinforce the
Antenna in Packaging
Some testers reported frustration with
the antenna bending or breaking while
removing it from the packaging.
A damaged antenna can lead to a
poor perception of product quality as
well as performance issues.
“While trying to remove the product from the box
the antenna snapped off. I did not even apply that
much pressure.”
“I did not realize that the antenna was broken off
until I heard it banging around in the box. Very
cheaply made. Plus the packaging was so hard to
remove, it is no wonder it broke off."
20%
RESPONDENTS
7
IMPACT SCORE
Critical
SEVERITY
FIX
RECOMMENDATION
Pro Tip: Create an “elevator pitch” for each significant issue
45. 45
Improve Recommendations Sample
Tester Quotes
SOURCE FEEDBACK
SUG-0041
Twist Ties Were
Difficult to unravel
“Everything was going great as I pulled stuff out of
the box, except the twist ties that held the ethernet
cord together were already broken.”
“I enjoyed pulling everything out of the box. Things
were neat and easily accessible. But, the twist ties
around the cords looked like twist ties on bread and
took the feel of quality out of most the experience.”
23%
RESPONDENTS
4
IMPACT SCORE
Minor
SEVERITY
IMPROVE
RECOMMENDATION
Provide Higher
Quality Twist Ties
Testers feel that the twist ties are
very cheap compared to the rest of
the contents in the packaging. Some
testers reported that twist ties were
broken upon first inspection.
This gives testers the mindset of a
product of lesser quality and can
result in a poor display.
46. 46
Promote Recommendation Sample
Tester Quotes
SOURCE FEEDBACK
PRA-0026
Onboarding was ultra fast
“WOW! That was probably the
simplest product setup I've ever done.
I was up and running in less than 5
minutes."
30%
RESPONDENTS
5
IMPACT SCORE
PROMOTE
RECOMMENDATION
Quick, Intuitive Setup
and Installation
Testers felt the onboarding
experience as a whole was very
simple and was done in a matter of
minutes with no major errors.
This should be shared with your
marketing team to fuel customer
testimonials and product messaging.
"I'm pretty impressed with how
quickly I was able to get set up. I
assumed it was going to take a
decent amount of my time, but was
pleasantly surprised.
Documentation, packaging, pretty
much everything was easy to get
through. I would expect my mother
could pick this thing up and install
it.
“I couldn't think of a way to make the
setup experience any better. I finished
just as soon as a started."
47. 47
Empowering Product Teams
Webinar Series
Fix Improve Promote
Deliver Actionable Recommendations
47
Engineering, QA,
& Support
Product Mgmt
& UX/CX
Marketing
& Sales
49. 49
Empowering Product Teams
Webinar Series
Map Product With Topics & Activities
49
Topics represent the individual
- Features
- Functions
- Experiences you want tested
Topic
Out of
the Box
Description
The initial experience opening
the product package.
Activity
Receive and unbox the product.
Priority
Med
Size
S
50. 50
Empowering Product Teams
Webinar Series
Announce
Weekly Topic
Activities
Gather
& Present
Results
Engage
& Support
Testers
Customer
Acceptance
Survey
1 2 3 4
50
Consistent Weekly Engagement
51. 51
Empowering Product Teams
Webinar Series
Fix Improve Promote
Deliver Actionable Recommendations
51
Engineering, QA,
& Support
Product Mgmt
& UX/CX
Marketing
& Sales
52. 52
Empowering Product Teams
Webinar Series
Goals Recap
• Strategically identify your top product areas that need attention
prior to release.
• Turn user data into strategic priorities that help the right teams
make the right decisions.
53. 53
Empowering Product Teams
Webinar Series
Achieve your goals by
1 Choosing an Achievable Feedback Goal
2 Organizing Experiences with Topics
3 Using a Simple Customer Acceptance Survey
4 Prioritizing Results with Satisfaction and Impact Scores
53
