Every product decision is an experiment. If we can reframe our thinking from launching campaigns and building features to challenging assumptions and running experiments, then we can increase our productivity and get better results with less effort. Grace will share a structured process for designing effective experiments, and talk about some of the common pitfalls and challenges teams face when running experiments in their companies. Using examples from her work helping product teams within large companies, Grace will reveal a few ways to overcome these challenges.
5. @uxceo
Minimum Viable Product!
An experiment that uses the least amount of effort
to get the most amount of customer feedback
6. @uxceo
Minimum Viable Product!
An experiment that uses the least amount of effort to test the
Riskiest Assumption against real customer feedback
7. Do people even have visual questions to post?
"
Riskiest Assumption: Have photo Qs to post.
Ask Question
Upload Photo
@uxceo
8. What kind of bird is this? (Is it even a bird??)
@uxceo
9. Will there be people who are able to answer the photo
questions that are posted?
"
"
Riskiest Assumption:
Someone can answer within given time.
@uxceo
10. What kind of bird is this? (Is it even a bird??)
@uxceo
33. How How to to MEASURE measure success?
success?
1. Cost
• Project Cost
• Opportunity Cost
"
2. Existing metrics
• Business KPI
• Specific user behavior
@uxceo grace@QuickMVP.com
@uxceo
36. Effective Interviews
1. I’m doing some research around...
2. Have you ever [experienced problem]?
3. Tell me about that time, what happened?
4. What did you do to resolve it?
5. How satisfied were you with the outcome?
6. If unsatisfied, how else have you tried to solve it?
@uxceo
37. Collect Data
• Level of pain/frustration
"
• Frequency of encounter with pain
"
• Existing behavior to solve problem
"
• Time/money spent searching for a solution
@uxceo
38. No
reliable
source
Interview:
6/10
9/12
Persevere
Friends.
Classes.
Pay for
UX help
@uxceo
43. No
reliable
source
Interview:
6/10
9/12
Persevere
Friends.
Classes.
Pay for
UX help
@uxceo
Pay for
virtual UX
feedback
Pitch:
8/10
10/10
Test
2nd side
of market
Don’t mind
virtual.
It’s cheap.
@uxceo
47. No
reliable
source
Interview:
6/10
9/12
Persevere
Friends.
Classes.
Pay for
UX help
@uxceo
Pay for
virtual UX
feedback
Pitch:
8/10
10/10
Test
2nd side
of market
Don’t mind
virtual.
It’s cheap.
@uxceo
Will
consult
virtually
Pitch:
8/10
0/10
Pivot!
48. No
reliable
source
Interview:
6/10
9/12
Persevere
Friends.
Classes.
Pay for
UX help
@uxceo
Pay for
virtual UX
feedback
Pitch:
8/10
10/10
Test
2nd side
of market
Don’t mind
virtual.
It’s cheap.
@uxceo
Will
consult
virtually
Pitch:
8/10
0/10
Pivot!
No!
5 yrs exp.
Established
49. Boot-strapped
startup
founders
Getting
UX
feedback
No
reliable
source
Interview:
6/10
Boot-strapped
startup
founders
Getting
UX
feedback
Virtual
UX
feedback
Pay for
virtual UX
feedback
Pitch:
8/10
@uxceo
UX
Designers
Getting
clients for
quick UX
consulting
Virtual
UX
feedback
Will
consult
virtually
Pitch:
8/10
Transition
to UX
designers
Getting
clients for
quick UX
consulting
50. Boot-strapped
startup
founders
Getting
UX
feedback
No
reliable
source
Interview:
6/10
Boot-strapped
startup
founders
Getting
UX
feedback
Virtual
UX
feedback
Pay for
virtual UX
feedback
Pitch:
8/10
@uxceo
UX
Designers
Getting
clients for
quick UX
consulting
Virtual
UX
feedback
Will
consult
virtually
Pitch:
8/10
Transition
to UX
designers
Getting
clients for
quick UX
consulting
Virtual
UX
feedback
Give quick
online
feedback
Pitch:
60%
55. 3 Data Points
@uxceo @uxceo @nycuxpa #ngnycuxpa
@uxceo
56. No
reliable
source
Interview:
6/10
9/12
Persevere
Friends.
Classes.
Pay for
UX help
@uxceo
Pay for
virtual UX
feedback
Pitch:
8/10
10/10
Test
2nd side
of market
Don’t mind
virtual.
It’s cheap.
@uxceo
Will
consult
virtually
Pitch:
8/10
0/10
Pivot!
No!
5 yrs exp.
Established
Give quick
online
feedback
Pitch:
60%
37/184
emails
Persevere
Started
referring
friends
57. Boot-strapped
startup
founders
Getting
UX
feedback
No
reliable
source
Interview:
6/10
Boot-strapped
startup
founders
Getting
UX
feedback
Virtual
UX
feedback
Pay for
virtual UX
feedback
Pitch:
8/10
@uxceo
UX
Designers
Getting
clients for
quick UX
consulting
Virtual
UX
feedback
Will
consult
virtually
Pitch:
8/10
Transition
to UX
designers
Getting
clients for
quick UX
consulting
Virtual
UX
feedback
Give quick
online
feedback
Pitch:
60%
58. Boot-strapped
startup
founders
Getting
UX
feedback
No
reliable
source
Interview:
6/10
Boot-strapped
startup
founders
Getting
UX
feedback
Virtual
UX
feedback
Pay for
virtual UX
feedback
Pitch:
8/10
@uxceo
UX
Designers
Getting
clients for
quick UX
consulting
Virtual
UX
feedback
Will
consult
virtually
Pitch:
8/10
Transition
to UX
designers
Getting
clients for
quick UX
consulting
Virtual
UX
feedback
Give quick
online
feedback
Pitch:
60%
Boot-strapped
startup
founders
Getting
UX
feedback
Virtual
UX
feedback
Online
platform is
sufficient
Concierge:
60%
satisfied
61. No
reliable
source
Interview:
6/10
9/12
Persevere
Friends.
Classes.
Pay for
UX help
@uxceo
Pay for
virtual UX
feedback
Pitch:
8/10
10/10
Test
2nd side
of market
Don’t mind
virtual.
It’s cheap.
@uxceo
Will
consult
virtually
Pitch:
8/10
0/10
Pivot!
No!
5 yrs exp.
Established
Give quick
online
feedback
Pitch:
60%
37/184
emails
Persevere
Started
referring
friends
Online
platform is
sufficient
Concierge:
60%
satisfied
20%
Iterate
Wanted
more
function-ality