Unbiased customer interviews v022 for slideshare kall consulting

1 Unbiased Customer UX Review
Interviews

Unbiased Customer Interviews:
What insight can we trust?

KALL Consulting
customer and user experience design and strategy

full version: 16Sep2011
© Kall Consulting 2011 --- +1 937-648-4966 --- darrenkall@kallconsulting.com 1

2
What’s the problem?
Unbiased Customer UX Review
Interviews

1. Informed business decisions need accurate customer
insight.

2. Customer interviews for insight are different than other
interactions with customers.

3. People accidentally introduce biases in customer
interactions.

4. These biases taint the information they bring back.

5. Business decisions are made on this tainted information.

6. Tainted decisions increase business costs
unnecessarily, and result in lost revenue.

3
Volunteers Needed
Interviews

Every customer interaction is not an interview
Selling Effect

Your approach can influence responses
That’s not it! Effect

How information is gathered makes a difference
Number Bias
Demand Characteristics Effect


4
Customer Interview Workshop
Interviews

Recording your Interviews

An Interview Coach

Interview Logistics

Biases in Interviewing

Picking an Interview Methodology

Question Design

Using Persona to Pre-categorize Interviewees

Data Analysis and Reporting

Learning to share: Customer insight database

5 Biases and Effects
12
Interviews

1. Selling Effect 7. Confirmation Bias
2. “That’s not it!” Effect 8. Framing Effect Bias
3. Number Bias 9. Encouragement Bias
4. Demand 10. Correlation and
Characteristic Effect Causality Bias
5. Scaling and Anchor 11. Ingrained Biases
Biases 12. Status Quo Bias
6. Consistency Bias


Extra
Interviews

• Not Remaining • Recency Effect
Neutral Effect • Anchor Bias
• Script Bias 1 • Impact Bias
• Script Bias 2 • Not Invented Here
• White Lie Effect Bias
• Not Listening Effect • Bandwagon Effect
• Value Bias


7 More Biases not Covered Today
22
Interviews

• Ambiguity effect • Information bias
• Black swans (attention) • Loss aversion bias
• Choice supportive bias • Misunderstanding
• Clustering illusion probability
• Contrast effect • Negativity bias
• Denomination effect • Omission bias
• Detection bias • Overconfidence effect
• Endowment effect • Reporting bias
• Funding bias • Scope neglect bias
• Halo effect • Unit bias
• Hindsight effect • Zero-risk bias
• Hyperbolic discounting • Etc. The list goes on.


8 or Effect Name
Bias
Interviews

The behavior, perception, or preference that
causes the problem.

How the bias gets introduced and/or
what happens when it does.

What you should do instead.

Rationale for why people behave this
way or why you should avoid this bias.


9
Selling Effect
Interviews


10 Effect
Selling
Interviews

Selling your product or your idea instead of
interviewing.

Customers respond to pitch, not to the
interview. Your insight is limited.

Don’t sell. Save the selling for another
interaction.

You’re not there to sell. You're trying to understand who the
customer is in order to get insight into their world.


11 not it!” Effect
“That’s
Interviews


12 not it!” Effect
“That’s
Interviews

Correcting, demeaning, blaming, etc. the
customer.

Treating the customer this way gets them
defensive or shuts them down.

Don't be corrective or pedantic. Reward the customer for
telling you things even if you disagree with them. Clarify
your understanding, capture the ideas, and move on.

The customer has their opinions and ideas. A customer
interview is not the place to be correcting them, even if
the idea is something the company will never do or that
you disagree with.


13
Number Bias
Interviews


14
Number Bias
Interviews

Thinking and reporting of numbers is highly
influenced without awareness.
13
People avoid some numbers. They “round” numbers to
align with their base system. Different cultures have
different biases. If asked for a number, people will make
up answers.

Be aware that the numbers you get back are not absolute
but are shaped. They are not data.

Culture, base 10, 6, 8 or 60 numbering systems, whole
numbers over fractions, having 24 hours, 7 days, all frame
the number bias.


15
Interviews


16
Interviews

You provide cues to your purpose and
interviewees give you a
“right” answer.

You don’t get a true answer, you get the biased answer you
“ask” for.

Remove demand characteristics, encourage open and
difficult responses.

Only by eliminating the expectation of a “right” answer can
you get to unbiased answers.


17 and Anchors Biases
Scaling
Interviews


18 and Anchors Biases
Scaling
Interviews

Making up scaling anchors, making up
scales, changing scales.

Introduces biases in customer
answers, make interpretation
impossible.
Get an expert to build your scales and set anchors. Don’t
make them up, change them, or try to be funny.

Inappropriate scales, changing them, etc. makes them
meaningless and impossible to compare.


19
Consistency Bias
Interviews


20
Consistency Bias
Interviews

Filtering out information accidentally to
make things consistent. Extrapolating
from one example to all cases.

Conclusions drawn won’t reflect the
reality of the customer’s variety.

Report what you observe not what you want to see. Report
all that you observe.

Forcing consistency where it does not exist is biasing the
information.


21
Confirmation Bias
Interviews


22
Confirmation Bias
Interviews


23
Confirmation Bias
Interviews

Find only supportive information, interpret or
avoid contradictions to fit your idea.

You only report supporting information
and you don’t SEE what you filtered
out with your bias.

Attempt to capture everything and make fair assessments.
Get someone else to review your interview recordings.

Confirmation bias is ingrained and hard to undo without
conscious attention and practice.


24
Framing Effect Bias
Interviews


25
Framing Effect Bias
Interviews

600 People with a fatal disease

Drug A saves 200 people's lives

Drug B has a 33% chance of saving all 600
people and a 66% possibility of saving no
one


26
Framing Effect Bias
Interviews

600 People with a fatal disease

If Drug C is taken, then 400 people die

If Drug D is taken, then there is a 33%
chance that no people will die and a 66%
probability that all 600 will die


27
Framing Effect Bias
Interviews

Framing a question or choice in a way that
manipulates its perceived value.

How a question or choice is presented biases the listener’s
answers.

Get an expert question designer to review your question
script. Explore different ways to present the question to
see if it influences the answer.

Framing effects work because of human nature preferring
gain over loss, certainty over doubt, positive over
negative.


28
Encouragement Bias
Interviews


29
Encouragement Bias
Interviews

Encouraging a particular result, giving
“good answer” cues.

Responses can be biased and shaped in
one direction or another.

Don’t reveal desired goal, don’t reward
positive or punish negative responses;
or vice versa. Remain consistent
customer to customer.

Encouragement sends information to the user as to the
desired outcome and they may respond without even
being aware.


30
Correlation and Causality Bias
Interviews


31
Correlation and Causality Bias
Interviews

Assuming that because things occur
together, or close in time, that one causes
the other.

When you ask interviewees for root
causes they will report correlations
which may not be the true root cause.

Correlations do not imply causality. Consider but don’t
blindly accept the rationale without triangulation.

General rules (heuristics) are built from experience and
applied incorrectly to all occasions.


32
Ingrained Biases
Interviews


33
Ingrained Biases
Interviews

Cultural, gender, national, geographic and
other backgrounds bias
behavior, preferences, and perceptions.

Both interviewers and interviewees
respond to each other filtered with
these biases.

Mitigate your own ingrained biases as much as possible.
Counter balance interviewers. Use interviewers from the
same culture and country.

You can’t get away from your accent or other culture you
wear. You cannot control how others see you.


34 Quo Bias
Status
Interviews


35 Quo Bias
Status
Interviews

Some people you will interview will have a
preference to keep things as they are
regardless of what you are proposing.

This biases the response away from new
things that will introduce change even
if they are better options giving them
false values.

Understand the motivations and loss for this persona and
address them.

Study on rats, cheese, hunger and shock grid: status quo
tipping point.


Extra
Interviews

• Not Remaining • Recency Effect
Neutral Effect • Anchor Bias
• Script Bias 1 • Impact Bias
• Script Bias 2 • Not Invented Here
• White Lie Effect Bias
• Not Listening Effect • Bandwagon Effect
• Value Bias


37 Bias 1
Script
Interviews


38 Bias 1
Script
Interviews

Skipping questions in the script and
improvising question variations.

Skipping questions and significant
improvisation force biases in
analysis when combining
information.

Hit all the questions. Don't improvise too much or you won't
have fair comparisons.

These are parts of normal conversation but interviews are
not normal conversations.


39 Bias 2
Script
Interviews


40 Bias 2
Script
Interviews

Ignoring or cutting off customer attempts to
roam away from script.

You can miss gems of insight. You are
blind-sided by your focus/belief in your
survey and your bias limits the scope
of information.

Allow enough time to let the customer lead you “off track”.
Explore, evaluate, capture, then get them back on track.

The best stuff may be in what the customer knows that you
did not think to ask about.


41 Lie Effect
White
Interviews


42 Lie Effect
White
Interviews

Telling white lies for
approval, support, encouragement, etc
..
Lies introduce biases in future
responses since based on
falsehoods.

Don't lie. Don't make false promises. Don’t confirm just
because they ask.

White lies seem like a kindness or an easy avoidance but
they have a devastating effect on the value of the
information. And you have to remember your lie.


43 Listening Effect
Not
Interviews


44 Listening Effect
Not
Interviews

Allowing interruptions such as
email, or thinking of next
appointment, to shift your focus
from the customer.

This biases the customer against
you and changes their
responses.

Stay focused. Take break if need one. Reschedule if needed.
Actively listen.

If you’re not listening, you’re missing things.


45 Bias
Value
Interviews


46 Bias
Value
Interviews

Using customer information as a weapon.
It and your conclusions are complete
perfect answers.

This blinds you to alternatives, gives
you false confidence in partial
information, & shuts down other
people.

Do interviews, bring back impressions, draw conclusions
and innovate ideas. But don’t treat them as data and facts.

They are only impressions; treat them that way.


47 Remaining Neutral Effect
Not
Interviews


48 Remaining Neutral Effect
Not
Interviews

Show positive or negative reactions to
customer responses, opinions, or actions.

Giving feedback teaches the customer
too much about your views, and
biases their responses.

Remain neutral and indifferent to their answers, opinions,
etc. Remain calm and non-committal. Show no value
reactions.

It is human nature to give feedback but doing so taints
responses.


49
Recency Effect
Interviews


50
Recency Effect
Interviews

Human memory biases toward the last item
in lists being remembered better.

The most recent interview or last
comments will influence your
conclusions or reporting of
findings more.

Take notes on all interviews, review all materials when
doing analysis.

This is a core way human memory works along with the
Primacy Effect.


51 Bias
Anchor
Interviews


52 Bias
Anchor
Interviews

Interviewees focus on one aspect
(speed, color, size, etc.) and give it more
importance making it a reference or anchor
for all other comparisons.

People are biased by this one aspect
skewing all their responses.

Discover and dismiss the anchor “Ignoring color…”. Use
paired comparisons. Define the anchor instead.

First exposed or obvious aspects often become the
comparison for all other judgments.


53 Bias
Impact
Interviews


54 Bias
Impact
Interviews

When people make judgments of future
impacts they exaggerate both the positive
and negative aspects.

The estimates you get back are biased to
overestimation in both directions.

This is not data, it is perspective and opinion information
that you should note on questions where you asked for
judgments of future impact.

This is a natural tendency without awareness. People
believe the numbers they give you.


55 Invented Here Bias
Not
Interviews


56 Invented Here Bias
Not
Interviews

Bias in interviewees and interviewers that
what we create is better than what someone
else creates.

Interviewees give false feedback on
ideas that are not theirs. Interviewers
miss better solutions that customers
may create.

Be aware of biases on both sides and keep consideration
open.

There are many reasons: fear of unknown, unwilling to
value others, lack of control, etc.


57
Bandwagon Effect
Interviews


58
Bandwagon Effect
Interviews

Some people influence group opinion and
others adopt it.

Strong personalities can influence
people interviewed in groups.

Not always a bad thing. If you want unbiased
opinions, interview alone. If you want to generate
conversation to get more information, use a group.

Group dynamics influence, behaviors, perceptions and
preferences.


59 Consulting Specialties
KALL
Interviews

Product and Service Development and Improvement:
Applying user experience techniques to the end-to-end
customer and user experience design

Product Integration:
Integrating the functionality of multiple products, combining
several products into one product, and integrating acquired
products into a product family

Merger and Acquisition User Experience:
User experience evaluations in due diligence for buyers, user
experience strategy as part of long term exit plan strategy for
sellers, integration of products post M&A


60
Caution
Interviews

What I’ve told you about today can also be used for

eviL
Please don’t.


61 You
Thank
Interviews

Darren Kall
• darrenkall@kallconsulting.com
• http://www.linkedin.com/in/darrenkall
• @darrenkall
• +1 (937) 648-4966

I’m glad to help your company become more
customer and user-centric.


62 Credits
Photo
Interviews

blood splat http://www.centrepatriotes.com

clock http://www.armyproperty.com/Resources/NSN-Listings/Clocks.htm

salesman http://www.expressionsofexcellence.com/GIFs/creative_selling.gif

buttons http://dragonlily.com.au/shop/popup_image.php?pID=154

script http://www.hearingreview.com/issues/articles/2003-10_02.asp

wrangler round up http://www.thebigquestions.com/wp-content/uploads/2010/04/roundup.jpg

liar http://www.mediabistro.com/fishbowldc/files/original/art_lie.jpg

scolding teacher http://2.bp.blogspot.com/_NjXJS2YMaYw/SyBiwnntCkI/AAAAAAAAAGY/hZjiRbF4TZg/s200/scold.gif

ear bud yeller http://quickbase.intuit.com/blog/wp-content/uploads/2010/08/92038203_5d8d68f920.jpg

knowledge is power
cartoon http://llenrock.com/blog/wp-content/uploads/2009/12/knowledge-is-power.gif

Home Depot survey http://www.homedepotopinion.com/

neutral shirt http://www.zazzle.com/

Rosie riveter http://en.wikipedia.org/wiki/Rosie_the_Riveter

confirmation bias http://www.agenarisk.com/resources/probability_puzzles/confirmation_bias.shtml

matrix choices http://imo-networker.com/two-roads-to-take/

brick framed http://twistedphysics.typepad.com/.a/6a00d8341c9c1053ef0148c85e67e9970c-pi

brain http://www.internettime.com/wp-content/uploads/2009/12/brain-300x226.png

anchor http://www.boatweb.com.au/images/admiralty.gif

fear status quo http://writingcreativenonfiction.files.wordpress.com/2011/07/fear-of-change.jpg

bingo balls http://www.bingo-balls.com/

family circus cartoon http://www.oxfamblogs.org/fp2p/wp-content/uploads/causation-v-correlation-cartoon-246x300.gif

cultural bias faces http://news.brown.edu/node/10388?size=large

flying car http://www.metro.co.uk/weird/58822-flying-cars-are-here-at-last

bridge jumpers http://diyblogger.net/wp-content/uploads/2010/12/jumping_bridge.jpg


Unbiased customer interviews v022 for slideshare kall consulting

Recomendados

Recomendados

Mais conteúdo relacionado

Semelhante a Unbiased customer interviews v022 for slideshare kall consulting

Semelhante a Unbiased customer interviews v022 for slideshare kall consulting (20)

Mais de Darren Kall

Mais de Darren Kall (10)

Último

Último (20)

Unbiased customer interviews v022 for slideshare kall consulting