Remote Research, The Talk.

ﬂickr/thetruthabout

Thursday, July 1, 2010

REMOTE RESEARCH

In person (95%)
Deﬁnition: Remote (5%)

Research where the
participant is not
sharing physical
space with the
researcher.
% of qualitative research
done remotely*

*Source: BP’s best guess


#UXREMOTE


WHEN DOES REMOTE WIN?


Marv.
Creative director UX designer product manager
developer IA.


Marv’s Boss.
He’s always on a plane and loves birds.


Hey Marv, I want to save
some money on research.


NATIVE ENVIRONMENT
CHEAP ETHNOGRAPHY, NOT CHEAP LAB.


TIME-AWARE
RESEARCH IS THE BEST

I’LL TRADE YOU TIME
FOR PHYSICAL PRESENCE


INVOLVED OBSERVERS
Easier to collaborate in real time.


Geography

TEXT


WHEN DOES REMOTE LOSE?


Hey Marv, I want feedback
on our MMO video game
from 13yo English speaking
kids in France & Latin
America with webcams.


I guess, Boss.
Might be hard
to do with
remote testing.


BANDWIDTH
Yes, it’s an issue.


TRANSLATION
It’s expensive and di cult to do live.


MINORS
Consent is a bear.
Especially if they’re under 13.


FACES
Most users don’t have webcams.
Webcams are a little invasive.


SUPER DUPER SECURITY


AN EPIC FAIL.


How To Find Actual
English Speakers:

What Happens When
You Let Go Of A
Balloon?


WHAT WENT WRONG
Coins were way more valuable
than we expected
People mis-assessed their English
levels
Bandwidth was really a problem
Nobody plays Habbo when their
parents are home to consent


DESIGNING
REMOTE STUDIES


Yo Marv, get me some
insight on WHY people are
bailing out of checkout.


Sure, Boss.
Let’s intercept
& observe 10
users live with
screen-sharing,
and do 10 self-
moderated with
usertesting.com


TIME-AWARE TASKS
Told to ﬁnd purple pumps. Has to wear this for sister’s wedding this
weekend.
Doesn’t care.
Whole diﬀerent interview.


REAL TIME OBSERVERS


PROTOTYPES, WIREFRAMES,
SKETCHES?


HECK YES.


10 YEARS
244 STUDIES
2,915 PARTICIPANTS (MODERATED)
95 ORGANIZATIONS
19,610 HOURS


60% PROTOTYPE RESEARCH


A DINOSAUR FAMILY
EXPLAINS INFORMATION
ARCHITECTURE


Marv-o: are their any UX
web apps we can use for
online research?


Sure, boss.
There is an
AVALANCHE
of UX TOOLS.


“At least a little
behavioral”
Surveys.
Web Analytics.


FIVE CATEGORIES

• Moderated (screen-sharing + audio)
• Self-moderated (usertesting.com)
• Automated (loop11, usabilla)
• Fancy Analytics on Steroids
(crazyegg)
• Diary (revelation)


Moderated

Self-Moderated

For A Long Time

Automated Live

Conceptual

Automated Static

Web Analytics


Talk To People

People Talk To A Machine

For A Long Time

Task Bar On A Live Site

Tasks On Concepts

Task Bar On Images

Fancy-Ass Analytics


Insight

Ammunition

For Ad Agencies

More Ammunition

I.A. & Navigation

Still More Ammunition

Just Better Analytics


Moderated

Self-Moderated

For A Long Time

Automated Live

Conceptual

Automated Static Deep
Behavioral UX
Web Analytics Insight

Moderated
Opinions

Self-Moderated

For A Long Time

Automated Live

Conceptual

Automated Static

Web Analytics


Moderated Ad Agencies
Love It
Self-Moderated

For A Long Time

Automated Live

Conceptual

Automated Static

Web Analytics


Moderated “Online
Qualitative
Self-Moderated
Research”

For A Long Time

Automated Live

Conceptual

Automated Static

Web Analytics


Moderated Jared Spool
Hates It
Self-Moderated

For A Long Time

Automated Live

Conceptual

Automated Static

Web Analytics


THAT WAS 293 WORDS


USABILLA

Clicks & Questions
Heatmaps
You do your own recruiting
$49 - $199


LOOP11 (UN-MODERATED)

TEXT


* $49/mo
* Integrated VOIP
* iPad Viewing


Moderated

Self-Moderated

Automated Live

Conceptual

Automated Static

Web Analytics


WHAT WE USE.

A LOT.

• Ethnio
• GoToMeeting + iShowYouHD
• Usabilla + Usertesting.com


RECRUITING
FOR REMOTE


So Marv, how do we ﬁnd
people for these studies?


Stop
harassing me
and get o
that weird
plane.


RECRUITING OPTIONS
Realness
Grab someone in the hallway
Grab someone who isn’t directly
involved in your project
IM your friends for a quick look
Panels and recruiting agencies
Real customers from marketing lists
Site intercepts


GOLD STANDARD
Live recruiting
Get as close as you
can


RECRUITING OPTIONS

Ethnio
Links or house ads to online
forms
Recruiting agencies
Customer email lists
Panels
Craigslist/online ads

USE ETHNIO WHEN
Website access
Some budget
Quotas are relatively
simple


USE FORMS WHEN

You can place a link but not code
Low budget
Quotas are relatively complex
Options:
Google Docs
Wufoo


USE AGENCIES WHEN

No website access
People whine about ﬁrst-time
visitors
Higher budget
Online recruiting failed


CUSTOMER LISTS

Sub-segments
Ownership
requirements
No website access


SORT OF

CRAIGSLIST

When other methods don’t work
No dough
Takes a while
Embarrassing to admit to your
friends


LIVE RECRUITING MATH

Target: 6 recruits/hour
Allow for: 1% response (low
average
You Need: 10,000 uniques/day


Paying Incentives

Collect minimal
personal info
Deliver quickly
Appropriate for
location


HOW TO BE
graceful
When All Your
TECHNOLOGY
BREAKS

WHAT THEY WORRY ABOUT


WHO ARE YOU AND CAN I
TRUST YOU?


IF YOU CAN SEE MY
SCREEN,
CAN YOU SEE MY FACE?


WILL YOU BE ABLE TO
TAKE
CONTROL OF MY
COMPUTER?


WILL YOU HELP ME GET
THE PLUG-IN
OFF MY COMPUTER AT
THE END?


WILL THIS WORK IF I HAVE A
FIREWALL/PROXY/STRICT IT
DEPARTMENT?


IS THIS GOING TO END
UP ON
YOUTUBE?


OXYMORON?
REMOTE EMPATHY


EMPATHY TIP

Get someone on the phone
Give them an impossible puzzle


HYBRID STUDIES

Numbers actually do lie
sometimes
And they don’t tell you the why
3-5 remote interviews is all you
need


MOBILE REMOTE
RESEARCH?


ADVANCED TECHNIQUES
One to many moderation

Multi-threading

PORTABLE RESEARCH


BURN THE SCRIPT
FIND OUT WHAT MATTERS.


ADVANCED TECHNIQUES
High Emotions


THE FUTURE


remoteusability.com


@boltron


Remote Research, The Talk.

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (6)

Similar to Remote Research, The Talk.

Similar to Remote Research, The Talk. (20)

More from bolt peters

More from bolt peters (20)

Recently uploaded

Recently uploaded (20)

Remote Research, The Talk.