UX Research

1. UX 리서치
(UX/UI 개발을 위한 사용자
조사의 이해 및 방법)
2012.11.16
InnoUX CEO 최병호
InnoUX@InnoUX.com, @ILOVEHCI

2. Table of Contents
• Definition
• Case Study
• Methodology
 Overview
 Field Study
 Card Sorting
 Heuristic Evaluation
 Usability Test
 Remote Test
 Focus Group
 Benchmarking
 Contextual Inquiry
 Eye Tracking
 Diary Study
 Cognitive Walkthrough
• References
UX 리서치 1 © 2012 InnoUX & Innodesign All rights reserved.

3. Definition

20. 4 Common Biases
in Customer Research
• Confirmation Bias
• Framing Effect
• Observer-expectancy Effect
• Recency Bias

21. Confirmation Bias
Your tendency to search for or interpret
information in a way that confirms your
preconceptions or hypotheses.

22. Framing Effect
When you and your team draw different
conclusions from the same data based on your
own preconceptions.

23. Observer-expectancy
When you expect a given result from your
research which makes you unconsciously
manipulate your experiments to give you that
result

24. Recency Bias
This results from disproportionate salience
attributed to recent observations (your very last
interview) – or the tendency to weigh more
recent information over earlier observations

27. Methodology:
Overview

39. Types of AgileUX Customer Research
Contextual Inquiry
Task Analysis
Card Sorting
Focus Groups
Surveys
Usability Testing

40. My bullshit customer insights graph
12
People
The most striking truth of this curve is that
zero users give zero insights
0 Insights Lots

41. You need to gather:
• Factual information
• Behavior
• Pain points
• Goals
You can document this on the persona validation board
As well as…
Photos, video, audio, journals…document everything

47. Types of Research Methods
Quantitative
12 fMRI Brain Imaging
10 Eye Tracking
9 Online Card Sorting
11 Online UX Concept Surveys
4 Focus Groups
7 Large Sample On-Line Behavior Testing
6 Lab-Based Testing
Generative 5 Ergonomic Observation
Evaulative
8 Professional
Heuristics
2 Remote Ethnography
1 Contextual Observation
Qualitative
46

48. 1 Methodology: Contextual Observation/Ethnography
► Business problem
► How are people actually using products versus how they were designed?
► Description
► In-depth, in-person observation of tasks & activities at work or home. Observations
are recorded.
► Benefits
► Access to the full dimensions of the user experience (e.g. information flow,
physical environment, social interactions, interruptions, etc)
► Limitations
► Time-consuming research; travel involved, Smaller sample size does not provide
statistical significance, Data analysis can be time consuming
► Data
► Patterns of observed behavior and verbatims based on participant response,
transcripts and video recordings
► Tools
► LiveScribe (for combining audio recording with note-taking)
Cost / respondent: Low – Moderate – High
Statistical validity: None – Some – Extensive
47

49. 1 Methodology: Contextual Observation/Ethnography
48

50. 2 Methodology: Remote Ethnography
► Business problem
► How are people actually using in their environment in real-time?
► Description
► Participants self-record activities over days or weeks with pocket video cameras or
mobile devices, based on tasks provided by researcher.
► Benefits
► Allows participants to capture activities as they happen and where they happen
(away from computer), without the presence of observers. Useful for longitudinal
research & geographically spread participants.
► Limitations
► Dependence on participant ability to articulate and record activities, Relatively high
data analysis to small sample size ratio
► Data
► Patterns based on participant response, transcripts and video recordings
► Tools
► Qualvu.com
Cost / respondent: Low – Moderate – High
Statistical validity: None – Some – Extensive
49

51. 4 Methodology: Focus Groups
► Business problem
► What are perceptions and ideas around products/concepts?
► Description
► Moderated discussion group to gain concept/product feedback and inputs; can
include screens, physical models and other artifacts
► Benefits
► Efficient method for understanding end-user preferences and for getting early
feedback on concepts , particularly for physical or complex products that benefit
from hands-on exposure and explanation
► Limitations
► Lacks realistic context of use; Influence of participants on each other
► Data
► Combination of qualitative observations (like ethnographic research) with
quantitative data (e.g. ratings, surveys)
► Tools
► See qualitative data analysis
Cost / respondent: Low – Moderate – High
Statistical validity: None – Some – Extensive
50

52. How To Think About UX Performance (Engagement Pyramid)
No Engagement
UX Testing
User cannot find critical features and cannot determine how to use
them if they find them.
Low Engagement
UX Testing
User can find critical features and understands how to use them but
has no interest in doing so.
Moderate Engagement
UX Testing
User can find critical features and understands how to use them
and has some interest in doing so.
Deep Engagement
UX Testing
User can find critical features and understands how to use them
and cannot stop doing so.
51

53. 10 Methodology: Eye-Tracking
Business Problem
Do users see critical content and in what order?
Description
Respondents view content on a specialized workstation or glasses.
Benefits
Very accurate tracking of eye fixations and pathways.
Limitations
Relatively high cost, analysis is complex, data can be deceiving.
Data
Live eye fixations, heat maps…etc.
Tools of Choice
Tobii - SMI
Cost / respondent: Low – Moderate – High
Statistical validity: None – Some – Extensive
52

54. 6 Methodology: Large-Sample Online Behavior Tracking
► Business problem
► Major redesign of a large complex site that is business-critical?
► Description
► 200-10,000+ respondents do tasks using online tracking / survey tools
► Benefits:
► Large sample size, low cost per respondent, extensive data possible
► Limitations
► No direct observation of users, survey design complex…other issues
► Data
► You name it (data exports to professional analysis tools).
► Tools of Choice
► Keynote WebEffective, UserZoom,
Cost / respondent: Low – Moderate – High
Statistical validity: None – Some – Extensive
53

55. Recommended Reading: Albert and Tullis
54

56. 7 Methodology: Lab-based UX Testing
► Business problem
► Are there show-stopper (CI) usability problems with your user experience?
► Description
► 12-24 Respondents undertake structured tasks in controlled setting (Lab)
► Benefits
► Relatively fast, moderate cost, very graphic display of major issues
► Limitations
► Small sample, study design, recruiting good respondents
► Data
► Summary data in tabular and chart format PLUS video out-takes
► Tools
► Leased testing room, recruiting service and Morae (Industry Standard)
Cost / respondent: Low – Moderate – High
Statistical validity: None – Some – Extensive
55

57. Recommended Reading: Tullis and Albert
56

58. 8 Methodology: Professional Heuristics
► Business problem
► Rapid feedback on UX design based on best practices or opinions
► Definition
► “Heuristic is a simple procedure that helps find adequate, though often imperfect,
answers to difficult questions (same root as: eureka)”
► Benefits
► Fast, low cost, can be very effective in some applications
► Limitations
► No actual user data, analysis only as good as expert doing audit
► Data
► Ranging from verbal direction to highly detailed recommendations
► Tools of Choice
► Written or verbal descriptions and custom tools by each experts.
Cost / respondent: NA
Statistical validity: None – Some – Extensive
57

59. 9 Methodology: Automated Online Card Sorting
► Business problem
► User’s cannot understand where content they want is located?
► Description
► Online card sorting based on terms you provide (or users create)
► Benefits
► Large sample size, low cost, easy to field
► Limitations
► Use of sorting tools confuse users, data hard to understand
► Data
► Standard cluster analysis charts and more
► Tools of Choice
► WebSort…and others
Cost / respondent: Low – Moderate – High
Statistical validity: None – Some – Extensive
58

60. 10 Methodology: fMRI (Brain Imaging)
► Business Problem?
► What areas of the brain are being activated by UX design
► Description
► Respondents given visual stimulus while in FMRI scanner
► Benefits
► Maps design variables to core functions of the human brain
► Limitations
► Expensive and data can be highly misleading
► Data
► Brain scans
► Tools
► Major medical centers and research services (some consultants)
Cost / respondent: Low – Moderate – High
Statistical validity: None – Some – Extensive
59

68. Methodology:
Field Study

79. Methodology:
Card Sorting

80. “It is important to use Card Sorting for the right reasons
and the right time in the project and to analyze the
results in combination with other inputs.”
- DONNA SPENCER 2009

81. STEPS IN A CARD SORT
1. Decide what you want to learn
2. Select the type of Card Sort (open vs closed)
3. Choose Suitable Content
4. Choose and invite participants
5. Conduct the sort (online or in-person)
6. Analyze Results
7. Integrate results

82. BEFORE YOU GET STARTED!
Inventory + Audit the Content

83. WHAT ARE YOU WANTING TO LEARN?
• New Intranet vs Existing?
• Section of Intranet?
• Whole organization vs single department?
• For a project? For a team?

84. OPEN VS CLOSED
CRM
Year in Product
Organizatio
Company
Review Targets
CRM
Vacation Christmas Projects
n Chart
OPEN Policy
Vacation
request Pay Days
Party
Walkathon
News
Meeting Project
Review
Results
SORT form Vacation
Policy
Human
Vacation
Resources
Events
Walkathon
Results
request Pay Days Christmas
CRM form Party
Product Organizatio
CRM
Christmas Targets in
Year n Chart
Vacation Project
Party Review
Policy Review
Meeting
Pay Days
CRM Vacation
Organization request
Chart CRM form
Year in
Project
Review
Review
Product Meeting
Walkathon
Targets
Results
Christmas Year in CRM
Party Review Project
Company
Company Walkathon Meeting Review
Product
Projects
Projects News
News Results Targets
Vacation
CLOSED Human
Resources
Departments
Human
Policy
Vacation
Resources
request Pay Days
Departments
CRM
Organizatio
form n Chart
SORT

85. SELECTING CONTENT

86. SELECTING CONTENT
Do’s Don’ts
•30 – 100 Cards • More than 100 cards
•Select content that can be • Mix functionality and
grouped content
•Select terms and concepts • Include both detailed and
that mean something to broad content
users

87. ANALYSIS

88. LOOK AT
• What groups were created
• Where the cards were placed
• What terms were used for labels
• Organization scheme used
• Whether people created accurate or inaccurate groups

89. INTEGRATE RESULTS: CREATE YOUR IA
Centers of
Our Company Projects Departments Excellence Employee Community &
Resources Groups
Project Management
Executive Blog Project Name 1 Executive Professionals Vacation & Holidays Events
Operations Engineering
New York Project Name 2 Expenses Charitable Campaigns
Operations Support Terminal Technologies
Project Name 3 Vancouver Carpool
Vancouver
Vessel Planning NAVIS
Project Name 4 Travel
Mission and Values
Yard Planning Lawson
Health & Safety
Rail Planning IT
Wellness
Finance & Yard Planning
Administration Benefits
Human Resources Facilities
Corporate
Communications Payroll
IT Communication Tools

90. Traditional Card Sort

91. Online Card Sorting

92. Card Sorting is as common as Lab based Usability
Testing
Source: 2011 UxPA Salary Survey

93. Terms & Concepts
• Open Sort: Users sort items into groups and give the
groups a name.
Closed Sort: Users sort items into previously defined
category names.
• Reverse Card Sort (Tree Test) : Users are asked to locate
items in a hierarchy (no design)
• Most Users Start Browsing vs Searching: Across 9
websites and 25 tasks we found on average 86% start
browsing
http://www.measuringusability.com/blog/card-sorting.php
http://www.measuringusability.com/blog/search-browse.php

94. Methodology:
Heuristic
Evaluation

95. Heuristic Eval is popular pick
UPA survey results for HE/expert review
% of respondents Survey year
77% 2007
74% 2009
75% 2011
Slide 94

96. Why so popular? Fact or myth?
Fast Cheap Easy
Effective Convenient
Slide 95

97. Care to comment?
Slide 96

98. HE output
• A list of usability problems
• Tied to a heuristic or rule of practice
• A ranking of findings by severity
• Recommendations for fixing problems
• Oh, and the positive findings, too
Slide 97

99. Nielsen’s 10 heuristics
1. Visibility of system status
2. Match between system and real world
3. User control and freedom
4. Consistency and standards
5. Error prevention
6. Recognition rather than recall
7. Flexibility and efficiency of use
8. Aesthetic and minimalist design
9. Help users recognize, diagnose, and recover from errors
10. Help and documentation
J. Nielsen and R. Mack, eds. Usability Inspection Methods, 1994
Slide 98

100. What do you do?
• Do you do it (or teach it)?
• How do you do it?
• Why do you do it?
• Do you do it alone or with others?
• How do you report it?
• How do you charge for it?
Slide 99

101. HE sample
findings
page
usability.spsu.edu UPA 201
100
1

102. Hyperspace, Shock, and Cardiac Arrest all require more clearly defined goals and objectives.
H = Hyperspace; C = Cardiac Arrest; S = Shock
Finding Description Recommendation H C S Severity Rating
Objectives/goals for the Reason content is being Develop a consistent structure that    3
modules presented defines what’s noted in the bulleted
Conciseness of presentation points, above.
Definitions required to work Avoid generic statements that don’t
with the module/content focus users on what they will be
Objectives/goals for Reason content is being Develop a consistent structure that    3
Evaluation criteria and
the modules presented accomplishing.
defines what’s noted in the
methods Conciseness of presentation Advise that there is an assessment
bulleted points, above.
Definitions requiredand used for evaluation and indicate if it’s
Direct tie between content to work Avoid generic statements that
with the module/content don’t focus users on what they will
assessment measurecriteria and
Evaluation at the end or interspersed in the
be accomplishing.
Sequence ofmethods
presentation module that there is an assessment
Advise
Direct tie between content Connect ideas in the indicateand
follows logically from used for evaluation and goals if
and assessment measure it’s at the end or interspersed in
introductionSequence of presentation objectives with outcomes in the
the module
Quizzes challenge logically from
follows users assessment in the goals and
Connect ideas
introduction Follow the with outcomes in the
objectives order of presentation
Quizzes challenge users assessment
defined the the beginning
Follow at order of presentation
Develop interesting and challenging
defined at the beginning
questionsinteresting and
Develop
challenging questions
Re-frame goals/objectives at the end
Re-frame goals/objectives at the
of theof the module
end module
101

103. usability.spsu.edu UPA 201
102
1

104. Steve Krug’s approach
• All sites have usability problems
• All organizations have limited resources
• You’ll always find more problems than you have
resources to fix
• It’s easy to get distracted by less serious problems
that are easier to solve . . .
• Which means that the worst ones often persist
• Therefore, you have to be intensely focused on fixing
the most serious problems first
Rocket Surgery Made Easy, New Riders, 2010
Slide 103

105. Krug’s maxims
• Focus ruthlessly on a small number of the most
important problems.
• When fixing problems, always do the least you can
do.
Slide 104

106. Methodology:
Usability Test

107. Learnability
Satisfaction Efficiency
Usability
Errors Memorability
Nielsen, Jakob. “Usabiilty 101: Introduction to Usability.” useit.com. Web. 26 Mar. 2012.

108. Highly Recommended

109. Usability Tests
Start new test
Make changes Identify 3-5
to site tasks to test
Observe test
Identify the 2-3
participants
easiest things to
performing
fix
tasks

110. Why Test at All?
User- Grounds
centered design
design debates
Surprising
Methodical
results

111. Identify Tasks for the Test
Known Most
problem common
areas activities
New
Popular
pages or
pages
services

112. Recommended Gear
Screen
Computer recording
(or paper prototype) software
(or observer)
Microphone
(or observer)

113. Tasks
You want to renew the loan period
on a book you’ve checked out.
You need to find an article on the
role of genes in schizophrenia.
You want to see what hours the
library is open next Monday.

114. Screen Recording Software
Price
Features

115. Staff of One
Recruits test
participants Runs the test
Records the Preps test
environment
test before & after
(screen recording
software & mic) test

116. “Guerilla testing!”
Photo and caption credit::
Amanda Etches
http://instagr.am/p/IuHhRri-wS/

117. Staff of Two
#1 #2
Recruits test Observes the
participants test
Preps test
environment
Runs the test
before & after
each test

118. Draft a Test Protocol
Goals Recruitment Tasks
Script Staff Roles Equipment

119. Before Each Test Participant Arrives
Clear
cache in
Check browser
screen
Check mic recording
software

120. Getting Each Test Participant Started
Don’t show
website until
Chat about first task is
websites introduced
Explain why they like
you’ll be
reading
from a script

121. Working Through Tasks
Don’t help
the
Encourage participant
participant
Stick to to speak
the script aloud

122. As Each Participant Finishes
End and
save the
Give screen
participant recording
Thank any payment
participant or reward
profusely

123. After Test Is Done
• ASAP (same day)
Make
Notes • Identify common problems
• Keep it simple
Write
Report • Focus on fixing just 2-3 things
• Might not be same tasks
Test Again • Lather, rinse, repeat

124. Focus on Fixing Easy Things
Small
changes
A website
that
keeps up
with users
Iterative
development
and testing

125. For More Info
Francoeur, Stephen. “Usability Testing Our New Website.” Beating the Bounds. 16 Jan.
2012. Web. 27 Mar. 2012. link
Hassenzahl, Marc and Noam Tractinsky. “User Experience - a Research Agenda.”
Behaviour & Information Technology 25.2 (2006): 91–97. link
Krug, Steve. Rocket Surgery Made Easy: The Do-It-Yourself Guide to Finding and Fixing
Usability Problems. Berkeley, Calif: New Riders, 2010. Print.
Nielsen, Jakob. “Usabiilty 101: Introduction to Usability.” useit.com. Web. 26 Mar. 2012. link
Reidsma, Matthew. “How We Do Usability Testing.” Matthew Reidsma. 15 Nov. 2011. Web.
27 Mar. 2012. link
Reidsma, Matthew. “Why We Do Usability Testing.” Matthew Reidsma. 25 Oct. 2011. Web.
27 Mar. 2012. link
University of Texas at Austin. “Usability Testing.” Web Publishing. University of Texas at
Austin. 27 Mar. 2012. link

126. Methodology:
Remote Test

127. New technologies and
techniques allow for
Remote:
– Moderated testing
– Unmoderated testing
– Observation
Irrelevance of
Place

128. Remote Moderated Testing
Products like GotoMeeting allow connections
to the test (or observation) computer to the
Internet. VoIP can carry voice cheaply.
Translator
Observers
Participant
Moderator
LiveMeeting
WebEx
GoToMeeting
For screen VoIP Audio
Skype
GoogleTalk

129. Remote Unmoderated Testing
Find the umbrella.
• Hundreds of users agree to participate in a study
• In their natural context
• From geographically spread locations
• Users try to complete tasks + answer questions
• No human moderation needed
• Browser bar connects users with secure servers

130. Remote Unmoderated Testing
‘Task-based’ Surveys Online Card Sorting
> Open or closed
> Online/remote Usability Studies
(unmoderated) > Stand alone or
> Benchmarking (competitive /comparison) > Integrated with task-based
studies & surveys
> UX Dash`boards (measure ROI)
Robust Set of Services
Online Surveys User Recruiting Tool
> Ad hoc research > Intercept real visitors (tab or layer)
> Voice of Customer studies > Create your own private panel
> Integrated with Web Analytics data > Use a panel provider*

131. When to do…
Remote Testing In Person
• Web/software UI • Physical artifact
• Need rich qualitative
• More quantitative based feedback
• Large, distributed • Need to have the human
connection
sample or low incidence • Ensure high-level of
• Low(er) budget consistency
• Uncertain of quality or
• High penetration of environment
Internet access

132. Introduction to remote usability testing

133. Defining Remote Testing Tools
What do we mean by “remote testing tools”?
• A remote testing tool is any technology that allows a researcher to collect data
from users about their experience in using an interface, without direct, face-to-
face contact.
There are two ways of organizing remote testing tools
• Moderated vs. Unmoderated
• Qualitative vs. Quantitative-based
Moderated Unmoderated
Qualitative-based Qualitative-based
Quantitative-based Quantitative-based
133

134. Unmoderated Testing Tools
Full-service
Self-service
Card
Quantitative- Sorting/IA
based
Unmoderated Surveys
tools
Click/Mouse
Video tools
Qualitative-
based Reporting
Expert
Reviews
134

135. Our UX Toolkit
135

136. Why Should You Care?
• Saves time
o Lab study takes 2-4 weeks from start to finish, unmoderated typically takes hours to
a few days*
• Saves money
o Participants compensation typically a lot less ($10 vs. $100)
o Tools are becoming very inexpensive
• Reliable metrics
o Only (reasonable) way to collect UX data from large sample sizes
• Geography is not a limitation
o Collect feedback from customers all over the world
• Greater Customer insight
o Richest dataset about the customer experience
136

137. Why is this important NOW?
• Questions from senior management are becoming more complex and
time sensitive
• Traditional usability is no longer enough
• Push to measure the UX
• Convergence with market research and web analytics to paint a more
complete picture of the UX
• Budgets/timeline constraints
137

138. Which one goes first?
Lab first, then Unmoderated Unmoderated first, then Lab
Identify/fix “low hanging fruit”, then Identify the most significant issues
focus on remaining tasks with large online through metrics, then use lab
sample size study to gather deeper qualitative
understanding of those issues
Generate new concepts, ideas, Collect video clips or more quotes of
questions through lab testing, then users to help bring metrics to life
test/validate online
Validate attitudes/preferences Gather all the metrics to validate
observed in lab testing design – if it tests well, then no need to
bring users into the lab
138

139. Quantitative-based tools

140. Unmoderated Testing Tools - Quantitative
Full-service
Self-service
Card
Quantitative- Sorting/IA
based
Unmoderated Surveys
tools
Click/Mouse
Video tools
Qualitative-
based Reporting
Expert
Reviews
140

141. Overview
Common Research Questions:
• What are the usability issues, and how big?
• Which design is better, and by how much?
• How do customer segments differ?
• What are user design preferences?
• Is the new design better than the old design?
• Where are users most likely to abandon a transaction?
Types of Studies: Typical Metrics:
• Comprehensive evaluation • Task success
• UX benchmark • Task time
• Competitive evaluation • Self-report ratings such as ease of use,
• Live site vs. prototype comparison confidence, satisfaction
• Feature/function test • SUS
• Discovery • Click paths
• Abandonment
141

142. Strengths / Limitations / Myths
Strengths:
• Comparing products
• Measuring user experience
• Finding the right participants
• Focusing on design improvements
• Insight into the user’s real experience
Limitations:
• Not well suited to rapid, iterative Myths:
design • Only test with websites
• Need a deep understanding of issues • Very expensive
• Studies that require long sessions • Only gather quantitative data
• Lose control over prototypes • A lot of noise in the data
• Internet access • Same as any market research study
• Comparing products
142

143. Full Service Tools – Online Usability Testing
143

144. Self Service Tools – Online Usability Testing
144

145. Card Sorting / IA Tools
145

146. Online Surveys
146

147. Click and Mouse Tools
147

148. Qualitative-based tools

149. Unmoderated Testing Tools - Qualitative
Full-service
Self-service
Card
Quantitative- Sorting/IA
based
Unmoderated Surveys
tools
Click/Mouse
Video tools
Qualitative-
based Reporting
Expert
Reviews
149

150. Overview
Common Research Questions:
• What are the big pain usability pain points, and wins?
• How do users think, and feel about the design?
• Are we on the right (design) track?
• What is the overall experience like?
• Why might users abandon a transaction?
Types of Studies:
• Low fidelity prototype Common Metrics:
• High fidelity prototype • Frequency of issues
• Competitive evaluation • Verbatim comments
• Comprehensive evaluation • Task success (via video)
• Feature/function test • Task time (via video)
• Discovery
150

151. Video Tools
151

152. Report Tools
152

153. Expert Review
153

154. Additional Resources
Online:
www.measuringUX.com
www.measuringusability.com
http://usability.bentley.edu
http://www.usefulusability.com/24-usability-testing-tools/
http://remoteresear.ch/tools/
154

155. Methodology:
Focus Group

156. During the interview
DO
Take notes
Smile
Ask open-ended questions Don’t
Get their story Talk about your product
Shut up and listen Ask about future behavior
Sell
Ask leading questions
Talk much

157. "It's really hard to design products by
focus groups. A lot of times, people don't
know what they want until you show it
to them.”
- Steve Jobs
What are
Focus Groups?

158. Let’s dispense with this little turd
blossom right up front: Henry Ford
never said, “If I'd asked customers
what they wanted, they would have
said "a faster horse,”
– it’s simply an myth

159. What are focus groups?
The concept of focus groups was developed in 1930’s by
psychoanalyst Ernest Dichter as a social research method
Focus groups are structured interviews that quickly and
inexpensively reveal a target audience’s desires,
experiences, attitudes, and priorities
Focus groups can be a useful technique when a company
needs a lot of insight from potential or existing customers
in a short amount of time

160. When to do Focus Groups?
In product design, focus groups are used early in the design
cycle when the team is generating ideas and seeking to
understand the needs of the target audience.
Early in the design cycle, focus groups can help the
company understand:
User’s fundamental issues and perceptions of the product
What users believe are the important features of the product
What types of problems users experience with the product
Where do users feel the product fails to meet their expectations

161. Focus Groups cannot be used to
unequivocally prove or disprove a
hypothesis about the user experience of
a product.

162. When NOT to do Focus Groups?
When the objective is to acquire usability information
A group of people can’t provide specific information regarding product
features without structured usability testing sessions
When seeking to understand the perspectives of the bigger
population
Quantitative data that is generalizable to the bigger population requires
surveys or other methodological approaches that require a large sample of
participants
There is no guarantee that proportion of responses in the group matches
that of larger population of users
Although focus groups are an excellent way to gather motivations
and insights from the users, it cannot be used to unequivocally
prove or disprove a hypothesis about the user experience

163. Types of
Focus Groups
Exploratory
Feature Prioritization
Competitive Analysis
Trend Explanations

164. Exploratory Focus Groups
Typically conducted in the beginning of a design cycle
Uncover users’ general attitudes on a given topic, allowing
product designs to
See how their users will understand their product
What words users will use to speak about it
What criteria they will use to judge it

165. Feature Prioritization Focus Groups
Generally held at the beginning of the product design cycle when
the outlines of the product are clear.
These groups focus on features of the product that are most
attractive to the users, with an emphasis on why they are
appealing.
Underlying assumption of this type of focus group is that that the
participants are interested in the product, with discussion
focusing on what kinds of things they would like the product to
do for them.

166. Competitive Analysis Focus Groups
Aims to uncover what attracts and repels users with respect to
competitor’s sites
What associations do users have with the competitor?
What aspects of the user experience they find valuable?
Where does the product satisfy users’s needs and where does it not suffice?
What emotions does the product evoke?
How do users identify with the product?
This type of focus group is often conducted anonymously

167. Trend Explanations Focus Group
Generally held in either a re-design part of the development
process, or in response to specific emotional or functional issues
in product development
Exploring the trends of users’ behaviors, needs, and expectations
within and across products.

168. How to conduct
Focus groups

169. How to conduct Focus Groups
Assemble your team
Make sure you have a good cross section of product, ux, marketing and
development.
Create a schedule
A good schedule provides sufficient time for recruiting, testing, analyzing
and integrating results
Define your users
Recruit participants who are your users and thus likely to provide the best
feedback – usually 6-8
Define the scope of your research
What is the complexity of your questions?
What is the depth at which you wish to explore the answers?
This will determine the number of people and the number of groups that
need to be conducted

170. How to run your Focus Groups
Choose a topics for discussion
On average, 3-5 topics per 90 minute focus group
Create a discussion guide
Consider the “core” questions you and your product team are
trying to answer and prioritize them
Establish roles
Who will moderate? Who will take notes? Who will lead the discussion
afterwards

171. Asking Good Questions
Questions should be:
 Carefully ordered, thus positioning participants within a
certain frame of mind, containing an intuitive flow
 Non-directed: should not imply an answer. Example: “How
difficult do you find this feature?”
 Open-ended: general enough not to constrain answers to a
specific responses (limit yes-no questions)
 Focused: focused on specific topics you are investigating
 Personal: people love to generalize their experience to the
bigger public; create questions that concentrate on person’s
current behavior and opinions without many opportunities to
project their experiences onto the general public
 Unambiguous: clear and concise, with few shades of meaning.

172. Example Discussion Guide
Warm up and introduction (approx. 15 min)
Introduction of moderation
Ice breakers for participants
Outline of the process
Main topic discussion
Moderate group discussion that focuses on specific questions you and your
company have regarding a product.
Wrap-Up
Final thoughts and reflections

173. Warm Up & Introduction Tips
 Telling participants that they were chosen to be part of the
group allows them to feel more comfortable with one another
 Informing the group of the purpose of the session focuses their
attention to the desired end goal
 Clearly set out expectations and “ground rules” for discussion
(no blocking, interruption, flow of discussion)
 Acknowledge any potentially anxiety provoking features of
the environment (camera, mirrored wall) to help people feel
more comfortable
• Inform participants of their rights to participation
Freedom to leave at any point, confidentiality of their thoughts

174. Main Discussion Tips
Probes and follow-up questions are extremely useful
They dig deeper into any given topic
They clarify what people mean when they state their opinions
My definition of “useful”, “clear”, and/or “good” may not mirror other
people’s definitions of there terms.
Probes help create a common definition of terms, and alleviate potential
misunderstandings between the researcher and participant.

175. Context Is King
Comfortable environment is key to a lively discussion
Limited interruptions
After the session begins, no new person should join the session so that the
dynamic isn’t altered by another person’s presence
Food is encouraged
Eating is an informal activity that often breaks tension in any group
No noisy snacks that disturb the conversation
Seating order
Have a 10-15 minute social time before the focus group starts so the
moderator can identify introverts, extroverts, and alpha-jerks
Videotaping advised
Human interaction is incredible complex. Since the moderator is part of
the group dynamic, it is helpful to videotape the sessions in order to
capture gestures, and other subtle interactions

176. John McLaughlin is a perfect
example of a bad moderator and
a douchebag: highly
opinionated, clearly biased, with
a tendency for dominating the
discussion
What about
the moderator?

177. What about the moderator?
Group moderation is a skill
Basic skills any moderator should embody are:
Respect for the participants
Ability to listen closely to other’s perspectives
Ability to think fast on multiple levels simultaneously.
The moderator must be able to predict the direction of the
conversation and drive it toward a desired direction, without the
participants realizing that they are being moderated.
This can be accomplished via moderator’s subtle cues, tone, and/or body
language

178. Effective Moderation
Control
Moderator should always be in control of leading discussion towards
answers to questions, and deterring tangents
Good time management
The flow should be monitored so that introduction of topics is at
appropriate times, transitions are intuitive and natural.
Participant-focused
A moderator should mediate the discussion, rather than expressing
opinions
Respect
All participants should feel comfortable and have a voice, alpha-jerks
managed
Preparation
Moderator should have sufficient knowledge of product space

179. Effective Moderation Tips
 Spend time with participants beforehand to get a sense of who is
quiet and might need more attention
 Stick to the guide but be flexible enough to stray away from the
script when necessary
 Engage ALL participants in the discussion
 Avoid introducing new terminology and concepts
 Be mindful of body language
 Clarify any comments & restate ideas and opinions to ensure
everyone is on the same page
 Probe for alternative opinions on any given topic
 Don’t dominate the discussion, allow the group to lead
 Provide the group with time to think & give a break when necessary
 Use humor when appropriate and keep the energy level high

180. Common
obstacles An electronics company was testing a new
boombox they hoped to start selling. Their
research included focus groups where they
showed the two colour options: yellow and
black. The participants were in agreement
that yellow was the best colour because it
is vibrant and energetic. At the end of the
focus group they were each allowed to take
a boombox home and could choose yellow
or black.
They all chose black.
– Steve Mulder, “The User is Always Right”

181. Common Obstacles
The moderator is not an objective observer
Moderator affects the group dynamics and discussion
Focus groups reveal the way people think and not the way they
actually behave
Opinions from focus groups may be limited to the participants
in the sessions
The sample may be biased for more reasons than just small size and
therefore cannot be adequately extrapolated to represent the bigger
population
Reticent individuals are often silenced by outspoken ones
Data may be biased toward those who speak up
“Vividness effect”
People often provide examples of situations that are most emotionally
vivid to them.

182. Common Obstacles
Overly talkative participants
When people are clearly talking without a purpose, ask them kindly to wrap up and
move on: Moderate the extroverts, probe the introverts.
Group dominance (The Alpha-Jerk Effect)
A single dominant/bullying participant can ruin the focus group
Unqualified participants
At times people misunderstand what the participation criteria are, or misrepresent
their experience
Tangents
They can be useful for discussion of values and ideas, but should be wrapped up
quickly and redirected to main discussion point
Hostility & Offensive Ideas
Vehement disagreement or offensive ideas can lead people in the group to feel
uncomfortable. The moderator should redirect conversation to the focusing on the
ideas behind any given perspective: Go Meta!

183. Analyzing
the data
Researchers must continually
be careful to avoid the trap of
selective perception
- Richard A. Krueger

184. What data to collect?
Focus groups produce a ton of potentially useful information
which can be extracted by means of:
Transcripts
Quotations
Observer opinions
Models
Videotapes
What information should be prioritized depends upon how
what form of data answers your question, and how quickly you
need to synthesize the results.

185. Analysis Steps
Capture the initial hypothesis
During the debriefing (occurring with little time lapse from focus group
to retain memory), discuss with other observers thoughts regarding the
groups' opinions and feedback
Transcribe and code
Video interaction should be transcribed and themes/trends of opinions
should be extracted via coding
Coding is a method of extrapolating ideas from the transcripts
and categorizing the responses (thus generating quantitative
data).
Your codes (general categories) should be short, concise, descriptive in
nature, and accurately depicting a users’ single idea.

186. Coding Framework
Top-down
A hypothesis of what types of themes the participants’ transcripts will
generate already exists and is confirmed using the data
A pre-existing model is applied to the data.
Bottom-up
The data is explored without pre-existing framework in mind, generating
themes based on the responses in the transcript.
The model is generated using the data.
This methodological approach to extracting themes from focus group
interviews is recommended since it is less biased and more true to the
data.

187. Extracting Trends in Data
Mental models
Mental representations of how your users understand the way the world
or a product works
Values
What do people like and dislike and what criteria do they use to establish
their opinions
Stories
Stories are a powerful way that people capture their unique, subjective
experiences, and provide details about their assumptions, order of doing
things and ways of solving problems
Product pitfalls
Brainstorms during focus groups can produce a list of problems that
users experience using the product

188. Getting the most out of the data
Questions to read for
 What are reasons behind people’s opinions?
 What terminology do people use and do products speak
their users’ language?
 Where do people contradict themselves?
 When do people change their minds and how does that
reveal their actual values and perspectives on a given
product?
 What do people consider to be important and is the
product that is popular actually important to them?

189. Methodology:
Benchmarking

197. Methodology:
Contextual Inquiry

198. What is
Contextual Inquiry?
198

199. What is contextual inquiry?
Contextual inquiry is a field data-gathering technique
that studies a few carefully selected individuals in
depth to arrive at a fuller understanding of the work
practice across all customers.
Through inquiry and interpretation, it reveals
commonalities across a product’s customer base.
~ Beyer & Holtzblatt

200. Contextual Inquiry:
When to do it
Every ideation and design cycle should start with a
contextual inquiry into the full experience of a customer and
his/her.
Contextual inquiry clarifies and focuses the problems a
customer is experiencing by discovering the
• Precise situation in which the problems occur.
• What the problem entails.
• How customers go about solving them.

201. Contextual Inquiry:
How to do it
What is your focus?
Who is your audience?
Recruit & schedule participants
Learn what your users do
Develop scenarios
Conduct the inquiry
Interpret the results
Evangelize the findings
Rinse, repeat (at least monthly)

202. Research
Process
202

203. Inquiry Process: Main Observation
Observe the participants
What are they doing?
What tools are they using?
How are they using the tools?
Have they developed any work-arounds?
Occasionally ask for the participant to describe what
they are doing, providing any necessary explanations,
clarifications, walk-through of actions.
Take copious notes. Video record, if possible.

204. Inquiry Process: Follow-Up & Wrap-Up
Follow-Up
After the main observations, ask the participant any follow-up
questions you may have regarding your observation which his/her
memory is still fresh.
Wrap-Up
Ask the participant about his/her experience and
perspective on the inquiry process
•Was anything anxiety provoking?
•Was there anything you would like to have been done differently?

205. Inquiry Results: What to collect
What tools do participants use?
Formal tools? Informal tools? What brands?
Behavior sequences
Order of actions is important to understanding how the participants think about
the task
Methods of organization
How do the participants organize the information they use? (out of necessity,
convenience, importance?)
What kinds of interactions do participants have?
What are the important parties in the transfer of knowledge? Are they people?
Are they processes? Is the information shared? What is the nature of the
interaction?
Collect Artifacts
Artifacts are the non-digital tools people use to help them accomplish the tasks
they’re trying to do. (ex: if you are interested in learning how people schedule
their events, photograph their daily calendar)

206. Analyzing
the results
“The output from customer research is not a
neat hierarchy; rather, it is narratives of
successes and breakdowns, examples of use
that entail context, and messy use artifacts”
Dave Hendry
206

207. Research Analysis
What are people’s values?
People are driven by their social and cultural contexts as much as their rational
decision making processes.
What are the mental models people build?
When the operation of a process isn’t apparent, people create their own models of
it
What are the tools people use?
It is important to know what tools people use since you are building new tools to
replace the current ones.
What terminology do people use to describe what they do?
Words reveal aspects of people’s mental models and thought processes
What methods do people use?
Flow is work is crucial to understanding what people’s needs are and where
existing tools are failing them.
What are people’s goals?
Understanding why people perform certain actions reveals an underlying
structure of their work that they may not be aware of themselves.

208. Affinity
Diagrams
“People from different teams engaged in affinity
diagramming is as valuable as tequila shots and
karaoke. Everyone develops a shared
understanding of customer needs, without the
hangover or walk of shame”
208

209. Research Analysis: Affinity Diagrams
Creates a hierarchy of all observations, clustering them
into themes.
From the video observations, 50-100 singular
observations are written on post-its
(observations ranging from tools, sequences, interactions,
work-arounds, mental models, etc)
With entire team, notes are categorized by relations into
themes and trends.

210. Methodology:
Eye Tracking

211. What is eye tracking?
Bill Albert
December 13, 2011
walbert@bentley.edu
781.891.2500 | www.bentley.edu/usability

212. Eye tracking records what people look at
212

213. See the user’s gaze - Live
You can follow what the
user pays attention to in
real-time.
The participant‟s gaze is
marked by red dots
and red lines.
A camera displays the
participant, so you can
see their facial
expressions and body
language.
213

214. Eye tracking results: Heatmaps
Heatmaps show what participants
focus on.
In this example, „hot spots‟ are the
picture of the shoes, the central entry
field and the two right-hand tiles
underneath.
The data of all participants is
averaged in this map.
214

215. Eye tracking results: Gazeplot
Gaze plots show the „visual path‟ of
individual participants. Each bubble
represents a fixation.
The bubble size denotes the length
or intensity of the fixation.
Additional results are available in
table format for more detailed
analysis.
More examples with
interpretations are coming up.
But before…
215

216. How does it work?
Bill Albert
December 13, 2011
walbert@bentley.edu
781.891.2500 | www.bentley.edu/usability

217. Pupils move a lot
Our pupils are constantly in
motion.
When the pupil is moving, it‟s
called a „saccade‟.
During a saccade, visual
perception is unlikely or even
impossible.
in here, somewhere…
217

218. How we ‘look’
Fixation
The pupil must focus on a
point in order to perceive
colour, faces, writing, etc.
That is called a „fixation‟.
Eye Tracking measures the speed of the
pupil and can thus detect when a fixation
Saccade
is happening!
218

219. What do these fixations tell us?
Fixations are linked to attention.
Moving your eyes means moving
attention.
A fixation does not mean that the
participant definitely perceived an
element.
But it is fair to say that elements that
draw visual attention have a higher
chance of being perceived (consciously
or subconsciously).
219

220. How can a monitor tell what I look at?
220
tobii

221. The red-eye effect
There‟s a layer in our eyes that
reflects infrared light.
This is where the red-eye effect in
photos comes from as photo flashes
use infrared light.
The eye tracking monitor makes
use of this effect!
221

222. What the eye tracker sees
The eye tracking monitor
uses infrared light to
make the pupils of the
person sitting in front
of it light up and so
become detectable.
This is what it looks like
for the monitor.
tobii
222

223. Monitors - No strings attached
It used to be like this:
Now it‟s all free and comfy. The monitor can
capture the gaze in a wide area, so the
participant can relax and move naturally:
kristenbell.org
223
tobii

224. Eye Tracking & User Research
Bill Albert
December 13, 2011
walbert@bentley.edu
781.891.2500 | www.bentley.edu/usability

225. You run user research to understand:
 Your user‟s actual behaviour.
 Your user‟s attitudes, feelings, and motivations.
 Your user‟s experience with your company/organisation (stories).
 What is and isn‟t working in your design/product.
225

226. You add eye tracking to get:
 A deeper understanding of user‟s actual behaviour.
 Insight into user‟s subconscious or instinctive behaviour.
 A better understanding of why your design does or doesn‟t work.
 Evidential (quantitative) data.
226

227. There’s 2 scenarios for eye tracking
The Check-Up Inform your design
How is my design, Use eye tracking data to
website or product support your design
performing? process
 How do users perceive my  Conceptual design: what basic
design/website/product? structure works best?
 Do users notice what I want  Wireframe stage: where shall I
them to notice? place my content or images?
 How is my design performing in  Detailed design: how does my
the context of typical usage visual design serve the
tasks? website‟s/product‟s design
purpose?
227

228. The Check-up
How is my Bill Albert
design, website or
December 13, 2011
product performing?
walbert@bentley.edu
781.891.2500 | www.bentley.edu/usability

229. Set-up of an eye tracking test
User tests are often run in 45 to 60
minute sessions with 6 to 15
participants:
1. Participants are give a number of
typical task to complete, using the
website, design or product you want
to test.
2. The user’s intuitive interaction is
observed, comments and reactions
are recorded.
3. The participant‟s impressions are
captured in an interview at the end
of the test.
229

230. What happens then?
The next step is to analyse the
eye tracking data and the user‟s
feedback. We focus on:
 what users saw,
 what users overlooked and
 what they thought and felt
about the website, design or
product.
The next slides are a couple of examples.
230

231. Examples: Testing website designs
What do you think
draws the user‟s
attention on this site?
The listed offers in
the centre or the
special offer
banners on the right?
231

232. The site suits browsers and focussed users
This participant This participant focusses
thoroughly reads the on the right-hand
listed offers. banners.
Whenever a destination He briefly gazes at the
sparks her interest, she listed offers, but shows
looks at the offer details, no reading behaviour
e.g. the price. there.
232

233. What drew most attention on this design?
233

234. The key visual and a box at the bottom
The key
visual got Surprising: This box got
lots of heaps of attention. It
attention. reads:
“If you are having trouble
getting through to us on
the phone, please click
here to email us, we‟ll get
The main back to you within 2
navigation and business days”.
its options got
almost no Participants got the
attention. impression that Telstra Clear
has trouble with their
customer service.
234
Note: Telstra Clear have since re-designed their homepage.

235. Inform your design
Use eye tracking data to support
Bill Albert
December 13, 2011
your design process
walbert@bentley.edu
781.891.2500 | www.bentley.edu/usability

236. Design process
There‟s lots of decisions to make in all stages of the development process:
236

237. Decision like these…
Where should the
‘Pay now’ button
be?
Will users notice
this if I put it
here?
237

238. … or these:
How does my
Does my design
design perform
draw enough
compared to
attention?
others?
visuality-group.co.uk
238

239. … and these:
Does
Design A
work
better?
… or
Design
B?
239

240. Design principles – revealed by eye
tracking
Bill Albert
December 13, 2011
walbert@bentley.edu
781.891.2500 | www.bentley.edu/usability

241. Face Effect
Humans are programmed to recognise
faces. Everywhere.
This effect can beAlbert
Bill seen in eye tracking.
December 13, 2011
Faces always draw attention!
walbert@bentley.edu
781.891.2500 | www.bentley.edu/usability

242. The Face effect – an example
Yep, there’s
attention on
certain… areas, … the face,
however, is the
strongest point
of focus!
242
bunnyfoot

243. Using the Face effect
The ‘Face effect’ can Version A Version B
be used to drive
perception.
Here‟s a great example
from humanfactors.com
2 versions of an ad
design were tested using
eye tracking.
The goal of the ad is of
course to draw attention
to the product name.
humanfactors.com
243

244. Using the Face effect
Eye tracking results for ad Version
A:
 We see a face effect: The model‟s face
draws a lot of attention.
 The slogan is the other hot spot of the
design. Participants will likely have read
it.
 The product and its name get some,
but not a lot of attention.
humanfactors.com 244

245. Using the Face effect
Eye tracking results for ad Version
B:
 Again, we see a strong face effect. BUT:
In this version, the models gaze is in line
with the product and its name.
 The product image and name get
considerably more attention!
 Additionally, even the product name at
the bottom is noticed by a number of
participants.
humanfactors.com 245

246. Ways to focus attention
Same effect: If the baby faces you, you‟ll look at the baby. But if the baby faces the ad
message, you pay attention to the message. You basically follow the baby‟s gaze.
usableworld.com.au
246

247. Banner Blindness
Did we learn to ignore them?
Bill Albert
December 13, 2011
walbert@bentley.edu
781.891.2500 | www.bentley.edu/usability

248. Central banners
Central banners are
used on a lot of
homepages.
They use prime real
estate on the homepage.
That means they must be
in the centre of attention,
right?
248

249. Banner blindness
… or are they?
In this test, participants were
given a task: Find the nearest
ATM.
Participants focused on the
main navigation and the
footer navigation– this is
where they found the „ATM
locator‟.
So, when visiting a site with a
task in mind – as you
normally do – the central
banner can be ignored!
249

250. Compare the visual paths: Task versus browse
When browsing, the central banner gets lots of attention. But how often do you visit a bank
website just to browse?
Participant was asked just to look at the homepage Participant was given a task („Find the nearest ATM‟)
250

251. Main focus: Navigation options
Eye tracking results show:
When looking for Task: „What concerts are happen in Auckland this month?‟ Task: „You want to send an email to customer service‟
something on a
website, the main
focus of attention are
the navigation options.
Maybe users have learned
that they‟re unlikely to
find what they‟re looking
for in a central banner
image.
251

252. When do users look at banners?
In this example, participants looked at the banner even though they were looking for
something specific. What‟s different?
Task: „You want to get in touch with customer service‟ Participant was asked just to look at the homepage
252

253. Banner Blindness: The trick is…
… don’t make your banners look like banners!
253

254. The bottom line:
User research + Eye tracking
=
a more complete understanding of
Bill Albert
your user’s13, 2011
December experience
walbert@bentley.edu
781.891.2500 | www.bentley.edu/usability

255. Methodology:
Diary Study

256. Customer Validation
Diary Studies help here

257. Diary Studies from 30,000 feet
What? How? Why?
Analyzing and gaining insights
Practical tips
Ideas on how to apply

258. A diary study involves participants reporting their
activities over a specific period of time, usually apart
from the researcher and in their normal daily lives.

259. Benefits
• Why users behave a certain way
• Right context & environment
• Remote
• Sample over a duration
• Bridge qualitative & quantitative
• Integrate with metrics & tools
• Flexibility

260. Users record thoughts,
comments, etc. over time Interview users
Gather feedback, data
Organise and analyse
http://www.flickr.com/photos/vanessabertozzi/877910821 (affinity maps, analytics)
http://www.flickr.com/photos/yourdon/3599753183/
http://www.flickr.com/photos/stevendepolo/3020452399/
http://www.flickr.com/photos/jevnin/390234217/

261. Participants keep a record of
“When” data “What" data
Date & time Activity / task
Duration Feelings / mood
Activity / task Environment / setting

262. No one right way to collect data
Structured
Yes/no
Select a category
Date & time
Multiple choice
Combine / mix & match
Unstructured
Open-ended
Opinions / thoughts / feelings
Notes / comments
http://www.flickr.com/photos/roboppy/9625780/
http://www.flickr.com/photos/vanessabertozzi/877910821

263. Organize before analyze
Affinity mapping (integrate with “To the whiteboard”* exercises)
•Extract notes from interview data
•Place similar notes together, repeat
•“tag” clusters according to topic, title, concept, etc.
•Extract learning and insights by illustrating the whole concept
•See “Rich Pictures” from Soft Systems Methodology
*See “Enterprise guide to customer development” by Brant Cooper & Patrick Vlaskovits

264. “Hygiene” aspects
At the beginning
•Introduction / get-to-know-you
•Demographics & psychographics, profiling
•Instructions / Setting expectations
At the end
•Follow-up
•Thanks / token gift
•Reflection

265. Pitfalls
• Belief bias
• Behavior adjustment
• Ramp-up time
• Failure to recall

266. Start small, be nimble
• Low-tech (email, SMS, twitter, paper/pen, phone
interviews)
• Simple, not complex, diary forms
• Let users just ‘dump’ facts, don’t force them to
think so much
• Catch up regularly & share often
• Tweak on the fly

267. Bring an observer or two
• Efficiency is shared experience
• Bring one or two key stakeholders
along to the interview sessions
(e.g. lead dev, mktg, sales, CEO,
etc.
• Share and contrast: Don’t be
precious about your own
viewpoints
http://www.flickr.com/photos/qkgirl/4837119913/

268. Map diary to session tracking
Validate diary data against your metrics.
Customers sometimes behave differently than
what they say. Make the data richer.

269. Integrate with business metrics
• Use diary studies to test
business assumptions
• Test against segments
• Useful for generative or
evaluative research
• Scale up or down
• Test with prototypes
• Etc.
Taken from “Enterprise guide to customer development” by Brant Cooper & Patrick Vlaskovits

270. ‘Hidden’ insights from qualitative data
• Useful for new ideas
• Generates more
questions
• Hidden facts about
customers
• Stuff you never knew to
look
• Store them for later

271. Take-aways
• Diary studies are flexible, tweak to your
requirements
• Use it to address gaps in your research data
• Integrate it with your existing measurement
machinery
• Refine and iterate your approach

272. More on diary studies
http://www.webcredible.co.uk/user-friendly-resources/web-usability/diary-study-guide.shtml
http://www.system-concepts.com/articles/usability-articles/2011/a-quick-start-guide-to-online-diary-studies.html
Analyzing using affinity maps
http://www.usabilitybok.org/methods/affinity-diagram?section=how-to
http://www.mindtools.com/pages/article/newTMC_86.htm
Gamestorming book by Dave Gray, Sunni Brown, James Macanufo

273. Methodology:
Cognitive
Walkthrough

279. References

280. 인용/참조 문헌
• Know Thy User: The Role of Research in Great Interactive Design (frog, Sep 2012)
• The Mobile Frontier (Rachel Hinman, Feb 2012)
• Introduction to AgileUX: Fundamentals of Customer Research (Will Evans, Jan 2012)
• Customer Research & Persona Development (Will Evans, Oct 2012)
• Introduction to UX Research: Conducting Focus Groups (Will Evans, Jan 2012)
• Midwest UX '12: Mapping the Experience (Chris Risdon, Jun 2012)
• Eye Tracking & User Research (Optimal Usability, Apr 2012)
• Taking it to the streets: Investigating mobile and web UX through field studies (Emma Rose, Jun 2012)
• NYTECH "Measuring Your User Experience Design“ (New York Technology Council, Mar 2012)
• How to Conduct UX Benchmarking (UserZoom, May 2012)
• Customer validation with Diary Studies (Boon Chew, Jan 2012)
• The Science of Great Site Navigation: Online Card Sorting + Tree Testing (UserZoom, Jul 2012)
• Introduction to Card Sorting (ThoughtFarmer, Sep 2012)
• Usability Testing Basics (Stephen Francoeur, Mar 2012)
• Storytelling: Rhetoric of heuristic evaluation (Southern Polytechnic State University, Mar 2012)
• Cognitive and pluralistic (Aarushi Mishra, Oct 2012)
• How to Quantitatively Measure Your User Experience (Richard Dalton, May 2012)
• Remote Testing Methods & Tools Webinar (UserZoom, Dec 2011)
• Beyond User Research (Louis Rosenfeld, Mar 2011)
• User Interview Techniques (Liz Danzico, May 2010)
• Design Research For Everyday Projects - UX London (leisa reichelt, Jun 2009)
UX 리서치 280 © 2012 InnoUX & Innodesign All rights reserved.

281. 경청해주셔서
고맙습니다!