90 minute training for experienced practitioners in best practices for analyzing and modeling qualitative user research, including KJ Analysis, personas, and scenarios. Tips and tricks and techniques included. Presented at the STC Summit 2010 on 3 May 2010.
8377087607, Door Step Call Girls In Kalkaji (Locanto) 24/7 Available
Â
Understanding Users Through Ethnography and Modeling - STC Summit 2010
1. Understanding Users Through Ethnography and Modeling
Jim Jarrett
Senior User Experience Architect and Manager
MedAssurantâą
STC Technical Communication Summit 2010
3 May 2010
5:00-6:15pm
Reunion C
2. Understanding Users Through Ethnography and Modeling
Great product design starts with a deep understanding of the work
that users do in the real world. Build your understanding through
observation, interviews, surveys, and artifact collection. Share your
understanding with KJ analysis, personas, and scenarios. Validate
and prioritize your understanding with follow-up surveys.
ï§ Introduction and Conference Context 5 minutes
ï§ Ethnography and Data Collection 20 minutes
ï§ Modeling and Validation 30 minutes
ï§ Tips, Tricks, Tools, and Resources 10 minutes
ï§ Q&A 10 minutes
3. Who is Jim Jarrett?
Senior User Experience Architect and Manager
MedAssurantâą
ï§ 15 years at Rockwell Automation
started as a multimedia developer in 1991
other companies: Chiron, Roadway, IDD
ï§ Dozens of on-site observations in domains of medical
informatics, industrial automation, and medical
diagnostics.
ï§ Largest single user research project:
30 sites on 3 continents in 7 countries
across 3 trips totaling 18 days on-site
ï§ Design for Six Sigma Blackbelt at Rockwell
$2 million+ in hard savings over 2 years
$24 million+ in revenue growth in same period
ï§ Multiple patents in UI for industrial automation
jim@jarrettinteractiondesign.com
www.JarrettInteractionDesign.com
LinkedIn.com/in/jarrettinteractiondesign
Twitter.com/JarrettUX
immersed in context
4. Ethnography and Modeling
Sketching
Expert Inspections
Usability Evaluation
Usability Institute
âUnderstanding Users Through Ethnography and Modelingâ
Monday 5:00-6:15pm Reunion C
âDiscovering Usability Defects Through Expert Inspectionsâ with Rich Gunther
Tuesday 8:00-9:15am Landmark B
âImproving Product Design Through Usability Evaluationâ with Scott Butler
Tuesday 9:45-11:00am Reunion F
âSketching User Experiences with the Design Studio Methodâ with Brian Sullivan
Wednesday 8:00-9:15am Reunion B
Inception
Elaboration
Construction
Transition
5. â⊠truth will sooner come out of error
than from confusion.â
Sir Francis Bacon
âBetter wrong than vague!â
âAn articulated guess beats an unspoken
assumption.â
Fred Brooks
6. Define Ethnography
ethno folk, people, nation
graphy writing, study of
âThe scientific description of nations or races
of men, with their customs, habits, and
points of difference.â
OED
âThe study and systematic recording of
human cultures.â
Merriam-Webster
ï§ Qualitative, language-based, empirical,
descriptive research.
ï§ Techniques include observation,
interviews, surveys, logs and journals,
gathering of artifacts and images.
ï§ Holistic and contextual.
âI prefer design by experts - by
people who know what they are
doing.â
Don Norman
â⊠pay attention to what users do,
not what they say⊠â
Jakob Nielsen
âA computer shall not harm your
work or, through inaction, allow
your work to come to harm.â
Jeff Raskin
âWhat do users do and how do
they talk about it?â
Jared Spool
7. Market Research vs. User Research
Market Research
ï§ Markets and segments
ï§ Best ways to influence
ï§ Target demographics
ï§ Buying behavior
ï§ Message definition
ï§ Focus groups, surveys,
and interviews
ï§ Competitive analysis
âWhy do people buy?â
User Research
ï§ Individuals and context
ï§ Best ways to work
ï§ User profiles
ï§ Goals and motives
ï§ Task definition
ï§ Observation, surveys,
and interviews
ï§ Domain analysis
âWhat do people do?â
8. Data Collection Process
1. Define the target of your research.
ï§ What questions are you trying to answer?
ï§ Who should you study?
2. Arrange visits.
ï§ Get permissions and release forms.
ï§ Schedule timing for your targets.
ï§ Coordinate with sales and marketing.
3. Prepare your site teams.
4. Two sites at a time, two observers at a time.
ï§ Introduce, observe, interview, wrap-up, thanks.
ï§ Collect artifacts and notice context.
ï§ 90 minutes.
5. Process with project team.
ï§ Immerse them in the experience and data.
6. Follow up.
ï§ Thanks + additional questions.
7. Retarget.
8. More visits, but not too many!
9. Example Observation 1
âI hate this [expletive deleted] history screen! I have to jump all
around and read everything to find out the patientâs latest A1C
measurement and see if sheâs visited the pedorthist recently.â
Option 1
âHow can we improve the history
screen?â
Option 2
âI noticed you came to this screen a
couple times while you were talking
to the patient.
What were you looking for?
Can you walk me through your
thinking?â
Donât ask users what they want, watch what they do.
Users are not designers, and you are not a user.
10. Example Observation 2
During the discussion with the patient, the nurse refers to a small spiral bound
note pad from time to time, sometimes making a quick notation. She also jots
some short notes on a piece of scrap paper. While you are paying attention to
those activities, you miss some of the conversation with the patient.
Option 1
âCan you repeat what you said to the
patient? I missed some of the
details.â
Option 2
âWhile you were talking with the
patient, you looked at your
notebook and sometimes wrote
something. Other times, you wrote
on the scrap paper.
What were you writing down?
Why in one place versus the
other?â
Synthesize a big picture with rich, textured detail.
Invest in more research, not post-processing.
11. Example Observation 3
BUT!
ï§ Get permission. Donât violate policies, regulations, or laws.
ï§ Get a signed release form if necessary.
ï§ Donât let the camera get in the way of really observing the work.
bunch of reference
material printed from
the web
support, clinic, and
social worker numbers
calendar with both
personal and work
events
brand name vs generic
reference for common
drugs
SOPs and
comprehensive clinic
and phsycian contacts
todayâs call list
monitor turned to
allow photo; avoid
HIPAA issues
ID badge worn at all
times
follow up notes from
last call
A picture is worth a
thousand memory slips.
ï§Triggers rich memories
from visit, immediately
afterward when
processing the data as well
as long term.
ï§Provides rich, visceral
context for team members
and stakeholders who
werenât there.
ï§Particularly memorable
ones can be used for
personas.
12. Data Collection Challenges
ï§ Confidentiality, privacy, and trust.
HIPAA, intellectual property, security, legal, regulatory.
ï§ Complex environments.
International sites, hazardous work, longitudinal or rare work.
ï§ Redundant data.
Repetitive work, rare exceptions, awkward times.
ï§ Too much data.
Many sites, trip logistics, team availability.
ï§ Exhaustion.
This is exhilarating work, but can be grueling.
13. Define Modeling
âA systematic description of an object or
phenomenon that shares important
characteristics with the object or
phenomenon.â
American Heritage Science Dictionary
âA simplified or idealized description or
conception of a particular system, situation, or
process⊠that is put forward as a basis for
theoretical or empirical understanding⊠a
conceptual or mental representation of
something.â
OED
ï§ Concrete and detailed representations of
observations.
ï§ Purposefully lossy: generalized but retaining
meaningful details.
ï§ Domain models include KJ Analysis, Personas,
Scenarios and a variety of others.
âmodels can frame the design
problem⊠help designers
understand the domainâ
Jared Spool
âThe most effective behavioral
models are distilled from
interview and observation data of
real users into an archetypal
description of how a particular
type of person behaves and what
their goals are.â
Kim Goodwin
âA mental model is a picture of
how your end users are
supported by what you are
creating.â
Indi Young
14. KJ Analysis
âAffinity on steroids.â
Outcomes:
ï§ Shared understanding
ï§ Relationships
ï§ Priorities
Process steps:
1. Define the question to be asked of the data.
2. Reduce the data set to 30 or fewer items
through multiple rounds of simple voting.
3. Group related items.
4. Title the groups.
5. Group related groups.
6. Create headlines for the higher level groups.
7. Visually lay out the groups with all data items.
8. Show relationships between groups.
9. Vote for the most important titleâlevel groups.
10. Draw a conclusion from the diagram.
We donât have a clear definition of
what a requirement is
No clear definition of what
the difference levels of
requirements are
There seems to be
a discrepancy on
how detailed the
rqmts need to be.
I canât delineate
between BRâs and
MRâs.
Itâs not clear
where a market
requirement stops
and an SR starts
I donât understand what the customer
needs; we donât understand what is
important to deliver
I like to be involved in the whole process
All roles should be
involved throughout the
entire process
VOCs are very valuable
Is seems like we are
spending a lot of time putting
the same data in multiple
places and coordinating to
ensure the data is in sync
The process shouldnât impede the
progress of the project
Weâre going to
need to do some
clarification of what is
design and what is a
requirements
We donât have a
good definition of
what a
requirement is
It seems like different
RAs have a different view
of what a requirement
should be
With requirements at
a high level it is difficult
to come up with a
good estimate
Perhaps we should
have two different
levels of use cases
We donât know what âgood
enoughâ is
How do you define
success at the end
of each iteration
We have lost the
concept of critical
release
requirements
We arenât validating what
we are building with the
customer
Need to get out of the
lab exercise of I know
what the answer is now
I need to go find a
customer to tell me they
have that problem
Need to know when
we come out the other
end, have we built the
feature that is necessary
for the customer.
You better think about
performance
requirements before you
start down the design
path
Need a better way
to bring customer
feedback in earlier in
the process
VOCs are very
valuable for
providing focus to
product issues
VOCS were very
valuable which gave
us a very good
process and means
to rank and prioritize
requirements
Get engineering and
test involved earlier on
before the rqmts are
finalized
The requirement
analyst needs to
own the rqmts all the
way through the
process.
Itâs difficult to look at
requirements in
ReqPro so everyone
gives up
I like the ability to
add attributes (Req
Pro)
Requirements tools are
difficult to work with
A lot of things you test
for will not show up in
requirements for
instance failure modes
If you read only the
SRs it is difficult to
understand what the
hell it is supposed to
do.
Hundreds of hours have been spend on
requirements with very specific detail for
consistency. The gap we have is that how
are we going to do a consistent design
without that level of detail
If the requirements donât
have the detail I need,
where do I get them?
When we took UI detail
out of the use case it
wasnât clear what the
rqmt really meant
Where do we put this
extra UI information
Itâs nice to have implementation details
as an example to understand what the
RA was thinking
UI Information is a good
way to communicate ideas
Anything so that people
arenât waiting for other
people to get stuff done
We should not have
to have everything
written down and
signed off to hand it
over the wall
No good way ot
fimplementing and
managing features access
products
We need to find a
better way to illustrate
requirements on a
features basis
All requirements
need to be
coordinated across
products
The process shouldnât
impede the progress of
the project
To be honest, itâs to
painful to do a change
I donât know where to go to get the information I need
Theme: Gaining a better understanding of the
requirement processes and how customers should be
involved
Lessons Learned: Better defections is needed for the
levels of requirements and where to get requirement
details,
15. Anatomy of a KJ Diagram
Headline
Relationship
cause and effect or contradiction
Title
Raw statement
Title with most votesTitle with second most votes
Title with third most votes
Question to ask?
Summary answer.
Time/date
Place
Participants
16. Personas
âThe aspect of a person's
character that is displayed to
or perceived by others.â
OED
ï§Tells a story
ï§Concrete and specific
ï§Memorable
ï§Day-in-the-life narrative
ï§Goals, tasks, and other
behavior
ï§Skills and demographics
âPersonas are helpful in creating and
iterating a design, building consensus,
marketing the product, and even
prioritizing bug fixes.â
Kim Goodwin
17. Persona: Identity
Memorable name with role or
user type
Memorable quote that captures
the essence of personaâs goals
and attitude
Memorable photo that strongly
represents persona and role
19. Persona: Skills
ï§ Identify skills that
differentiate the
different personas.
ï§ Rate the personaâs skill
level for each.
ï§ Present visually and
simply.
20. Persona: Details
ï§ Extend, generalize, and add specifics to narrative.
ï§ Enumerated and prioritized.
ï§ More analytical and processed than narrative.
21. Persona Quality
P Primary research
Is the persona based on contextual interviews with real customers?
E Empathy
Does the persona evoke empathy by including a name, a photograph and a
product-relevant narrative?
R Realistic
Does the persona appear realistic to people who deal with customers day-to-
day?
S Singular
Is each persona unique, having little in common with other personas?
O Objectives
Does the persona include product-relevant high-level goals and include a
quotation stating the key goal?
N Number
Is the number of personas small enough for the design team to remember the
name of each one, with one of the personas identified as primary?
A Applicable
Can the development team use the persona as a practical tool to make design
decisions?
http://www.userfocus.co.uk/articles/personas.html
22. Scenarios
âAn outline or model of an
expected or supposed sequence of
events.â
American Heritage Dictionary
ï§Envision the future.
ï§Personas drive the action and
point of view.
ï§Begin with a triggering event,
describes a sequence of actions,
and the results of those actions.
ï§Technology and solution-free.
ï§A few key scenarios are better
than a comprehensive set.
âA scenario supplies the context-
of-use necessary to set the
stage on the subtlety and
nuance you'll need to get the
mix just right.â
Jared Spool
23. Scenario Example
First step is the trigger that
begins the scenario
Step by step, who does what,
when, and why
Call out questions or
observations each step brings
to mind
24. Scenario vs. Use Case vs. User Story
ï§A scenario is a concrete,
sequential narrative about a
specific workflow.
ï§A use case is a generalization
of a number of scenarios and
can generate multiple
scenarios.
ï§A user story is a single
statement that captures who,
what, and why. Usually
names a use case or scenario,
or is a single step in one.
âAn outline or model of an
expected or supposed sequence
of events.â
American Heritage Dictionary
â⊠a collection of possible
sequences of interactions
between the system under
discussion and its external actors,
related to a particular goal.â
Alistair Cockburn
âA description of desired
functionality told from the
perspective of the user or
customer.â
Mike Cohn
25. Other Models
Contextual Inquiry
ï§ Flow Models
ï§ Sequence Models
ï§ Artifact Models
ï§ Cultural Models
ï§ Physical Models
Organizational Models
Process Models
Stakeholder Models
Concept Models
ï§ Choose models that
communicate the most
important information to the
most important stakeholders.
ï§ Keep the raw data around for
many purposes:
ï§ Explain or justify the models.
ï§ Future data mining.
ï§ Explorable body of knowledge.
26. Validate the Models
ï§ Validate KJ priorities with
surveys.
ï§ Rate and rank.
ï§ Much larger audience.
ï§ Validate personas with
interviews.
ï§ âWhich do you identify with?â
ï§ âWhat parts donât work?â
ï§ âDescribe a day in your work life.â
ï§ Iterate throughout the
lifecycle and across multiple
projects.
ï§ Over-communicate the
models and advocate their
use.
surveymonkey
Yahoo! personas poster from article at UIE.com
27. Tools, Tips, and Tricks of the Trade
ï§ Get really good at taking notes while
watching and talking.
ï§ Donât use a laptop.
ï§ Donât bother recording.
Do another visit instead.
ï§ Exception: Livescribe Pulse Smartpen
ï§ Take lots of pictures (if permitted).
ï§ âRevolving Doorâ technique.
ï§ Leah Rader and Beth Toland
ï§ Card sorting.
ï§ Information Architecture
ï§ Controlled Vocabularies
ï§ Journaling.
28. What To Do with All This Research
ï§ Share it!
Accessible repository, posters, walkthroughs with stakeholders,
conversations with SMEs, users, and customers.
ï§ Invent new products.
How could current work be transformed or eliminated?
ï§ Frame all downstream work.
Use cases, UI designs, and test scenarios based on the real world.
ï§ Validate the system.
Does it support real people doing real work? Does it help them achieve
their goals?
ï§ Simplify the system.
Eliminate functions and workflows that donât support the work.
ï§ Discover new avenues.
What more research paths are opened up?
29. Resources
The Design of Design: Essays from a Computer Scientist
Frederick P. Brooks, Jr.
Contextual Design: Defining Customer-Centered Systems
Hugh Beyer, Karen Holtzblatt
Rapid Contextual Design: A How-to Guide to Key Techniques for User-Centered Design
Karen Holtzblatt, Jessamyn Burns Wendell, Shelley Wood
Commercializing Great Products with Design for Six Sigma, chapter 16 âKJ Analysisâ
Randy C. Perry, David W. Bacon
Designing for the Digital Age: How to Create Human-Centered Products and Services
Kim Goodwin, Alan Cooper
The Persona Lifecycle: Keeping People in Mind Throughout Product Design
John Pruitt, Tamara Adlin
AgileProductDesign.com
Jeff Patton
Writing Effective Use Cases
Alistair Cockburn
Notas do Editor
If you are in the wrong room, stick around anyway!
The point of this slide is two-fold:
You can contact me any time.
Iâve been doing this for a while, even before I knew what I was doing.
I am a practitioner.
I am indebted to many others who have taught and inspired me. I have included quotes from some of them.
I offer advice on how to be successful at this; Iâve done so.
Inception define product direction
Elaboration prove out the direction and architecture
Construction build it
Transition deploy it
Always iterative, especially so in an Agile environment.
The activities largely happen outside the âsprint teamsâ and in parallel; may have their own sprints.
Keep ahead of the sprints; define the backlog.
Love these!
Be concrete
Take risks
Make decisions
Itâs okay to be wrong
Quotes from Fred Brooksâs new book The Design of Design
We study, record, and analyzeâŠ
People doing real workâŠ
In their native habitatsâŠ
In order to define better systems that transform the work.
Overlapping goals and techniques.
But distinct in purpose.
Donât confuse one for the other.
Marketing people are experts in something other than user research.
But they arenât â necessarily â evil.
2. Competitors + installed base
5. Workflow, breakdowns, artifacts, photos, sketches, memorable momentsrely on your notes and photographs
6. Share key findings, but be commercially sensitive good experiences lead to great participation in future and loyalty/trust
8. avoid redundant data
Option 1: Unproductive rat hole. Limited scope of learning. Sub-optimize a design that isnât supporting the actual work.
Option 2: Probing questions. 5 whys to root cause. Make implicit thoughts and actions explicit. Repeat work, donât recount from memory.
Option 1: Unproductive rat hole: specific conversational details likely not relevant; looking for patterns, breakdowns, unexpected findings
Worse yet: going back and listening to a recording to get the details; unproductive
Option 2: Exploring implicit actions and reasoning. Opportunity for the system to support functions that she is working around today using different paper mechanisms. At least two different functions going on here.
I will focus on these three today: KJ, personas, scenarios
GOAL: Immerse your organization and team in knowledge of users, their work, and their context.
Raw data hard to communicate. Models make it palatable and operational.
Developed by anthropologist Jiro Kawakita in the 60âs, the method proceeds through a series of simple group process steps.
Reducing the set is key!
Diagram is great for walking others through the data
I recently ran across this article by User Focus. They capture everything I wanted to.
User Focus is a great British UX consulting firm.
Lots of other articles besides this one.
Actual scenario runs to about 50 steps.
I use all three.
Alistair Cockburnâs books and web site are full of info on them.
Raw data repository should make using/finding them easy. Searchable, tagging, accessible.
Be aware of data/document obsolescence policies. Sarbanes Oxley, etc.
Rating and ranking
Ranking is hard
Pseudo-quantitative
Guides decisions, doesnât make them
Have I mentioned not to record audio/video?
All of these authors also have web sites (and many have other books as well)