Try It! Learn to Improve Guides and Websites Using Design Research Methods

Try It! Learn to
Improve
Guides and
Websites Using
Design
Research
Methods
Georgia Libraries Conference
October 3, 2018
Ashley Hoffman
eLearning Librarian
Amy Gratz Barker
Learning & Teaching Services Librarian
Horace W. Sturgis Library
Kennesaw State University

Why Use Design Research?
“…the challenge is not
organizing information,
but organizing it in a way
that works well for other
people – the users.”
~Donna Spencer,
Card Sorting, p. 146
• Design Research
Methods refers to user-
focused research that
informs the design
process
• Provides insight into
user experience and
needs
• Allows you to test a
design and assess
impact on function,
aesthetics, and overall
usability
2

Why We Did Design Research
• Research guide template
redesign
• Old template was:
o From 2013
o Based on v1, not v2 of
LibGuides
o Not often used
o Limiting for librarians
• Wanted a student-
driven design
3

Design
Method 1:
Card Sorting
Amy Gratz Barker
Card Sorting: Designing Usable Categories
Donna Spencer
2009, Rosenfeld Media
ISBN: 978-1-933820-02-6
Additional Resources, including analysis spreadsheets
and instructions:
https://rosenfeldmedia.com/books/card-sorting/

Card Sorting in a Nutshell
• Cards:
o Index cards with descriptions of content on the test website
• Sorting options:
o Open: participants determine their own groups and name
them
o Closed: uses a set of predetermined categories
• What you learn:
o Different ways information could be grouped
o How users conceptualize groups
o What goes together - and what doesn’t
o How users name groups
5

Designing a Card Sort Study
1. Identify what you want
to learn
2. Design the study:
o Choose method
o Create cards
o Create other materials
o Design recruitment
3. Submit for approval
Potential Uses For Card Sorting:
• Brainstorming categorization models
• Exploring how people think about a
certain topic
• Finding out what categories seem similar
or complementary
• Learning about what goes together and
what doesn’t
• Gathering lists of words people use to
describe groups of information
Card Sorting, p. 7
6

Creating Cards
• Use descriptive, simple
language
• Try to make cards at a
similar “level”:
o Participants can’t easily
sort broad and detailed
cards at the same time
• How many? Usually 30
to 100
7

Running a Card Sort
• Prepare the room:
o Plenty of space &
supplies
• Walk participants
through step-by-step:
o Don’t tell participants
they will be labeling their
groups – until they’re
done sorting cards
Image “Card sorting” by Luca Mascaro (CC BY-SA 2.0) 2009.
https://flic.kr/p/6y2zAf
8

Running a Card Sort
• Record what participants
are saying:
o Take notes or film their
work
Image “Card sorting” by Luca Mascaro (CC BY-SA 2.0) 2009.
https://flic.kr/p/6y2zAf
• Photograph each sort:
o Exactly as each
participant left it
o With every card clearly
visible
9

Card Sorting Activity
Part 1: Create Cards
5 MINUTES
1. Review the assigned website.
2. Review the 12 current cards.
3. Create 2 to 3 additional cards.
• We have provided a suggested area of the website to
create cards for!

Part 2: Sort Cards
10 MINUTES
1. Exchange your cards with a team from the other side
of the room.
2. As a team, organize the other team’s cards into
groups of your own choosing.
3. AFTER you have grouped cards, come up with a name
for each group. Write names on envelopes and place
at the head of the group.

Part 3: Review Sorts
5 MINUTES
1. Look at how your cards were sorted by your partner
team.
• How many groups did they create?
• What names did they give their groups?
2. Look at another sort for the same website.
• How similar, or dissimilar, is the sort?
3. If you have time, look at additional sorts.

Analyzing the Results
• Two main options:
o Exploratory and Statistical
• Exploratory analysis:
o First, create standardized category names
o Analyze frequency of cards in a certain category
o Analyze frequency of category for a certain card
• End goal:
o Create a categorization scheme based on the data – but not
exactly following it!
We strongly recommend Donna Spencer’s spreadsheets for getting started!
13

Top Five Tips!
1. Recruit extra participants
2. Record participants’ thought process
3. Balance # of cards with time available
4. Create a system for data collection/entry
5. Take photos immediately when finished
14

“I would never suggest [card sorting] is the
only technique you should use in a project.
Even though it is particularly good at [what it
does], it doesn’t help you to learn about what
people need or how they undertake tasks.”
~Donna Spencer, Card Sorting, p.10
Break

Rocket Surgery Made Easy: The Do-It-Yourself
Guide to Finding and Fixing Usability Problems
Steve Krug
2010, New Riders
ISBN: 978-0-321-65729-9
Additional Resources, including demo videos of usability
testing: www.sensible.com/rsme.html
Design
Method 2:
Usability Testing
Ashley Hoffman

What is Usability Testing?
• Usability Testing:
o Observing others use
your website
o Noting problems that
users encounter
o Fixing those problems
• DIY Usability Testing:
o Inexpensive
o Easily adapted to your
situation
“Watching people try to use
what you’re creating/
designing/building… with the
intention of (a) making it easier
for people to use or (b) proving
that it is easy to use.”
-Steve Krug,
Rocket Surgery Made Easy, p. 13
17

Planning Your Usability Test
1. Identify targets for
testing
2. Create authentic tasks
3. Recruit participants
4. Conduct the test
5. Analyze results
6. Make improvements
Repeat often!
Identify
Create
Recruit
Conduct
Analyze
Improve
18

1. Identify What to Test
• Can start with:
o a prototype
o your existing site
o or someone else’s site!
• What do users actually
need to do on your site?
o List 10 essential things
19

2. Create Authentic Tasks
Sample Scenario
You’re a student in English 1101. Your
professor is requiring you to use a
scholarly article for a paper.
Where would you go on this site to get
started?
• Develop scenarios:
o Who are they?
o What’s their motivation?
o What do they need to
do?
• Create a script
20

3. Recruit Participants
• Recruit:
o “Loosely” - actual users
not necessary!
o Aim for at least 3
participants
• Advertise:
o Banner on your website
o Email listservs
o In-person
• Incentives
WANTFREE
CANDY?
HELP THE
LIBRARY!
Your participation is completely
voluntary and will not affect your
grades. All your responses will be
kept confidential. No personally
identifiable information will be
associated with your responses to
any reports from this survey. This
survey has been approved by the
KSU Institutional Review Board and
the KSU Library Administration.
Study #18-306
The Library is looking for students to help
us improve our online Research Guides.
The study will only take
15 MINUTES
and in return, you will help the
library improve our online
resources for all students!
(Plus, at the end, you will
receive a small piece of
deliciousness for your
trouble)
21

4. Conduct the Test
• What You Need:
o Computer
o Microphone
o Screen recording
software
o Private room for testing
Sample Probing Questions
What do you think this is?
What are you thinking?
What are you looking at now?
Is this what you expected to happen?
Did you find what you were looking for?
Why did you click on that?
What would you do if I weren’t here?
Sample Usability Test Video: https://youtu.be/QckIzHC99Xc22

4. Conduct the Test
• Your roles:
o Tour Guide
– Lead participant through
scenarios
– Keep them on track
o Therapist
– Keep them talking!
Sample Usability Test Video: https://youtu.be/QckIzHC99Xc
• Considerations:
o Participant well-being
o Participant privacy
o IRB guidelines
23

4. Conduct the Test
• DO:
o Be flexible with tasks
o Encourage the
participant to talk
o Remain neutral
Sample Usability Test Video: https://youtu.be/QckIzHC99Xc
• DON’T:
o Let them use search
o Let them leave the site
o Distress them!
24

5. Analyze Results
• Don’t get hung up on
analyzing results!
• The point is to learn
about things you didn’t
know or things that you
can fix
Key
0 Fail
1
Succeed very slowly, in
a roundabout way
2 Succeed a little slowly
3 Succeed quickly
Task Completion Times/Success
Participant Task 1 Task 2 Task 3 Task 4
A 2 0 2 1
B 2 0 3 0
C 0 1 3 0
D 2 2 3 1
E 2 0 1 3
F 2 2 2 2
G 1 1 2 1
H 0 0 3 1
I 0 0 2 0
J 1 1 3 2
K 0 0 3 0
Average: 1.1 0.6 2.5 1.0
25

6. Make Improvements
• Identify the biggest
problems, not just the
easiest to fix
• Commit to a timeline
• Try to improve even if
you can’t fix completely
“What’s the smallest,
simplest change we can
make that’s likely to keep
people from having the
problem we observed?”
~Steve Krug,
Rocket Surgery Made Easy, p. 146
26

Usability Testing Activity
Part 1: Identify What to Test
5 MINUTES
1. Break into groups of 2-4.
2. Browse your assigned website. Ask yourselves these
questions:
• What is this site’s main purpose?
• What is most essential for users of this site to be able
to do?
3. Identify 3 potential areas for testing.

Part 2: Create Authentic Tasks
5 MINUTES
1. In the same group, create scenarios to test your 3
targets.
2. For each scenario, identify:
• Who the participant is “playing.”
• What is the participant’s motivation.
• What the participant needs to accomplish.
3. Write the scenarios on your worksheet.

Part 3: Conduct a Test
10 MINUTES
• Volunteers will take turns leading a volunteer from
the opposite group in a sample usability test.
• Audience members will observe and take notes.

Top Six Tips!
1. Do small tests, frequently
2. Design tasks your users actually need
3. Recruit loosely
4. Your jobs are: Tour Guide and Therapist
5. Be neutral and ask probing questions
6. Focus on the biggest problem, not easiest to fix
30

Conclusion
• Design research isn’t as
time-intensive or
complicated as you may
think
• You can adapt a variety
of methods to suit your
library
• Allows you to design
sites with user needs in
mind
31

References
Krug, S. (2010). Rocket surgery made easy: The do-it-yourself guide to finding and
fixing usability problems. Berkeley, CA: New Riders.
Spencer, D. (2009). Card sorting: Designing usable categories. New York: Rosenfeld
Media.
Also Recommended:
Krug, S. (2014). Don’t make me think, revisited: A common sense approach to web
usability (3rd ed.). San Francisco, CA: New Riders.
Kuniavsky, M. (2003). Observing the user experience: A practitioner’s guide to user
research. San Francisco, Calif.: Morgan Kaufmann.
Little, J. J. (2010). Cognitive load theory and library research guides. Internet
Reference Services Quarterly, 15(1), 53–63. doi.org/10.1080/10875300903530199
Sinkinson, C., Alexander, S., Hicks, A., & Kahn, M. (2012). Guiding design: Exposing
librarian and student mental models of research guides. Portal: Libraries and
the Academy, 12(1), 63–84. doi.org/10.1353/pla.2012.0008
Thorngate, S., & Hoden, A. (2017). Exploratory usability testing of user interface
options in LibGuides 2. College & Research Libraries, 78(6), 844–861.
doi.org/10.5860/crl.78.6.844
32

“[Techniques] provide insights and help you create
great solutions. But you are the one who puts it all
together into a great solution.”
~Donna Spencer,
Card Sorting, p. 150
Questions?

Try It! Learn to Improve Guides and Websites Using Design Research Methods

Recomendados

Recomendados

Mais conteúdo relacionado

Semelhante a Try It! Learn to Improve Guides and Websites Using Design Research Methods

Semelhante a Try It! Learn to Improve Guides and Websites Using Design Research Methods (20)

Mais de Georgia Libraries Conference (formerly Ga COMO).

Mais de Georgia Libraries Conference (formerly Ga COMO). (20)

Último

Último (20)

Try It! Learn to Improve Guides and Websites Using Design Research Methods