The objectives of the evaluation was to asses the usability of the websites book borrowing process. The methodologies used to test were a background survey gathering information about the New York Public Library members and website users , interview questions regarding the participants web usage and their familiarity with the NYPL website, real-world tasks combined with a think aloud protocol including finding and borrowing a digital book, finding a physical book and borrowing it, finding the due date of a book and finding out the hold state of a book, a LikertScale questioner filled by the participants upon completion of a task and an exit question asking what is the first thing they would change on the website. The main results were that participants had a difficulty finding out the book format and reported difficulty in downloading ebooks. Participants also struggled with finding out the location of the branches in which to pick up or place holds for books. Another area users found difficult was the filtering of search results in which all the filter options were not noticed by the user and took discovery time. Participants also struggled in finding the my NYPL menu in which the information regarding due dates and holds appeared but once the menu was found most participants navigated it easily. The recommendations are to assist users in discovering the my NYPL menu by adding it to the area most participants searched for it, in the personal account drop down menu at the top of the page. It is also recommended to explore the visibility of the side menus and filters as they appear in the search results page and the my NYPL areas as well as further test the downloading process of ebooks to assure that users can complete the process effectively and efficiently.

1. User Testing the
borrowing process on
The New York Public
Library website
nypl.org
Galit Lurya

2. BACKGROUND
In 2012, the New York
public library's website
had 32 million visits.
Gives access to the
library's catalogs and
on-line collections
Has information about
the library's events,
exhibitions, and classes
Library cardholders can
request books as well as
download e-books and
audibooks

3. GOALS
Focusing on identifying the problem
areas in the New York Public Library
website book borrowing process

4. GOALS
Discovering and finding books the user is interested in

5. GOALS
Requesting to borrow books, check due dates of borrowed
books and status of books on hold

6. PARTICIPANTS
Focusing on users that already have a library card
and are divided into two subgroups:
Individuals that have
used the NYPL website
to borrow books
Individuals that have not
used the NYPL website
to borrow books

7. WHAT TO TEST- TASKS
Users should be able to:

Find books they want to borrow

Discover new books

See if a book is available for borrowing

Place a hold for a book

Check the state of a hold or a borrowed book

8. METHODOLOGY
Background
Questionnaire
Tasks
Talk out loud
LikertScale
Interview

9. BACKGROUND QUESTIONNAIRE
GENDER
AGE
2 | male
4 | female
3 | 18-25
1 | 26-39
2 | 40-59
60+

10. BACKGROUND QUESTIONNAIRE
ETHNICITY
LANGUAGE
4 | english
3 | white/caucasian
2 | asian/pascific islander
2 | asian
black / african american
spanish
american indian / alaska native
other
native hawaiian / other pacific islander
1 | other

11. BACKGROUND QUESTIONNAIRE
HOUSEHOLD INCOME
2 | under $25,000
1 | between $25,000 to $60,000
2 | more than $60,000
EDUCATION
less than high school
high school
some college
2 year college degree (associate)
5 | 4 year college degree (BS,BA)
1 | masters degree
doctoral degree
professional degree (MD, JD)

12. BACKGROUND QUESTIONNAIRE
BOOKS READ (MONTHLY)
less than 1
3 | 1-2
2 | 3-4
1 | more than 4
LIBRARY VISITS (MONTHLY)
1 | less than 1
3 | 1-2
1 | 3-4
1 | more than 4

13. BACKGROUND QUESTIONNAIRE
HOW TO DECIDE WHAT TO READ
1 | bestseller rankings
2 | books that everyone is talking about
4 | direct recommendations from someone else
2 | new titles/recently released books
4 | other

14. INTERVIEW QUESTIONS
HOURS SPENT ONLINE WEEKLY
3
2
1
less than
15
between
15-30
50+

15. INTERVIEW QUESTIONS
VISITS TO THE NYPL
WEBSITE (YEARLY)
LOCATION WHEN USING
THE NYPL WEBSITE
5
3
2
1
1
1-2
3-9
10+
home
on the
go

16. INTERVIEW QUESTIONS
OTHER WEBSITES/PLATFORMS USED FOR BOOKS

17. TASK 1
Find a book (digital format) you recently
read or want to read and borrow it
4
1
success
partial
success
fail

18. TASK 1
Find a book (digital format) you recently
read or want to read and borrow it
2 experienced users
4 new users
4
1
success
1
partial
success
fail

19. TASK 1 LIKERTSCALE | EXPERIENCED USERS
Find a book (digital format) you recently
read or want to read and borrow it
2 experienced users
Strongly
disagree
Disagree
Neutral
Finding a book was easy
2
1
Finding the book format was easy
Finding the book availabilty was easy
Downloading the book to a device was easy
Agree
Strongly
agree
1
1
1
1

20. TASK 1 LIKERTSCALE | NEW USERS
Find a book (digital format) you recently
read or want to read and borrow it
4 new users
Strongly
disagree
Disagree
Neutral
Agree
Finding a book was easy
1
1
2
Finding the book format was easy
1
3
Finding the book availabilty was easy
1
1
Downloading the book to a device was easy
1
2
Strongly
agree

21. TASK 2
Find a book (physical format) that you haven't
heard of and interests you and borrow it
1 experienced user
0 new users
1
success
partial
success
fail

22. TASK 2 LIKERTSCALE | EXPERIENCED USERS
Find a book (physical format) that you haven't
heard of and interests you and borrow it
1 experienced users
Finding a book was easy
Strongly
disagree
Disagree
Neutral
Agree
1
Finding the book format was easy
1
Finding the book availabilty was easy
1
Finding where to pick up the book was easy
1
Strongly
agree

23. TASK 3
Find out when a book you borrowed is
due for return

24. TASK 3
Find out when a book you borrowed is
due for return
2 experienced users
3 new users
4
1
success
partial
success
fail

25. TASK 3 LIKERTSCALE | EXPERIENCED USERS
Find out when a book you borrowed is
due for return
2 experienced users
Strongly
disagree
Disagree
Finding my borrowed books was easy
Finding the due date of the book was easy
Neutral
Agree
2
1
1
Strongly
agree

26. TASK 3 LIKERTSCALE | NEW USERS
Find out when a book you borrowed is
due for return
2 new users
Finding my borrowed books was easy
Finding the due date of the book was easy
Strongly
disagree
Disagree
1
Neutral
Agree
Strongly
agree
1
1
1

27. TASK 4
Find out how long you need to wait for
a book on hold
2 experienced users
2 new users
3
1
success
partial
success
fail

28. TASK 4 LIKERTSCALE | EXPERIENCED USERS
Find out how long you need to wait for
a book on hold
2 experienced users
Strongly
disagree
Disagree
Neutral
Finding my book on hold was easy
Finding how many people are before me in
line was easy
Agree
2
1
1
Strongly
agree

29. TASK 4 LIKERTSCALE | NEW USERS
Find out how long you need to wait for
a book on hold
1 new users
Finding my book on hold was easy
Finding how many people are before me in
line was easy
Strongly
disagree
Disagree
Neutral
1
1
Agree
Strongly
agree

30. CLOSING QUESTIONS
If you were the developer of the website
what is the first thing you would change?

31. Thank you