1. What do students want from discovery tools?
Keren Mills, The Open University
@mirya

2. Presentation outline
•Distance learners as co-designers
•Discovery project overview
•Research methods used
•Key findings
#ili2014 @mirya

3. OU students
73%
29
42%
http://www.open.ac.uk/about/main/strategy/facts-and-figures
20,000


4. https://www.flickr.com/photos/31167076@N07/5471047557

5. Student perspectives
I think what I find very difficult or challenging about finding information is that things are never as straightforward as you think they are going to be. And you kind of click on one thing and then it’s never quite what you expect it to be. It’s never full text, it’s never…I don’t know. It’s very frustrating.”
OU student, Arts level 3

6. STUDENTS AS CO-DESIGNERS
•In late 2012 OU Library Services set up a student panel to make it easier to contact students who might be willing to take part in our user research.
•Students are recruited to the panel twice a year and we consistently maintain a cohort of 400-500 students.
•Response rates to participate in research activities is high, ranging between 24% to 82%

7. Library Futures project outline
The discovery research was a nine month project as part of the Library Futures project. The bigger project included 2 other strands – procuring a new LMS (Alma/Primo) and developing a “Digital Skills Passport”, which will be a modified version of “librarygame” from Running in the Halls.

8. Discovery research
•The aim of the research was to understand student expectations of library resource search tools
•The research ran from Autumn 2013 to Summer 2014
•There was a core team of project lead, project officer, developer with involvement from other library staff from learning and teaching, and systems
•We had advice and support from the Institute of Educational Technology’s Usability Labs team
•Outcomes have fed into procurement and setup of new library systems

9. RESEARCH METHODS USED

10. Literature review
“… students do indeed want a Google like experience, where they are taken directly to a search interface that is easy to use and had immediately relevant results” (Mussell, 2013)
The discovery work started with a literature review, which gave us confidence that a single search was the approach preferred by students around the world. Combined with a summary of past feedback, this helped us frame what we needed to know about the student experience of discovery.

11. Data analysis of search logs
One stop search (current Discovery tool)
•only 25% of searches use author or title radio buttons
•23% keyword search
•19% article titles
Feedback and log files of search terms weren’t enough to help us understand what did or didn’t work well about discovery tools or how students interact with them.

12. Staff workshops
What academics want students to use
What librarians want students to use

13. Remote online interviews
All of the different phases of testing were conducted through remote online interviews
•Students in own home, 1.5 hour interviews
•Researcher in lab on campus using Teamviewer software to see student screens
•Interviews were recorded with the participants’ permission
•Comments and behaviour were analysed

14. Observing student searches
We started by giving 18 students a selection of tasks to do using 2 or 3 different discovery tools each. This way we were able to test a combination of search tasks in 8 different discovery tools, using both our own and search tools from other universities. The students undertook these tasks during a remote online interview, allowing me to observe their methods of searching and their interaction with the search results pages.

15. Successful searches

16. Wireframe prototyping
Having analysed student behaviour and their suggestions for what they wanted from library search, we then developed a series of wireframes to help us verify what we’d learned and allow students to compare interface designs for search boxes, facets and results page layouts. The advantage of using the principles of paper prototyping is that test participants don’t get distracted by commenting on the colour or brand elements and pay more attention to the functional features of the interface.
The wireframes were also tested through one-to-one remote interviews.

17. Wireframe examples

18. Working prototype
Finally we developed a working prototype using the EDS API and undertook iterative testing through yet more remote online interviews, tweaking the interface and results ranking as we went along. At this point we included some personalisation features such as showing students a list of resources they had previously accessed through the library, offering a bookshelf feature, and offering settings to allow them to customise the interface.
•Construct prototype search interfaces
•Several cycles of iterative development
•Analysis of behaviour and feedback

19. Working prototype examples

20. CHALLENGES
Tension – cable/rope, climber?
#ili2014 @mirya

21. KEY FINDINGS
https://www.flickr.com/photos/30925581@N02/3688091593/

22. What do students want from library search?
1. Students want a simple search interface and it must be clear what is being searched
“The simpler it can be and the faster it is to search the better.”
2. Students want the search results page to open in a new tab/window
“On the module website could you have a search box above the TMA screens, but the search results open in another tab?”
FELS level 2 student
3. Students want a clear and uncluttered search results page
“[The prototype search tool] seems quite clear, the way it’s laid out. It’s very easy to find if what you’re looking for is there. There’s a lot of information, but it’s not cluttered or anything.”
MCT level 2 student

23. What do students want from library search?
4. The search tool must cope with finding the right result from a full or partial reference and offer best matches
26% of participants entered whole references and 44% entered partial references (e.g. author, year and title) into the search box when looking for an item from a reference.
5. There must be a clear indication whether the full text is available, there must be a clear link to it and it should open in a new window
“I’d expect another window to open with the full text, but I’d still expect the search results to stay open.”
Social Sciences level 1 student
6. Students want a library search box on Student Home, or at least a link
“On Student Home – click on a little icon, anywhere down the left hand side where the links are… A search box is the ultimate.”
MCT level 2 student

24. What do students want from library search?
7. Students expect an autocomplete feature for the search box
“Very much so, especially if you can’t quite remember the full title of an item or correct spelling of an author’s name.”
Social Sciences level 1 student
8. Students want to see their previous searches and an indication if they have viewed the item before
“That would be brilliant, especially if you’ve forgotten to note down the details of something you’ve referred to.”
Social Sciences level 1 student
“What I like to do is find things upfront and save them somewhere so I can come back to it. I’d like it if I could mark it as read so I know which ones I’ve read already.”

25. What do students want from library search?
9. Students like the idea of a personal library ‘shelf’
“That would be fantastic! I presume it means you can store articles and e-books and come back to them.”
MCT level 2 student

26. What do students want from library search?
10. Students expect Google-like relevance ranking
“We’re all used to Google and we just put in one, two or three words.”
Social Sciences level 3 student
“Google brings up results with most of the keywords.”
FELS level 2 student

27. Lessons learned
User research is definitely worth doing
•It helped us decide which supplier to go with
•The evidence gained will shape how Primo Discovery will be implemented

28. What do students want from discovery tools?
Keren Mills (@mirya)
Digital Services Development Officer
Library Services, The Open University
Keren.Mills@open.ac.uk
www.slideshare.net/mirya