Presentation of an analysis of Oria queries at the University of Oslo. (see also: http://www.ub.uio.no/om/prosjekter/the-visualisation-project/news/oria-analysis)
Not available, or not found? Lessons from user queries in the Oria catalog at UiO
1. Not available, or not found?
Lessons from user queries in
the Oria catalog at UiO
07/12/2016 & 08/12/2016
Hugo Huurdeman, Visual Navigation Project (@BookNavigation)
2. Visual Navigation Project
interaction methods with digital content have to be rethought
Increasing degree of digital access to materials
Loss of rich interactions in physical space
Photo:metalocus.es
3. Visual Navigation Project
interaction methods with digital content have to be rethought
Increasing degree of digital access to materials
Loss of rich interactions in physical space
1
Photo:metalocus.es
4. Visual Navigation Project
interaction methods with digital content have to be rethought
Increasing degree of digital access to materials
Loss of rich interactions in physical space
1
2
photo: Marina Tofting
Photo:metalocus.es
5. Visual Navigation Project
interaction methods with digital content have to be rethought
Increasing degree of digital access to materials
Loss of rich interactions in physical space
1 3
2
photo: Marina Tofting
Photo:metalocus.es
6. Introduction
• More insights needed of how current catalogs
are used in practice, what goes right and what
goes wrong
• Opportunity: analyze data gathered at UiO
during the last two years
7.
8.
9.
10. Primo Analytics
• Actions
• Device usage
• Facets
• Queries (most popular, monthly)
• Timeframes: Jan-Jun 2015; Nov-Dec 2015; Jan-Sep 2016
• Sessions
• Zero result queries (daily)
• Aug 7 2015-Sep 30 2016
11.
12. Processing & Analysis
• Basic normalization of queries
• "stanislav andreski" -- stanislav andreski
• Manual annotation of
• (1) top 50 popular queries
• (2) random selection of 50 "0 result" queries
• If derivable from query, determine
• nature of query?
• for which resource type?
• pensum-related? [in pensum lists UiO]
• succesful? [result on first results page]
13. 1. Most popular queries
• 5,776 different
queries, 115,590
searches (UiO)
• monthly totals
• 2015 & 2016
• Analyzing top 50
• Together issued
almost 20,000
times (17%)
16. Top 50
• What types of queries are most popular?
• titles
• det kvalitative
forskningsintervju
• det norske samfunn
• topics (course names)
• spesialpedagogikk
• neurology
• “databases”
• atekst
• pubmed
• duo
17. Top 50
• What resource types are sought for?
• books
• det kvalitative
forskningsintervju
• det norske samfunn
• journals
• science
• lancet
• “databases”
• atekst
• pubmed
• duo
18. Pensum?
• Were the queries related to pensum books?
• Quite often: at least 57%!
• menneskets fysiologi
• det kvalitative
forskningsintervju
• Or other things (9%) :)
• aftenposten;
morgenbladet
• harry potter
19. Pensum?
• Were the queries related to pensum books?
• Quite often: at least 57%!
• menneskets fysiologi
• det kvalitative
forskningsintervju
• Or other things (9%) :)
• aftenposten;
morgenbladet
• harry potter
20. Pensum?
• Were the queries related to pensum books?
• Quite often: at least
57%!
• menneskets fysiologi
• det kvalitative
forskningsintervju
• Or other things (9%) :)
• aftenposten;
morgenbladet
21. Were the queries successful?
• Often, yes:
• main result in first 10
results: 51%
• not (easily) found: 19%
• For example:
• pubmed
• nature
• science
23. 2. “Zero result” queries
• 39,925
different
queries
• In total 52,257
searches
• Aug 2015-now
• Process:
• Normalizing
queries
• Annotating
random
sample (50)
24. 2. “Zero result” queries
• 39,925
different
queries
• In total 52,257
searches
• Aug 2015-now
• Process:
• Normalizing
queries
• Annotating
random
sample (50)
25. 2. “Zero result” queries
• 39,925
different
queries
• In total 52,257
searches
• Aug 2015-now
• Process:
• Normalizing
queries
• Annotating
random
sample (50)
al azm sadik «orientalisme og omvendt orientalisme» 28
curr eye res 27
allmenningen olaf 24
12136173x 22
am j ophthalmol 21
821017268 21
askim j. 19
900317809 19
cheng 19
(direkte krav or direktekrav) and subrogasjon 17
151582998 16
142734500 16
direkte krav or direktekrav 16
961675616 16
9780549547303 16
agirdag phalet 16
andresen steinar elin l. boasson og geir hønneland (2012) international environmental agreements 15
26. Queries without results
author(s)
16%
citation
43%
isbn
6%
title
31%
topic
4%
types&of&queries
What is the nature of the “zero result” queries performed in Oria?
• Pasted citation (43%)
• Browning, N. (2015). The
ethics of two-way symmetry
and the dilemmas of dialogic
kantianism. Journal of Media
Ethics
• Title (31%)
• Sentralbankens oppgaver i
dag og i fremtiden
• “Folk er vanligvis ikke særlig
villige til å snakke om de
døde”
• Author (16%)
• Christopher Hotchens
28. Degree of pensum books?
yes
28%
no
38%
_unknown
34%
Pensum'book?
At least 28% of the
unsuccessful queries
are for pensum books
• Fukuyama, F. (2013): What Is
Governance? Governance,
Vol. 26, No. 3, July 2013 (s.
347–368).
• basic immubology
• Adcock + Collier /
Measurement Validity: A
Shared Standard for
Qualitative and Quantitative
Research
30. Why no results?
• Query being too specific (pasted citation, pasting quote from
text) [22%]
• Browning, N. (2015). The ethics of two-way symmetry and the
dilemmas of dialogic kantianism. Journal of Media Ethics
• Misspellings, reference mistakes (e.g. wrong year) [20%]
• why students underacheive
• svennevig j.: ledelsesretorikk i nedbemanningssituasjoner
2009
• Using incorrect query syntax [2%]
• "Psykisk helsevern" + "tvang" + "umenneskelig"
• "McLuhan" AND/OR "Understanding media"
• 978-147996410-9
31. Why no results (2)
• Searching for an ISBN number, which may just refer to a specific
edition of a book, but not the one in the library
• 9780618721566
• Searching for journal titles, ISSNs, DOIs
• journal of speech and hearing disorders
• Searching for course codes
• MED1100
• Resource not available in Oria [24%]
• Haugianerliberalistene: En analyse av haugianere som
politikere og næringslivsaktører
• "Limp Wrists and Laser Guns: Male Homosexuality and
Science Fiction"
32. 3. Learned Lessons (1)
• Importance of aiding students finding better results, e.g.
• Enhanced spell check / query corrections
• why students underacheive → 0 results
• why students underachieve → 463 results
• Query suggestions and autocomplete
• Could be based on previous (successful) queries
• especially recurring queries should be supported (50% of popular
queries!)
• ISBN suggestions
• Automatically search for the book title and author (using
information derived from ISBN number?)
33. 3. Learned Lessons (1)
• Importance of aiding students finding better results, e.g.
• Enhanced spell check / query corrections
• why students underacheive → 0 results
• why students underachieve → 463 results
• Query suggestions and autocomplete
• Could be based on previous (successful) queries
• especially recurring queries should be supported (50% of popular
queries!)
• ISBN suggestions
• Automatically search for the book title and author (using
information derived from ISBN number?)
34. 3. Learned Lessons (2)
• Importance of "pensum" queries
• Detecting curriculum queries
• E.g. widget which suggests pensum
materials directly // referring students
to UiO fagsider // etc.
• On the cataloging side:
• Course codes (e.g. INF2260)
"to catalog or not to catalog"
• More support for Database queries
• Atekst can be found, but pubmed cannot
be found...
35. 3. Learned Lessons (2)
• Importance of "pensum" queries
• Detecting curriculum queries
• E.g. widget which suggests pensum
materials directly // referring students
to UiO fagsider // etc.
• On the cataloging side:
• Course codes (e.g. INF2260)
"to catalog or not to catalog"
• More support for Database queries
• Atekst can be found, but pubmed cannot
be found...
36. Summary & Future work
• Conclusion
• Lots of Oria queries, lots of opportunities for
improvement :-)
• Plans for further analysis
• Comparing queries with most frequently borrowed items
• Alma Analytics
• Looking at common exploratory queries
• medicine; math; united nations; economics
• how can we support those types of queries better?
• Possibility to obtain more data?
• e.g. analyzing "struggling sessions", stats per location, etc
38. Not available, or not found?
Lessons from user queries in
the Oria catalog at UiO
07/12/2016 & 08/12/2016
Hugo Huurdeman, Visual Navigation Project (@BookNavigation)
