The NISO Update provides the latest news about NISO's current efforts, including standards, recommended practices and community meetings covering many areas of interest to the library community. Working group members will provide updates on projects newly underway or recently completed.
Open Discovery Initiative (ODI), Laura Morse, Director, Library Systems, Harvard University
1. ODI - Open Discovery Initiative
Laura Morse
ALA Annual NISO Update – June 29, 2014
2. The context for ODI
• Based on a meeting at ALA Annual Conference in New
Orleans on Sunday June 26, 2011. Recognition of the
following trends and issues:
– Emergence of Library Discovery Services solutions
• Based on index of a wide range of content
• Commercial and open access
• Primary journal literature, ebooks, and more
– Adopted by thousands of libraries around the world, and impact
millions of users
– Agreements between content providers and discovery providers
ad-hoc, not representative of all content, and opaque to
customers.
2
3. General Goals
• Define ways for libraries to assess the level of
content providers’ participation in discovery services
• Help streamline the process by which content
providers work with discovery service vendors
• Define models for “fair” linking from discovery
services to publishers’ content
• Determine what usage statistics should be collected
for libraries and for content
providers
3
4. Balance of Constituents
Libraries
Publishers
Service Providers
4
Marshall Breeding, Independent Consultant
Jamene Brooks-Kieffer, Kansas State University
Laura Morse, Harvard University
Ken Varnum, University of Michigan
Sara Brownmiller, University of Oregon
Lucy Harrison, Florida Virtual Campus (D2D
liaison/observer)
Michele Newberry, Independent
Lettie Conrad, SAGE Publications
Jeff Lang, Thomson Reuters
Linda Beebe, American Psychological Assoc
Aaron Wood, Alexander Street Press
Roger Schonfeld, JSTOR, Ithaka
Jenny Walker, Independent Consultant
John Law, Proquest
Michael Gorrell, EBSCO Information Services
David Lindahl, University of Rochester (XC)
Jeff Penka, OCLC (D2D liaison/observer)
5. Recommended Practice
• Open Discovery Initiative: Promoting Transparency in
Discovery (NISO RP-19-2014) - June 26th, 2014
– Vocabulary
– NISO Recommended Practice
– Mechanisms to evaluate conformance with recommended
practice
5
6. Vocabulary
1.2 Terms and Definitions – Key areas
– Search Models and Related Concepts
– Data Definitions
– Methods of Data Exchange
– Actors
– License Terms
6
7. Vocabulary (Examples)
• central index - The result of storing and indexing content in a central location.
Disparate content sources are aggregated with consistent formatting, indexing,
and ranking algorithms. Also referred to as centrally indexed search engine.
• enhanced metadata Metadata that is augmented by textual descriptions extracted
directly from the work including some or all of abstracts, tables of contents, and
sample pages
• syndication -Method of pushing content to remote indexing, abstracting, or
display services.
• content provider - The organization providing dissemination of the content
(literature or information). May be a publisher, aggregator, OA or institutional
repository, or A&I service provider. The same content may be available from
multiple content providers.
• market product - A specific, defined collection of resources made available for
license or use by a content provider to a library. This could be a citation database,
a set of e-journals or e-books, or other constellation of titles or objects commonly
licensed as a single entity.
7
8. Recommended Practices
3.1 General
• Oversight Group – education, promotion, ongoing
discussion
• Conformance Disclosure – via checklists included in
the best practice
8
9. Recommended Practices
3.2 Content Providers
• Participation – provide core metadata and full-text/original
content, as well as full text and enriched content
• Core metadata elements – basic citation metadata
(author, title, publisher, date, type, format, etc)
• Enriched content – indexing data (A&I data like subject
headings), full text or transcript, abstracts/description
• Disclosure – provide information to libraries related to level
of participation
• Technical formats – use existing standards to facilitate
data exchange
9
10. Recommended Practices
3.3 Discovery Service Providers
• Disclosure – provide key information in a consistent,
usable form to libraries about content indexed to
facilitate evaluation
• Linking – linking and relevancy methods should not
introduce bias to particular content providers;
libraries should determine linking choices; annual
disclosure related to neutrality
• Data transfer – use existing protocols and provide
documentation, preferences, and indication on
impact on different processes to content providers
10
11. Recommended Practices
3.4 Usage Statistics
Provide simple and frequent statistics to content
providers
– Total number of searches
– Result clicks
– Total number of click-throughs
Provide simple and frequent statistics to libraries
– Total number of searches per month
– Total number of unique visitors per month
– Total number of click-throughs per month
– Top 500 search queries for the last period
– Top 100 referring URLs to the discovery service for the last period
11
12. Next Steps
• Collaborative Discussion
• Application Programming Interfaces (APIs)
• Managing “Restricted” Content in Discovery Services
• Reporting on Discovery Service Content at a Collection
Level
• On-Demand Lookup by Discovery Service Users
• Research and Analysis to Monitor Discovery Services for
Fair/Unbiased Linking
• Further Interaction with COUNTER
• Identification of Features/Functionality of Discovery
Services to Address Needs of A&I Service Providers
12
15. Summary
Web-scale discovery systems are increasingly important
to the work of libraries in service of their users.
Fall 2013 Harvard Library User Survey
• 434 undergraduates, graduate students, faculty, staff, and other non-
Harvard researchers across all disciplines. 297 (68%) ranked single search
as important.
.
15
16. Summary
“One form, one search, one syntax.” – faculty member survey
respondent
Content Providers & Discovery System Providers, please follow
these best practices to ensure that your data and services are
available to meet the needs of customers and users.
Libraries, please engage in conversations with content providers
and discovery system providers early and often to ensure that
discovery systems best serve our users now and in the future.
Please use the conformance checklists with all types of content
providers and discovery vendors to inform your decision making.
16
17. Summary
Please read the full recommended practice, available
in the ODI Workroom:
http://www.niso.org/workrooms/odi/
17
In order to ensure consistent use of terminology, one of the early tasks of ODI was to create a glossary of
definitions relevant to the field of investigation. The terms as used in this recommended practice, have the
meanings indicated below.
Create an oversight group (Standing Committee or Working Group) to promote educational
opportunities about adoption of these recommended practices, provide support for content providers
and discovery providers during adoption, provide a forum for ongoing discussion related to all aspects
of discovery platforms for all stakeholders (content providers, discovery providers, libraries), and
determine timing for next steps for ongoing work.
2. Content providers and discovery service providers can take specific measures to assert their
conformance with the recommended practices of the ODI. These measures will be voluntarily made
by the organizations. Self-check conformance lists are provided in Appendix B and Appendix C. In
this phase, conformance will not be assessed by NISO or any other formal organization, but through
general community review
ODI recommends that discovery services implement the following protocols to ensure fair linking:
1. Discovery services should not discriminate, based on business relationships, among content providers
or products (especially their own) in the methods that are used to generate results, relevance rankings,
or link order.
2. In cases where the same content is available through multiple content platforms (such as a primary
publisher and a secondary database, or multiple secondary databases), discovery service providers
should provide mechanisms to enable libraries implementing the service to establish preferences
regarding which platforms to present to users as link targets, and in what order or priority.
3. Discovery service providers should issue a statement annually to all customers (or generically on their
website in an area available to all customers) explaining their business connections with content
providers, including those with direct or indirect ownership relationships and those with which they
have negotiated paid or other agreements for metadata deposit, direct linking, or other special
arrangements.
4. Discovery service providers should offer an affirmative statement of the neutrality of their algorithms
for generating result sets, relevance rankings, and link order with respect to ideological/political
viewpoint, content provider source (especially but not limited to any content that it or its parent
organization may provide), and any other relevant factors.
5. Discovery service providers should make other aspects of link presentation associated with a given
result (including the number of links presented, the order in which links are presented within a given
result, and how libraries’ “get the full text” link is labeled / branded) configurable options by
libraries.
6. Discovery service providers should offer a seamless link (not necessarily a full-text link) from the
discovery screen(s) to the A&I interface(s) utilized in the discovery service index, when such link can
be provided by the A&I provider. NISO RP-19-2014, Open Discovery Initiative
26
7. Discovery service providers should supply content providers and libraries with information when
material changes are made to the discovery service that could impact the result set or relevance
rankings or link order of results.
When requested by libraries, Content Providers can use the table to indicate their ODI compliance.
A ‘Y’ (for Yes) in column 1 indicates compliance with the indicated paragraph of this Recommended
Practice. A ‘P’ response indicates Partial compliance for which explanatory comments should be entered
in the last column. An ‘N’ (No) response indicates that the content provider does not comply with the
recommendation. Explanatory comments may be added for any response.
When requested by libraries, Discovery Service Providers can use the table below to indicate their ODI
compliance. A ‘Y’ (for Yes) in column 1 indicates compliance with the indicated paragraph of this
Recommended Practice. A ‘P’ response indicates Partial compliance for which explanatory comments
should be entered in the last column. An ‘N’ (No) response indicates that the content provider does not
comply with the recommendation. Explanatory comments may be added for any response.
In a open response to an open ended question in a Fall 2013 survey “What ONE change to library search tools (HOLLIS, HOLLIS Classic, E-Research@Harvard, and the Harvard Library web site) would have the biggest impact on your research?”, 46 users from a mix of disciplines and roles responded with single search across all materials.
In a open response to an open ended question in a Fall 2013 survey “What ONE change to library search tools (HOLLIS, HOLLIS Classic, E-Research@Harvard, and the Harvard Library web site) would have the biggest impact on your research?”, 46 users from a mix of disciplines and roles responded with single search across all materials.