Presented at http://scig.acrlnec.org/content/november-2013-program-altmetrics-help-your-researchers-measure-their-full-impact

1. NISO’s Altmetrics Initiative
Nettie Lagace (@abugseye)
ACRL New England
Scholarly Communication Interest Group
November 14, 2013

2. What’s NISO?
• Non-profit industry trade association accredited by ANSI
with 150+ members
• Mission of developing and maintaining standards related to
information, documentation, discovery and distribution of
published materials and media
• Represent US interests to ISO TC46 (Information and
Documentation) and also serve as Secretariat for ISO
TC46/SC 9 (Identification and Description)
• Responsible for standards like ISSN, DOI, Dublin Core
metadata, DAISY digital talking books, OpenURL, SIP, NCIP,
MARC records and ISBN (indirectly)
• Volunteer driven organization: 400+ spread out across the
world

3. Premise of “Standards”
• Consensus standards created by a community with
various stakeholders
• Trust
• Leading to broader acceptance
• Standards as plumbing
• Standards facilitate trade, commerce and innovation
• Standards reduce costs
• Standards support better communication and
interoperability across systems

4. http://www.flickr.com/photos/72143877@N00/5090281853

5. The Altmetrics Project
• Came out of breakout session at (ACM)
altmetrics12 meeting in Chicago, June 2012
• Funded by $207,500 grant from Alfred P. Sloan
Foundation
• Premise of grant award: To help facilitate the
adoption of altmetrics, development of
community consensus standards can help
address limitations and gaps that currently exist
• Identify and prioritize needs and requirements

6. Why worth funding?
• Scholarly assessment is critical to the overall
process
– Which projects get funded
– Who gets promoted and tenure
– Which publications are prominent
• Assessment has been based on citation since the
60s
• Today’s scholars multiple types of interactions
with scholarly content are not reflected
– Is “non-traditional” scholarly output important too?

7. Why worth funding?
• In order to move out of “pilot” and “proof-of-
concept” phases …
• Altmetrics must coalesce around commonly
understood definitions, calculations and data
sharing practices
• Altmetrics must be able to be audited
• Organizations who want to apply metrics will
need to understand them and ensure consistent
application and meaning across the community

8. What’s to Standardize?
• Field is too new … no existing practice to build
on; standards should not be “invented” /
codified – a diversion from the real work that
needs to be done?
– No community practice: administrators, granting
orgs etc. need accepted, trustworthy, verifiable
info
– This project is not taking place overnight; two
stages for a reason
– NISO open process invites engagement from all

9. A few Questions about Altmetrics
• What gets measured (counted)?
• What are criteria for assessing quality of these
measures?
• How granular should the metrics be to enable
computation and analysis?
• What period(s) should the metrics cover?
• What technical infrastructure is necessary to
enable the exchange of the metrics data?

10. http://www.flickr.com/photos/93352468@N00/4053127701

11. 2 Phases
• Phase 1: Hold meetings of stakeholders to define a
high-level list of issues
– October 2013, San Francisco
– December 2013, Washington, DC
– January 2014, Philadelphia
– Public Webinars
– White paper output, public presentations, public feedback
• Phase 2: Create Working Group within NISO structure,
to create recommended practice(s) and/or standard(s)
– Education/training efforts to ensure implementation
• Final report to Sloan due November 2015

12. Steering Committee
• Euan Adie, Altmetric
• Amy Brand, Harvard University
• Mike Buschman, Plum Analytics
• Todd Carpenter, NISO
• Martin Fenner, Public Library of Science (PLoS) (Chair)
• Michael Habib, Reed Elsevier
• Gregg Gordon, Social Science Research Network (SSRN)
• William Gunn, Mendeley
• Nettie Lagace, NISO
• Jamie Liu, American Chemical Society (ACS)
• Heather Piwowar, ImpactStory
• John Sack, HighWire Press
• Peter Shepherd, Project Counter
• Christine Stohn, Ex Libris
• Greg Tananbaum, SPARC (Scholarly Publishing & Academic Resources Coalition)

13. Meeting #1 – San Francisco
• Held in conjuction with PLOS ALM Meeting
• Livestreamed (video recordings now available)
• Morning: lightning talks, post-it brainstorming
• Afternoon: discussion groups
– Business and use cases
– Quality and data science
– Definitions

14. Meeting #1 – General outputs
• The importance of best practices for media
coverage of science (using DOIs, etc.)
• More Altmetrics research is needed and could
be promoted through this group
• Providing a standard set of research outputs
that we can use to compare different services
• The importance of use cases for specific
stakeholder groups in driving the discussion
forward

15. Business & Use Case Breakout
• What people are doing and why
• Publishers want to serve authors, make money
• People don’t value a standard, they value something that helps
them
• Researchers have noticed differences between different altmetric
services
• How can altmetrics be used in collection development
• How do altmetrics interact with traditional metrics
• Transparency: people want to know where the data comes from
(gaming)
• Libraries, research funders, institutions want trust
• Fear of altmetrics eliminating journal brand
• … Couldn’t identify a logical standard need that actors in the space
would value, and best practices are of interest

16. Quality & Data Science Breakout
• Themes: context, validation, provenance, quality,
description & metadata
• What’s out there and what does it mean?
– Studies of the subject; find correlations (what do positive
correlations/lack of correlations mean?)
• Implicit and derived metrics
• Discussion of experiments for Saturday hackfest
• Need to balance signal/noise – want to have metrics
accepted, and accepted soon
• Regional views – Harvard doesn’t weight citations more
than their judgment; China does
• We'll never get to the point where assessment can be done
without a human in the loop, but discovery and
recommendation can

17. Definitions Breakout
• Define “ALM” and “Altmetrics”
• Map the landscape
• Disseminate the glossary and definitions
• Simplifying and aggregating altmetrics
• Define axes for measuring altmetrics
• Create gold standard data set
• Promote research
• Who gets a seat in standards building?

18. Upcoming Meetings
• December 11, Capital Hilton, Washington DC
(with CNI)
• January 23, Site TBD, Philadelphia (with ALA)
• Open meetings, RSVP on NISO site
– Livestreaming both meetings

19. ANSI Standards
• Openness, lack of dominance, balance =
everyone affected has a voice
• Notifications, consideration of views and
objections
• Consensus votes and appeals
• ANSI patent policy, commercial terms and
conditions
• Publication and maintenance
• Credible & have Integrity

20. Thank you! Questions?
nlagace@niso.org
@abugseye