June 18 NISO Virtual Conference: Transforming Assessment: Alternative Metrics and Other Trends Keynote Speaker: Altmetrics at the Portfolio Level - Paul Groth, Ph.D., Assistant Professor at the VU University Amsterdam

1. Paul Groth @pgroth
Web & Media Group
Department of Computer Science
The Network Institute
VU University Amsterdam
http://www.few.vu.nl/~pgroth
Altmetrics at the portfolio level

2. NIH Data Book – (http://report.nih.gov/ndb/index.aspx) Data provided by the Division of Information Services, Reporting Branch
Research Project Grants
Applications, awards, and success rates

3. Metrics: Do metrics matter?
Nature 2010
http://doi.org/10.1038/465860a
"Outside letters basically trump
everything," says Robert Simoni,
chairman of the biology
department at Stanford University
in California.

5. “Imagine how the academic appointment process
might change if search and review committees had
access—within an appropriately tagged or linked
online CV, for example, or via the ORCID system—
to information about the specific contributions
made by a candidate to each of his/her works,
including contributions that might not otherwise have
qualified for ‘authorship’ status?”
Point of view: Faculty appointments and the record of
scholarship
Amy Brand
http://dx.doi.org/10.7554/eLife.00452

6. Point of view: Faculty appointments and the record of scholarship
Amy Brand
http://dx.doi.org/10.7554/eLife.00452
Slide 3: http://article-level-metrics.plos.org/files/2013/10/Brand.pptx
Opportunities
• Individuals and institutions need better tools
for curating and networking their own record
of scholarship
• Institutions need more information about
scholarly contribution
• ALMs that reliably differentiate sources of
input (general; academic; expert; etc.) would
be more useful”

7. ENTER ALTMETRICS

8. Altmetrics is the study and use of scholarly
impact measures based on activity in online
tools and environments.
http://doi.org/10.1371/journal.pone.0048753

9. http://blog.peerj.com/post/65345738206/changing-the-currency-of-science-to-solve-our-greatest

10. Thanks Ian Mulvany

13. ALTMETRICS AS MEASURES
OF IMPACT?

14. “It took approximately a generation (20
years) for bibliographic citation analysis to
achieve acceptability as a measure of
academic impact."
(Vaughan and Shaw,
2003)

15. The Research is Happening
http://jasonpriem.org/self-archived/5uni-poster.png
(Birkholtz et al. 2013)
(Fausto et al 2012)
http://ploscollections.org/altmetrics
http://asis.org/Bulletin/Apr-13/AprMay13_Piwowar.html

16. http://www.cwts.nl/pdf/CWTS-WP-2013-003.pdf

17. Altmetrics 2014
Zahedi, Zohreh; Fenner, Martin; Costas, Rodrigo (2014): How
consistent are altmetrics providers? Study of 1000 PLOS ONE
publications using the PLOS ALM, Mendeley and
Altmetric.com APIs.
figshare. http://dx.doi.org/10.6084/m9.figshare.1041821

18. http://www.slideshare.net/paulwouters1/issi2013-wg-pw

19. Bottom Line: use altmetrics as
evidence in a larger story

20. WHAT’S THE LARGER STORY?

23. DIFFERENT AGGREGATIONS

24. Aggregations
• Institutional Level
– See the next talk
• Faculty
• Department
• Group

25. Aggregations
• Institutional Level
– See the next talk
• Faculty
• Department
• Group
• Most interesting: adhoc Groups

26. Projects
• Typical lifetime 2-5
years
• Networks of sub-groups
• Networks of projects
• Different measures of
success
Snia Bpd Fiat Group
http://cordis.rkbexplorer.com/id/
a Valenc
ization-fc758
LASER ZENTRUM HANNOVER E.V.
http://cordis.rkbexplorer.com/id/organization-9c08bc3c346074
GEMETEC Aachen Gesellschaft f¸
http://cordis.rkbexplorer.com/id/organization-54f41a7353577
ELLINOGERMANIKI AGOGI S.A.
http://cordis.rkbexplorer.com/id/organization-1fa32c75c
Aalborg University
http://cordis.rkbexplorer.com/id/
FRAUNHOFER GESELLSCHAFT ZUR FO
http://cordis.rkbexplorer.com/id/organization-c4bab301e9e31a63cdebaed13af6c19

29. 29
Published AND discussed AND cited

30. 30

31. 31

32. 32

33. Events
• Interval lifetime
• Changing networks
• Virtual teams
• Affiliation is transient
• Aggregates
• Central to scholarly communication

39. Summary: a broader view
• Almetrics gives a broader view
• Altmetrics is must be situated in context
• We should begin looking at the portfolio
level
– Multiple aggregations
– What cannot be measured

40. Challenge: Architecture

41. Thanks
Collaborators: Peter van den Besselaar, Julie
Birkholz, Frank van Harmelen, Shenghui Wang,
Rinke Hoekstra, Thomas Gurney, Mike Taylor,
Anita de Waard, Jason Priem, Dario Taraborelli,
Cameron Neylon, Ian Mulvany