This document discusses new ways of measuring research impact beyond traditional citations. It describes how Mendeley collects data on researcher behavior directly from their platform to provide faster and more comprehensive metrics on researcher engagement. This includes data on document views, saves, annotations and more. It also discusses how this broader dataset could enable new services for stakeholders to better understand research impact and discovery.

1. Redefining research impact
William Gunn
Head of Academic Outreach, Mendeley
@mrgunn
https://orcid.org/0000-0002-3555-2054

3. Citations are too slow

4. Google Analytics for research

5. New data on research impact
Get better data on researcher engagement with research
Get it faster
Serve all the stakeholders in research

6. ...and aggregates research data in the cloud
Mendeley extracts research data…
Mendeley is not just a reference manager!
Install Mendeley Desktop
Collecting rich signals
from domain experts.

7. Rich user profile data

8. Instrumenting the research workflow
•2.6 Million users
•470 M documents
•4-700K uploads per day
•90% coverage of Pubmed
•Accessible alternative to citations

9. Instrumenting the Research Workflow
Search
Read
Annotate
Organize
Write
Import

10. http://arxiv.org/html/1203.4745v1
altmetrics show broader impact

11. http://www.asis.org/Bulletin/Apr-13/AprMay13_Lin_Fenner.html
altmetrics show broader impact

12. What would people build if they could get the data?
•Impact Story – get credit for all your work
•PLOS ALM – article-level metrics for papers
•Plum Analytics – bespoke analytics for libraries (EBSCO)
•Altmetric.com – altmetrics for publishers. (Digital Science)

13. New data on research impact
Get better data on researcher engagement with research
Get it faster
Serve all the stakeholders in research

14. Personalized Impact

15. New forms of discovery
•Mendeley Suggest
–personalized recommendations based on reading history
•related articles
–relatedness based on document similarity
•third-party platforms
–PubChase, ScienceScape, myScienceWork, Docphin

16. Secondary literature
Off-line
•Correspondence to journals
•peer review
Online
•peer review
•comments at publishers website (PLOS, PeerJ)
•third-party platforms
–Pubmed Commons, Publons, Stack Exchange, Quora, blog posts

17. Secondary publication

18. Secondary publication

19. How authors use Pubmed Commons
•Went public on Dec 19.
•~700 comments
•1 in 5 are from authors
•Commenters must have a paper in Pubmed to comment, but they’re publicly viewable
•Comments can be cited by PMID
http://pubmedcommonsblog.ncbi.nlm.nih.gov/2013/12/13/expanding-and-updating-the-record-authors-using-pubmed- commons/

20. •50% of comments
–add study data
–link to full free text
–give updates related specifically to that publication
–link to newer publications
•10% used it to add errata or notify readers that the conclusions no longer hold
•only 2 used it to talk about how great their own paper is
How authors use Pubmed Commons

21. How authors use Pubmed Commons

22. Issues To Be Addressed
•Identity
•Privacy
•Attribution
•Gaming
•Filtration

23. Identity
•Mendeley - http://www.mendeley.com/profiles/jonathan-eisen/
•Scopus - http://www.scopus.com/authid/detail.url?authorId=35247902700
•Pubmed - http://www.ncbi.nlm.nih.gov/myncbi/jonathan.eisen@gmail.com/comments/
•ORCID - http://orcid.org/0000-0002-0159-2197

24. Building a reproducibility dataset
•Mendeley and Science Exchange have started the Reproducibility Initiative
•working with Figshare & PLOS to host data & replication reports
•building open datasets backing high-impact work
•http://cos.io/cancerbiology

25. www.mendeley.com
william.gunn@mendeley.com @mrgunn