An introduction to altmetrics, the complementary metrics of research impact. The presentation covers some of the challenges with more traditional measures, and the potential of and challenges with altmetrics. The presentation gives a brief overview of the background to a new research project about measuring the societal impact of open science.
5. Goodhart’s Law:
When a measure becomes
the target, it ceases to be
a good measure
Goodhart, 1975
6. Outreach and Impact Leader, - Ref:1455401
“ We are seeking to recruit … to support, develop
and promote outreach and impact activities …
Our aim is to maximize the widest possible
impact of the research programme in whichever
way is most appropriate …
The salary range will be £41,430 - £48,873 per
annum...”
Taking impact seriously
9. 0 2 4 6 8 10 12 14 16 18 20
Impact factor window
Immediacy index window
Cited half-life
50% of
citations
50% of
citations
Time after publication (Years)
Citations
10. 0 2 4 6 8 10 12 14 16 18 20
Time after publication (Years)
Citations
Time of research assessment
Option 1: Journal impact factor, Julkaisufoorumi, …
Potential
future
impact
11. 0 2 4 6 8 10 12 14 16 18 20
Time after publication (Years)
Citations
Time of research assessment
Option 2: Citations (after n years)
Past
success and
impact
12. 0 2 4 6 8 10 12 14 16 18 20
Time after publication (Years)
Citations How can we measure
impact in realtime?
16. Fenner, M. & Lin, J. (2013). Altmetrics in Evolution: Defining & Redefining the Ontology of Article-Level Metrics.
http://dx.doi.org/10.3789/isqv25no2.2013.04
2
32. Altmetrics
the study and use of scholarly
impact measures based on activity
in online tools and environments
Priem, J. (2014). Altmetrics. In Cronin, B. & Sugimoto, C.R.
(Eds.). Beyond Bibliometrics: Harnessing Multidimensional
Indicators of Scholarly Impact. MIT Press, 2014.
40. Rank Title Altmetric score News Blogs Tweets
1 Experimental evidence of massive-scale emotional
contagion through social networks
5044 301 130 3801
2 Variation in Melanism and Female Preference in Proximate
but Ecologically Distinct Environments
4823 4 11 6669
3 Artificial sweeteners induce glucose intolerance by
altering the gut microbiota
3500 127 34 3303
4 Stimulus-triggered fate conversion of somatic cells into
pluripotency
2986 159 92 3295
5 Dogs are sensitive to small variations of the Earth's
magnetic field
2956 54 33 2668
6 Christmas 2013: Research: The survival time of chocolates
on hospital wards: covert observational study
2734 3 4 3084
7 Epidemiological modeling of online social network
dynamics
2392 156 19 1649
8 Searching the Internet for evidence of time travelers 2246 34 22 2262
9 Conscious Brain-to-Brain Communication in Humans Using
Non-Invasive Technologies
2160 82 32 1660
10 Were James Bond's drinks shaken because of alcohol
induced tremor?
2146 127 14 1559
Top 10 articles according to the Altmetric Score (Source: http://www.altmetric.com/top100/2014/)
41. 0
50
100
150
200
250
300
350
0
1000
2000
3000
4000
5000
6000
7000
8000
1 2 3 4 5 6 7 8 9 10
Altmetric score
Tweets
News
Blogs
The volume of tweets, blog posts, and news stories for the top ten research articles
from 2014 as measured with altmetrics, and their Altmetric Score (number of news
stories and blog posts are depicted on the secondary axis to the right in the figure)
45. Citations Tweets
Older publications Newer publications
The effects of time on the volume of citations and tweets
(Haustein, Peters, Sugimoto, Thelwall, & Larivière, 2014)
The effects of time
48. Bollen J, Van de Sompel H, Hagberg A, Bettencourt L, Chute R, Rodriguez MA, et al. (2009) Clickstream
Data Yields High-Resolution Maps of Science. PLoS ONE 4(3): e4803. doi:10.1371/journal.pone.0004803
Disciplinary differences
49. Bots?
Haustein, S., Bowman, T.D., Holmberg, K., Tsou, A., Sugimoto, C.R. & Lariviére, V. (in press). Tweets as impact indicators: Examining the implications of
automated “bot” accounts on Twitter. To appear in JASIST. arXiv version available at: http://arxiv.org/ftp/arxiv/papers/1410/1410.4139.pdf.
Gaming altmetrics?
52. Measuringthe Societal Impact of Open Science
AvoimenTieteenYhteiskunnallisenVaikuttavuudenMittarit
Project is financed by the Ministry of Education and Culture in Finland (2015-2016)
53. Goals of the research project:
1) Investigate altmetrics (at country-level, i.e.
Finland) and create a prototype to
demonstrate them
2) Develop methods to map societal impact of
research (and to analyze the stories behind
the altmetrics)