The document discusses a conference on open peer review, innovative dissemination, and alternative metrics. It summarizes the objectives to explore the current landscape of altmetrics indicators, generate a taxonomy of scientific knowledge dissemination channels and altmetrics indicators, and suggest indicators to assess research impact and quality. It provides results of the conference including a bibliometric analysis of altmetrics research and a final taxonomy of linkages between dissemination channels based on expert interviews. It discusses challenges like normalization issues in altmetrics and outlines an outlook for making altmetrics more entrenched in the current value system of science.
2. 2018-09-12 2
Objectives
• Explore the current landscape of altmetrics and open
metrics, including a systematic overview of current
indicators
• generate a validated taxonomy of channels of scientific
knowledge dissemination and altmetrics indicators
• … and suggest indicators enabling assessing impact
and quality of the underlying research.
5. 2018-09-12 5
State-of-play
• Altmetrics is extremely heterogeneous
• Scrutinization attempts are (still) “en vogue”
• There are some crucial challenges to solve
• Normalization issues - in an open universe (vs. closed universe
of bibliographic databases)
• Coverage (Differences between data providers)
• Conceptual scrutinization and construct validity (Impact Flavor
debate, taxonomic attempts)
6. 2018-09-12 6
Results
• Bibliometric analysis of altmetrics-related research.
• Implementation of tracking infrastructure for altmetrics-related Twitter activities (Corpus of
more than 54k Twitter messages).
• Desk research to identify current trends and state of the art in altmetrics, condensing a
total of 416 publications.
• Desk research on research impact current situation, issues, and opportunities >SWOT
analysis based on the state of the art of altmetrics
• In-depth analysis of survey data to inform taxonomy development to assess the overall role
and relevance of altmetrics (Inform, Disseminate, Appreciate, Impact)
• Final taxonomy of linkages between altmetrics and innovative dissemination, conducting
more than 20 expert interviews
9. 10
D5.4: Example of taxonomy table
Dimension: Doing Science and
Science made public
Service 1
Blogging
Service 2
twitter
Service 3
Faceboo
k
Service 4
RG
Attention (expert interviews, desk
research)
Medium High High Medium
General scholarly appreciation
(Survey data)
Medium Low Very low Medium
Scholarly relevance (literature
review, expert interviews)
Yes No No Yes
Field specific appreciation (expert
interviews)
Yes No No Yes
Metrics (desk research) Citations
Downloads
Views
Mentions
Followers
Mentions
Views
Likes
Activities
within the
network
Provider coverage (desk research) Altmetric,
Plum
Altmetric,
Plum, PloS,
Impactstory
Altmetric
., Plum
None
Technical Accessibility (API) Partly Yes (but
restricted)
Yes (but
restricte
d)
No
Intergenerational differences in
assessment (survey)
High Medium Low NA
10. 2018-09-12 11
OpenUP Hub
• Overview of main content in OpenUP hub for this
area:
• Altmetrics providers (Platforms, which collect data from
different sources & produce Altmetrics)
• Altmetrics features (SWOT analysis, Data sources & the
Taxonomy of Altmetrics)
• Training videos: Current issues in Altmetrics
• …blog posts on Altmetrics
11. 2018-09-12 12
Outlook
• Altmetrics is a booming, but heterogeneous field
• Potential for assessing novel forms of dissemination and accounting for
dynamics within science
• But: so far not entrenched within the current value system of science
What’s likely to be relevant in the future?
• Content-based perspective (What do we count? Who engages? With what?
How? Why?)
• Reflexive perspective (How do altmetrics feed back into science?)
• Understanding gaming effects (Offerbots, (Re-)tweet farms, “social media
suppliers”)
13. Backup
1) "Which of these channels listed do you use most
frequently to disseminate your own research to reach your
target groups?" (Disseminate)
2) "How frequently do you use these sources to inform your
professional work as a researcher?" (Inform)
3) "Please rate how being represented in these dissemination
activities is generally appreciated within your field of
research" (Appreciate)
4) "What potential do the following dissemination channels
have to lead to a wider societal impact?" (Society)
For each question, a battery of 19 dissemination channels
was assessed on a 5-item end-point-scaled Likert scale.
Editor's Notes
Can these be compared in a useful manner?
Information behaviour
Reward system
Society
Asked respondents...
Darker the higher the share of respondents ticking „high“ or „very high“
Gutenberg press vs. social meda
Types of similar patterns of how inf, reward and soc resonance and disonance with each other. Within the higher they more similar the resonance
Substitution
Traditional academia channels (Academic Publishing, Conferences)
Traditional PUSH Channels (Press, Radio, TV, Popular Science Books, Public Events)
Non-traditional Science 2.0 PUSH channels (Art, Wikis, Generic Social Networks, Podcasts & Videos)
Progress Channels (Blogs, Project Websites, Newsletters, Print Media)
Sharing Channels (Open Lab Books, OA Repositories, Academic Social Networks, Code Repositories)