1. Bibliometrics as Evidence
Paul Murphy, RCSI Library pjmurphy@rcsi.ie
A presentation to the Health Science Libraries Group Annual Conference, 11 April 2013

2. Royal College of Surgeons in
Ireland

3. Research evaluation contexts

4. Stakeholders and supports
Current Research
Information System
Research process
management
Reporting & analytics
Bibliometrics
Institutional
repositories

5. Royal College of Surgeons in
Ireland
• Medicine, Pharmacy,
Physiotherapy, Nursing,
Faculties
• Ireland, Bahrain, Malayasia
• 1,850 u/grads, 365 p/grads,
350 research staff
• 5,935 publications, Web of
Science, 1996-
• 535 publications in 2012 of
which 289 articles
Subject areas
• Medicine General Internal 786
• Psychiatry 537
• Pharmacology 515
• Surgery 473
• Neurosciences 398
• Biochemistry Molecular Biology
374
• Peripheral Vascular Disease 349
• Clinical Neurology 293
• Oncology 270
• Obstetrics Gynecology 225

6. Individual profiling
RCSI services
Initiative:
 position librarians as experts in bibliometrics
 establish Library as stakeholder in research management
 consolidate repository and publication advisory support
 Initiated targeted personal profiling for Principal Investigators,
senior academics and clinicians
 Demonstrated Departmental profiling for quality assurance,
programme accreditation, institutional review
 Promoted self profiling, publications management for tenure and
promotion purposes with repository and EndNote information
skills sessions
Initiative:
 position librarians as experts in bibliometrics
 establish Library as stakeholder in research management
 consolidate repository and publication advisory support

7. Individual profiling - tools
Citation databases
• Web of Knowledge
Citation Databases
• Conference Proceedings
Index
• InCites
• Scopus
• Google Scholar
( + Publish or Perish)
Window services
• Researcher ID
• ORCID
• ResearchGate
• Google Scholar

8. Individual profiling –
subscription databases

11. Link to repository

13. Google Scholar profile

14. Individual profiling tools:
Google Scholar
• Comparisons of citations in Web of Science, Scopus, and
Google Scholar for articles published in general medical
journals. JAMA. 2009 Sep 9;302(10):1092-6. doi: 10.1001/jama.2009.1307.
• Comparison of PubMed and Google Scholar literature searches.
Respir Care. 2010 May;55(5):578-83.
• Medical literature searches: a comparison of PubMed and
Google Scholar. Health Info Libr J. 2012 Sep;29(3):214-22. doi:
10.1111/j.1471-1842.2012.00992.x.

16. Individual profiling tools: Google
Scholar + Publish or Perish

17. Individual profiling tools: Google
Scholar + Publish or Perish

18. Individual profiling tools: Google
Scholar + Publish or Perish

19. MyRI
http://www.ndlr.ie/myri/

20. MyRI
http://www.ndlr.ie/myri/

21. Institutional profiling
Who? What? Where? When?
• Who is an RCSI researcher?
• Who was an RCSI researcher?
• Specify inclusion / exclusion
criteria
• Collaborator / co-authorship
guideline
• Reckonable sites
• Schedule of RCSI schools,
academic, clinical, research
centres
• Peer reviewed journal papers?
• Peer reviewed conference
paper?
• Meeting Abstract?
• Report or specialist publication?
• Review paper?
• Meeting abstract?
• Letter?
• Outputs quantified by specific
years?
• Which years?
• Whole of life output or RCSI
only output?

22. Analysis – the metrics
• Items published
• By funding agency
• By document type
• By author ranked
• By subject area ranked
• By journal title ranked
• By collaborating institutions /
co-author addresses
• By countries
• By Items cited
• By items not cited
• Rankings by internal centre or
location
• By H-index

23. The Evidence – by times cited
• Peripheral vascular disease 9,923
• Psychiatry 9,094
• Pharmacology & pharmacy 8,769
• Neurosciences 7,217
• Biochemistry & molecular biol
5,888
• Cardiac & cardiovascular 5,481
• Hematology 4,273
• Genetics & heredity 2,813
• Oncology 2,757
• Surgery 2,656
• Clinical neurology 2,551
• Cell biology 2,508
• Microbiology 2,210
• Immunology 1,844
• Public, environmental &
occupational health 1,811
• Endocrinology & metabolism1,753
• Biochem research methods 1,587
• Nutrition & dietetics 1,557
• Medicine, general & internal 1,551
• Obstetrics & gynecology 1,445
• Pathology 1,441
• Infectious diseases 1,389
• Biophysics 1,321
• Psychology 1,250
• Respiratory system 1,127

24. Sourcing & quantification
Sources
• Research database – Current
Research Information System
• PubMed
• Embase
• Web of Science Citation
Indexes
• PsychInfo
• CINAHL
• Agency publisher: DoH, WHO
• Institutional repositories
Items per source
Validation and normalisation
• Personal name variants
• Institutional name variants
Processing: time, resources
• Who?
• What? EndNote, CRIS?

25. Retrieval and processing
RCSI affiliated
Source Time Results: all
items, all types
All items, year
2010
Web of Science 1945- 4,879 492
PubMed 1950s- 2,034 254
Embase and Medline combined 1974- 3,904 538
CINAHL 1937- 265 53
Records to validate and de-duplicate 11,082 1,337
Estimated records annually from 2010- annual 350

27. http://www.webometrics.info/

29. SCImago - W Eur - Medicine

31. Institutional metrics
• Spend on teaching
• Graduation time/ rates
• Progression / success
rates
• Student:staff ratio
• Student profile
• Student satisfaction
• Programme
development
• Spend on research
• Research income
• Publication output &
impact
• Industry linkage
• Doctoral productivity
• Knowledge exchange
• Licences / patents
• International staff

32. 2007
http://www.hea.ie/files/files/file/ResearchStrengthsInIreland(2).pdf
http://www.hrb.ie/health-information-in-house-research/disability/ddu-
publications/ddu-publication/publications//278/

33. http://www.hea.ie/files/files/file/ResearchStrengthsInIreland(2).pdf p82

34. http://www.hea.ie/files/files/file/ResearchStrengthsInIreland(2).pdf p83

35. Institutional profiling -
comparative
• Using the TR database Essential Science Indicators ESI
• ESI uses different subject categories to Web of Science
• Only valid comparator subject category = “Clinical Medicine”

36. Essential Science Indicators
2002-2012
Rank from 3,739
in Clinical
Medicine
Institution name in database Papers Citations Citations per
paper
355 TRINITY COLL DUBLIN 2,731 42,479 15.55
366 UNIV COLL DUBLIN 2,699 41,320 15.31
658 NATL UNIV IRELAND UNIV COLL CORK 1,428 18,985 13.29
779 ROYAL COLL SURGEONS IRELAND 816 14,868 18.22
1,241 NATL UNIV IRELAND GALWAY 594 7,716 12.99
2,267 DUBLIN CITY UNIV 272 3,133 11.52
3,367 UNIV LIMERICK 274 1,827 6.67
Harvard Johns Hopkins Mayo Clin U Calif SF
U Washington U Calif LA U Michigan U Toronto U Penn

37. Institutional profiling –
comparative and analytic
National Reseach Data Project
• Co-ordinated by Director of Research, Irish Universities
Association
• Higher Education Authority funded to 2009 -2012 now
institutionally subscribed
• UCC UL UCD NUIG NUIM TCD DIT RCSI
• Creating a national citation dataset 1980-
• TR InCites engine
• Data quality: normalization and validation
• Analytics to informing national research prioritization exercises
held by Higher Education Authority, Science Foundation Ireland,
Office of Science, Technology and Innovation

38. National Research Data Project
Outcomes and targets
1. To employ locally the resources and metrics which are used to
evaluate our universities internationally
2. To provide a reliable source of quality-controlled hard data upon
which to base our research analyses
3. To map and analyse collaborations (based on author affiliations
and addresses associated with individual publications)
4. To produce ready data for research audits and research
evaluations to support the emerging cross-institutional research
and graduate school collaborations

39. National Research Data Project
National Citation Report Ireland dataset at Feb 2013
Contains 201,471 source articles from 209,842 authors published
between 1979 and 2013.
Also contains 1,487,103 articles that have cited the 201,471
source articles in this dataset.

40. InCites

41. InCites
http://researchanalytics.thomsonreuters.com/incites/

42. RCSI InCites NRDP

46. Global comparison
Ireland - Medicine

47. InCites Comparison
Ireland and EU 15 Medicine General

49. Altmetrics
• Altmetrics (alternate or alternative metrics) is
the study of newer web-based metrics in
scholarship that look at total influence of
someone via social media.
• Priem J, Taraborelli D, Groth P, Neylon C. Alt-metrics: a
manifesto. 26 October 2010. http://altmetrics.org/manifesto
Altmetric App for Scopus
http://hlwiki.slais.ubc.ca/index.php/Altmetrics
Toward a basic framework for webometrics
L Björneborn, P Ingwersen – JASIST, 2004
We define webometrics within the framework of informetric
studies and bibliometrics as belonging to library and
information science.
Cited by 224 Related articles

50. Conclusions
• Librarians established as research support stakeholders
• Librarians established as experts on bibliometrics
• Bibliometrics, especially citation distribution, are validated
measures of research evaluation and performance
• Bibliometrics to be reported and incorporated into Current
Research Information System
• Bibliometrics an element in comparative and benchmarking
assessments using multiple dimensions: geographic, research
front, expected relative performance indicators.
• Subscription products are the core tools but free resources and
alternative resources merit further investigation

51. Bibliometrics as Evidence
Paul Murphy, RCSI Library pjmurphy@rcsi.ie
A presentation to the Health Science Libraries Group Annual Conference, 11 April 2013