Data-Drive Solutions to Support your Research Enterprise from the 2016 AAUDE Conference.

1. Data-Driven Solutions to Support
Your Research Enterprise
Ann Beynon
THOMSON REUTERS
Simon Pratt
UNIVERSITY OF TORONTO
AAUDE conference
April 2016

2. THE WEB OF SCIENCE CORE COLLECTION-
OUR FOUNDATION FOR ANALYTICS
• High-quality, peer-reviewed content; independent, unbiased selection
• 61 million records: the largest and most consistent citation database, with
1 billion cited references, back to 1900, all searchable
• Multidisciplinary: Journals (12,760), Conferences (161,000), and Books (70,000)
indexed in the Sciences, Social Sciences, and Arts & Humanities
• Comprehensive metadata due to our cover-to-cover indexing, all
authors/addresses, funding acknowledgements, editorial policies and oversight
• Used by 7,000+ institutions globally, including major research institutions,
governments, international rankings…and all AAU universities and AAU
administration
• Emerging Sources Citation Index (new)

3. InCites-
analyze and benchmark your research productivity and
impact

4. INCITES-bibliometric analysis of people, organizations, and funding agencies
Users:
• Institutional
Research
• Faculty Affairs
• Library
• Office of
Research
• Provost
• President

5. 4
—
INCITES

6. 5
INCITES BENCHMARKING AND ANALYTICS
5
Web of Science Core Collection data – normalized for
analytics
• Multidisciplinary content, indexed metadata, and true citation indexing
• Science Citation Index Expanded, Social Science Citation Index, and
Arts & Humanities Citation Index (12,760 high impact scholarly journals)
• Conference Proceedings Citation Index
• Book Citation Index
• Normalized data and benchmarks updated bimonthly
• Unified institutions (5,900 +); Unified funders (700)
• 1980 to present
• All document types (articles, reviews, editorials, book chapters, etc.)
• Subject area schema for comparing:
Web of Science, Essential Science Indicators and 8 regional schemes
• Save to InCites – import Web of Science results sets of up to 50,000 records for analysis!

7. InCites: Normalized citation data
6
Busine
ss Oncology
Substance Abuse
Respiratory System
Biolog
yLaw
Agronomy
CATEGORY
citation patterns differ by
subject category
e.g. nanotechnology vs
law
Review Article Editorial
DOCUMENT TYPE
citations differ by document
type within a journal
e.g. reviews are generally
more heavily cited than
articles, and editorials, book
reviews etc. may go uncited
NORMALIZATION PUTS DATA INTO CONTEXT— IS AN ENTITY DOING BETTER OR WORSE
THAN WOULD BE EXPECTED IN A JOURNAL OR CATEGORY?
TIME
citations accumulate over
time and at different rates
depending on article age
and category
e.g. new articles may
accumulate citations quickly,
older ones more slowly or
not at all
2015
timescited
1990

8. Is 20 citations good or bad?
Article Article Article
Article Article Article
Article Article Article
Article Article Article
Article Article Article
also ORGANIZATIONS, COUNTRIES, RESEARCH AREAS
NORMALIZED
RATIO OF ACTUAL / EXPECTED CITATIONS
EXPECTED
NUMBER OF
CITATIONS FOR
ARTICLE SET*
AVERAGE CALCULATED FROM
SUM OF ALL DOCUMENTS IN
THE SELECTED GROUP
20
2 32
5
0
189 2
1218 1
1
87 23
158 Total Cites in group
15 documents in group
10.5 Expected cites
Article 12
Article
12 Total
10.5 Expected
1.14 Normalized
8 People
8 Total
10.5 Expected
0.76 Normalized
*for journal/category, publication year, and document type

9. 8
INSTITUTIONAL BENCHMARKING
• AAU top ten
• 2006-2015
• # Web of Science Core
Collection documents

10. 9
INSTITUTIONAL BENCHMARKING
• AAU top ten
• 2006-2015
• Document trend

11. 10
INSTITUTIONAL BENCHMARKING
• AAU top ten
• 2006-2015
• Category
normalized
citation impact

12. 11
FACULTY EVALUATION AND RECRUITMENT
• University of Toronto
authors
• Use radar charts to
compare multiple
indicators

13. 12
COLLABORATION ANALYSIS
• University of Toronto
• Government and
corporate
collaborators
• Ranked by #
documents
• Indicator shown is
category normalized
citation impact

14. 13
IMPACT OF FUNDED RESEARCH
• University of Toronto
• Documents funded by
NSERC
• Ranked by %
documents in top 1%
of category
• Web of Science
categories

15. 14
IMPACT IN A TOPIC- GOLD NANOPARTICLES
• AAU top 5
• Ranked by % hot
papers (published in
last two years; high
citations in last two
months)

16. Customized reports-
support your strategic decisions with expert consultancy

17. CUSTOMIZED REPORTS
• Compare your research activity to peers
• Identify topical strengths/weaknesses/opportunities/threats
• Recommendations on improving rankings or meeting research goals

18. • How much research is taking place and in what field(s) at my institution?
• Where is this research being published?
• What are the disciplinary strengths and opportunities in this body of research?
• How does our publication output and impact compare against competitors?
• Who most commonly funds this disciplinary research?
CUSTOMIZED REPORTS- Bibliometric Performance Report
Bibliometric overview of your university vs. select peers

19. CUSTOMIZED REPORTS- Emerging Research Areas Report
Topical analysis of your strengths/weaknesses/opportunities/threats
• In what research areas do we have strengths, opportunities, weakness, or threats?
• What are the emerging research areas relevant for my portfolio? What are the global trends
in these areas?
• Who at my institutions performs research in these areas?
• Who is funding this research?

20. Tailored reports-
Answer your specific questions and provide recommendations on improving rankings or
meeting research goals
GLOBAL INSTITUTIONAL PROFILES DATA
Publication output share by year from US
Top 20 Journals by Impact Factor
BIBLIOMETRIC DATA

21. Converis-
manage faculty data and reporting efficiently

22. CONVERIS – a faculty information system
Growing client base – 100+ globally

23. • Converis is designed for easy integration of existing data
• Example of systems for integration:
– Login servers (e.g. Shibboleth, LDAP, Kerberos, CAS, often with SSO)
– HR systems (e.g. Resource Link, SAP-HR, PeopleSoft)
– Finance systems (e.g. SAP, PeopleSoft, Agresso, Oracle, Raindance,
MACH)
– Student record systems (e.g. BANNER, PeopleSoft CS, SITS, Ladok, AIS)
– Grants systems (e.g. Cayuse, Huron, Kuali, InfoEd, etc.)
– Institutional repositories (e.g. DSpace, EPrints, Fedora, Digital Commons)
– External publication databases (e.g. Web of Science, InCites, PubMed,
Scopus, ORCID, etc.)
– Build and maintain researcher profiles for research networking like VIVO,
Harvard Profiles, Loki, etc.
CONVERIS
INTEGRATION OF EXISTING SYSTEMS

24. CONVERIS – A FACULTY ACTIVITY REPORTING SYSTEM
Converis has a comprehensive data model, covering activities across all departments:
You can also add your own activity types
FACULTY ACTIVITIES
Publication Type Type of Activity Type of Creative Work Type of Teaching Activity Type of Service Type of Project
Annotation Award Artwork Advising Clinical Service Charity Funded
Book Chapter Lecture Audio/Visual Work Continuing Education Clinical Trial Clinical Trial
Confererence
Proceeding
Poster Musical Performance Graduate Editor/Reviewer Contract
Research
Journal Article Presentation Software Multimedia Mentoring Professional
Societies
Fellowship
Journal Review Professional
Membership
Other Creative Work Professional Student Group
Advising
(NEA) Project
Report Seminar Undergraduate Univ Committee (NIH) Project
Thesis Speech (NSF) Project

25. We can add your
specific faculty
review templates.
Configure the approval
workflow for your local
faculty management
needs.
-promotion and tenure
-annual review
-sabbatical
CONVERIS – A FACULTY ACTIVITY REPORTING SYSTEM

26. Data integration-
integrate trusted Web of Science and InCites data into
university systems

27. 26
DATA INTEGRATION- via APIs or data delivery
• Multidisciplinary
• Cover-to-cover indexing
• Funding
acknowledgements
• Citations back to 1900
• Unbiased selection
• Normalized bibliometric
indicators back to 1980
• Collaboration indicators

28. 27
DATA INTEGRATION- via APIs or data delivery
Example: populating publication and citation data into a research
portal (VIVO)

29. 28
DATA INTEGRATION- via APIs or data delivery
Example: populating publication and citation data into an
institutional repository
http://hub.hku.hk/

30. University of Toronto Case Study
Simon Pratt
Director, Policy and Analysis

31. University of Toronto
30
 We use bibliometric data in a variety of ways:
– Reports to Executive Committee and
Governing Council
– Promoting our excellence
– Grant support
– Research Management

32. Reports to Executive Committee and
Governing Council
31

33. Reports to Executive Committee and
Governing Council
32

34. Promote our excellence
33
Collaboration Landscape

35. Promote our excellence
34
Collaboration Landscape

36. 35
Top 30
Urban Regions
Research Publications
2011-13
Research Publications,
% Change 1996-1998 to 2011-13
Promote our excellence

37. Grant support
36
 Canada First Research Excellence Fund
(CFREF-2)
 $900 million funds available (this round)
 U of T submission is on
“Data Driven Solutions” (DDS) an initiative to
transform the way data is applied to the most
complex and urgent social, environmental,
health and economic challenges.

38. Grant support
37
 Searched the Web of Science using a set of
keywords related to DDS
– Vetted and refined with input from academics
in the field
 Exported the results to InCites for analysis
– Identified collaborating countries and
institutions
– Created trend benchmarks to predict future
performance

39. Grant support
38

40. Grant support
39
 Identified 400 scholars across the university
with expertise in the field.
– Many faculty/departments represented
– Multidisciplinary
– Created “bio-sketches” of each, with some
bibliometric data included (h-index).

41. Grant support
40

42. Research Management
 The University of Toronto is a very large
university and the individual faculty have a
high degree of autonomy.
 The Faculty of Medicine maintains a variety
of resources to assist researchers secure
funding
 One of the biggest challenges is maintaining
accurate lists of publications and integrating
those lists with the Canadian Common CV
41

43. Research Management
 Historically, the Faculty of Medicine has done
a huge amount of work to manually clean the
data and to feed the cleaned data back to
Thomson Reuters for integration with InCites
 However, we are currently implementing
Converis as our research information system
which will automate many of these processes
and link with external resources.
42

44. Data-Driven Solutions to Support Your Research
Enterprise
ann.beynon@thomsonreuters.com
Visit our table for a complimentary research report on
your university
Visit our website:
http://stateofinnovation.thomsonreuters.com/