1.
MahendraMahey (UKOLN)<br />Stephanie Taylor (UKOLN)<br />Niamh Brennan (TCD)<br />Kevin Kiely (TCD)<br />http://cerify.ukoln.ac.uk<br />

2.
Current and past CRIS/CERIF/RIM projects<br />Research Information ManagementProcess Mapping and Gap Analysis<br />TCD CRIS <br />Questionnaires Site Visits<br />Business Process Analysis‘as is’ and ‘to be’<br />4 Priority Processes<br />User requirements elicitation<br />Prioritization of RIM Processes / Identification,<br />Mapping, Gap and Stakeholder Analysis<br />Data from at least 2 partners<br />Thomson CRIS<br />Research in View and InCites<br />Increasing level of Engagement with a CRIS / CERIF<br />Evaluation of CERIF and CRIS<br />Import and Export of Data and Facilitation of data exchange<br />CERIFy Approach<br />People Centered <br />Socio Cognitive<br />TCD – CERIF People and Publications<br />Institutions see value of CERIF and CRIS, Tell their regional institutions at regional and national events.CERIF and CRIS Engagement Increases<br />Priority<br />Process Areas – ‘as is’ and ‘to be’<br />CERIF<br />TCD<br />XML<br />2008<br />Mappings and Cross Walks relevant CERIF Schema<br />InCites<br />CERIF<br />TCD<br />XML<br />2002<br />CERIF Health<br />Check<br />Other, e.g. PURE?<br />Reducing duplication<br />Research in View<br />Data Surgery<br />Mahendra Mahey<br />Stephanie Taylor<br />Rosemary Russell<br />Michael Day<br />Talat Chaudhri<br />NiamhBrennan<br />Kevin Kiely<br />Jimmy Tang<br />Roisin Croker<br />http://cerify.ukoln.ac.uk<br />

3.
InCites Exchange of Data<br />http://cerify.ukoln.ac.uk<br />

4.
How Is This Used?<br />RAE requirement<br />Comparison with other universities (e.g. Russell Group)<br />Global comparisons (updated annually)<br />Citation information generated from this tool provides a lot of key information<br />http://cerify.ukoln.ac.uk<br />

5.
Collection of Data<br />Two-way process between Thomson Reuters & individual institution<br />Requires a lot of work by institution<br />Institutional data collected in local database of staff research information<br />http://cerify.ukoln.ac.uk<br />

6.
User Issues<br />“There is a lot of effort involved in understanding the data - there would be much value in standardising the data.”<br />E.g. - problems with author ID - duplicates & null fields<br />http://cerify.ukoln.ac.uk<br />

7.
User Requirements<br />“If the data was in a standardised format it would make life much easier!”<br />Nightly updates<br />Internationalised data sets<br />Relational database with mapping research output onto staff ID number for returned data<br />Better mapping to institutional schools structure<br />http://cerify.ukoln.ac.uk<br />

8.
Measures of Esteem<br />http://cerify.ukoln.ac.uk<br />

9.
How Is This Used?<br />RAE requirement<br />Personal review for promotion process<br />Assessment & benchmarking - internal & external<br />As key information in the assessment in judging the overall quality of research outputs<br />Drafting REF documentation<br />Part of institution-wide annual planning cycle <br />http://cerify.ukoln.ac.uk<br />

10.
Collection of Data<br />From individual researchers on adhoc basis<br />Systematic for RAE<br />Continuous tracking for REF<br />Ask individual schools to provide this on regular basis<br />http://cerify.ukoln.ac.uk<br />

11.
User Issues<br />Process is ‘woolly’ - difficult to provide meaningful data<br />Perception varies depending on given point in a career path<br />Citations alone are not accurate enough<br />Context is needed to make sense of raw data<br />Defining this process is very difficult<br />Researchers can undersell themselves and don’t make information available<br />http://cerify.ukoln.ac.uk<br />

12.
User Requirements<br />Systematic way of capturing this is required<br />Bringing a bibliometrics perspective to esteem-related information <br />Take into account that one measurement doesn’t fit all<br />Acknowledge differences in different subjects<br />When one size doesn’t fit all, find where to plug the gaps in information <br />Personalised audit tool to be built into RIM system<br />http://cerify.ukoln.ac.uk<br />

13.
Thomson Reuters data exchange – what’s InCites?<br />

14.
Why CERIFy it?<br />The community expressed interest in this process<br />Current data is unsatisfactory ; the dataset is a specific view<br />Institutional Citation Report is ‘historic’: based on affiliation at the time of authorship. <br />Does not show Schools, Research Centres etc. (WoS API will not provide this either).<br />Author, institution name variants (no automatic DAI in WoS/InCites)<br />Missing papers (aside from general coverage issues)<br />But.. institutions want the metrics + institutional structure in InCites + to embed data in local CRIS/IR:<br />Sample<br />Current exchange process is unsatisfactory: non-standard schema, heavy demands on institution, disappointing results<br />Opportunity to create a CERIF data model for exchange of people, institutional affiliations, publication data & metrics<br />Opportunity to demonstrate a 360⁰ data exchange using CERIF<br />If successful, it should help to show value and help to engage institutions with CERIF<br />Model can be re-used for other similar exchange processes<br />

15.
Process 1: InCites‘as is’ –Based on Queen’s University Belfast<br />

16.
Data drilldown<br />Creation of CERIF Data Model for ‘InCites’ Exchange<br />Process 1: InCites ‘to be’ –Generic (based on CERIFy Data Surgery) <br />3 Institutional Data automatically matched with TR records for authors & publications<br />4. InCites 'Institutional Profile' established and updated automatically<br />2. TR import and update deltas<br />Thomson Reuters<br />EXTERNAL<br />[6] 5 TR send report to institution on results of matching process (including matched Author ID)<br />1. Online export of CERIF-based data to TR from RIM/CRIS via web services (nightly updates)<br />MIS/CRIS developers<br />[7] TR data integrated with CRIS data (if required) Note: may use TR API instead or alongside<br />Research Office / Library<br />Business Intelligence Officers/Institutional Analysts<br />5. Reporting on Research<br /> 8.1 Internal Reporting 8.2 External Reporting<br />EXTERNAL<br />DATA EXCHANGE PARTNERS<br />

17.
Over to Kevin…<br />

18.
Where we are – results<br />CERIF Data model<br />Data exchange next week!<br />Includes: extended ‘publication types’ list<br />Multiple identifiers (Researcher ID; Institutional ID, HESA ID, Publication ID etc<br />Full metrics <br />= standardisation<br />Next steps<br />QUB<br />Thomson Reuters are working on automating the process using Web Services…<br />

19.
Notes on Esteem & REF<br /> “‘Esteem’ is no longer included as a distinct element in the assessment”<br /> <br />67. All information provided by HEIs in submissions to the 2014 REF must be capable of verification [note: can CERIF help provide the reference or source of data?]<br /> <br />Under ‘Definition of Impact for the REF’:<br /> <br />“a. Impacts on research or the advancement of academic knowledge within the higher education sector (whether in the UK or internationally) are excluded. (The submitted unit’s contribution to academic research and knowledge is assessed within the ‘outputs’ and ‘environment’ elements of REF.)”<br />Part 3 Section 5: Environment template (REF5) <br />183. Information is required about the research environment for each submitting unit relating to the period 1 January 2008 to 31 July 2013. Each submission must include a single completed REF5 form, consisting of the following sections: <br />• Overview. <br />• Research strategy. <br />• People, including: <br />– Staffing strategy and staff development. <br />– Research students. <br />• Income, infrastructure and facilities. <br />• Collaboration and contribution to the discipline.<br />From: Assessment framework and guidance on submissions, HEFCE, July 2011<br /> <br />http://www.hefce.ac.uk/research/ref/pubs/2011/02_11/02_11.pdf<br />REF5 ‘Environment’:<br /><ul><li>Maximum score reduced from 30%</li></ul>(RAE 2008) to 15% (REF 2014)<br /><ul><li>Indicators of Esteem = Collaboration & </li></ul>Contribution to the Discipline<br />

20.
Process 1: Esteem ‘as is’ –Based on Aberystwyth University<br />1 Completes ‘Other Activities’ annual form on Research Activity Database (RAD) SharePoint interface<br />Academic<br />4. Hardcopies printed for Research Monitoring meetings <br />2. Details of form transferred to RAD<br />3. Pdf report produced<br />Research Monitoring Team<br />5. Electronic copy deposited within RAD SharePoint site Departmental document folder<br />HoD/DDR<br />6. Access electronic copy deposited within RAD SharePoint site Departmental folder<br />Deans<br />7. Hardcopies printed for Research Monitoring meetings <br />DATA EXCHANGE PARTNERS<br />

21.
Process 1: Esteem ‘to be’ –CERIFy Data Surgery<br />1. CRIS/RIMacademic profile is automatically populated with elements of 'Measures of Esteem'<br />MIS/Library<br />Academic<br />2. Academic updates CRIS/RIM with other elements of Measures of Esteem<br />3. CRIS / RIM system data is queried and reported by Research Monitoring Team<br />Research Monitoring Team<br />HoD/DDR<br />4. Reports generated by HoDs, Deans etc.<br />Deans <br />5. Reports generated by HoDs, Deans etc.<br />External agencies/Funders <br />6. Data Exported to external agencies as required<br />

22.
Indicators of Esteem:<br />based on CERIFy Data<br />Surgery, current practice<br />In partner institutions + <br />a number of external sources<br />

23.
Indicators of Esteem - <br />the extended version:<br />based on original CERIFy IE, <br />With additions to <br />publications & outputs<br />Please see spreadsheet <br />for latest list CERIFy Indicators of<br />Esteem<br />

24.
CERIFy Esteem Indicators Model – based on MICE (Impact) Note: current focus for Esteem is more on Indicators rather than on Measures<br />Esteem Indicator<br />language table<br />Link tables<br />cfOrgUnitId etc.cfEsteemIndicatorId<br />cfClassID<br />cfClassSchemeId<br />cfStartDate<br />cfEndDate<br />Esteem Indicator table<br />OrgUnit<br />Project<br />Person<br />Publ.<br />Etc.<br />OrgUnit<br />Project<br />Person<br />Publ.<br />Etc.<br />Link table<br />Esteem Measure table<br />cfEsteemMeasureIdcfEsteemIndicatorId<br />cfStartDate<br />cfEndDate<br />cfEsteemMeasureIdcfEsteemMeasureKind<br />cfCountInteger<br />cfValueFloatingPoint<br />cfValueJudgementalNumeric<br />cfCountIntegerChange<br />cfValueFloatingPointChange<br />cfValueJudgementalChange<br />cfValueJudgementalText<br />Link tables<br />Link tables<br />Link tables<br />cfOrgUnitId etc.cfEsteemIndicatorId<br />cfClassID<br />cfClassSchemeId<br />cfStartDate<br />cfEndDate<br />cfOrgUnitId etc.cfEsteemIndicatorId<br />cfClassID<br />cfClassSchemeId<br />cfStartDate<br />cfEndDate<br />cfOrgUnitId etc.cfEsteemIndicatorId<br />cfClassID<br />cfClassSchemeId<br />cfStartDate<br />cfEndDate<br />NB: this adaptation has yet to be finally validated by Brigitte<br />

25.
Esteem Indicators mapped to UK RAE 2008, Australian ERA<br />

26.
Esteem Indicators for REF5: Contribution to Discipline/Collaborations<br />

27.
The Issue with Esteem Indicators [as with Impact]<br />The processes associated with the capture and reporting of Esteem Indicators can benefit more if there is an effort to standardise at least some of the data required. <br />Most of the data is not currently available from sources other than narratives or reports supplied by members of academic staff<br />To reduce time and effort & assist with verification:<br /><ul><li>Where data can be imported from external sources it should.

28.
‘Authority-controlled’, taxonomies and definitions lists should be agreed and built into the system</li></ul>NOTE: the RAE 2008 submissions under ‘Research Environment, Measures of Esteem’ provide an extremely valuable source of information on the type of information provided by institutions in this area. This body of knowledge is not available for other REF-relevant areas such as ‘Impact’. The RAE 2008 ‘Measures of Esteem’ is a particularly rich source of data from the disciplinary viewpoint. <br />The analysis of these ‘texts’ has great potential to assist with developing the type of ‘authority lists’/taxonomies described above. <br />The following slides show the results from analysing the RAE 2008 University of Bath ‘Measures of Esteem’ submission in the field of Pharmacy. This text was analysed using Many Eyes and Open Calais. The results of using Open Calais for semantic analysis and entity extraction even for this small sample are very positive.<br />

29.
Acknowledgements<br /><ul><li>The CERIFY Team & Partners

30.
Brigitte Jorge, EuroCRIS

31.
Thomson Reuters InCitesCERIFy team, Philadelphia

32.
Bo Alroe, Atira /UK Pure User Group (Esteem Indicators)

33.
http://cerify.ukoln.ac.uk</li>