O slideshow foi denunciado.
Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.

Bibliometrics, Webometrics, Altmetrics, Alternative metrics.

302 visualizações

Publicada em

A. Scharnhorst (2016) Bibliometrics, Webometrics, Altmetrics, Alternative metrics. Presentation given at the COST Action TD1210 Knowescape Workshop “Alternative metrics or tailored metrics: Science dynamics for science policy”, November 9-10, 2016 Warsaw

Publicada em: Educação
  • Seja o primeiro a comentar

  • Seja a primeira pessoa a gostar disto

Bibliometrics, Webometrics, Altmetrics, Alternative metrics.

  1. 1. dans.knaw.nl DANS is an institute of KNAW en NWO Bibliometrics, Webometrics, Altmetrics, Alternative metrics A possible Zeno effect for science metrics, and why we nevertheless look for metrics? Andrea Scharnhorst www.knowescape.org Workshop “Alternative metrics or tailored metrics: Science dynamics for science policy”, November 9-10, 2016 Warsaw
  2. 2. NARCIS - http://www.narcis.nl/
  3. 3. EASY: https://easy.dans.knaw.nl/ui/home
  4. 4. Motivation PhD on math models of science dynamics – measurement – scientometrics (e.g., # researcher in a field; # PhD students in a field) Use of metrics in science policy – EastEurope in the mirror of bibliometrics – Matthew effect of countries (Bonitz) New practices, new metrics Web indicators for scientific, technological and innovation research – WISER 2002-5 Academic Careers Understood through Measurement and Norms - ACUMEN 2011-14 Impact-EV - Evaluation of SSH 2013-17 Visualisation of structure and evolution of science Visualising NARCIS Mapping Digital Humanities Digital Observatory for DH (Pilot) Semantic web technologies - Open Data CEDAR Dutch Historic Census New practices Research Data - FAIR
  5. 5. Growth of science and indicator systems – How metrics came about?
  6. 6. Growth of science and indicator systems – How metrics came about? 1950 1960 1970 1980 1990 2000 2010 NSF (1950) https://nsf.gov/statistics/ i.e., PhDs per field OECD (1961) Frascati Manual 63 EuroCRIS (2002) CERIF Standard Data Model VIVITI (1952) RZH ISI (1960) WoK, Citation indexing Altmetrics.com (2011) VIVO Open source software/ontology for scholarship wikipedia Google Scholar (2004) CASRAI (2006) Open standards RI, CA
  7. 7. Box model of research Output journal articles; citation impact; patents Input Human capital: authors; …. ?students? Expenditures: projects; ...?infrastructures? Process
  8. 8. Tailored metrics or all-in metrics? Perhaps counter-intuitively, when it comes to metrics more is not necessarily always better. When deciding what to record, you should picture yourself at operationally significant periods within the year like year-end, budget submission time, and month end, imagining the information you would ideally like to report upwards or use to make operational decisions for your department. For example a handy technique is to design your ideal annual departmental report and then work backwards asking whether at present you have the necessary data to produce the report. The annual report should talk to your firm’s strategic goals if it is to be effective and well received. Of course you won’t collect metrics solely for upward reporting to management, you’ll also collect metrics to help run your department better. Differentiate between external and internal metrics – those meant to help you and your team run things better, and those meant to communicate your value externally within the firm. Peter Borchers, Managing Director http://priorysolutions.com/articles/law-firm-library-metrics-aall-session-summary/
  9. 9. Metrics - What for? Questions To better understand science dynamics To better monitor science dynamics How have disciplines developed over centuries? Do innovation, institutionalisation, education operate on different time scales? What is the dynamic of the academic job market? How much ‘small fields’ does an university need? How adequate are national portfolios to team science? Impact of large scale infrastructure investment Who does re-use research data?
  10. 10. Blind spots – infrastructure and new fields Start of large scale digitization projects at the Royal Library Start of the "Cultural memory of The Netherlands" Start of Staten-Generaal Digitaal - Parlamentary Debates LifeCoursesInContext NWO - Mega RIS - Digital Databank for Newspapers (DDD) PoliticalMashup CEDAR - Dutch Historic Census EliteNetworkShifts DELPHER - Portal to digital sources of the KB ExPoSe Digging into Linked Parliamentary Data 1-Jan-99 31-Dec-00 31-Dec-02 30-Dec-04 30-Dec-06 29-Dec-08 29-Dec-10 28-Dec-12 28-Dec-14 27-Dec-16 From Digitization to Digital Humanities
  11. 11. Get inspiration
  12. 12. Evidence Analytics & Information Systems
  13. 13. But be aware Local (geo, topic, institutional) science measurement Global, cross-domain, long-term ResearchInformation Systems Not all measurement should be pursuit on all levels of granularity and all time! Up-scaling comes with a price!
  14. 14. Take away Understanding Monitoring Combine qualitative and quantitative research Make sure to refer to standard data models – re-use ontologies RI data are ‘just’ data – use the FAIR principles (findable, accessible, interoperable, re-usable) When experimenting with new Research Information Systems communicate where they are located (local-global; incidental-long- time;….) Communicate about error margin’s, uncertainty and ambiguity – visualise!
  15. 15. References Godin, B. (2005). Measurement and statistics on science and technology: 1920 to the present. London: Routledge. Godin, B. (2001). The Emergence of Science and Technology Indicators: Why Did Governments Supplement Statistics With Indicators? (No. 8). Montreal. Retrieved from http://www.csiic.ca/PDF/Godin_8.pdf - (annex: NSF indicators (scores/feasibility), considered by not recommended) Priem, J., Taraborelli, D., Groth, P., & Neylon, C. (2010). Alt-metrics: a manifesto. October. Retrieved from http://altmetrics.org/manifesto/ Diana Hicks, & Wouters, P. (2015). The Leiden Manifesto for research metrics. Use these ten principles to guide research evaluation... Nature, 520(7548), 9–11. doi:10.1038/520429a Borgman, C. L. (2015). Big data, little data, no data: Scholarship in the networked world. Cambridge, Mass: MIT Press Börner, K. (2010). Atlas of science: Visualizing what we know. Cambridge, Mass: MIT Press. Börner, K. (2015). Atlas of knowledge: Anyone can map. Cambridge, Mass: MIT Press. Wilkinson, M. D., Dumontier, M., Aalbersberg, I. J., Appleton, G., Axton, M., Baak, A., ... Mons, B. (2016). The FAIR Guiding Principles for scientific data management and stewardship. Nature, 3, 160018. DOI: doi:10.1038/sdata.2016.18
  16. 16. dans.knaw.nl DANS is an institute of KNAW en NWO Thanks for your attention! Andrea.scharnhorst@dans.knaw.nl @ScharnhorstA @knowescape Dans.knaw.nl