SlideShare a Scribd company logo
1 of 19
Download to read offline
NISO Altmetrics Initiative:
A Project Update
Martin Fenner, http://orcid.org/0000-0003-1419-2405
Technical Lead, PLOS Article-Level Metrics
Chair, NISO Alternative Assessment Metrics Project Steering Group
http://fivethirtyeight.com/interactives/world-cup/
Phase I
• Describe the current state of the
altmetrics discussion
• Identify potential action items for
further work on best practices and
standards
Steering Committee
• Euan Adie (Altmetric)
• Amy Brand (Harvard University/Digital Science)
• Mike Buschman (Plum Analytics)
• Todd Carpenter (NISO)
• Martin Fenner (Public Library of Science, Chair)
• Gregg Gordon (Social Science Research Network)
• William Gunn (Mendeley)
• Michael Habib (Reed Elsevier)
• Nettie Lagace (NISO)
• Jamie Liu (American Chemical Society)
• Heather Piwowar (ImpactStory)
• John Sack (HighWire Press)
• Peter Shepherd (Project Counter)
• Christine Stohn (Ex Libris, Inc.)
• Greg Tananbaum (Scholarly Publishing & Academic Resources Coalition)
In-Person Meetings
• October 9, 2013 in San Francisco
• December 11, 2013 in Washington, DC
• January 23, 2014 in Philadelphia
All meetings were streamed and
recorded
NISO Altmetrics Initiative: A Project Update - Martin Fenner, Technical Lead for the PLOS Article-Level Metrics project
Interviews
• Thirty researchers, administrators,
librarians, funders (and others)
• Semi-structured interview
• March – April 2014
Approach
• Open format: lightning talks,
brainstorming, breakout groups, etc.
• Include all stakeholders
• Focus on collecting unstructured input
• Make all material (including audio
recordings of steering group) publicly
available
NISO Altmetrics Initiative: A Project Update - Martin Fenner, Technical Lead for the PLOS Article-Level Metrics project
NISO Altmetrics Initiative: A Project Update - Martin Fenner, Technical Lead for the PLOS Article-Level Metrics project
Should metrics be hidden to prevent herd mentality? (yes, like Reddit) Quality and Gaming San Francisco
Define credible sources for inclusion: Twitter, Facebook Quality and Gaming San Francisco
A problem is data quality and provenance Quality and Gaming San Francisco
Quality assessment of studied data Quality and Gaming San Francisco
Validity of reliability of altmetrics Quality and Gaming San Francisco
Data quality & validity: How valid the altmetrics data … ? have to assess? Quality and Gaming San Francisco
Approaches to factor out the effects of gaming (e.g., not counting self
citation)
Quality and Gaming San Francisco
Define acceptable promotion versus gaming Quality and Gaming San Francisco
Auto-spam detection; trolling Quality and Gaming San Francisco
Interactions with more traditional altmetrics Quality and Gaming San Francisco
Define what behaviors are considered as cheating/gaming Quality and Gaming San Francisco
Define what reaction to gaming should be public sharing? Data tainting?
We ingnore it?
Quality and Gaming San Francisco
Are there any “ungameable” systems out there at all and can we learn
anything from them?
Quality and Gaming San Francisco
What are effective measures to audit published altmetrics for accuracy? Quality and Gaming San Francisco
NISO Altmetrics Initiative: A Project Update - Martin Fenner, Technical Lead for the PLOS Article-Level Metrics project
Potential Action Items
Data Quality and Gaming
• Promote and facilitate use of persistent identifiers.
• Research issues surrounding the reproducibility of metrics across providers.
• Develop strategies to improve data quality through normalization of source
data across providers.
• Explore creation of standardized APIs or download or exchange formats to
facilitate data gathering.
• Develop strategies to increase trust, e.g., openly available data, audits, or a
clearinghouse.
• Study potential strategies for defining and identifying systematic gaming of
new metrics.
NISO Altmetrics Initiative: A Project Update - Martin Fenner, Technical Lead for the PLOS Article-Level Metrics project
NISO Altmetrics Initiative: A Project Update - Martin Fenner, Technical Lead for the PLOS Article-Level Metrics project
Next Steps
• Finalize and release white paper and draft new work
item proposal for standards/best practices based on
the study
• Proposal vetted by NISO leadership and members
• Proposal approved and working groups formed for
Phase II of the project
Lessons Learned?
Definitions
• Altmetrics
• Article-Level Metrics
• Research Assessment
• Scholarly Impact
• Quality of Research
Data Quality and Gaming
• Disconnect between frequent concerns by users and
lack of (public) activity by altmetrics providers
• Strategies to improve data quality include standards,
audits, open data, and a central clearinghouse
• Users of altmetrics data (researchers, institutions,
funders) should be more proactive in demanding
data quality checks and transparency
Grouping and Aggregation
• Common practice for a single research output, e.g.
for multiple versions, multiple locations, or grouping
of all metrics into a single score
• Common practice for multiple research outputs, e.g.
by journal, by author, or by institution
• Challenge of uniquely identifying authors, institutions
and funders associated with these research outputs
• Aggregation remains problematic, more empirical
evidence and transparency needed

More Related Content

What's hot

Embedding ORCID across researcher career paths
Embedding ORCID across researcher career pathsEmbedding ORCID across researcher career paths
Embedding ORCID across researcher career pathsORCID, Inc
 
Case studies for open science
Case studies for open scienceCase studies for open science
Case studies for open scienceIUPUI
 
Altmetrics for Team Science
Altmetrics for Team ScienceAltmetrics for Team Science
Altmetrics for Team ScienceIUPUI
 
Citation & altmetrics - a comparison
Citation & altmetrics - a comparisonCitation & altmetrics - a comparison
Citation & altmetrics - a comparisonIUPUI
 
The Right Metrics for Generation Open [Open Access Week 2014]
The Right Metrics for Generation Open [Open Access Week 2014]The Right Metrics for Generation Open [Open Access Week 2014]
The Right Metrics for Generation Open [Open Access Week 2014]Impactstory Team
 
Scholarly Metrics in Specialized Settings
Scholarly Metrics in Specialized SettingsScholarly Metrics in Specialized Settings
Scholarly Metrics in Specialized SettingsElaine Lasda
 
Your Systematic Review: Getting Started
Your Systematic Review: Getting StartedYour Systematic Review: Getting Started
Your Systematic Review: Getting StartedElaine Lasda
 
January 13, 2016 NISO Webinar: Ensuring the Scholarly Record: Scholarly Retra...
January 13, 2016 NISO Webinar: Ensuring the Scholarly Record: Scholarly Retra...January 13, 2016 NISO Webinar: Ensuring the Scholarly Record: Scholarly Retra...
January 13, 2016 NISO Webinar: Ensuring the Scholarly Record: Scholarly Retra...DeVonne Parks, CEM
 
Understanding impact through alternative metrics: developing library-based as...
Understanding impact through alternative metrics: developing library-based as...Understanding impact through alternative metrics: developing library-based as...
Understanding impact through alternative metrics: developing library-based as...Kristi Holmes
 
Librarians Conducting Research: Researcher Librarian Partnerships
Librarians Conducting Research: Researcher Librarian PartnershipsLibrarians Conducting Research: Researcher Librarian Partnerships
Librarians Conducting Research: Researcher Librarian PartnershipsSusanMRob
 
Altmetrics 101 - Altmetrics in Libraries
Altmetrics 101 - Altmetrics in LibrariesAltmetrics 101 - Altmetrics in Libraries
Altmetrics 101 - Altmetrics in LibrariesIUPUI
 
The good, the efficient and the open - changing research workflows and the ne...
The good, the efficient and the open - changing research workflows and the ne...The good, the efficient and the open - changing research workflows and the ne...
The good, the efficient and the open - changing research workflows and the ne...Bianca Kramer
 
Frontiers: Five Year Plan
Frontiers: Five Year PlanFrontiers: Five Year Plan
Frontiers: Five Year PlanFrontiersIn
 
Frontiers' Collaborative Review
Frontiers' Collaborative ReviewFrontiers' Collaborative Review
Frontiers' Collaborative ReviewFrontiersIn
 

What's hot (20)

Embedding ORCID across researcher career paths
Embedding ORCID across researcher career pathsEmbedding ORCID across researcher career paths
Embedding ORCID across researcher career paths
 
Roth "Tools to support systematic review research"
Roth "Tools to support systematic review research"Roth "Tools to support systematic review research"
Roth "Tools to support systematic review research"
 
Carelli "Promoting Content Discovery Within the Reader/Researcher Workflow"
Carelli "Promoting Content Discovery Within the Reader/Researcher Workflow"Carelli "Promoting Content Discovery Within the Reader/Researcher Workflow"
Carelli "Promoting Content Discovery Within the Reader/Researcher Workflow"
 
Warren-Jones "Using text-mining and summarisation technology to manage the gr...
Warren-Jones "Using text-mining and summarisation technology to manage the gr...Warren-Jones "Using text-mining and summarisation technology to manage the gr...
Warren-Jones "Using text-mining and summarisation technology to manage the gr...
 
Case studies for open science
Case studies for open scienceCase studies for open science
Case studies for open science
 
Altmetrics for Team Science
Altmetrics for Team ScienceAltmetrics for Team Science
Altmetrics for Team Science
 
Citation & altmetrics - a comparison
Citation & altmetrics - a comparisonCitation & altmetrics - a comparison
Citation & altmetrics - a comparison
 
The Right Metrics for Generation Open [Open Access Week 2014]
The Right Metrics for Generation Open [Open Access Week 2014]The Right Metrics for Generation Open [Open Access Week 2014]
The Right Metrics for Generation Open [Open Access Week 2014]
 
Scholarly Metrics in Specialized Settings
Scholarly Metrics in Specialized SettingsScholarly Metrics in Specialized Settings
Scholarly Metrics in Specialized Settings
 
Cooper "Simplicity is the Ultimate Sophistication: Accessible, Ubiquitous Tec...
Cooper "Simplicity is the Ultimate Sophistication: Accessible, Ubiquitous Tec...Cooper "Simplicity is the Ultimate Sophistication: Accessible, Ubiquitous Tec...
Cooper "Simplicity is the Ultimate Sophistication: Accessible, Ubiquitous Tec...
 
Your Systematic Review: Getting Started
Your Systematic Review: Getting StartedYour Systematic Review: Getting Started
Your Systematic Review: Getting Started
 
January 13, 2016 NISO Webinar: Ensuring the Scholarly Record: Scholarly Retra...
January 13, 2016 NISO Webinar: Ensuring the Scholarly Record: Scholarly Retra...January 13, 2016 NISO Webinar: Ensuring the Scholarly Record: Scholarly Retra...
January 13, 2016 NISO Webinar: Ensuring the Scholarly Record: Scholarly Retra...
 
Understanding impact through alternative metrics: developing library-based as...
Understanding impact through alternative metrics: developing library-based as...Understanding impact through alternative metrics: developing library-based as...
Understanding impact through alternative metrics: developing library-based as...
 
Librarians Conducting Research: Researcher Librarian Partnerships
Librarians Conducting Research: Researcher Librarian PartnershipsLibrarians Conducting Research: Researcher Librarian Partnerships
Librarians Conducting Research: Researcher Librarian Partnerships
 
Altmetrics 101 - Altmetrics in Libraries
Altmetrics 101 - Altmetrics in LibrariesAltmetrics 101 - Altmetrics in Libraries
Altmetrics 101 - Altmetrics in Libraries
 
Stephanie Faulkner - Using Metrics Beyond Citations to Demonstrate the Impact...
Stephanie Faulkner - Using Metrics Beyond Citations to Demonstrate the Impact...Stephanie Faulkner - Using Metrics Beyond Citations to Demonstrate the Impact...
Stephanie Faulkner - Using Metrics Beyond Citations to Demonstrate the Impact...
 
The good, the efficient and the open - changing research workflows and the ne...
The good, the efficient and the open - changing research workflows and the ne...The good, the efficient and the open - changing research workflows and the ne...
The good, the efficient and the open - changing research workflows and the ne...
 
Attribution From Res Lib Perspective - Micah Altman, MIT
Attribution From Res Lib Perspective - Micah Altman, MITAttribution From Res Lib Perspective - Micah Altman, MIT
Attribution From Res Lib Perspective - Micah Altman, MIT
 
Frontiers: Five Year Plan
Frontiers: Five Year PlanFrontiers: Five Year Plan
Frontiers: Five Year Plan
 
Frontiers' Collaborative Review
Frontiers' Collaborative ReviewFrontiers' Collaborative Review
Frontiers' Collaborative Review
 

Viewers also liked (6)

June 18 NISO Virtual Conference: Transforming Assessment: Alternative Metrics...
June 18 NISO Virtual Conference: Transforming Assessment: Alternative Metrics...June 18 NISO Virtual Conference: Transforming Assessment: Alternative Metrics...
June 18 NISO Virtual Conference: Transforming Assessment: Alternative Metrics...
 
Alternative Metrics and Other TrendsThree Publishers' Perspective on Using Al...
Alternative Metrics and Other TrendsThree Publishers' Perspective on Using Al...Alternative Metrics and Other TrendsThree Publishers' Perspective on Using Al...
Alternative Metrics and Other TrendsThree Publishers' Perspective on Using Al...
 
June 18, 2014 NISO Virtual Conference: Transforming Assessment: Alternative M...
June 18, 2014 NISO Virtual Conference: Transforming Assessment: Alternative M...June 18, 2014 NISO Virtual Conference: Transforming Assessment: Alternative M...
June 18, 2014 NISO Virtual Conference: Transforming Assessment: Alternative M...
 
Identifying the Invisible Impact of Scholarly Publications: A Multi-Disciplin...
Identifying the Invisible Impact of Scholarly Publications: A Multi-Disciplin...Identifying the Invisible Impact of Scholarly Publications: A Multi-Disciplin...
Identifying the Invisible Impact of Scholarly Publications: A Multi-Disciplin...
 
eLife and Article Metrics: Ruth Ivimey-Cook, Web Application Developer, eLife...
eLife and Article Metrics: Ruth Ivimey-Cook, Web Application Developer, eLife...eLife and Article Metrics: Ruth Ivimey-Cook, Web Application Developer, eLife...
eLife and Article Metrics: Ruth Ivimey-Cook, Web Application Developer, eLife...
 
Virginia Tech’s Response to Research Data Needs: The Center for Digital Resea...
Virginia Tech’s Response to Research Data Needs: The Center for Digital Resea...Virginia Tech’s Response to Research Data Needs: The Center for Digital Resea...
Virginia Tech’s Response to Research Data Needs: The Center for Digital Resea...
 

Similar to NISO Altmetrics Initiative: A Project Update - Martin Fenner, Technical Lead for the PLOS Article-Level Metrics project

How to use your data science team: Becoming a data-driven organization
How to use your data science team: Becoming a data-driven organizationHow to use your data science team: Becoming a data-driven organization
How to use your data science team: Becoming a data-driven organizationYael Garten
 
Ethical Priniciples for the All Data Revolution
Ethical Priniciples for the All Data RevolutionEthical Priniciples for the All Data Revolution
Ethical Priniciples for the All Data RevolutionMelissa Moody
 
NCME Big Data in Education
NCME Big Data  in EducationNCME Big Data  in Education
NCME Big Data in EducationPhilip Piety
 
Measuring Progress: Indicators, Data Sources and Assessment | Laszlo Pinter, ...
Measuring Progress: Indicators, Data Sources and Assessment | Laszlo Pinter, ...Measuring Progress: Indicators, Data Sources and Assessment | Laszlo Pinter, ...
Measuring Progress: Indicators, Data Sources and Assessment | Laszlo Pinter, ...NAP Global Network
 
Approaching Big Data: Lesson Plan
Approaching Big Data: Lesson Plan Approaching Big Data: Lesson Plan
Approaching Big Data: Lesson Plan Bessie Chu
 
Measuring Progress: Indicators, data sources and assessment | Laszlo Pinter, ...
Measuring Progress: Indicators, data sources and assessment | Laszlo Pinter, ...Measuring Progress: Indicators, data sources and assessment | Laszlo Pinter, ...
Measuring Progress: Indicators, data sources and assessment | Laszlo Pinter, ...NAP Global Network
 
Evaluating Systems Change
Evaluating Systems ChangeEvaluating Systems Change
Evaluating Systems ChangeNoel Hatch
 
Recommender system introduction
Recommender system   introductionRecommender system   introduction
Recommender system introductionLiang Xiang
 
20190527_Dietmar Lampert _ New indicators for Open Sciene
20190527_Dietmar Lampert _ New indicators for Open Sciene20190527_Dietmar Lampert _ New indicators for Open Sciene
20190527_Dietmar Lampert _ New indicators for Open ScieneOpenAIRE
 
How metadata drives data sharing; UK Data Archive
How metadata drives data sharing; UK Data Archive How metadata drives data sharing; UK Data Archive
How metadata drives data sharing; UK Data Archive Louise Corti
 
20160607 citation4software opening
20160607 citation4software opening20160607 citation4software opening
20160607 citation4software openingDaniel S. Katz
 
Researcher profiles and metrics that matter
Researcher profiles and metrics that matterResearcher profiles and metrics that matter
Researcher profiles and metrics that matterLibrary_Connect
 
ACRL 2011 Data-Driven Library Web Design
ACRL 2011 Data-Driven Library Web DesignACRL 2011 Data-Driven Library Web Design
ACRL 2011 Data-Driven Library Web DesignAmanda Dinscore
 
Introduction to data science
Introduction to data scienceIntroduction to data science
Introduction to data scienceSpartan60
 

Similar to NISO Altmetrics Initiative: A Project Update - Martin Fenner, Technical Lead for the PLOS Article-Level Metrics project (20)

NISO-Altmetrics-NE-ACRL-ScholComIG-Nov2013
NISO-Altmetrics-NE-ACRL-ScholComIG-Nov2013NISO-Altmetrics-NE-ACRL-ScholComIG-Nov2013
NISO-Altmetrics-NE-ACRL-ScholComIG-Nov2013
 
Falk-Krzesinski, "Administrator (Institutional Use of the Data): Data-informe...
Falk-Krzesinski, "Administrator (Institutional Use of the Data): Data-informe...Falk-Krzesinski, "Administrator (Institutional Use of the Data): Data-informe...
Falk-Krzesinski, "Administrator (Institutional Use of the Data): Data-informe...
 
How to use your data science team: Becoming a data-driven organization
How to use your data science team: Becoming a data-driven organizationHow to use your data science team: Becoming a data-driven organization
How to use your data science team: Becoming a data-driven organization
 
Ethical Priniciples for the All Data Revolution
Ethical Priniciples for the All Data RevolutionEthical Priniciples for the All Data Revolution
Ethical Priniciples for the All Data Revolution
 
NCME Big Data in Education
NCME Big Data  in EducationNCME Big Data  in Education
NCME Big Data in Education
 
NISO's Altmetrics Initiative
NISO's Altmetrics InitiativeNISO's Altmetrics Initiative
NISO's Altmetrics Initiative
 
Lagace - Copyright Clearance Center April 2, 2015
Lagace - Copyright Clearance Center April 2, 2015Lagace - Copyright Clearance Center April 2, 2015
Lagace - Copyright Clearance Center April 2, 2015
 
Measuring Progress: Indicators, Data Sources and Assessment | Laszlo Pinter, ...
Measuring Progress: Indicators, Data Sources and Assessment | Laszlo Pinter, ...Measuring Progress: Indicators, Data Sources and Assessment | Laszlo Pinter, ...
Measuring Progress: Indicators, Data Sources and Assessment | Laszlo Pinter, ...
 
Approaching Big Data: Lesson Plan
Approaching Big Data: Lesson Plan Approaching Big Data: Lesson Plan
Approaching Big Data: Lesson Plan
 
ALA 2016 NISO Altmetrics
ALA 2016 NISO AltmetricsALA 2016 NISO Altmetrics
ALA 2016 NISO Altmetrics
 
Assessing Digital Output in New Ways
Assessing Digital Output in New WaysAssessing Digital Output in New Ways
Assessing Digital Output in New Ways
 
Measuring Progress: Indicators, data sources and assessment | Laszlo Pinter, ...
Measuring Progress: Indicators, data sources and assessment | Laszlo Pinter, ...Measuring Progress: Indicators, data sources and assessment | Laszlo Pinter, ...
Measuring Progress: Indicators, data sources and assessment | Laszlo Pinter, ...
 
Evaluating Systems Change
Evaluating Systems ChangeEvaluating Systems Change
Evaluating Systems Change
 
Recommender system introduction
Recommender system   introductionRecommender system   introduction
Recommender system introduction
 
20190527_Dietmar Lampert _ New indicators for Open Sciene
20190527_Dietmar Lampert _ New indicators for Open Sciene20190527_Dietmar Lampert _ New indicators for Open Sciene
20190527_Dietmar Lampert _ New indicators for Open Sciene
 
How metadata drives data sharing; UK Data Archive
How metadata drives data sharing; UK Data Archive How metadata drives data sharing; UK Data Archive
How metadata drives data sharing; UK Data Archive
 
20160607 citation4software opening
20160607 citation4software opening20160607 citation4software opening
20160607 citation4software opening
 
Researcher profiles and metrics that matter
Researcher profiles and metrics that matterResearcher profiles and metrics that matter
Researcher profiles and metrics that matter
 
ACRL 2011 Data-Driven Library Web Design
ACRL 2011 Data-Driven Library Web DesignACRL 2011 Data-Driven Library Web Design
ACRL 2011 Data-Driven Library Web Design
 
Introduction to data science
Introduction to data scienceIntroduction to data science
Introduction to data science
 

More from National Information Standards Organization (NISO)

More from National Information Standards Organization (NISO) (20)

Mattingly "Text and Data Mining: Building Data Driven Applications"
Mattingly "Text and Data Mining: Building Data Driven Applications"Mattingly "Text and Data Mining: Building Data Driven Applications"
Mattingly "Text and Data Mining: Building Data Driven Applications"
 
Mattingly "Text and Data Mining: Searching Vectors"
Mattingly "Text and Data Mining: Searching Vectors"Mattingly "Text and Data Mining: Searching Vectors"
Mattingly "Text and Data Mining: Searching Vectors"
 
Mattingly "Text Mining Techniques"
Mattingly "Text Mining Techniques"Mattingly "Text Mining Techniques"
Mattingly "Text Mining Techniques"
 
Mattingly "Text Processing for Library Data: Representing Text as Data"
Mattingly "Text Processing for Library Data: Representing Text as Data"Mattingly "Text Processing for Library Data: Representing Text as Data"
Mattingly "Text Processing for Library Data: Representing Text as Data"
 
Ross and Clark "Strategic Planning"
Ross and Clark "Strategic Planning"Ross and Clark "Strategic Planning"
Ross and Clark "Strategic Planning"
 
Mattingly "Data Mining Techniques: Classification and Clustering"
Mattingly "Data Mining Techniques: Classification and Clustering"Mattingly "Data Mining Techniques: Classification and Clustering"
Mattingly "Data Mining Techniques: Classification and Clustering"
 
Straza "Global collaboration towards equitable and open science: UNESCO Recom...
Straza "Global collaboration towards equitable and open science: UNESCO Recom...Straza "Global collaboration towards equitable and open science: UNESCO Recom...
Straza "Global collaboration towards equitable and open science: UNESCO Recom...
 
Lippincott "Beyond access: Accelerating discovery and increasing trust throug...
Lippincott "Beyond access: Accelerating discovery and increasing trust throug...Lippincott "Beyond access: Accelerating discovery and increasing trust throug...
Lippincott "Beyond access: Accelerating discovery and increasing trust throug...
 
Kriegsman "Integrating Open and Equitable Research into Open Science"
Kriegsman "Integrating Open and Equitable Research into Open Science"Kriegsman "Integrating Open and Equitable Research into Open Science"
Kriegsman "Integrating Open and Equitable Research into Open Science"
 
Mattingly "Ethics and Cleaning Data"
Mattingly "Ethics and Cleaning Data"Mattingly "Ethics and Cleaning Data"
Mattingly "Ethics and Cleaning Data"
 
Mercado-Lara "Open & Equitable Program"
Mercado-Lara "Open & Equitable Program"Mercado-Lara "Open & Equitable Program"
Mercado-Lara "Open & Equitable Program"
 
Ratner "Enhancing Open Science: Assessing Tools & Charting Progress"
Ratner "Enhancing Open Science: Assessing Tools & Charting Progress"Ratner "Enhancing Open Science: Assessing Tools & Charting Progress"
Ratner "Enhancing Open Science: Assessing Tools & Charting Progress"
 
Pfeiffer "Enhancing Open Science: Assessing Tools & Charting Progress"
Pfeiffer "Enhancing Open Science: Assessing Tools & Charting Progress"Pfeiffer "Enhancing Open Science: Assessing Tools & Charting Progress"
Pfeiffer "Enhancing Open Science: Assessing Tools & Charting Progress"
 
Hahnel “Mapping Progress: Reflections and Charting Future Pathways"
Hahnel “Mapping Progress: Reflections and Charting Future Pathways"Hahnel “Mapping Progress: Reflections and Charting Future Pathways"
Hahnel “Mapping Progress: Reflections and Charting Future Pathways"
 
Stall "Open Science: The Journey of a Scholarly Society"
Stall "Open Science: The Journey of a Scholarly Society"Stall "Open Science: The Journey of a Scholarly Society"
Stall "Open Science: The Journey of a Scholarly Society"
 
Hrynaszkiewicz "A Publisher's Perspective on Open Science"
Hrynaszkiewicz "A Publisher's Perspective on Open Science"Hrynaszkiewicz "A Publisher's Perspective on Open Science"
Hrynaszkiewicz "A Publisher's Perspective on Open Science"
 
Carter "Higher Education Leadership Initiative for Open Scholarship (HELIOS)"
Carter "Higher Education Leadership Initiative for Open Scholarship (HELIOS)"Carter "Higher Education Leadership Initiative for Open Scholarship (HELIOS)"
Carter "Higher Education Leadership Initiative for Open Scholarship (HELIOS)"
 
Webster "NISO Roundtable: Open Science and Libraries"
Webster "NISO Roundtable: Open Science and Libraries"Webster "NISO Roundtable: Open Science and Libraries"
Webster "NISO Roundtable: Open Science and Libraries"
 
Ratan "Are we there yet? Keeping the promise of open science"
Ratan "Are we there yet?  Keeping the promise of open science"Ratan "Are we there yet?  Keeping the promise of open science"
Ratan "Are we there yet? Keeping the promise of open science"
 
Tananbaum "NISO Tech Summit: Reflections on Reflections Upon The Year of Open...
Tananbaum "NISO Tech Summit: Reflections on Reflections Upon The Year of Open...Tananbaum "NISO Tech Summit: Reflections on Reflections Upon The Year of Open...
Tananbaum "NISO Tech Summit: Reflections on Reflections Upon The Year of Open...
 

Recently uploaded

P4C x ELT = P4ELT: Its Theoretical Background (Kanazawa, 2024 March).pdf
P4C x ELT = P4ELT: Its Theoretical Background (Kanazawa, 2024 March).pdfP4C x ELT = P4ELT: Its Theoretical Background (Kanazawa, 2024 March).pdf
P4C x ELT = P4ELT: Its Theoretical Background (Kanazawa, 2024 March).pdfYu Kanazawa / Osaka University
 
Prescribed medication order and communication skills.pptx
Prescribed medication order and communication skills.pptxPrescribed medication order and communication skills.pptx
Prescribed medication order and communication skills.pptxraviapr7
 
CAULIFLOWER BREEDING 1 Parmar pptx
CAULIFLOWER BREEDING 1 Parmar pptxCAULIFLOWER BREEDING 1 Parmar pptx
CAULIFLOWER BREEDING 1 Parmar pptxSaurabhParmar42
 
How to Use api.constrains ( ) in Odoo 17
How to Use api.constrains ( ) in Odoo 17How to Use api.constrains ( ) in Odoo 17
How to Use api.constrains ( ) in Odoo 17Celine George
 
Easter in the USA presentation by Chloe.
Easter in the USA presentation by Chloe.Easter in the USA presentation by Chloe.
Easter in the USA presentation by Chloe.EnglishCEIPdeSigeiro
 
How to Manage Cross-Selling in Odoo 17 Sales
How to Manage Cross-Selling in Odoo 17 SalesHow to Manage Cross-Selling in Odoo 17 Sales
How to Manage Cross-Selling in Odoo 17 SalesCeline George
 
CHUYÊN ĐỀ DẠY THÊM TIẾNG ANH LỚP 11 - GLOBAL SUCCESS - NĂM HỌC 2023-2024 - HK...
CHUYÊN ĐỀ DẠY THÊM TIẾNG ANH LỚP 11 - GLOBAL SUCCESS - NĂM HỌC 2023-2024 - HK...CHUYÊN ĐỀ DẠY THÊM TIẾNG ANH LỚP 11 - GLOBAL SUCCESS - NĂM HỌC 2023-2024 - HK...
CHUYÊN ĐỀ DẠY THÊM TIẾNG ANH LỚP 11 - GLOBAL SUCCESS - NĂM HỌC 2023-2024 - HK...Nguyen Thanh Tu Collection
 
Clinical Pharmacy Introduction to Clinical Pharmacy, Concept of clinical pptx
Clinical Pharmacy  Introduction to Clinical Pharmacy, Concept of clinical pptxClinical Pharmacy  Introduction to Clinical Pharmacy, Concept of clinical pptx
Clinical Pharmacy Introduction to Clinical Pharmacy, Concept of clinical pptxraviapr7
 
PISA-VET launch_El Iza Mohamedou_19 March 2024.pptx
PISA-VET launch_El Iza Mohamedou_19 March 2024.pptxPISA-VET launch_El Iza Mohamedou_19 March 2024.pptx
PISA-VET launch_El Iza Mohamedou_19 March 2024.pptxEduSkills OECD
 
Diploma in Nursing Admission Test Question Solution 2023.pdf
Diploma in Nursing Admission Test Question Solution 2023.pdfDiploma in Nursing Admission Test Question Solution 2023.pdf
Diploma in Nursing Admission Test Question Solution 2023.pdfMohonDas
 
The Singapore Teaching Practice document
The Singapore Teaching Practice documentThe Singapore Teaching Practice document
The Singapore Teaching Practice documentXsasf Sfdfasd
 
3.21.24 The Origins of Black Power.pptx
3.21.24  The Origins of Black Power.pptx3.21.24  The Origins of Black Power.pptx
3.21.24 The Origins of Black Power.pptxmary850239
 
Ultra structure and life cycle of Plasmodium.pptx
Ultra structure and life cycle of Plasmodium.pptxUltra structure and life cycle of Plasmodium.pptx
Ultra structure and life cycle of Plasmodium.pptxDr. Asif Anas
 
General views of Histopathology and step
General views of Histopathology and stepGeneral views of Histopathology and step
General views of Histopathology and stepobaje godwin sunday
 
2024.03.23 What do successful readers do - Sandy Millin for PARK.pptx
2024.03.23 What do successful readers do - Sandy Millin for PARK.pptx2024.03.23 What do successful readers do - Sandy Millin for PARK.pptx
2024.03.23 What do successful readers do - Sandy Millin for PARK.pptxSandy Millin
 
5 charts on South Africa as a source country for international student recrui...
5 charts on South Africa as a source country for international student recrui...5 charts on South Africa as a source country for international student recrui...
5 charts on South Africa as a source country for international student recrui...CaraSkikne1
 
Patient Counselling. Definition of patient counseling; steps involved in pati...
Patient Counselling. Definition of patient counseling; steps involved in pati...Patient Counselling. Definition of patient counseling; steps involved in pati...
Patient Counselling. Definition of patient counseling; steps involved in pati...raviapr7
 

Recently uploaded (20)

P4C x ELT = P4ELT: Its Theoretical Background (Kanazawa, 2024 March).pdf
P4C x ELT = P4ELT: Its Theoretical Background (Kanazawa, 2024 March).pdfP4C x ELT = P4ELT: Its Theoretical Background (Kanazawa, 2024 March).pdf
P4C x ELT = P4ELT: Its Theoretical Background (Kanazawa, 2024 March).pdf
 
Prelims of Kant get Marx 2.0: a general politics quiz
Prelims of Kant get Marx 2.0: a general politics quizPrelims of Kant get Marx 2.0: a general politics quiz
Prelims of Kant get Marx 2.0: a general politics quiz
 
Finals of Kant get Marx 2.0 : a general politics quiz
Finals of Kant get Marx 2.0 : a general politics quizFinals of Kant get Marx 2.0 : a general politics quiz
Finals of Kant get Marx 2.0 : a general politics quiz
 
Prescribed medication order and communication skills.pptx
Prescribed medication order and communication skills.pptxPrescribed medication order and communication skills.pptx
Prescribed medication order and communication skills.pptx
 
CAULIFLOWER BREEDING 1 Parmar pptx
CAULIFLOWER BREEDING 1 Parmar pptxCAULIFLOWER BREEDING 1 Parmar pptx
CAULIFLOWER BREEDING 1 Parmar pptx
 
How to Use api.constrains ( ) in Odoo 17
How to Use api.constrains ( ) in Odoo 17How to Use api.constrains ( ) in Odoo 17
How to Use api.constrains ( ) in Odoo 17
 
Easter in the USA presentation by Chloe.
Easter in the USA presentation by Chloe.Easter in the USA presentation by Chloe.
Easter in the USA presentation by Chloe.
 
How to Manage Cross-Selling in Odoo 17 Sales
How to Manage Cross-Selling in Odoo 17 SalesHow to Manage Cross-Selling in Odoo 17 Sales
How to Manage Cross-Selling in Odoo 17 Sales
 
CHUYÊN ĐỀ DẠY THÊM TIẾNG ANH LỚP 11 - GLOBAL SUCCESS - NĂM HỌC 2023-2024 - HK...
CHUYÊN ĐỀ DẠY THÊM TIẾNG ANH LỚP 11 - GLOBAL SUCCESS - NĂM HỌC 2023-2024 - HK...CHUYÊN ĐỀ DẠY THÊM TIẾNG ANH LỚP 11 - GLOBAL SUCCESS - NĂM HỌC 2023-2024 - HK...
CHUYÊN ĐỀ DẠY THÊM TIẾNG ANH LỚP 11 - GLOBAL SUCCESS - NĂM HỌC 2023-2024 - HK...
 
Clinical Pharmacy Introduction to Clinical Pharmacy, Concept of clinical pptx
Clinical Pharmacy  Introduction to Clinical Pharmacy, Concept of clinical pptxClinical Pharmacy  Introduction to Clinical Pharmacy, Concept of clinical pptx
Clinical Pharmacy Introduction to Clinical Pharmacy, Concept of clinical pptx
 
PISA-VET launch_El Iza Mohamedou_19 March 2024.pptx
PISA-VET launch_El Iza Mohamedou_19 March 2024.pptxPISA-VET launch_El Iza Mohamedou_19 March 2024.pptx
PISA-VET launch_El Iza Mohamedou_19 March 2024.pptx
 
Diploma in Nursing Admission Test Question Solution 2023.pdf
Diploma in Nursing Admission Test Question Solution 2023.pdfDiploma in Nursing Admission Test Question Solution 2023.pdf
Diploma in Nursing Admission Test Question Solution 2023.pdf
 
The Singapore Teaching Practice document
The Singapore Teaching Practice documentThe Singapore Teaching Practice document
The Singapore Teaching Practice document
 
3.21.24 The Origins of Black Power.pptx
3.21.24  The Origins of Black Power.pptx3.21.24  The Origins of Black Power.pptx
3.21.24 The Origins of Black Power.pptx
 
Ultra structure and life cycle of Plasmodium.pptx
Ultra structure and life cycle of Plasmodium.pptxUltra structure and life cycle of Plasmodium.pptx
Ultra structure and life cycle of Plasmodium.pptx
 
General views of Histopathology and step
General views of Histopathology and stepGeneral views of Histopathology and step
General views of Histopathology and step
 
Personal Resilience in Project Management 2 - TV Edit 1a.pdf
Personal Resilience in Project Management 2 - TV Edit 1a.pdfPersonal Resilience in Project Management 2 - TV Edit 1a.pdf
Personal Resilience in Project Management 2 - TV Edit 1a.pdf
 
2024.03.23 What do successful readers do - Sandy Millin for PARK.pptx
2024.03.23 What do successful readers do - Sandy Millin for PARK.pptx2024.03.23 What do successful readers do - Sandy Millin for PARK.pptx
2024.03.23 What do successful readers do - Sandy Millin for PARK.pptx
 
5 charts on South Africa as a source country for international student recrui...
5 charts on South Africa as a source country for international student recrui...5 charts on South Africa as a source country for international student recrui...
5 charts on South Africa as a source country for international student recrui...
 
Patient Counselling. Definition of patient counseling; steps involved in pati...
Patient Counselling. Definition of patient counseling; steps involved in pati...Patient Counselling. Definition of patient counseling; steps involved in pati...
Patient Counselling. Definition of patient counseling; steps involved in pati...
 

NISO Altmetrics Initiative: A Project Update - Martin Fenner, Technical Lead for the PLOS Article-Level Metrics project

  • 1. NISO Altmetrics Initiative: A Project Update Martin Fenner, http://orcid.org/0000-0003-1419-2405 Technical Lead, PLOS Article-Level Metrics Chair, NISO Alternative Assessment Metrics Project Steering Group http://fivethirtyeight.com/interactives/world-cup/
  • 2. Phase I • Describe the current state of the altmetrics discussion • Identify potential action items for further work on best practices and standards
  • 3. Steering Committee • Euan Adie (Altmetric) • Amy Brand (Harvard University/Digital Science) • Mike Buschman (Plum Analytics) • Todd Carpenter (NISO) • Martin Fenner (Public Library of Science, Chair) • Gregg Gordon (Social Science Research Network) • William Gunn (Mendeley) • Michael Habib (Reed Elsevier) • Nettie Lagace (NISO) • Jamie Liu (American Chemical Society) • Heather Piwowar (ImpactStory) • John Sack (HighWire Press) • Peter Shepherd (Project Counter) • Christine Stohn (Ex Libris, Inc.) • Greg Tananbaum (Scholarly Publishing & Academic Resources Coalition)
  • 4. In-Person Meetings • October 9, 2013 in San Francisco • December 11, 2013 in Washington, DC • January 23, 2014 in Philadelphia All meetings were streamed and recorded
  • 6. Interviews • Thirty researchers, administrators, librarians, funders (and others) • Semi-structured interview • March – April 2014
  • 7. Approach • Open format: lightning talks, brainstorming, breakout groups, etc. • Include all stakeholders • Focus on collecting unstructured input • Make all material (including audio recordings of steering group) publicly available
  • 10. Should metrics be hidden to prevent herd mentality? (yes, like Reddit) Quality and Gaming San Francisco Define credible sources for inclusion: Twitter, Facebook Quality and Gaming San Francisco A problem is data quality and provenance Quality and Gaming San Francisco Quality assessment of studied data Quality and Gaming San Francisco Validity of reliability of altmetrics Quality and Gaming San Francisco Data quality & validity: How valid the altmetrics data … ? have to assess? Quality and Gaming San Francisco Approaches to factor out the effects of gaming (e.g., not counting self citation) Quality and Gaming San Francisco Define acceptable promotion versus gaming Quality and Gaming San Francisco Auto-spam detection; trolling Quality and Gaming San Francisco Interactions with more traditional altmetrics Quality and Gaming San Francisco Define what behaviors are considered as cheating/gaming Quality and Gaming San Francisco Define what reaction to gaming should be public sharing? Data tainting? We ingnore it? Quality and Gaming San Francisco Are there any “ungameable” systems out there at all and can we learn anything from them? Quality and Gaming San Francisco What are effective measures to audit published altmetrics for accuracy? Quality and Gaming San Francisco
  • 12. Potential Action Items Data Quality and Gaming • Promote and facilitate use of persistent identifiers. • Research issues surrounding the reproducibility of metrics across providers. • Develop strategies to improve data quality through normalization of source data across providers. • Explore creation of standardized APIs or download or exchange formats to facilitate data gathering. • Develop strategies to increase trust, e.g., openly available data, audits, or a clearinghouse. • Study potential strategies for defining and identifying systematic gaming of new metrics.
  • 15. Next Steps • Finalize and release white paper and draft new work item proposal for standards/best practices based on the study • Proposal vetted by NISO leadership and members • Proposal approved and working groups formed for Phase II of the project
  • 17. Definitions • Altmetrics • Article-Level Metrics • Research Assessment • Scholarly Impact • Quality of Research
  • 18. Data Quality and Gaming • Disconnect between frequent concerns by users and lack of (public) activity by altmetrics providers • Strategies to improve data quality include standards, audits, open data, and a central clearinghouse • Users of altmetrics data (researchers, institutions, funders) should be more proactive in demanding data quality checks and transparency
  • 19. Grouping and Aggregation • Common practice for a single research output, e.g. for multiple versions, multiple locations, or grouping of all metrics into a single score • Common practice for multiple research outputs, e.g. by journal, by author, or by institution • Challenge of uniquely identifying authors, institutions and funders associated with these research outputs • Aggregation remains problematic, more empirical evidence and transparency needed