On Sep 17th Catalyst brought a panel of academic and industry thought leaders for a lively discussion on the issue of data reproducibility in academic research. Moderated by Cathy Tralau-Stewart, head of the Therapeutics track of the Catalyst Awards, the panel explored causes and potential solutions for a problem that has been receiving national attention in both scientific and popular media. Panelists included Keith Yamamoto, Vice Chancellor for Research at UCSF; Larry Tabak, Principle Deputy Director, NIH; John Ioannidis, Professor of Health Research Policy at Stanford School of Medicine; Elizabeth Iorns, Co-Founder, Science Exchange; Parker B. Antin, Board of Directors President, FASEB; Amanda Halford, MBA, VP of Research, Sigma-Aldrich. http://ctsi.ucsf.edu/news/about-ctsi/data-reproducibility-preclinical-research-and-discovery

1. Clinical and Translational
Science Institute / CTSI
at the University of California, San Francisco
Data Reproducibility in Preclinical Discovery,
Is it a Real Problem?
Cathy Tralau-Stewart PhD FRSC
Head, Therapeutics Track, UCSF Catalyst

2. Thank you to our sponsors

3. Why is Reproducibility
Currently
Hitting The Headlines ?

4. ‘The push to replicate findings could
shelve promising research and
unfairly damage the reputations of
careful, meticulous scientists’ Mina
Bissell 2013
‘the challenge of increasing
reproducibility and addressing the
costs associated with the lack of
reproducibility in life science
research is simply too important and
costly to ignore’. Freedman 2015
‘the guidelines could make preparing,
submitting and reviewing research
papers more burdensome, and that
the checklists and other requirements
imposed by journals could make it
more difficult to recruit high-quality
peer reviewers’ Haywood 2015
‘The recent evidence showing the irre-
producibility of significant numbers of
biomedical-research publications demands
immediate and substantive action’ Collins &
Tabak 2015
‘Our observations indicate
that literature data on
potential drug targets
should be viewed with
caution’ Prinz et al 2014
The Debate

5. Potential Implications of Non Reproducibility
Globally
• Estimated $28bn wasted per year (50% of US Pre-Clinical spend) due
to reproducibility issues
• Reduction in;
– products
– Patient benefit
– Return on Investment (ROI)
– Innovation
• Exposure of patients to ineffective or dangerous new drugs
For The Research Community
Proof of reproducibility may soon be a requirement for
• Publication
• Securing funding & Investment

6. Evidence for Poor Reproducibility
Freedman LP et al PLOS 2015

7. ~50% of Annual US Pre-Clinical Research Spend Are
Subject to Reproducibility Errors
Freedman LP et al PLOS 2015

8. Cathy Tralau-Stewart, PhD
Head, Therapeutics Track,
UCSF Catalyst
Moderator:
UCSF–CTSI Catalyst Awards presents
Data Reproducibility in Preclinical
Discovery, Is it a Real Problem?
Panelists:
Keith Yamamoto, PhD
Vice Chancellor for Research
Executive Vice Dean,
UCSF School of Medicine
John Ioannidis, MD, DSc
Professor of Health Research Policy
Stanford School of Medicine
Parker Antin, PhD
FASEB
Board of Directors President
Amanda Halford, MBA
VP of Academic Research
Sigma-Aldrich
Elizabeth Iorns, PhD
Co-Founder & CEO
Science Exchange
Lawrence Tabak, DDS, PhD
Principal Deputy Director
NIH
Questions for Panel: Catalyst@ucsf.edu