This document discusses the importance of reproducibility and transparency in data analysis. It notes that an analysis can be fully reproducible but still produce incorrect results. Proper training, not just tools, is needed to ensure reliable research. While tools for reproducibility exist, scaling training to large groups is challenging. Prevention of problems through transparency is emphasized over later treatment of issues found in analyses.

1. Evidence based data analysis
Jeff Leek
@simplystats

2. What Roger said but with more pictures.
Jeff Leek
@simplystats

4. The tools for reproducibility
already exist.*

8. I think that it is ironic that a paper that
claims to prove the reliability of the
literature had completely messed up
the two main figures that represent the
core of all its data and its main results.
“
”

9. An analysis can be fully
reproducible and still be
wrong.
“
”

10. http://bioinformatics.mdanderson.org/Supplements/ReproRsch-All/

11. Training not tools

12. Fisher et al. (2014) http://dx.doi.org/10.7717/peerj.589

13. Fisher et al. (2014) http://dx.doi.org/10.7717/peerj.589

14. Prevention not treatment

15. Leek and Peng (2015) doi: 10.1073/pnas.1421412111

17. Scaling training

19. 1,750,000+
Enrolled

20. Even simple things are
hard at scale

21. Leek and Peng (2015) DOI: 10.1126/science.aaa6146