Overview

  • What is reproducible research?
  • Why do we care?
  • Why reproducibility questions arise?
  • The cost of reproducibility
  • Reproducibility and statistics
  • Current status of reproducibility
  • What can we do?

What is reproducible research?

Reproducibility and scientific progress

  • Science is the systematic enterprise of gathering knowledge about the universe and organizing and condensing that knowledge into testable laws and theories

  • The success and credibility of science are anchored in the willingness of scientists to expose their ideas and results to independent testing and replication by other scientists.

http://www.aps.org/policy/statements/99_6.cfm

What is reproducible research?

  • Reproducibility
  • Replicability
  • Repeatability
  • Reliability
  • Robustness
  • Generalizability

 

  • Transparency    
  • Open Science    
  • TRUTH

What is reproducible research?

Reproducible research is the ultimate standard for strengthening scientific evidence by independent:

  • Investigators
  • Data
  • Analytical methods
  • Laboratories
  • Instruments

The beginning of reproducible research

Galileo Galilei

Why do we care?

More data = more chance for errors

  • High-throughput biology generates volumes of data

  • Data-generating technologies are increasingly used to make clinical recommendations and treatment decisions

  • A problem may be overlooked .. Published .. Get in clinical trials

More data = more chance for errors

Clinical trials based on flawed and fraudulent data

Clinical trials based on flawed and fraudulent data

  • Described drug response “gene signatures” in NCI60 cell lines
  • Demonstrated these “signatures” correspond to patient-specific signatures and can be used to predict patient response to the drugs