• Current state of clinical research

• What the scientific community can do to improve clinical research

• What you can do to improve your own research

• Provide an overview of complex problems and solutions to clinical research

• Focus on methodological and statistical issues

• False findings may be "the majority of published research claims"

• Due to statistical likelihood of finding a positive (or false positive) result

• Bias in studies

• Testing by several independent teams

• Different:
  • Statistics
    
  • Methods
  • Measures
  • Populations
  • Locations
  • Times
    
    

• It is difficult to account for heterogeneity

• Bias is usually unintentional

1. P-values can indicate how incompatible the data are with a specified statistical model.
2. P-values do not measure the probability that the studied hypothesis is true, or the probability that the data were produced by random chance alone.
3. Scientific conclusions and business or policy decisions should not be based only on whether a p-value passes a specific threshold.
4. Proper inference requires full reporting and transparency.
5. A p-value, or statistical significance, does not measure the size of an effect or the importance of a result.
6. By itself, a p-value does not provide a good measure of evidence regarding a model or hypothesis.

Example

See Greenland et al., (2016) for mor information about interpreting and using pvalues, confidence intervals and power.

• 85% of research investments are wasted (Macleod et al., 2014)

• Bias in studies leads to impediments in meta-research

• Researchers are not incentivized appropriately

• Novel publications and grants over replication studies

• Positive over null findings (journals)

• Quantity over quality

• Problem base: Is there a health problem that is big/important enough to fix?

• Context placement: Has prior evidence been systematically assessed to inform (the need for) new studies?

• Information gain: Is the proposed study large and long enough to be sufficiently informative?

• Pragmatism: Does the research reflect real life? If it deviates, does this matter?

• Patient centeredness: Does the research reflect top patient priorities?

• Value for money: Is the research worth the money?

• Feasibility: Can this research be done?

• Transparency: Are methods, data, and analyses verifiable and unbiased?

• Large-scale studies

• Reproducibility practices (Begley & Ioannidis, 2015)

• More appropriate analyses

• Improved study design standards

• Change incentives for researchers

• Preregister studies/results

• Utility in p-values
  • Focus more on precision of effect sizes
  • Consider Bayesian Credible Intervals
    
    
• Missing data
  • Missing data mechanisms
  • Avoid missing data!
    
    
• Meta-analysis
  • Garbage in garbage out
  • Fairly simple to conduct
  • A good way to learn about a subject area and gaps

• Use theories when possible

• Adapt theories based on evidence

• Recognize the limitations of a theory (Prestwich et al., 2014)

• “The idea that data analyses, and more generally, scientific claims, are published with their data and software code so that others may verify the findings and build upon them” (from Coursera)

• http://ropensci.github.io/reproducibility-guide/

• Clearly delineate how data are managed, manipulated, and analyzed in order to allow others to replicate findings with no (or minimal) assistance

• Why this is hard to achieve
  • Tight deadlines
  • Focus on quantity over quality (on both ends)
  • Not how most of us were trained
  • Heavy up front costs (e.g., time)

• Stay up to date on the literature
  • Feedly
  • Google Alerts
    
    

• Be comfortable with not having a comprehensive understanding of complex topics

• Learn to work with data analysts/biostatisticians

• Get feedback

• Collaborate with people who know more than you and are good teachers

• Participate in transdisciplinary research

• Familiarize yourself with research outside of your discipline

• Follow researchers who do good research and emulate them
  • John Ioannidis
  • Andrew Gelman
    
    

• Get really good at writing (Zinsser) and write a lot (Silva)

• Methods and statistics change just as substantive areas do

• How do you learn best
  • Didactics
  • Books
  • Practice
  • Online training (DataCamp, Coursera, Udemy)

• Research training should focus on:
  • Philosophy
  • English/writing
  • Methods
  • Statistics
    
    

• Focus on learning how to ask and answer questions

• Substantive information is easier to learn

• Doing good science is really hard

• We can do better
  • Structural changes (granting agencies, journals, academic institutions)
  • More personal responsibility
    
    
• Chase ideas instead of grant dollars
  • Focus on programmatic and collaborative research