Section 1 Describe General Problem

When considering whether a patient is likely to benefit from a therapy, the most relevant measure of treatment effect is the absolute risk reduction (ARR) of a treatment. It is well known that a study’s overall ARR or NNT will often not reflect a treatment’s true ARR for many people in the trial, since a 25% relative risk reduction (RRR) in high risk patients produces much more benefit than it does in low-risk patients resulting in substantial heterogeneity in treatment effect (HTE). Identifying a subgroup of patients who will benefit is a difficult task.

(Kent et al. 2010) Assessing and reporting heterogeneity in treatment effects in clinical trials: a proposal.

  • Authors propose reporting on the risk-based heterogeneity at the end of each trial

  • HTE can be shown with multivariable modeling.

(Ebrahim and Smith 1999) The ‘number need to treat’: does it help clinical decision making?

(???) Can overall results of clinical trials be applied to all patients?

(Furukawa, Guyatt, and Griffith 2002) Can we individualize the ‘number needed to treat’? An empirical study of summary effect measures in meta-analyses

(Kent et al. 2016) Risk and treatment effect heterogeneity: re-analysis of individual participant data from 32 large clinical trials

  • Report from 32 trials where they collected patient level data, and showed HTE

Section 2 Overview of Existing Methods

  • Enrolling all participants

    • Enroll all and show heterogeneity among all patients (Kent et al. 2010)
    • Enroll all and investigate interactions, for example by baseline risk (TBBI: Treatment-by-biomarker interaction)

  • Enrichment

    • Enroll high risk patients more likely to benefit from a therapy
    • Adaptive Enrichment Enrollment (Simon 2015)
    • Biomarker-Adaptive Threshold Design (Jiang, Freidlin, and Simon 2007)

  • I want to incorporate adaptive enrichment designs, with the reporting of HTE by multivariable modeling suggested by (Kent et al. 2010), and Bayesian sequential designs to adaptively enroll patients in a trial who will benefit (i.e. who are above the individualized NNT based on their baseline risk and the estimated effect size).

  • Bayesian Sequential Designs

    • (Spiegelhalter, Freedman, and Parmar 1994) Bayesian Approaches to Randomized Trials

    • For review: Why a Bayesian Approach to Drug Development and Evaluation? Frank E Harrell Jr (not published)

    • Controlling “Type I/II Error” in Bayesian Sequential Designs

      • (Shi and Yin 2019) Control of Type I Error Rates in Bayesian Sequential Designs
      • (Ryan et al. 2020) Do we need to adjust for interim analyses in a Bayesian adaptive trial design?
        • Authors show that yes, you must control for the number of looks

  • Other useful references

    • (Kent, Steyerberg, and Klaveren 2018) Personalized evidence based medicine: predictive approaches to heterogeneous treatment effects

    • (Howard, Coffey, and Cutter 2005) Is Bayesian Analysis Ready for Use in Phase III Randomized Clinical Trials?

  • Example Trial Data

    1. Trial shown to have significant HTE that I think I can get patient-level data on (Sussman et al. 2015)
    2. I just finished work on a trial that did not show an effect (N ~ 1800). There is heterogeneity in risk in the population, and I wonder if there were fewer low risk patients enrolled we could have shown an effect of the intervention. I have full access to use this set of data.

References

Ebrahim, S, and G Davey Smith. 1999. “The ‘number need to treat’: does it help clinical decision making?” Journal of Human Hypertension 13 (11): 721–24. https://doi.org/10.1038/sj.jhh.1000919.

Furukawa, Toshiaki A, Gordon H Guyatt, and Lauren E Griffith. 2002. “Can we individualize the ‘number needed to treat’? An empirical study of summary effect measures in meta-analyses.” International Journal of Epidemiology 31 (1): 72–76. https://doi.org/10.1093/ije/31.1.72.

Howard, George, Christopher S Coffey, and Gary R Cutter. 2005. “Is Bayesian Analysis Ready for Use in Phase III Randomized Clinical Trials?” Stroke 36 (7): 1622–3. https://doi.org/10.1161/01.str.0000170638.55491.bb.

Jiang, Wenyu, Boris Freidlin, and Richard Simon. 2007. “Biomarker-Adaptive Threshold Design: A Procedure for Evaluating Treatment With Possible Biomarker-Defined Subset Effect.” JNCI: Journal of the National Cancer Institute 99 (13): 1036–43. https://doi.org/10.1093/jnci/djm022.

Kent, David M, Jason Nelson, Issa J Dahabreh, Peter M Rothwell, Douglas G Altman, and Rodney A Hayward. 2016. “Risk and treatment effect heterogeneity: re-analysis of individual participant data from 32 large clinical trials.” International Journal of Epidemiology 45 (6): dyw118. https://doi.org/10.1093/ije/dyw118.

Kent, David M, Peter M Rothwell, John P A Ioannidis, Doug G Altman, and Rodney A Hayward. 2010. “Assessing and reporting heterogeneity in treatment effects in clinical trials: a proposal.” Trials 11 (1): 85. https://doi.org/10.1186/1745-6215-11-85.

Kent, David M, Ewout Steyerberg, and David van Klaveren. 2018. “Personalized evidence based medicine: predictive approaches to heterogeneous treatment effects.” BMJ 363: k4245. https://doi.org/10.1136/bmj.k4245.

Ryan, Elizabeth G, Kristian Brock, Simon Gates, and Daniel Slade. 2020. “Do we need to adjust for interim analyses in a Bayesian adaptive trial design?” BMC Medical Research Methodology 20 (1): 150. https://doi.org/10.1186/s12874-020-01042-7.

Shi, Haolun, and Guosheng Yin. 2019. “Control of Type I Error Rates in Bayesian Sequential Designs.” Bayesian Analysis 14 (2): 399–425. https://doi.org/10.1214/18-ba1109.

Simon, Noah. 2015. “Adaptive enrichment designs: applications and challenges.” Clinical Investigation 5 (4): 383–91. https://doi.org/10.4155/cli.15.9.

Spiegelhalter, David J, Laurence S Freedman, and Mahesh K B Parmar. 1994. “Bayesian Approaches to Randomized Trials.” Journal of the Royal Statistical Society. Series A (Statistics in Society) 157 (3): 357. https://doi.org/10.2307/2983527.

Sussman, J B, D M Kent, J P Nelson, and R A Hayward. 2015. “Improving diabetes prevention with benefit based tailored treatment: risk based reanalysis of Diabetes Prevention Program.” BMJ 350 (feb19 2): h454–h454. https://doi.org/10.1136/bmj.h454.