Introduction

In clinical trials, triple-blinding is a state where the participants, people running the trial data and those assessing the data are all blinded to treatment assignment. The statisticians that run the trial analyses need to be aware of which records belong to which group. However, in simple scenarios, they do not need to know which arm is the SOC and which arm is the active treatment because the parameter estimates have symmetry. For example, imagine a trial with an active control standard of care and a new treatment where the endpoint is 7 day recovery from illness and the effect estimate is a difference in proportions. The analysis could be implemented via a beta-binomial model. We can simulate some trial data and then take draws from the posterior distribution for the proportion recovering by day 7 in each arm and the treatment effect as follows:

set.seed(2)
nsim <- 1000
n <- c(10, 10)
y <- c(3, 7)
smpls <- sapply(1:2, function(i){rbeta(nsim, 1 + y[i], 1 + n[i] - y[i])})
dens <- lapply(1:2, function(i){density(smpls[, i])})
delta <- smpls[, 2] - smpls[, 1]
densdel <- density(delta)

Posterior for proportion recovered by treatment arm.

Posterior distribution for treatment effect (difference in proportions).

We could report the treatment effect as having a median of 0.33 (95% CI -0.05 to 0.67) and \(Pr(\delta > 0) \approx 0.96\). If we needed 95% confidence that the treatment arm had a greater recovery rate than the SOC arm, then we could declare this finding on the basis of the evidence we have collected.

quantile(delta, probs = c(0.5, 0.025, 0.975))
##         50%        2.5%       97.5% 
##  0.32951944 -0.05315228  0.66745496
mean(delta > 0)
## [1] 0.962

In this case it makes no difference if we hadn’t of known the treatment assignment. If the reference arm was the treatment arm then we would get an effect estimate of -0.33 (95% CI -0.67 to 0.05) and \(Pr(\delta > 0) \approx 0.04\).

delta <- smpls[, 1] - smpls[, 2]
quantile(delta, probs = c(0.5, 0.025, 0.975))
##         50%        2.5%       97.5% 
## -0.32951944 -0.66745496  0.05315228
mean(delta > 0)
## [1] 0.038

In other words, saying that the treatment arm is better than the SOC is just the same as saying that the SOC is worse than the treatment arm. Symmetry is our friend.

Adaptive Trials

In an adaptive setting, the symmetry that is present in the simple case discussed above might not exists. For example, a trial with an adaptive sample size with the adaptations occurring at interim analyses might adopt the following procedure.

  1. Compute the posterior distribution for the treatment effect at the current interim sample size.
  2. Use the posterior predictive distribution to simulate many trials run to the maximum sample size.
  3. In each simulated trial determine whether a treatment effect was detected at a given probability threshold.
  4. If the proportion of simulated trials showing a treatment effect is low then abandon the trial for futility, otherwise progress to the next interim.

The third bullet is important. We need to make a decision, each simulated trial either showed a treatment effect or it didn’t at the nominated probability threshold.

Assuming a maximum sample size of 20 in each arm, we can extend the earlier example as shown below. However, this time assume that the data only shows a marginal improvement in one arm. The first interim did not achieve sufficient confidence to stop the trial so we run an interim analysis to see if it is worth randomising another 10 people to each arm.

set.seed(2)
nsim <- 1000
n <- c(10, 10)
y <- c(4, 6)
smpls <- sapply(1:2, function(i){rbeta(nsim, 1 + y[i], 1 + n[i] - y[i])})
dens <- lapply(1:2, function(i){density(smpls[, i])})
delta <- smpls[, 2] - smpls[, 1]
densdel <- density(delta)
# simulated recovery events from future enrollments
newdata <- sapply(1:2, function(i){rbinom(nsim, 10, smpls[, i])})
win <- unlist(lapply(1:nsim, function(i){
  newsmpls <- sapply(1:2, function(j){
    rbeta(nsim, 
          1 + y[j] + newdata[i, j], 
          1 + 20 - y[j] - newdata[i, j])
    })
  newdel <- newsmpls[, 2] - newsmpls[, 1]
  mean(newdel > 0) > 0.95
}))
mean(win)
## [1] 0.267

Given that only 27% of simulated trials we achieved the level of evidence required, we might or might not choose to continue to the trial.

When the arms are unknown, the decision rule breaks down at the level of evidence required here. This can be seen below where we switch the reference arm and end up concluding that none of the simulated trials were successful, which would imply that the trial should be stopped for futility.

# simulated recovery events from future enrollments
newdata <- sapply(1:2, function(i){rbinom(nsim, 10, smpls[, i])})
win <- unlist(lapply(1:nsim, function(i){
  newsmpls <- sapply(1:2, function(j){
    rbeta(nsim, 
          1 + y[j] + newdata[i, j], 
          1 + 20 - y[j] - newdata[i, j])
    })
  newdel <- newsmpls[, 1] - newsmpls[, 2]
  mean(newdel > 0) > 0.95
}))
mean(win)
## [1] 0

However, for this trial, declaring futility would be the wrong thing to do. A way around the problem of the lack of symmetry is to compute the predicted probability of success for both \(\delta = \theta_1 - \theta_0\) and \(\delta = \theta_0 - \theta_1\). If either of these results suggest that a statistical trigger has occurred then the DSMC might want to request that they be unblinded in order to make a decision on what to do.

Regardless of how this was to be operationalised, the situation is not as simple as the introductory two arm trial where the results were based solely on the posterior distribution from the observed data.