Use of Baseline in RCT Guideline

Unreleased befor revision ## CAPTOPRIL DRUG RCT

On RCT Drug researcher aim to found an effect on drug by testing an active arm treated with that drug against a Placebo one.

Before having the sample and result in hand end of Phase I we have to design the experiments gathers some preliminary works , data of preecedings test and historic reviewing. It is worth to mention that already at this stage power has to be calculated on presumed parameters and desired-estimated biophysical targer value of treatment effect Power must be calculated NOT AFTER the result of the data experiment in hand as good RCT practice.

Drugs is all about nature _Love Snakes

Snake are our best friend in biopharma/or worst deadly enemies in nature

A bite of bitis jararaca (Viperidae/Bitisdae of Brazil) can cause severe hematostase problems such as:

Pro Coagulation (Clots) or
Anti Coagulation (hemoragia) -
Collapse of blood pressure (BP)

This last effect was the aim of the proposed data and research.

The venom has the power to act and inhibit the cascade of renine angiotensine enzymatic activated reactions and if so produce a major vaso dilatation , relaxes soften the stiffness of arteries promoting collapse of blood pressure. For more detail please refer to physiology litterature.

Such effect is called ENZYME CONVERSION INHIBITING and that is extracted from the venom and synthesized as drug (after changing some pattern of branching of radicals).

A Special case of an expirment with low power who turn as a success story

At the early stage of a design of RCT experiment some parameters were given u.e in a systematic review of blood pressure research: Let might guess some:

sd of population under investigation about 10 mmHg at first measure but if repeated and averaged can be lowered to 5 mmHG (averaging kill the autocorrelation). The population mean is higher than normal let say by 10-20 mmHG. We hope for a drop of at least 10 mmHG in avg meaning that a patient in Hypetensive class HT1A will return to the first week of treatment to a unpathological value. To be a commercial success and for a long term perspective we hope reaching to a drop of 20 mmHg to be a very efficient drug in the market.

Note:several outcomes variables is possible Here clearly(?) stated : THE PRIMARY outcome is BP measure in mmHg.

HOWEVER IT IS NOT REPORTED BY WHICH TECHNICAL PROCESS IT HAS BEEN MEASURED WHICH HAS A MAJOR IMPACT ON PRECISION.

Although a standadrized Cohen d to be at 1.0 can presumably be assumed to ease our calculation.

A Two sided Hypothesis approach?

Of course ! if no systematic review nor meta analysis of CAPTOPRIL exist and proved a drop in BP the gold standard of statistic is to used a two sided approach to be on safe side. In 1986 no such analysis exist and Captopril came onto the market.

In Biopharma we prefer to use instead of one sided “lower” a non inferiority or equivalent trial design even for the same drug.

Power parameters estimates before all

Let’s investigate and calculate the two sample power t test ( the most use test in RCT) to have an id of either n or the reached power:

power.t.test(power=0.80,delta=1)

## 
##      Two-sample t test power calculation 
## 
##               n = 16.71477
##           delta = 1
##              sd = 1
##       sig.level = 0.05
##           power = 0.8
##     alternative = two.sided
## 
## NOTE: n is number in *each* group

power.t.test(power=0.60,delta=1)

## 
##      Two-sample t test power calculation 
## 
##               n = 10.83754
##           delta = 1
##              sd = 1
##       sig.level = 0.05
##           power = 0.6
##     alternative = two.sided
## 
## NOTE: n is number in *each* group

power.t.test(power=0.40,delta=1)

## 
##      Two-sample t test power calculation 
## 
##               n = 6.900428
##           delta = 1
##              sd = 1
##       sig.level = 0.05
##           power = 0.4
##     alternative = two.sided
## 
## NOTE: n is number in *each* group

Well we aim all to reach a 80 % power to detect an effect when it exist but there is still a 1/5 at that stage that the effect can be missed [Type II] (H1 vs Ho).

we don’t know the reason but it was recruited n1=9 patients in the Treated Group under CAPTOPRIL and n2 = 7 for the arm of the PLACEBO group

Note that unbalanced arms increase variance estimate and power calculation.
Anova type II -III main effect and applied contrast should be applied
Note that despite 𝛔² common to groups, s² although unbiased mathematically (n-1) might be quite different when n is low.

The presented Data come from:

*Hommel, E. et al. (1986), Effect of Captopril on kidney func tion in insulin-dependent diabetic patients with nephropathy, British Med ical Journal, 293, 467–470*

## 
##  Descriptive statistics by group 
## group: BaselineP
##    vars n   mean    sd median trimmed   mad min max range  skew kurtosis   se
## X1    1 7 146.57 12.29    152  146.57 13.34 129 161    32 -0.28    -1.84 4.64
## ------------------------------------------------------------ 
## group: trt1Wpla
##    vars n   mean   sd median trimmed  mad min max range skew kurtosis   se
## X1    1 7 141.86 6.94    139  141.86 7.41 134 151    17 0.19    -2.02 2.62

## 
##  Descriptive statistics by group 
## group: BaselineC
##    vars n mean    sd median trimmed   mad min max range  skew kurtosis   se
## X1    1 9  148 11.43    151     148 10.38 129 164    35 -0.32    -1.38 3.81
## ------------------------------------------------------------ 
## group: trtCapto
##    vars n   mean   sd median trimmed  mad min max range  skew kurtosis   se
## X1    1 9 135.33 8.43    137  135.33 5.93 120 144    24 -0.79    -1.07 2.81

                         **SUMMARY DATA TABLE**

*mmHg*	MEAN	SD	N
CAPTOPRIL1W	135.33	8.43	9
PLACEBO1W	141.86	6.94	7
TRT DIFF	- 6.53	7.82*
BASELINE CAPTO	148.00	11.43	9
BASELINE PLACEBO	146.57	12.29	7
BASELINE DIFF	1.43

*POOLED SD

Note that TRT C-P are independant RV that is no cov is removed by difference
Note that every Baseline and trt in each arm are correlated as repeated measurments on same ID.

PLOTS

XY REGRESSION PLOTS

On the 1:1 xyplot we note that:

CAPTOPRIL data show systematic lowering after 1 week of treatment that is away from red line
1/9 Point of CAPTCAPTOPRIL after 1Week trt is higher than his baseline
2/7 points are at higher value the PLACEBO 1 week” trt” especially at lower value. These pattern “might?” be typical of a regression to the mean, algtough the TRUE the effect of PLACEBO might exist too (Cofounded here because treatment should lower the x_i)
PLACEBO group are closer to the 1:1 RED line.Higher value tend to be closer to the red line too,lower value higher .(see RTM comments)
Both regression lines are about the same slope.

This is not the best plot to easealy– understand — capture the value of the data and trt effect however it is sometimes presented in systematic review.

PLOT MEANS AND SD

## 
## Attachement du package : 'gplots'

## L'objet suivant est masqué depuis 'package:stats':
## 
##     lowess

As stated before the lower mean is reached after 1 week of CAPTOPRIL trt and is clearly seen ; the slope angle between the two arm from their baseline is also noticeable ,

But it let no doubt of a mean difference.

Baseline is however a bit higher in Captopril trt group

The sd seems constantly lower after 1 Week of trt. (RTM also affect the sd!)

CHANGE FROM BASELINE ; - DIFFERENCE PLOT

It is clearly seen here that:

Captorpil lowered (8/9 values) mmHG to a Higher value range and often above the targeted value of 10mmg (7/9Values) times.
Placebo lowered 5 out 7 to a lower value and not above the targeted value of 10mmg 2 out of 7 times.

A binomial test is let at the discretion !

BOXPLOTS AND VIOLIN PLOTS

par(mfrow=c(1,2))
boxplot(Placeboolong$Pla~Placeboolong$groupPlace,ylim=c(120,160),col=c(8,2),main="PLA")
boxplot(Capto$Basecapto,Capto$Capto1W,col=c(8,3),ylim=c(120,160),main="CAPT",xlab="Baseline-TRT")#outliers and se with N0T not informative

vioplot::vioplot(Placeboolong$Pla~Placeboolong$groupPlace,col=c(8,2),ylim=c(120,160))
vioplot::vioplot(Capto$Basecapto,Capto$Capto1W,col=c(8,3),ylim=c(120,160),xlab="Baseline-TRT")

They are far less important with such low sample size expect for Baseline:

boxplot(Placbaseline,Capto$Basecapto,main="BASELINE BOXPLOT",xlab="Placebo vs Captopril",ylab="mmHg")

Violin are given for an estimate of Normality check but not necessary.

Note 1: Worrying about the normality on that range of X value despite the skew and kurtosis is not assessable and should be considered as normal (low N ,shapito etc) nor no transformation needed-
Note 2: The regression to the mean occur when a statistical unit is measure a second time (repeated measure) as just because , aside of all other effect,- the probability in Normal law is higher to be closer to the common mean that to the extreme(See the Normal density curve)
Note 3 Although RTM will exsit when using the baseline measurment we suppose the they are the same magnitude in both arm therfore cancelling each other when making a difference comparison in both arms and when including their baseline.
Note 3+ It seems that RTM occurs has sd has dimnished after controlling for baesline in both arm that is measurements closer to their mean possibly? but is also cofounded with the effecz of removing covaviance when substraction 2 RV (Baseline-Trt_1Week) .Please refer to last paragraph about conditional variance for some explanation.
Note 4: When we will perform some t test var equal TRUE because one assumptions of RCT coming from same population but of course questionable as when trt ocurs urss it might affect the variability in some sense b< biological timed response for each individuals.Please refer to last paragraph about conditional variance. This subject It is at let at the interpretation of the researcher.

ASSESING THE TREATMENT

Some assumption before all in RCT :

The population shared a common mean and variance at start. SRS ensure that randomization is corrretly performed.

Samples might differs due to random error in sampling.

THE UNIVARIATE APPROACH

The easiest way to compare the effect of 2 groups and trt effect is to perform an independant t test against the trt:

Univariate Independant samples CAPTO VS PALCEBO at 1 week time points:

t.test(Capto1W,Plac1W,var.equal = TRUE)

## 
##  Two Sample t-test
## 
## data:  Capto1W and Plac1W
## t = -1.6547, df = 14, p-value = 0.1202
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
##  -14.979791   1.932172
## sample estimates:
## mean of x mean of y 
##  135.3333  141.8571

135.333-141.857###mmHG diff

## [1] -6.524

The two sample T test after 1 week of CAPTOPRIL VS PLACEBO give us a two sided p value of:

PVal= 0.12 with 14 Df

Note :If the trail were one sided the p value would have been P=0.06 Therefore we cannot reject the null:

They are no significant treatment difference in means expect a random variation in the difference of the groups.

NO drug effect? Dont stop the job….

Note the difference in MEANS is in favor of Captopril by a drop value of 6.53mmHG ! what ever the significance!

Confidence interval

Let guess what would be the interval around -6.53mmHG that could include the true value of the difference assuming a p value of 0.05 that is a z quantile value of :

qt(0.975,14)

## [1] 2.144787

A z-value of 2.145 instead of 1.66547 would have produce a p value of 0.05

If you construct this intervals you will found:

-15 mmHg to 2 mmHg

Note that zero is included in the range of course because the diff between Captopril1W and Placbo isnt significant !

So it might be that we have not enough power to detect the true effect due to low sample size?

But: for the drug a 2mmHG increase by CAPTOPRIL isn’t a big trouble for the Patient

HOWEVER a drop a 15 mmHG of BP can be reached at the limit of the 95% []

The true value of the drop might be above the targeted value and HAS a MAJOR impact on Blood Pressure lowering isn’t?

This is why we always report the confidence interval to evaluate the biophysical benefit of a drug trial

A CONDITIONAL APPROACH:

USING THE BASELINE INFORMATION

Has stated on the assumptions the baseline has a common population mean and if any diff. in the sample (here 1.43 higher in CAPTOPRIL arm) is just due to random sampling error nothing else.

In any doubt of the randomization process it can be still confirmed by a independant T test between the baseline.

Also to ocnfirm : The regression Slope of baseline are about to be equal:

lm(Plac1W~Placbaseline)

## 
## Call:
## lm(formula = Plac1W ~ Placbaseline)
## 
## Coefficients:
##  (Intercept)  Placbaseline  
##      75.5763        0.4522

lm(Capto1W~Basecapto)

## 
## Call:
## lm(formula = Capto1W ~ Basecapto)
## 
## Coefficients:
## (Intercept)    Basecapto  
##     66.8515       0.4627

All about Variance:

When for the Captopril group you make the difference between hi baseline and the trt after 1 week depiste a common population variance that is:

VAR (TRT-BASELINE)=VAR(TRT)+VAR(BASE)-2COV(TRT-BASELINE)

VAR(DIFF_capto)= 𝛔^2 + 𝛔² - 2𝞺(𝛔2)

So if 𝞺 is higher that +0.5 you reach a LOWER VARIANCE that is the trick!

The covariance come from the repeated measurement on the same ID after 1 week! This break the independance required in statistics and some change apply.

This times we will take a greater advantages in information of correlated data!

The same variance change apply to the Placebo arm-

CORRELATION WITH BASELINE

cor(Capto$Basecapto,Capto$Capto1W)##CAPTO

## [1] 0.6279215

cor(Placbaseline,Plac1W)##PLACEBO

## [1] 0.8007418

DOD T-TEST

Let see what a two samples T test between the mean difference between Trt_i-baseline_i in each arm can bring us as result:

t.test(Capto$Capto1W-Capto$Basecapto,Plac1W-Placbaseline,var.equal = TRUE)

## 
##  Two Sample t-test
## 
## data:  Capto$Capto1W - Capto$Basecapto and Plac1W - Placbaseline
## t = -1.8474, df = 14, p-value = 0.08592
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
##  -17.184764   1.280002
## sample estimates:
##  mean of x  mean of y 
## -12.666667  -4.714286

Now bringing the conditional information of BASELINE the mean diffrence from Baseline in Captopril arm is of 12.7 mmHg drop vs a 4.7 mmHg in the Placebo arm. that is a 8 mmHG mean difference. Recall that the pooled variance was 7.82 mmHg above but now is much lower as the correlation-information has been implemented in the test design .

The two independant T test has a Pvalue = 0.08592 two sided at the 5% level with df 14.

Note: If a one sided was elected at early stage the p value would have been significant of half of 0.08592.

CONFINT

Now the confidence interval has broaden in favor of Captopril and we might expect a drop of more than 17 mmHg

ANCOVA APPROACH:

Usually and broadly speaking ANCOVA is a linear model where the interaction of a factor with an continuous variable make a slope estimate between the level of that factor (TRT) and a controlling variable.

Tricky Question:

However here the baseline is not a controlling factor per se as it as no direct connection with the administration of the trt thereafter and neither has a longitudinal effect. However in Placebo group has no trt is given it is hard to justify the above reasoning.

So we might NOT expect a significant intercation effect within trt*baseline if set in the linear model (lm).

Lets proove it by an ANOVA type I (Sequential ANOVA):¹

## trt
## CAP PLA 
##   9   7

## 
## Call:
## lm(formula = dfca$Y ~ dfca$Baseline + dfca$trt + dfca$trt * dfca$Baseline)
## 
## Coefficients:
##               (Intercept)              dfca$Baseline  
##                  66.85150                    0.46272  
##               dfca$trtPLA  dfca$Baseline:dfca$trtPLA  
##                   8.72484                   -0.01051

## Analysis of Variance Table
## 
## Response: dfca$Y
##               Df Sum Sq Mean Sq F value  Pr(>F)  
## dfca$Baseline  1 374.66  374.66  8.0723 0.01308 *
## Residuals     14 649.78   46.41                  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

## Analysis of Variance Table
## 
## Response: dfca$Y
##               Df Sum Sq Mean Sq F value   Pr(>F)   
## dfca$Baseline  1 374.66  374.66  10.878 0.005768 **
## dfca$trt       1 202.04  202.04   5.866 0.030791 * 
## Residuals     13 447.75   34.44                    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

## Analysis of Variance Table
## 
## Response: dfca$Y
##                        Df Sum Sq Mean Sq F value   Pr(>F)   
## dfca$Baseline           1 374.66  374.66 10.0424 0.008085 **
## dfca$trt                1 202.04  202.04  5.4154 0.038271 * 
## dfca$Baseline:dfca$trt  1   0.05    0.05  0.0014 0.970391   
## Residuals              12 447.69   37.31                    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Because both Baseline have about same slope they cannot account for an effect due SRS design. Therefore including such interaction doesn’t make really sense and is not recommended.

TREATMENT AND CONFINT IN ANCOVA

Well lets calculate the mean TRT effect: CAPTO-PLA after controlling for baseline :In a contrast treatment regression it is given in the coefficient, the PLA included in the intercept.

We obtain a difference of 7.17 mmHg in favor of Captorpil with a two sided p value of 0.031.

For teh confint we need to take the SE of the coefficient regression and compute the quantile of a Student with t = 13 thats is 2.160

Hence the confint extend from 0.77 mmHg to 13.57 mmHg.

summary(mo2)

## 
## Call:
## lm(formula = dfca$Y ~ dfca$Baseline + dfca$trt)
## 
## Residuals:
##    Min     1Q Median     3Q    Max 
## -9.129 -3.445  1.415  2.959 11.076 
## 
## Coefficients:
##               Estimate Std. Error t value Pr(>|t|)   
## (Intercept)    67.5731    19.7577   3.420  0.00456 **
## dfca$Baseline   0.4578     0.1328   3.446  0.00434 **
## dfca$trtPLA     7.1779     2.9636   2.422  0.03079 * 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 5.869 on 13 degrees of freedom
## Multiple R-squared:  0.5629, Adjusted R-squared:  0.4957 
## F-statistic: 8.372 on 2 and 13 DF,  p-value: 0.004608

qt(0.975,13)

## [1] 2.160369

7.17+2.160*2.9636

## [1] 13.57138

7.17-2.160*2.9636

## [1] 0.768624

CONTRAST : CARE FOR ANOVA III

Here trt is a Treatment contrast factor (default in R::) that is 1 or 0 reference level:

library(car)

## Le chargement a nécessité le package : carData

contrasts(trt)#rfef is CAPTO so expect an increase

##     PLA
## CAP   0
## PLA   1

Anova(mo3)##Type II recommended

## Anova Table (Type II tests)
## 
## Response: dfca$Y
##                        Sum Sq Df F value   Pr(>F)   
## dfca$Baseline          409.11  1 10.9659 0.006207 **
## dfca$trt               202.04  1  5.4154 0.038271 * 
## dfca$Baseline:dfca$trt   0.05  1  0.0014 0.970391   
## Residuals              447.69 12                    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Anova(mo3,type="III")#WREONG HERE CONTAST Caution due to ortogonality TYPE III required contr sum

## Anova Table (Type III tests)
## 
## Response: dfca$Y
##                        Sum Sq Df F value  Pr(>F)  
## (Intercept)            212.29  1  5.6903 0.03442 *
## dfca$Baseline          223.95  1  6.0029 0.03059 *
## dfca$trt                 1.69  1  0.0454 0.83480  
## dfca$Baseline:dfca$trt   0.05  1  0.0014 0.97039  
## Residuals              447.69 12                  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Anova type III is WRONG HERE due to ortogonality break up .

In SAS the contrast is SUM and for anova type III adj SS it is mandatory to set the contrast to SUM in R if one want type use this Anova type .

The wrong way to test TRT in a RCT by pairing in each arm and comapring P value

t.test(Captolong$trt~Captolong$groupBC,paired=TRUE)#test captopril vs his baseline

## 
##  Paired t-test
## 
## data:  Captolong$trt by Captolong$groupBC
## t = 4.2288, df = 8, p-value = 0.002881
## alternative hypothesis: true mean difference is not equal to 0
## 95 percent confidence interval:
##   5.759338 19.573995
## sample estimates:
## mean difference 
##        12.66667

t.test(Placeboolong$Pla~Placeboolong$groupPlace,var.equal=TRUE,paired=TRUE)#test Placebo vs his baseline

## 
##  Paired t-test
## 
## data:  Placeboolong$Pla by Placeboolong$groupPlace
## t = 1.5768, df = 6, p-value = 0.1659
## alternative hypothesis: true mean difference is not equal to 0
## 95 percent confidence interval:
##  -2.601439 12.030010
## sample estimates:
## mean difference 
##        4.714286

Such test is a flaw because: In this kind of test it is missing the main aim of the research:

” you need to test an effect between two treatments” (or one if placebo is considered with no effect changer)

you are not testing one arm against another but you check only baseline vs trt

two paired t test don’t estimate the treatment effect form Placebo to the active CAPTOPRIL.
Comparing p-values for an evidence of trt effect has no validity in RCT.

CONCLUSION

Summary of mean drop in favor of Captopril (all cases and mmHg)

Statistic test	Mean Drop	Confint
T test TRT alone	-6.53	-15 to +2
DOD	- 8.00	-17.18 to +1.28
ANCOVA	-7.17	- 13.57 to -0.77

Definitely Using Baseline improve your power of detection , lower the conditional variance and given more meaning on the confidence interval.

Now the statistical Technic is a paramount importance where the statistical world is driven by p-value: But the biophysical of Drugs world by intervals. where the true value of the treatment could lie at a 95% Trust.

However one of these technich must be chosen before the data in hand as a good statistical practice.

In our next blog we will demonstrate mathematically our preferences for the ANCOVA.

ABBREVATION

TRT Treatment CAPTOPRIL

PLA Placebo

RCT Randomized Clinical Trial

RTM Regression to the mean

SS Sum of Square

SRS Simple Random Sampling

REFERENCES

Matthews, J. N. (2006). Introduction to randomized controlled clinical trials. Chapman and Hall/CRC. https://doi.org/10.1201/9781420011302

Not recommended in RCT unblanaced as trt the marginal effect might not be accouted fully in SSQ Use type II of Type III with contrast sum required:: car Anova (J.Fox)↩︎