The perfect doctor

Lab specs can be found here

Exercise 1

Question: Recreate table 1 shown below, inserting values appropriately for the three empty colums: (i) The column labeled \(\delta_i\): please enter each patient’s treatment effect (ii) The column labeled \(D\): the optimal treatment for this patient (iii) The column labeled \(Y\): the observed outcomes. Calculate the average treatment effect (ATE) and the average treatment effect for the treated (ATT) when comparing the outcome of the ventilators treatment with that of the bedrest treatment and comment as to which type of intervention is more effective on average. Finally, explain under which conditions might SUTVA be violated for treatments of covid-19 in the scenario described above.

# Write your code here

df <- tribble(
  ~patient, ~`Y(0)`, ~`Y(1)`, ~Age, ~delta, ~D, ~Y,
  1, 10, 1,  29, -9, 0, 10,
  2,  5, 1,  35, -4, 0, 5,
  3,  4, 1,  19, -3, 0, 4,
  4,  6, 5,  45, -1, 0, 6,
  5,  1, 5,  65, 4, 1, 5,
  6,  7, 6,  50, -1, 0, 7, 
  7,  8, 7,  77, -1, 0, 8, 
  8, 10, 7,  18, -3, 0, 10, 
  9,  2, 8,  85, 6, 1, 8,
  10, 6, 9,  96, 3, 1, 9,
  11, 7, 10, 77, 3, 1, 10) 

df %>% 
  gt()
patient Y(0) Y(1) Age delta D Y
1 10 1 29 -9 0 10
2 5 1 35 -4 0 5
3 4 1 19 -3 0 4
4 6 5 45 -1 0 6
5 1 5 65 4 1 5
6 7 6 50 -1 0 7
7 8 7 77 -1 0 8
8 10 7 18 -3 0 10
9 2 8 85 6 1 8
10 6 9 96 3 1 9
11 7 10 77 3 1 10

Answer: Average treatment effect (ATE) = E[Y(1)-Y(0)] = -0.545

Average treatment effect for the treated (ATT) = E[Y(1)-Y(0)|T=1] = 4

Average treatment effect for the untreated (ATU) = E[Y(1)-Y(0)|T=0] = -3.14

Based on the above average treatment effect, treatment using a ventilator would on average not be more effective.

Conditions of SUTVA which might be violated for treatments of covid-19 in the scenario described above are;

  1. Spillover effects: e.g. vaccination could reduce transmission which can affect the outcome for those who did not receive the vaccine.
  2. Resource availability could have lead to different allocation of treatment.
  3. Noncompliance: if the patient could not adhere to the ventilator treaatment

Exercise 2

Question: Calculate the simple difference in outcomes (SDO), showing the details of your calculation. Is the SDO a good estimation for the ATE? Finally, check whether the SDO is equal to the sum of the ATT and the selection bias, \(E[Y(0)|T=1] - E[Y(0)|T=0]\).

# Write your code here
df %>%
  summarize(mn.Y = mean(Y), .by = D)

SDO= E[Y|T=1] - E[Y|T=0] = (5+8+9+10)/4 - (10+5+4+6+7+8+10)/7 = 8-7.14 = 0.86

Answer: The SDO is not a good estimate for ATE

SDO is also not equal to the sum of the ATT + selection bias

Exercise 3

Question: Compare the treatment effect for both groups: for those treated with a ventilator and for those treated with bedrest. What explains the difference in the average effect? Now compare all four measures of effects. What are the advantages and disadvantages of each? Is the ATE equal to the mean of the ATU and the ATT? Why or why not?

# Write your code here
df %>%
  summarize(mn.Y = mean(`Y(0)`), .by = D)
df %>%
  summarize(mn.Y = mean(delta), .by = D)

Answer: The average effect for those treated with a ventilator is now higher than those treated with bedrest. The difference could be due to heterogenous treatment effects, differences in compliance, or selection bias.

ATE Advantages - more generalisable, simple Disadvantages - can be misleading if there are subgroups responding differently to treatment

ATT and ATU Advantages - can be more relevant to the treated population as it focusses on the effect of the treatment in those requiring treatment Disadvantages - less generalisable

Using regression to estimate effects

The following exercises demonstrate that regression is a useful tool to estimate average outcomes and treatment effects in the different groups. Notice that in contrast to the role that regressions play in traditional statistics, here standard errors and significance are not of primary concern. Instead, we are interested in using regression to calculate average effects proper.

Exercise 4

Question: Calculate the outcome, conditional on getting the bedrest treatment \(\mathbb{E}[Y|D=0]\). Now estimate the following regression, comparing the coefficients \(\alpha\) and \(\delta\) to the statistics you’ve previously calculated. What did you find? How would you explain these finding?

# Write your code here

  lm(formula = Y ~ delta, data = subset(df, D==0))
## 
## Call:
## lm(formula = Y ~ delta, data = subset(df, D == 0))
## 
## Coefficients:
## (Intercept)        delta  
##      6.1228      -0.3246

Answer: The result is comparable to the ATT and not the ATE, in that we see that bestrest is worse than receiving treatment with a ventilator. There is likely confounders affecting the overall average effect.

Exercise 5

Question: Now estimate the same regression, but this time, controlling for age, again comparing, the coefficient \(\delta\) to the statistics you’ve previously calculated. What did you find? How do you explain these results?

# Write your code here
lm(formula = Y ~ delta + Age, data = subset(df, D==0))
## 
## Call:
## lm(formula = Y ~ delta + Age, data = subset(df, D == 0))
## 
## Coefficients:
## (Intercept)        delta          Age  
##     4.47915     -0.44119      0.03275

Answer:

When you control for age the delta becomes more negative and the intercept lower.

This means that age is a confounder in the relationship between treatment and outcome.

Exercise 6

Question: Estimate the following three regression models. The first model is the same as the one above. The second equation is the auxiliary regression of \(D\) onto \(X_{age}\). The third equation regresses \(Y\) onto \(\tilde{D}\) which is the residual from the second equation. Compare the coefficient on \(D\) from the first equation to the coefficient on \(\tilde{D}\) in the third equation. What does this tell you about how to interpret multivariate regressions?

# Write your code here
lm(formula = Y ~ delta + Age, data = subset(df, D == 0))
## 
## Call:
## lm(formula = Y ~ delta + Age, data = subset(df, D == 0))
## 
## Coefficients:
## (Intercept)        delta          Age  
##     4.47915     -0.44119      0.03275
glm(formula= D ~ Age, family=binomial(link="logit"), data=df)
## 
## Call:  glm(formula = D ~ Age, family = binomial(link = "logit"), data = df)
## 
## Coefficients:
## (Intercept)          Age  
##     -8.5254       0.1261  
## 
## Degrees of Freedom: 10 Total (i.e. Null);  9 Residual
## Null Deviance:       14.42 
## Residual Deviance: 5.819     AIC: 9.819
aux<-glm(formula= D ~ Age, family=binomial(link="logit"), data=df)
df$residual<-resid(aux)

equation3<-lm(Y ~ residual, data=df)
summary(equation3)
## 
## Call:
## lm(formula = Y ~ residual, data = df)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -3.4615 -1.8843  0.3126  2.0553  2.6487 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)    
## (Intercept)   7.4524     0.7009  10.632 2.14e-06 ***
## residual     -0.1381     0.9637  -0.143    0.889    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 2.324 on 9 degrees of freedom
## Multiple R-squared:  0.002277,   Adjusted R-squared:  -0.1086 
## F-statistic: 0.02054 on 1 and 9 DF,  p-value: 0.8892

Answer: Multivariate regression allows us to control for age and by regressing on the residual we see the true effect of the treatment on the outcome.