The perfect doctor

Lab specs can be found here

Exercise 1

Question: Recreate table 1 shown below, inserting values appropriately for the three empty colums: (i) The column labeled \(\delta_i\): please enter each patient’s treatment effect (ii) The column labeled \(D\): the optimal treatment for this patient (iii) The column labeled \(Y\): the observed outcomes. Calculate the average treatment effect (ATE) and the average treatment effect for the treated (ATT) when comparing the outcome of the ventilators treatment with that of the bedrest treatment and comment as to which type of intervention is more effective on average. Finally, explain under which conditions might SUTVA be violated for treatments of covid-19 in the scenario described above.

# Write your code here

df = tribble(
  ~patient, ~`Y(0)`, ~`Y(1)`, ~Age, ~delta, ~D, ~Y,
  1, 10, 1,  29, -9, 0, 10,
  2,  5, 1,  35, -4, 0, 5,
  3,  4, 1,  19, -3, 0, 4,
  4,  6, 5,  45, -1, 0, 6,
  5,  1, 5,  65, 4, 1, 5,
  6,  7, 6,  50, -1, 0, 7, 
  7,  8, 7,  77, -1, 0, 8, 
  8, 10, 7,  18, -3, 0, 10, 
  9,  2, 8,  85, 6, 1, 8,
  10, 6, 9,  96, 3, 1, 9,
  11, 7, 10, 77, 3, 1, 10) 


df%>% 
  gt()
patient Y(0) Y(1) Age delta D Y
1 10 1 29 -9 0 10
2 5 1 35 -4 0 5
3 4 1 19 -3 0 4
4 6 5 45 -1 0 6
5 1 5 65 4 1 5
6 7 6 50 -1 0 7
7 8 7 77 -1 0 8
8 10 7 18 -3 0 10
9 2 8 85 6 1 8
10 6 9 96 3 1 9
11 7 10 77 3 1 10

Answer: This is if the sample presented above is not representative of the population.

Exercise 2

Question: Calculate the simple difference in outcomes (SDO), showing the details of your calculation. Is the SDO a good estimation for the ATE? Finally, check whether the SDO is equal to the sum of the ATT and the selection bias, \(E[Y(0)|T=1] - E[Y(0)|T=0]\).

# Write your code here
df %>%
  summarize(mn.Y=mean(Y), .by = D)
## Calculate the SDO
D_0 <- filter(df, D == 0)
D_1<-filter(df, D==1)
mean_Y_for_D0 <- mean(D_0$Y, na.rm = TRUE)
mean_Y_for_D1 <- mean(D_1$Y, na.rm = TRUE)
SDO=mean_Y_for_D1-mean_Y_for_D0
# Print the result
print("SDO=")
## [1] "SDO="
print(SDO)
## [1] 0.8571429
## Calculate the ATE
print("ATE=")
## [1] "ATE="
ATE=(mean(df$delta))
print(ATE)
## [1] -0.5454545
## Calculate the ATT
ATT <- mean(D_1$delta, na.rm = TRUE)
print("ATT=")
## [1] "ATT="
print(ATT)
## [1] 4
## Calculate the ATU
ATU <- mean(D_0$delta, na.rm = TRUE)
print("ATU=")
## [1] "ATU="
print(ATU)
## [1] -3.142857
## Calculate the Bias
D_1<-filter(df, D==1)
mean_Y_0_D1<-(mean(D_1$`Y(0)`, na.rm = TRUE))
D_0 <- filter(df, D == 0)
mean_Y_0_D0<-(mean(D_0$`Y(0)`, na.rm = TRUE))
Bias=(mean_Y_0_D1-mean_Y_0_D0)
print("Bias=")
## [1] "Bias="
print(Bias)
## [1] -3.142857
ATT+Bias
## [1] 0.8571429

**The above calculations show that ATT+Bias=0.8571429=SDO

Exercise 3

Question: Compare the treatment effect for both groups: for those treated with a ventilator and for those treated with bedrest. What explains the difference in the average effect? Now compare all four measures of effects. What are the advantages and disadvantages of each? Is the ATE equal to the mean of the ATU and the ATT? Why or why not?

# Write your code here

Answer: Treatment Effect Comparison:

Explanation of Differences in Average Effects:

The difference in the average treatment effects could be due to the different treatment methods (ventilator vs. bedrest) and their direct impact on the outcome. It appears that the treatment with bedrest had a slightly higher average treatment effect in this scenario.

ATE: Advantages - Captures the overall treatment effect on the entire population. Disadvantage - Might not reveal subgroup differences. ATT & ATU: Advantages - Focus on treatment effects within specific subgroups. Disadvantages - Can be subject to selection bias if treated and untreated groups are not comparable. SDO: Advantages - Helps understand the differences in treatment effects between different subgroups. Disadvantages - May not fully capture all sources of bias.

Using regression to estimate effects

The following exercises demonstrate that regression is a useful tool to estimate average outcomes and treatment effects in the different groups. Notice that in contrast to the role that regressions play in traditional statistics, here standard errors and significance are not of primary concern. Instead, we are interested in using regression to calculate average effects proper.

Exercise 4

Question: Calculate the outcome, conditional on getting the bedrest treatment \(\mathbb{E}[Y|D=0]\). Now estimate the following regression, comparing the coefficients \(\alpha\) and \(\delta\) to the statistics you’ve previously calculated. What did you find? How would you explain these finding?

# Write your code here
df %>%
  filter(D==0)%>%
  summarize(mn.Y=mean(Y))
nd1=lm(Y~D,data=df)
summary(nd1)
## 
## Call:
## lm(formula = Y ~ D, data = df)
## 
## Residuals:
##    Min     1Q Median     3Q    Max 
## -3.143 -1.643  0.000  1.500  2.857 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)    
## (Intercept)   7.1429     0.8624   8.282 1.68e-05 ***
## D             0.8571     1.4302   0.599    0.564    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 2.282 on 9 degrees of freedom
## Multiple R-squared:  0.03838,    Adjusted R-squared:  -0.06847 
## F-statistic: 0.3592 on 1 and 9 DF,  p-value: 0.5637

Answer: E[Y|D=0]=7.1428571

lm(formula = Y ~ D, data = df)

Residuals: Min 1Q Median 3Q Max -3.143 -1.643 0.000 1.500 2.857

Coefficients: Estimate Std. Error t value Pr(>|t|)
(Intercept) 7.1429 0.8624 8.282 1.68e-05 *** D 0.8571 1.4302 0.599 0.564
— Signif. codes: 0 ‘’ 0.001 ‘’ 0.01 ‘’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 2.282 on 9 degrees of freedom Multiple R-squared: 0.03838, Adjusted R-squared: -0.06847 F-statistic: 0.3592 on 1 and 9 DF, p-value: 0.5637

Remark: The values are the same. Intercept The intercept matches the expected value of 𝑌E[Y|D=0]=7.1428571, confirming that the regression model is correctly specifying the average outcome for the bedrest treatment group.

Slope The small t-value and high p-value suggest that the difference in outcomes between those who did and did not receive bedrest is not statistically significant. This means that there is no strong evidence that bedrest has a significant effect on the outcome𝑌

Exercise 5

Question: Now estimate the same regression, but this time, controlling for age, again comparing, the coefficient \(\delta\) to the statistics you’ve previously calculated. What did you find? How do you explain these results?

df %>%
  filter(D==0)%>%
  summarize(mn.Y=mean(Y))
nd2=lm(Y ~ D + Age, data=df)
summary(nd2)
## 
## Call:
## lm(formula = Y ~ D + Age, data = df)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -2.7390 -1.6630 -0.0858  1.3839  3.2811 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)  
## (Intercept)  6.35543    1.90736   3.332   0.0104 *
## D            0.01419    2.33997   0.006   0.9953  
## Age          0.02019    0.04309   0.469   0.6519  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 2.388 on 8 degrees of freedom
## Multiple R-squared:  0.06407,    Adjusted R-squared:  -0.1699 
## F-statistic: 0.2738 on 2 and 8 DF,  p-value: 0.7673

Answer: Call: lm(formula = Y ~ D + Age, data = df)

Residuals: Min 1Q Median 3Q Max -2.7390 -1.6630 -0.0858 1.3839 3.2811

Coefficients: Estimate Std. Error t value Pr(>|t|)
(Intercept) 6.35543 1.90736 3.332 0.0104 * D 0.01419 2.33997 0.006 0.9953
Age 0.02019 0.04309 0.469 0.6519
— Signif. codes:
0 ‘’ 0.001 ‘’ 0.01 ‘’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 2.388 on 8 degrees of freedom Multiple R-squared: 0.06407, Adjusted R-squared: -0.1699 F-statistic: 0.2738 on 2 and 8 DF, p-value: 0.7673

Remark: The coefficient𝛿for D decreased significantly from 0.8571 to 0.01419 after controlling for age. This substantial change indicates that age may be a confounding variable in the relationship between Dand Y. When age is not controlled for, it might have inflated the apparent effect of D.

Exercise 6

Question: Estimate the following three regression models. The first model is the same as the one above. The second equation is the auxiliary regression of \(D\) onto \(X_{age}\). The third equation regresses \(Y\) onto \(\tilde{D}\) which is the residual from the second equation. Compare the coefficient on \(D\) from the first equation to the coefficient on \(\tilde{D}\) in the third equation. What does this tell you about how to interpret multivariate regressions?

# Write your code here
df$y_pred <- predict(nd2)
df$dbar <- df$Y - df$y_pred
nd2=lm(Y ~ D + Age + dbar, data=df)
summary(nd2)
## 
## Call:
## lm(formula = Y ~ D + Age + dbar, data = df)
## 
## Residuals:
##        Min         1Q     Median         3Q        Max 
## -4.027e-15 -4.341e-16  3.134e-16  4.962e-16  3.101e-15 
## 
## Coefficients:
##              Estimate Std. Error   t value Pr(>|t|)    
## (Intercept) 6.355e+00  1.615e-15 3.935e+15   <2e-16 ***
## D           1.419e-02  1.981e-15 7.162e+12   <2e-16 ***
## Age         2.019e-02  3.648e-17 5.534e+14   <2e-16 ***
## dbar        1.000e+00  2.994e-16 3.340e+15   <2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 2.022e-15 on 7 degrees of freedom
## Multiple R-squared:      1,  Adjusted R-squared:      1 
## F-statistic: 3.974e+30 on 3 and 7 DF,  p-value: < 2.2e-16

Answer: Controlling for Confounders: The significant change in the coefficient when we move from Model 1 to Model 3 indicates the importance of controlling for confounding variables. In Model 1, age is included as a covariate to control for its potential confounding effect on the relationship between D and Y.

Residuals (Dbar) Capture Unique Variation: In Model 3, the residuals Dbar represent the part of D that is orthogonal (uncorrelated) to age. The perfect correlation (coefficient = 1.000) of Dbar with Y highlights that once we control for age, the remaining variation in D has a direct and perfect relationship with the outcome.

Multivariate Interpretation: Multivariate regressions help to isolate the effect of the primary explanatory variable (in this case, D) by controlling for other potential confounders (in this case, age). The large difference in coefficients between the first and third models underscores how omitting important variables can bias the estimates of the effect of interest.