Lab specs can be found here
Question: Recreate table 1 shown below,
inserting values appropriately for the three empty colums: (i) The
column labeled \(\delta_i\): please
enter each patient’s treatment effect (ii) The column labeled \(D\): the optimal treatment for this patient
(iii) The column labeled \(Y\): the
observed outcomes. Calculate the average treatment effect (ATE) and the
average treatment effect for the treated (ATT) when comparing the
outcome of the ventilators treatment with that of the
bedrest treatment and comment as to which type of
intervention is more effective on average. Finally, explain under which
conditions might SUTVA be violated
for treatments of covid-19 in the scenario described above.
# Write your code here
df <- tribble(
~patient, ~`Y(0)`, ~`Y(1)`, ~Age, ~delta, ~D, ~Y,
1, 10, 1, 29, -9, 0, 10,
2, 5, 1, 35, -4, 0, 5,
3, 4, 1, 19, -3, 0, 4,
4, 6, 5, 45, -1, 0, 6,
5, 1, 5, 65, 4, 1, 5,
6, 7, 6, 50, -1, 0, 7,
7, 8, 7, 77, -1, 0, 8,
8, 10, 7, 18, -3, 0, 10,
9, 2, 8, 85, 6, 1, 8,
10, 6, 9, 96, 3, 1, 9,
11, 7, 10, 77, 3, 1, 10)
df %>%
gt()| patient | Y(0) | Y(1) | Age | delta | D | Y |
|---|---|---|---|---|---|---|
| 1 | 10 | 1 | 29 | -9 | 0 | 10 |
| 2 | 5 | 1 | 35 | -4 | 0 | 5 |
| 3 | 4 | 1 | 19 | -3 | 0 | 4 |
| 4 | 6 | 5 | 45 | -1 | 0 | 6 |
| 5 | 1 | 5 | 65 | 4 | 1 | 5 |
| 6 | 7 | 6 | 50 | -1 | 0 | 7 |
| 7 | 8 | 7 | 77 | -1 | 0 | 8 |
| 8 | 10 | 7 | 18 | -3 | 0 | 10 |
| 9 | 2 | 8 | 85 | 6 | 1 | 8 |
| 10 | 6 | 9 | 96 | 3 | 1 | 9 |
| 11 | 7 | 10 | 77 | 3 | 1 | 10 |
#Average treatment effect (ATE) = E[Y(1)-Y(0)]
ATE <- df %>% #
summarize(ATE = mean(`Y(1)`-`Y(0)`)) %>% #mean of Y1 -Y0 to get ATE
pull(ATE)
ATE## [1] -0.5454545#Average treatment effect of treated (ATT) = E[Y(1)|D=1]-E[Y(0)|D=1]
ATT <- df %>%
filter(D == 1) %>% #filter on treatment is 1
summarize(ATT = mean(`Y(1)`-`Y(0)`)) %>% #ATT
pull(ATT)
ATT## [1] 4#OR by hand
ATE <- (1 + 1 + 1 + 5 + 5 + 6 + 7 + 7 + 8 + 9 + 10) / 11 - (10 + 5 + 4 + 6 + 1 + 7 + 8 + 10 + 2 + 6 + 7) / 11
#OR
ATT <- (5 + 8 + 9 + 10) / 4 - (1 + 2 + 6 + 7) / 4Answer: Based on the calculated ATE, if all individuals were receiving the same treatment, bedrest seems to have a higher post-treatment survival average as compared to ventilator treatment due to the negative ATE. Meaning that if we treated all the patients, on average, we would have decreased their life span by -0.5454545 years on average, compared to not treating them. Based on the ATT, among the treated, it seems that on average the ventilator treatment is more effective compared to when they would be treated with bedrest (difference of on average 4 years post-treatment survival).
The SUTVA (stable unit treatment value assumption) might be violated, because the treatments are not well defined, which might violate the consistency assumption. Important things such as: - (in)consistent application of treatment - Interference between patients - Individual effects/characteristics affecting treatment
Question: Calculate the simple difference in
outcomes (SDO), showing the details of your calculation. Is
the SDO a good estimation for the ATE? Finally, check
whether the SDO is equal to the sum of the ATT
and the selection bias, \(E[Y(0)|T=1] -
E[Y(0)|T=0]\).
#SDO calculation
SDO <- df %>%
group_by(D) %>% #group by treatment
summarize(mean.Y = mean(Y)) %>% #calculate mean per group
summarize(SDO = mean.Y[D == 1] - mean.Y[D == 0]) %>% #Substract means Y(1) - Y(0) to get SDO
pull(SDO)
SDO #SDO## [1] 0.8571429#ATT calculation
ATT <- df %>%
filter(D == 1) %>% #filter on treatment is 1 (for ATT)
summarize(ATT = mean(`Y(1)`-`Y(0)`)) %>% #Substract means Y(1) - Y(0)
pull(ATT)
ATT #ATT## [1] 4#Selection bias
BIAS <- df %>%
group_by(D) %>% #group by treatment
summarize(mean.Y = mean(`Y(0)`)) %>% # mean of Y1- Y0 per group
summarize(BIAS = mean.Y[D == 1] - mean.Y[D == 0]) %>% #Substract means T(1) - T(0) to get BIAS
pull(BIAS)
BIAS #Selection bias## [1] -3.142857## [1] TRUE#OR by hand
#Check SDO
SDO <- (5 + 8 + 9 + 10)/ 4 - (10 + 5 + 4 + 6 + 7 + 8 + 10) / 7 #E[Y(1)|D=1]-E[Y(0)|D=0]
#Check ATT + BIAS OR difference with ATE
ATT <- (5 + 8 + 9 + 10) / 4 - (1 + 2 + 6 + 7) / 4 #E[Y(1)|D=1]-E[Y(0)|D=1]
BIAS <- (1 + 2 + 6 + 7) / 4 - (10 + 5 + 4 + 6 + 7 + 8 + 10) / 7 #E[Y(0)|T=1] - E[Y(0)|T=0]Answer: The SDO is not a good estimation of the Average Treatment Effect (ATE) because they are not equal in this case. SDO calculates average treatment outcomes over the treated minus the average treatment outcomes of the untreated. Confounding factors, such as selection bias could be present. The SDO is therefore 0.8571429, which is positive, whereas that the ATE we calculated before is -0.5454545, negative. This means that if we had treated the entire sample, we would reduce, on average, the lifespan of our patients. However, since we applied the treatment selectively, treating only the patients that would benefit from the procedure, we have increased the average lifespan of the patients in our sample. This is also indicated by the selection bias. The presence of selection bias means that differences between the groups are due not only to the treatment but also to other factors that affect the outcome, which is why SDO is not a reliable estimate of the ATE.The SDO can be explained by ATT + Selection bias.
Question: Compare the treatment effect for both groups: for those treated with a ventilator and for those treated with bedrest. What explains the difference in the average effect? Now compare all four measures of effects. What are the advantages and disadvantages of each? Is the ATE equal to the mean of the ATU and the ATT? Why or why not?
diff <- df %>%
group_by(D) %>%
summarize(mean.Y = mean(`Y(1)`-`Y(0)`)) %>%
summarize(mean = mean(mean.Y)) %>%
pull(mean)
diff## [1] 0.4285714## [1] FALSE#OR by hand
#Average treatment effect (ATE) = E[Y(1)-Y(0)]
ATE <- (1 + 1 + 1 + 5 + 5 + 6 + 7 + 7 + 8 + 9 + 10) / 11 - (10 + 5 + 4 + 6 + 1 + 7 + 8 + 10 + 2 + 6 + 7) / 11
#ATT E[Y(1)|D=1]-E[Y(0)|D=1]
ATT <- (5 + 8 + 9 + 10) / 4 - (1 + 2 + 6 + 7) / 4
#ATU E[Y(1)|D=0]-E[Y(0)|D=0]
ATU <- (1 + 1 + 1 + 5 + 6 + 7 + 7) / 7 - (10 + 5 + 4 + 6 + 7 + 8 + 10) / 7Answer:
The Average Treatment Effect (ATE) is calculated as the difference in the average outcomes between the two treatment groups. In this case, the ATE represents the overall effectiveness of ventilator treatment compared to bedrest. The treatment effect for ventilator (ATT) is higher than that for bedrest, indicating that ventilator treatment has a greater impact on the outcome for those who receive it. The difference in the average effect between ventilator and bedrest can be attributed to various factors such as the nature of the treatments, patient characteristics, and potential confounders.The ATE considers the entire population, while ATU and ATT focus on specific subgroups.If there is no effect modification, i.e., the treatment effect is consistent across all individuals, then ATE may approximate the average of ATU and ATT. In this case, the ATE is not equal to the mean of ATT and ATU, possibly due to other confounding factors.
ATT is 4, indicating that among the treated, ventilator treatment results in an average of 4 years higher post-treatment survival compared to bedrest treatment. In other words, among those who received ventilator treatment, the average post-treatment survival is 4 years longer than it would have been had they instead received bedrest treatment. This suggests that ventilator treatment is beneficial for those who received it.
ATU is -3.14, indicating that among the untreated, bedrest results in an average of 3.14 years higher post-treatment survival compared to ventilator treatment. This means that for those who received bedrest, bedrest was the better option compared to ventilator treatment, as it provided a 3.14-year longer post-treatment survival on average than if they had received ventilator treatment.
The flaw of SDO is explained in the previous answer.
Question: Calculate the outcome, conditional on
getting the bedrest treatment \(\mathbb{E}[Y|D=0]\). Now estimate the
following regression, comparing the coefficients \(\alpha\) and \(\delta\) to the statistics you’ve
previously calculated. What did you find? How would you explain these
finding?
# Given data in data frame format
# Calculate E[Y|D=0]
df %>%
filter(D == 0) %>%
summarize(Y_0_D_0 = mean(Y))# Estimate regression model using conditional outcome
regression_model <- lm(Y ~ D, data = df)
summary(regression_model)##
## Call:
## lm(formula = Y ~ D, data = df)
##
## Residuals:
## Min 1Q Median 3Q Max
## -3.143 -1.643 0.000 1.500 2.857
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 7.1429 0.8624 8.282 1.68e-05 ***
## D 0.8571 1.4302 0.599 0.564
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 2.282 on 9 degrees of freedom
## Multiple R-squared: 0.03838, Adjusted R-squared: -0.06847
## F-statistic: 0.3592 on 1 and 9 DF, p-value: 0.5637## (Intercept)
## 7.142857## D
## 0.8571429The intercept coefficient (α) represents the conditional outcome E[Y∣D=0], which is the average outcome for the untreated group. The coefficient D represents the difference in the average outcome between the treated and untreated groups. Therefore, E[Y∣D=1]=α+D. In this case, with α=7.1429 and D=0.8571, the expected outcome for the treated group is 7.1429+0.8571=8 (E[Y∣D=1]).
Question: Now estimate the same regression, but this time, controlling for age, again comparing, the coefficient \(\delta\) to the statistics you’ve previously calculated. What did you find? How do you explain these results?
##
## Call:
## lm(formula = Y ~ D + Age, data = df)
##
## Residuals:
## Min 1Q Median 3Q Max
## -2.7390 -1.6630 -0.0858 1.3839 3.2811
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 6.35543 1.90736 3.332 0.0104 *
## D 0.01419 2.33997 0.006 0.9953
## Age 0.02019 0.04309 0.469 0.6519
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 2.388 on 8 degrees of freedom
## Multiple R-squared: 0.06407, Adjusted R-squared: -0.1699
## F-statistic: 0.2738 on 2 and 8 DF, p-value: 0.7673## (Intercept)
## 6.355427## D
## 0.01418918Answer: The age behaves as a confounder, and by controlling for age, we are removing the confounding and measuring only the effect of the treatment on the outcome conditional on age E[Y|T,Age], whereas in the previous equation we looked at age E[Y|T]. Taking the age into account, we see that we overestimated the effect of the treatment in the previous question (the conditional effect of the treatment given age is much smaller). Both the intercept (α) and the coefficient (D) are affected when age is included in the model. The intercept, which represents the expected outcome for the untreated group (bedrest), becomes lower, indicating that when accounting for age, the average post-treatment survival for the untreated group decreases. This suggests that older individuals may respond differently to treatment compared to younger individuals. Specifically, the interaction between age and treatment indicates that age modifies the treatment effect, leading to different outcomes for different age groups. Also, the coefficient D became smaller (from 0.8571 to 0.01419) when age was included in the model. This change seems to indicate that the difference in outcomes between the treated and untreated groups decreased when age was taken into account.
Question: Estimate the following three regression models. The first model is the same as the one above. The second equation is the auxiliary regression of \(D\) onto \(X_{age}\). The third equation regresses \(Y\) onto \(\tilde{D}\) which is the residual from the second equation. Compare the coefficient on \(D\) from the first equation to the coefficient on \(\tilde{D}\) in the third equation. What does this tell you about how to interpret multivariate regressions?
# First regression model: Y regressed onto D and Age
regression_model_Y_D_Age <- lm(Y ~ D + Age, data = df)
# Second regression model: D regressed onto Age
regression_model_D_Age <- lm(D ~ Age, data = df)
# Calculate residuals from the second regression model
df$D_tilde <- residuals(regression_model_D_Age)
# Third regression model: Y regressed onto the residuals
regression_model_Y_D_tilde <- lm(Y ~ D_tilde, data = df)
# Compare coefficients
coef(regression_model_Y_D_Age)["D"]## D
## 0.01418918## D_tilde
## 0.01418918Answer: The coefficient on D_tilde in the third equation is equal to the coefficient on D in the first equation, it indicates that age does not act as a mediator or modifier of the relationship between the treatment and outcome variables. By regressing the outcome variable on these residuals, the relationship of the remaining unexplained variation in the treatment variable to the outcome is examined. Because the residuals of the auxillary regression (the part of treatment variable not explained by Age, regression model 2), explains the treatment effect similar as the first model with age, there is no effect of age.