Question 4: Quartic Treatment Effect with Variance Imbalance

Data Generation

# Sample sizes
n_treat   <- 40
n_control <- 100

# Generate X1 with variance imbalance
# Treatment: mean=10, SD=1; Control: mean=10, SD=2
X1_treat   <- rnorm(n_treat,  mean = 10, sd = 1)
X1_control <- rnorm(n_control, mean = 10, sd = 2)

X1 <- c(X1_treat, X1_control)
W <- c(rep(1, n_treat), rep(0, n_control))

# Generate observed outcome Y
# Baseline: Y = 5 + X1
# Treatment effect: adds 2*X1^4 if treated (QUARTIC)
Y <- 5 + X1 + W * (2 * X1^4)

Estimands: ATT, ATC, ATE

# True treatment effect for each unit: tau_i = 2 * X1^4
tau <- 2 * X1^4

# Calculate estimands
ATT <- mean(tau[W == 1])  # Average treatment effect on treated
ATC <- mean(tau[W == 0])  # Average treatment effect on controls

# ATE as weighted average of ATT and ATC
p_treat <- mean(W)  # Proportion treated
ATE <- p_treat * ATT + (1 - p_treat) * ATC

cat("ATT (treatment effect on treated):  ", round(ATT, 2), "\n")
## ATT (treatment effect on treated):   21314.73
cat("ATC (treatment effect on controls): ", round(ATC, 2), "\n")
## ATC (treatment effect on controls):  24178.16
cat("ATE (weighted average):             ", round(ATE, 2), "\n\n")
## ATE (weighted average):              23360.03
cat("ATE = p*ATT + (1-p)*ATC\n")
## ATE = p*ATT + (1-p)*ATC
cat("    = ", round(p_treat, 3), "*", round(ATT, 2), " + ", 
    round(1-p_treat, 3), "*", round(ATC, 2), "\n")
##     =  0.286 * 21314.73  +  0.714 * 24178.16
cat("    = ", round(ATE, 2), "\n")
##     =  23360.03

Key Finding: With the quartic treatment effect: - ATT = 21,314.73 - ATC = 24,178.16 - ATE = 23,360.03

The difference between ATT and ATC is much larger than in the linear case due to the nonlinear (quartic) relationship.

Naive Difference and Bias Decomposition

# Naive difference in means
naive_diff <- mean(Y[W == 1]) - mean(Y[W == 0])

# Selection bias: baseline difference between groups
Y0_treated <- 5 + X1[W == 1]
Y0_control <- 5 + X1[W == 0]
selection_bias <- mean(Y0_treated) - mean(Y0_control)

# Heterogeneity bias: difference between ATT and ATE
heterogeneity_bias <- ATT - ATE

# Total bias
bias <- naive_diff - ATE

cat("Naive difference in means:                ", round(naive_diff, 2), "\n")
## Naive difference in means:                 21314.79
cat("Selection bias (E[Y0|W=1] - E[Y0|W=0]):  ", round(selection_bias, 4), "\n")
## Selection bias (E[Y0|W=1] - E[Y0|W=0]):   0.0596
cat("Heterogeneity bias (ATT - ATE):          ", round(heterogeneity_bias, 2), "\n")
## Heterogeneity bias (ATT - ATE):           -2045.31
cat("Total bias (naive_diff - ATE):           ", round(bias, 2), "\n\n")
## Total bias (naive_diff - ATE):            -2045.25
# Verify decomposition 1: naive_diff = ATT + selection_bias
decomp1 <- ATT + selection_bias
cat("Decomposition 1: ATT + selection_bias\n")
## Decomposition 1: ATT + selection_bias
cat("                 ", round(ATT, 2), " + ", round(selection_bias, 4), 
    " = ", round(decomp1, 2), "\n")
##                   21314.73  +  0.0596  =  21314.79
cat("                 Should equal naive_diff = ", round(naive_diff, 2), "\n")
##                  Should equal naive_diff =  21314.79
cat("                 Match? ", isTRUE(all.equal(decomp1, naive_diff)), "\n\n")
##                  Match?  TRUE
# Verify decomposition 2: naive_diff = ATE + selection_bias + heterogeneity_bias
decomp2 <- ATE + selection_bias + heterogeneity_bias
cat("Decomposition 2: ATE + selection_bias + heterogeneity_bias\n")
## Decomposition 2: ATE + selection_bias + heterogeneity_bias
cat("                 ", round(ATE, 2), " + ", round(selection_bias, 4), 
    " + ", round(heterogeneity_bias, 2), " = ", round(decomp2, 2), "\n")
##                   23360.03  +  0.0596  +  -2045.31  =  21314.79
cat("                 Should equal naive_diff = ", round(naive_diff, 2), "\n")
##                  Should equal naive_diff =  21314.79
cat("                 Match? ", isTRUE(all.equal(decomp2, naive_diff)), "\n")
##                  Match?  TRUE

Summary: The naive difference of 21,314.79 can be decomposed as:

  1. ATT + Selection Bias = 21,314.73 + 0.0596 = 21,314.79

  2. ATE + Selection Bias + Heterogeneity Bias = 23,360.03 + 0.0596 + -2,045.31 = 21,314.79

Key Insight: The heterogeneity bias (-2,045.31) is much larger with the quartic treatment effect than it was with the linear effect. This is because the quartic function amplifies differences in X1 - even though the groups have similar means, the variance difference causes the treated group (with lower variance, so values closer to the mean of 10) to have systematically different X1^4 values than the control group.