How does RDD work?

RDD exploits situations where treatment is assigned based on whether some running variable crosses a known cutoff, like a test score determining a scholarship or age determining Medicare eligibility. Since people just above and just below the cutoff are essentially identical except for treatment status, comparing outcomes right around that threshold approximates a randomized experiment. The key intuition (going back to Thistlethwaite and Campbell) is that any jump in the outcome at the cutoff can be attributed to the treatment, since nothing else should be discontinuous there.

Am I convinced RD design can give causal estimates?

Yes, more so than most other quasi-experimental designs, because the “local randomization” result doesn’t require assuming the design is as-good-as-random, it derives that from the weaker assumption that people can’t precisely manipulate the running variable near the cutoff.

When is RDD valid?

RDD is valid when individuals have imprecise control over the running variable, meaning they can influence it in general but can’t finely manipulate exactly where they land relative to the cutoff. That imprecision is what generates local randomization. It’s testable too: baseline covariates (things determined before treatment) should show no discontinuity at the cutoff, which is basically a placebo check.

When can RD design be invalid?

It breaks down when people can precisely sort around the threshold, like the scholarship-exam example where only the students who know about the reward bother double-checking their answers. If there’s a density discontinuity in the running variable itself (a “stacking” of observations just past the cutoff), that’s a red flag that manipulation is happening and the design isn’t credible.

Article critique / your insights

I looked at Lee and McCrary (2005) on criminal offenders in Florida. The running variable is age at the time of offense, treatment is the switch from juvenile to adult sanctions at age 18, and the outcome is arrest/recidivism rates. It makes sense as an RD because the legal system draws a hard line at 18 that nobody can manipulate (you can’t choose your own birthdate), so offenders just under and over 18 should be comparable in every way except which justice system they face.

Is it sharp or fuzzy? It’s sharp, since turning 18 deterministically switches which legal system applies, there’s no partial compliance issue.

Would I have done anything differently? This is actually a case the paper itself flags as a “non-randomized” RD, because everyone ages into 18 eventually, treatment isn’t a coin-flip the way it is for something like a scholarship cutoff. That means the usual smoothness-in-covariates test is basically uninformative here (baseline characteristics are mechanically identical right before and after the cutoff for a single cohort). If I were running it, I’d want to dig more into anticipation effects, whether offenders who know they’re close to 18 change behavior beforehand, since that’s a real threat to interpretation that a standard RD doesn’t have to worry about.

A small RDD model

To make this concrete I simulated a small RD dataset based on the exact merit-scholarship setup from Thistlethwaite and Campbell that the reading opens with. 2,000 “students” get an entrance exam score (the running variable), a scholarship kicks in at score 50 (sharp cutoff), and I built in a true causal effect of 0.35 GPA points so I’d have a known answer to check my estimate against.

set.seed(42)

n <- 2000
cutoff <- 50
true_tau <- 0.35   # true causal effect (GPA points), known because it's simulated

df <- tibble(
  X = runif(n, 0, 100),            # running variable: exam score
  D = as.integer(X >= cutoff)      # treatment: scholarship awarded
) %>%
  mutate(
    Y = 2.20 + 0.010 * (X - cutoff) - 0.00015 * (X - cutoff)^2 +
      true_tau * D + rnorm(n, 0, 0.35),
    # placebo / baseline covariate that should NOT jump at the cutoff
    W = 55 + 0.08 * (X - cutoff) + rnorm(n, 0, 6)
  )

Checklist item 1 (Section 4.6): check the density of the running variable for signs of manipulation.

ggplot(df, aes(X)) +
  geom_histogram(binwidth = 2, fill = "#6b8cae", color = "white") +
  geom_vline(xintercept = cutoff, linetype = "dashed", color = "#c0392b") +
  labs(title = "Density of the running variable around the cutoff",
       x = "Entrance exam score (X)", y = "Count") +
  theme_minimal()

No bunching right at the cutoff, which is what you’d want to see before trusting anything downstream.

Main RD plot: binned local averages with a quadratic fit on each side.

rd_plot <- function(data, outcome, cutoff, binwidth, title, ylab) {
  data <- data %>%
    mutate(bin = floor((X - cutoff) / binwidth) * binwidth + binwidth / 2 + cutoff)
  binned <- data %>% group_by(bin) %>% summarise(m = mean(.data[[outcome]]), .groups = "drop")

  ggplot(data, aes(X, .data[[outcome]])) +
    geom_point(data = binned, aes(bin, m), size = 2, color = "black") +
    geom_smooth(data = filter(data, X < cutoff), method = "lm",
                formula = y ~ poly(x, 2), se = FALSE, color = "#c0392b") +
    geom_smooth(data = filter(data, X >= cutoff), method = "lm",
                formula = y ~ poly(x, 2), se = FALSE, color = "#2c7a4b") +
    geom_vline(xintercept = cutoff, linetype = "dotted") +
    labs(title = title, x = "Entrance exam score (X)", y = ylab) +
    theme_minimal()
}

rd_plot(df, "Y", cutoff, 2.5,
        "RD plot: scholarship (D) and graduation GPA (Y)",
        "Graduation GPA")

The binned averages show a clear jump right at the cutoff, and the quadratic fits stay pretty flat away from it, which is what you’d want to see before trusting the jump as a treatment effect rather than just curvature in the underlying relationship.

Checklist item 5: placebo test on a baseline covariate.

rd_plot(df, "W", cutoff, 2.5,
        "Placebo check: baseline covariate vs. score",
        "Family income ($000s)")

No discontinuity there, which is reassuring, family income was determined before the exam so it shouldn’t jump if the design is valid.

Estimation: local linear regression with a triangular kernel, across a few bandwidths.

local_linear_rd <- function(data, cutoff, h) {
  d <- data %>%
    filter(X >= cutoff - h, X <= cutoff + h) %>%
    mutate(Xc = X - cutoff,
           DXc = D * Xc,
           w = 1 - abs(Xc) / h)
  m <- lm(Y ~ D + Xc + DXc, data = d, weights = w)
  tidy(m, conf.int = TRUE) %>%
    filter(term == "D") %>%
    mutate(h = h, n = nrow(d)) %>%
    select(h, n, estimate, std.error, conf.low, conf.high)
}

bandwidths <- c(10, 15, 20, 25)
results <- map_dfr(bandwidths, ~ local_linear_rd(df, cutoff, .x))
knitr::kable(results, digits = 3,
             col.names = c("Bandwidth", "N used", "Estimate", "SE", "CI low", "CI high"))
Bandwidth N used Estimate SE CI low CI high
10 393 0.325 0.063 0.200 0.449
15 578 0.286 0.052 0.184 0.387
20 768 0.279 0.044 0.192 0.366
25 969 0.302 0.039 0.225 0.379

All four bandwidths recover something close to the true 0.35 effect.

sens <- map_dfr(6:40, ~ local_linear_rd(df, cutoff, .x))

ggplot(sens, aes(h, estimate)) +
  geom_line(color = "#2c3e50", linewidth = 1) +
  geom_ribbon(aes(ymin = conf.low, ymax = conf.high), alpha = 0.15) +
  geom_hline(yintercept = true_tau, linetype = "dashed", color = "#c0392b") +
  labs(title = "Sensitivity of the RD estimate to bandwidth choice",
       x = "Bandwidth (h)", y = "Estimated treatment effect") +
  theme_minimal()

What surprised me a little is how much the standard error shrinks as the bandwidth widens, since you’re using more data, but the point estimate barely moves. That stability is basically the whole selling point of RD when the continuity assumption holds. Obviously this is a toy example where I get to cheat and know the true effect ahead of time. It made me appreciate how much of the actual work in a real RD paper (like Lee and McCrary) is in defending that continuity assumption, since with real data you never get to check your estimate against ground truth the way I just did here.

I’m still a little unsettled on how much weight to put on RD estimates that only apply to the population right at the threshold. Is a “local” causal effect at age 18 actually the number policymakers should care about, or does that external validity problem undercut the whole appeal of RD?