ATE, ATT, ATC

• Individual Causal / Treatment Effect:
  \(\delta_i = Y_i^T - Y_i^C\)

• Average Treatment Effect (ATE) for the entire population:

  \[ \text{ATE} = \text{Average}(\delta) = E\!\left[Y_i^T - Y_i^C\right] = \color{red}{E[Y_i^T] - E[Y_i^C]} \]

• Average Treatment Effect for the Treated (ATT):

  \[ \text{ATT} = \text{Average}(\delta \mid D = T) = E(Y_i^T - Y_i^C \mid D = T) = \color{red}{E(Y_i^T \mid D = T) - E(Y_i^C \mid D = T)} \]

• Average Treatment Effect for the Controls (ATC) or sometimes ATUT:

  \[ \text{ATC} = \text{Average}(\delta \mid D = C) = E(Y_i^T - Y_i^C \mid D = C) = \color{red}{E(Y_i^T \mid D = C) - E(Y_i^C \mid D = C)} \]

ATE, ATT, ATC: numerical example

ATE, ATT, ATC: numerical example

Naive estimate vs. ATE: Selection Baseline Bias

Naive estim. vs. ATE: Differential Treatm. Effect

Naive estimator vs. ATE: numerical example

\[ \begin{align} 8.8 &= -0.5 && (\text{ATE}) \\ &\quad + \frac{44}{4} - \frac{25}{6} && (= 6.8:\ \text{selection (baseline) bias}) \\ &\quad + 0.6 \cdot (2 + 2.17) && (= 2.5:\ \text{ATT--ATC bias}) \end{align} \]

Randomisation as the solution

• Conditional independence: (D) is given randomly to individuals
• In this case, selection bias \(= 0\), \(ATT - ATC = 0\), thus:
• \(\widehat{\delta}_{\text{naive}} = \underbrace{\text{Avrg}(\delta)}_{\text{ATE}}\)
• So randomisation solves various biases and the fundamental problem of causal inference (on average)

Key assumtion when randomising: SUTVA

When relying on the Rubin/Neyman Causal Model with potential outcomes, we rely on SUTVA:

• SUTVA: Stable Unit Stable Unit Treatment Value Assumption.

• An observation’s outcome is not affected by other observations’ assignments.

  • If your neighbor is treated or not, that cannot affect your potential outcomes (e.g. make your \(\delta_i\) larger).
  • This means that there cannot be any spill-over effects or general equilibrium effects.

• Example: Immunization randomised control trial may violate SUTVA because immunization also has a group effect.

Forms of validity

Bias in the naive estimator when trying to reach our estimand (ATE): baseline differences (under the control condition), and differential response to the treatment (under the exposure condition).

Types of experiments

• Laboratory experiments: Usually conducted with a small sample (of undergraduate psychology students), many times involving games in a computer. Helpful for cognitive/behavioral questions.

• Field experiments: In order to obtain more externally valid results, experiments conducted in the field (i.e., under real-world conditions) are the way to go. Definitely more expensive though. Audit studies are a particular type of field experiment.

• Survey experiments: One can randomize treatment conditions in a survey to evaluate how participants change their responses based on certain stimulus. Vignettes and list experiments are examples of this approach.

• (Bonus) Quasi-experiments: Researchers usually call quasi-experiments to real-world situations that offer as-if random variation in a treatment of interest. For example, earthquakes, change in laws, date of birth, etc.

How to randomise?

How much dispersion (i.e. uncertainty) is in our distribution will be affected by the level at which randomization (i.e. treatment) happens: at the individual or cluster/group level?

Conditional randomization (i.e., blocking) increases efficiency, when we have variables that are highly predictive of the outcome of interest

Blocking

Similar to the intuition for stratified random sampling in the context of surveys, blocking may increase precision in experimental design

Precision gains are similar to increasing the sample size

• Collect background information on covariates relevant to the outcome

• Pre-stratify your sample, then randomise within blocks

  • This ensures that, with respect to the blocked factors, both treatment arms are identical

  • It is essentially the same as running a separate experiment in each strata

Blocking

For estimation, obtain block-specific effects, and average according to population shares. With \(J\) strata:

\[ \tau_{\text{block}} = \sum_{j=1}^{J} \frac{N_j}{N} \tau_j \]

Short activity (3 mins)

It may be hard to imagine an experiment that would be relevant for the type of questions we care about.

Some even say that experiments tend to emphasize “small” versus “big” questions, promoting incremental/testable policies.

However, there are examples of experiments addressing big and difficult questions. Can you think of any example or a proposal? ( hint, see https://graemeblair.com/teaching/UCLA_PS200E_Syllabus.pdf )

What is your estimand?

Researchers tend to formalize their effect of interest as regression coefficients (i.e., their hypotheses are formulated within a statistical model)

This is too restrictive!

What is your estimand?

Statistics/ML vs Causal Inference

The ladder of causation

The ladder of causation

Why should we care about causal inference?

What should you expect from this class?

This class is designed as a first course in causal inference, so we focus on essentials:

• Familiarize yourself with the most widely used causal inference frameworks

• Understand the role of randomisation to tackle causal questions

• Use potential outcomes (and the do-operator) to formalize causal estimands

• Use directed acyclic graphs (DAGs) to encode qualitative assumptions and derive testable implications

• Selection on observables (regression, imputation, matching, weighting, doubly robust methods and flexible estimation using machine learning)

• Difference-in-difference, synthetic controls, and extensions

• Instrumental variables and regression discontinuity designs

• Sensitivity analysis

Logistics

• Lectures: Wed 2-4pm

• Exercises: hands-on practices that accompany the lectures (self-study), released weekly

• Bi-weekly drop-ins: optional, lecturer and TA present for any questions, Fri 12-1pm weeks 2/4/6/8

• Summative assessments: to be released bi-weekly

Correlation vs causation

Selection bias