class: center, middle, inverse, title-slide .title[ # Advanced quantitative data analysis ] .subtitle[ ## Difference in Difference II ] .author[ ### Mengni Chen ] .institute[ ### Department of Sociology, University of Copenhagen ] --- <style type="text/css"> .remark-slide-content { font-size: 20px; padding: 20px 80px 20px 80px; } .remark-code, .remark-inline-code { background: #f0f0f0; } .remark-code { font-size: 14px; } </style> #Let's get ready ```r #install.packages("did") library(tidyverse) # Add the tidyverse package to my current library. library(haven) # Handle labelled data. library(splitstackshape) #transform wide data (with stacked variables) to long data library(plm) #linear models for panel data library(did) #for difference in difference analysis ``` --- #Standard difference in difference `\(Y_{i}=b_{0} + δ_{G}G_{treatid} + δ_{T}T_{treattime} + δ_{GT}(G_{treatid}*T_{treattime})\)`, where `\(δ_{GT}\)` is the average treated effect on the treated. - Two groups: one control group, one treated group - There is only one treatment, and the treatment happens at the same time to all members in the treated group - Two period of observations: one period before the treatment, one period after the treatment. <img src="https://github.com/fancycmn/slide13/blob/main/S13_Pic1.PNG?raw=true" width="40%" style="display: block; margin-left:20px;"> --- #Standard difference in difference - The model can be modified by using two-way fixed effects (TWFE) - `\(Y_{i}=α_{i} + λ_{t} + δ^{DD}D_{it}+ ε_{it}\)` - where `\(α_{i}\)` and `\(λ_{t}\)` are the individual and time fixed effect, and `\(D_{it}\)` is similar to `\(G_{treatid}*T_{treattime}\)`. - `\(δ^{DD}\)` is the DID ATT estimate. <img src="https://github.com/fancycmn/slide13/blob/main/S13_Pic1.PNG?raw=true" width="40%" style="display: block; margin-top:20px;"> --- #Difference in differnce: time-varying treatment - More then two group: one control, one treated group is treated earlier, one treated group is treated later - More than two periods - The treatment time is not the same for all members in the treated group - This is the so-called **staggered treatment** - Using a two-way fixed effect (TWFE) to estimate the average treatment effect (ATT) is problematic (Goodman-Bacon 2018) - Problem 1: `\(δ^{DD}\)` becomes a very strange weighted average, due to the strange weight - Problem 2: the treatment effects may be heterogeneous across groups and over time <img src="https://d33wubrfki0l68.cloudfront.net/53a39a756721843acc5f97dc13f07b72991a38aa/716a0/post/2019-09-25-difference-in-differences-methodology_files/figure-html/unnamed-chunk-3-1.png" width="50%" style="display: block; margin-top:20px;"> --- # TWFE is problematic when the treatment is staggered - Goodman-Bacon(2018) identifies that the `\(δ^{DD}\)` is a [strange weighted average](https://www.youtube.com/watch?v=aUHCAG98G-o) of 2x2 comparisons (see the following graphs for different 2x2 comparisons). - Difficult to interpret what the ATT means from the twoway fixed effect - Later treated groups become a control group for early treated groups - Earlier treated groups become also a control group of late treated groups - Heterogeneous treatment effects may lead to severe bias - Estimate depends on many factors: variation in treatment time, group size, etc <img src="https://d33wubrfki0l68.cloudfront.net/53a39a756721843acc5f97dc13f07b72991a38aa/716a0/post/2019-09-25-difference-in-differences-methodology_files/figure-html/unnamed-chunk-3-1.png" width="40%", style="position:absolute; left:50px; top:330px;"> <img src="https://d33wubrfki0l68.cloudfront.net/5ea38f7e27db439faeb5eafecb24d7a2d48fe581/7c116/post/2019-09-25-difference-in-differences-methodology_files/figure-html/unnamed-chunk-4-1.png" width="40%", style="position:absolute; right:150px; top:330px;"> --- # Two-way fixed effect is problematic - `\(δ^{DD}\)` from two-way fixed effect estimate depends on the weights for the three groups - treated group vs untreated group - earlier treated vs later treated - later treated vs earlier treated - Greater weight will be given to - Big groups (i.e. many observations) - Groups that are treated closer to th middle of the sample period - Therefore, due to the strange weight, it is very difficult to explain what `\(δ^{DD}\)` it means --- #Solutions: Callaway and Sant’Anna (2021) - Give a group-period specific estimate - ATT(g,t): average treatment effect of group G and at time t - Use the "never treated" group as the comparison for every treated group and every period - Summarize the ATT(g,t)s, using weights flexibly and meaningful to you --- #Using DID package - [prepare the data](https://rpubs.com/fancycmn/979788) - three codes - estimate a group-period specific effect: `att_gt()` - plot the result: `ggdid()` - summarize the ATT(g,t)s: `aggte()` --- #Estimate a group-period specific effect ```r did <- att_gt(yname = "sat", #dependent variable tname = "wave", #time variable idname = "id", #id gname = "partnerwave1", #the variable in data that contains the first period when a particular observation is treated. xformla = ~ hlt, #when you don't have any covariates to control, use "~ 1"; if yes, you can add covariates here by ~ x1+x2 data = long_data #specify your data ) ``` ``` ## Warning in pre_process_did(yname = yname, tname = tname, idname = idname, : ## Dropped 1844 observations while converting to balanced panel. ``` ```r summary(did) ``` --- #Estimate a group-period specific effect ``` ## ## Call: ## att_gt(yname = "sat", tname = "wave", idname = "id", gname = "partnerwave1", ## xformla = ~hlt, data = long_data) ## ## Reference: Callaway, Brantly and Pedro H.C. Sant'Anna. "Difference-in-Differences with Multiple Time Periods." Journal of Econometrics, Vol. 225, No. 2, pp. 200-230, 2021. <https://doi.org/10.1016/j.jeconom.2020.12.001>, <https://arxiv.org/abs/1803.09015> ## ## Group-Time Average Treatment Effects: ## Group Time ATT(g,t) Std. Error [95% Simult. Conf. Band] ## 2 2 0.4247 0.2057 -0.1748 1.0242 ## 2 3 0.5519 0.1995 -0.0293 1.1331 ## 2 4 0.3989 0.2054 -0.1996 0.9974 ## 2 5 0.3846 0.2201 -0.2568 1.0260 ## 2 6 0.3509 0.2486 -0.3734 1.0752 ## 3 2 0.2135 0.1873 -0.3322 0.7593 ## 3 3 0.3585 0.1942 -0.2074 0.9244 ## 3 4 -0.0110 0.2902 -0.8568 0.8348 ## 3 5 -0.0818 0.2165 -0.7126 0.5491 ## 3 6 0.2470 0.2169 -0.3850 0.8790 ## 4 2 -0.2275 0.2310 -0.9006 0.4457 ## 4 3 0.3435 0.2219 -0.3030 0.9900 ## 4 4 0.1018 0.1965 -0.4708 0.6744 ## 4 5 0.2113 0.1704 -0.2853 0.7080 ## 4 6 0.3577 0.1941 -0.2079 0.9233 ## 5 2 0.2324 0.2501 -0.4966 0.9613 ## 5 3 -0.2236 0.2103 -0.8364 0.3891 ## 5 4 0.3300 0.1957 -0.2402 0.9002 ## 5 5 0.2728 0.2102 -0.3397 0.8852 ## 5 6 0.1340 0.1905 -0.4210 0.6890 ## 6 2 -0.4108 0.3027 -1.2927 0.4711 ## 6 3 0.0123 0.2222 -0.6352 0.6599 ## 6 4 -0.0745 0.3212 -1.0106 0.8616 ## 6 5 0.1747 0.3191 -0.7551 1.1044 ## 6 6 0.4392 0.2427 -0.2680 1.1465 ## --- ## Signif. codes: `*' confidence band does not cover 0 ## ## P-value for pre-test of parallel trends assumption: 0.27657 ## Control Group: Never Treated, Anticipation Periods: 0 ## Estimation Method: Doubly Robust ``` --- #Plot group-period specific effect ```r ggdid(did, ylim=c(-2,2)) ``` <img src="https://github.com/fancycmn/slide13/blob/main/S13_Pic6.png?raw=true" width="45%" style="display: block; margin-left:20px;"> --- #Summarize the ATT(g,t)s: get an overall one ATT ```r #"simple" (this just computes a weighted average of all group-time average treatment effects with weights proportional to group size) agg.ovearll <- aggte(did, type = "simple") summary(agg.ovearll) ``` ``` ## ## Call: ## aggte(MP = did, type = "simple") ## ## Reference: Callaway, Brantly and Pedro H.C. Sant'Anna. "Difference-in-Differences with Multiple Time Periods." Journal of Econometrics, Vol. 225, No. 2, pp. 200-230, 2021. <https://doi.org/10.1016/j.jeconom.2020.12.001>, <https://arxiv.org/abs/1803.09015> ## ## ## ATT Std. Error [ 95% Conf. Int.] ## 0.288 0.096 0.0999 0.4761 * ## ## ## --- ## Signif. codes: `*' confidence band does not cover 0 ## ## Control Group: Never Treated, Anticipation Periods: 0 ## Estimation Method: Doubly Robust ``` --- #Summarize the ATT(g,t)s: get an group-specific ATT ```r #average treatment effects across different groups agg.group <- aggte(did, type = "group") summary(agg.group) ``` ``` ## ## Call: ## aggte(MP = did, type = "group") ## ## Reference: Callaway, Brantly and Pedro H.C. Sant'Anna. "Difference-in-Differences with Multiple Time Periods." Journal of Econometrics, Vol. 225, No. 2, pp. 200-230, 2021. <https://doi.org/10.1016/j.jeconom.2020.12.001>, <https://arxiv.org/abs/1803.09015> ## ## ## Overall summary of ATT's based on group/cohort aggregation: ## ATT Std. Error [ 95% Conf. Int.] ## 0.2834 0.0804 0.1257 0.441 * ## ## ## Group Effects: ## Group Estimate Std. Error [95% Simult. Conf. Band] ## 2 0.4222 0.1877 -0.0476 0.8920 ## 3 0.1282 0.1989 -0.3696 0.6260 ## 4 0.2236 0.1559 -0.1667 0.6139 ## 5 0.2034 0.1748 -0.2341 0.6409 ## 6 0.4392 0.2391 -0.1593 1.0378 ## --- ## Signif. codes: `*' confidence band does not cover 0 ## ## Control Group: Never Treated, Anticipation Periods: 0 ## Estimation Method: Doubly Robust ``` --- #Summarize the ATT(g,t)s: plot a group-specific ATT ```r ggdid(agg.group) ``` <img src="https://github.com/fancycmn/slide13/blob/main/S13_Pic7.png?raw=true" width="45%" style="display: block; margin-left:20px;"> --- #Summarize the ATT(g,t)s - Get a time-dynamic ATT ```r agg.dynamic <- aggte(did, type = "dynamic") summary(agg.dynamic) ``` ``` ## ## Call: ## aggte(MP = did, type = "dynamic") ## ## Reference: Callaway, Brantly and Pedro H.C. Sant'Anna. "Difference-in-Differences with Multiple Time Periods." Journal of Econometrics, Vol. 225, No. 2, pp. 200-230, 2021. <https://doi.org/10.1016/j.jeconom.2020.12.001>, <https://arxiv.org/abs/1803.09015> ## ## ## Overall summary of ATT's based on event-study/dynamic aggregation: ## ATT Std. Error [ 95% Conf. Int.] ## 0.2978 0.1223 0.0581 0.5375 * ## ## ## Dynamic Effects: ## Event time Estimate Std. Error [95% Simult. Conf. Band] ## -4 -0.4108 0.2838 -1.1755 0.3539 ## -3 0.1375 0.1611 -0.2966 0.5716 ## -2 -0.1875 0.1378 -0.5590 0.1839 ## -1 0.2744 0.1078 -0.0162 0.5650 ## 0 0.3141 0.0885 0.0756 0.5527 * ## 1 0.2494 0.1077 -0.0408 0.5396 ## 2 0.2484 0.1205 -0.0763 0.5730 ## 3 0.3263 0.1594 -0.1033 0.7559 ## 4 0.3509 0.2421 -0.3015 1.0033 ## --- ## Signif. codes: `*' confidence band does not cover 0 ## ## Control Group: Never Treated, Anticipation Periods: 0 ## Estimation Method: Doubly Robust ``` --- #Summarize the ATT(g,t)s: plot a time-dynamic ATT ```r ggdid(agg.dynamic) ``` <img src="https://github.com/fancycmn/slide13/blob/main/S13_Pic10.png?raw=true" width="45%" style="display: block; margin-left:20px;"> --- #About balanced or unbalanced data ```r did_ub <- att_gt(yname = "sat", #dependent variable tname = "wave", #time variable idname = "id", #id gname = "partnerwave1", #the variable in data that contains the first period when a particular observation is treated. xformla = ~ hlt, #when you don't have any covariates to control, use "~ 1"; if yes, you can add covariates here by ~ x1+x2 data = long_data, #specify your data allow_unbalanced_panel = TRUE #allow Rstudio to use unbalanced data to do the estimation ) agg.ovearll_ub <- aggte(did_ub, type = "simple") summary(agg.ovearll_ub) ``` <img src="https://github.com/fancycmn/slide13/blob/main/S13_Pic9.png?raw=true" width="45%" style="display: block; margin-left:20px;"> - a struggle between efficiency and bias: - on one hand, unbalanced data could lead to bias result - on the other hand, too few observations in balanced data may not give you enough sample size and inefficient in using data --- #Take home - What is staggered treatment - What is the problem of using two-way fixed effect to estimate the ATT of staggered treatment - Using `did` package to do staggered DID - `att_gt()`: estimate the group-time specific effect - `ggdid()`: plot the effect - `aggte()`: aggregate the group-time specific effect --- class: center, middle #[Exercise](https://rpubs.com/fancycmn/979794)