Licença

This work is licensed under the Creative Commons Attribution-ShareAlike 4.0 International License. To view a copy of this license, visit http://creativecommons.org/licenses/by-sa/4.0/ or send a letter to Creative Commons, PO Box 1866, Mountain View, CA 94042, USA.

License: CC BY-SA 4.0

Citação

Sugestão de citação: FIGUEIREDO, Adriano Marcos Rodrigues. Exemplo Diferenças em diferenças do Torres-Reyna (2015): R e Python. Campo Grande-MS, Brasil: RStudio/Rpubs, 2021. Disponível em https://rpubs.com/amrofi/diff_in_diff_torres_reyna.

1 Introdução

Neste exemplo, reproduzimos o script de aula do Prof. Torres-Reyna (TORRES-REYNA, 2015). Os dados vem das notas de aula (https://www.princeton.edu/~otorres/DID101R.pdf). Um exemplo parecido utilizando dados em painel pode ser visto em Figueiredo (2019).

# Exemplo de Dados de Corte Transversal
library(tidyr)
library(dplyr)
library(DT)
library(magrittr)

Os dados em painel, ou de combinação de seção cruzada e série temporal (SCST), ou também chamados de dados longitudinais, associam dados de diferentes unidades ou indivíduos para diferentes períodos de tempo.

Segue o exemplo de Torres-Reyna (2015) para os dados em Panel101.dta, para 10 anos (1990-99) e 7 países (aqui designados pelas letras A até G). Primeiro chamaremos os dados.

library(foreign)
mydata <-read.dta("http://dss.princeton.edu/training/Panel101.dta")
datatable(mydata)

Agora, criaremos a variável dummy indicadora do ano em que o tratamento foi iniciado. Encorajamos os leitores a ler Waldinger (sd) e Wooldridge (2007) para o tratamento teórico.

Neste caso, assumiremos, como em Torres-Reyna (2015), que o tratamento iniciou em 1994. A variável dummy terá o nome time e será criada em mydata com valores 0 para year menor que 1994 e valores 1 para year >= 1994.

mydata$time = ifelse(mydata$year >= 1994, 1, 0)

Portanto, agora o mydatatem 10 variáveis, e a variável time como se vê abaixo:

datatable(mydata[,c(1:2,10)])

Agora faremos uma variável para identificar os grupos “tratados”, chamada treated. Ela é criada também como dummy com valores iguais a 1 para os grupos de países E, F e G, e zero para os demais. Assim, temos o código.

mydata$treated = ifelse(mydata$country == "E" | 
                        mydata$country == "F" | 
                        mydata$country == "G", 1, 0)

O leitor pode verificar que agora existem 11 variáveis em mydata, e que os países E, F e G tem valores de treated iguais a 1.

datatable(mydata[,c(1:2,10:11)])

Criaremos agora uma variável que fará a interação entre time e treated, ou seja, os anos em que houve tratamento e os países que sofreram o tratamento. Esta será a variável chave para o modelo diff-in-diff, a variável did.

mydata$did = mydata$time * mydata$treated
write.csv(mydata,'mydata.csv')

Tipicamente, o modelo diff-in-diff será uma regressão do tipo (WALDINGER, sd):

\[ Resultado_{it} = \beta_1 + \beta_2.Tratamento_i + \beta_3.Periodo_t + + \beta_4.(Tratamento.Periodo)_{it} + \varepsilon \] em que:
* \(Resultado\) é a nossa variável dependente;
* \(Tratamento\) é a dummy que indica a observação do grupo de tratamento;
* \(Periodo\) é a dummy que indica o período pós-tratamento;
* \(\beta\) são parâmetros a serem estimados;
* \(i\) e \(t\) são subscritos para a observação e o ano;
* \(\varepsilon\) é o resíduo ruído branco.

2 Estimando o DID (diff-in-diff)

didreg = lm(y ~ treated + time + did, data = mydata)
summary(didreg)
## 
## Call:
## lm(formula = y ~ treated + time + did, data = mydata)
## 
## Residuals:
##        Min         1Q     Median         3Q        Max 
## -9.768e+09 -1.623e+09  1.167e+08  1.393e+09  6.807e+09 
## 
## Coefficients:
##               Estimate Std. Error t value Pr(>|t|)  
## (Intercept)  3.581e+08  7.382e+08   0.485   0.6292  
## treated      1.776e+09  1.128e+09   1.575   0.1200  
## time         2.289e+09  9.530e+08   2.402   0.0191 *
## did         -2.520e+09  1.456e+09  -1.731   0.0882 .
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 2.953e+09 on 66 degrees of freedom
## Multiple R-squared:  0.08273,    Adjusted R-squared:  0.04104 
## F-statistic: 1.984 on 3 and 66 DF,  p-value: 0.1249

3 Estimação do DID pelo método multiplicativo

Neste caso, não precisamos fazer a interação, o programa faz automaticamente.

didreg1 = lm(y ~ treated*time, data = mydata)
summary(didreg1)
## 
## Call:
## lm(formula = y ~ treated * time, data = mydata)
## 
## Residuals:
##        Min         1Q     Median         3Q        Max 
## -9.768e+09 -1.623e+09  1.167e+08  1.393e+09  6.807e+09 
## 
## Coefficients:
##                Estimate Std. Error t value Pr(>|t|)  
## (Intercept)   3.581e+08  7.382e+08   0.485   0.6292  
## treated       1.776e+09  1.128e+09   1.575   0.1200  
## time          2.289e+09  9.530e+08   2.402   0.0191 *
## treated:time -2.520e+09  1.456e+09  -1.731   0.0882 .
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 2.953e+09 on 66 degrees of freedom
## Multiple R-squared:  0.08273,    Adjusted R-squared:  0.04104 
## F-statistic: 1.984 on 3 and 66 DF,  p-value: 0.1249

O coeficiente de ‘treated:time’ é a estimativa do differences-in-differences (did no exemplo anterior). O efeito foi significativo a 10% com o tratamento tendo efeito negativo.

4 Exemplo em Python

Chamarei o arquivo csv salvo no início do exemplo.

import numpy as np
import csv
import pandas
mydata = pandas.read_csv("mydata.csv")
print(mydata.head())
##    Unnamed: 0 country  year           y  ...  op  time  treated  did
## 0           1       A  1990  1342787840  ...   1     0        0    0
## 1           2       A  1991 -1899660544  ...   0     0        0    0
## 2           3       A  1992   -11234363  ...   0     0        0    0
## 3           4       A  1993  2645775360  ...   0     0        0    0
## 4           5       A  1994  3008334848  ...   0     1        0    0
## 
## [5 rows x 13 columns]

Como gerei o csv após criar as variáveis em R, já tenho todas as variáveis para a regressão linear no Python.

Código para instalar o sklearn:

library(reticulate)
py_install("scikit-learn")
py_install("statsmodels")
py_install("statsmodels.formula.api")
import numpy as np
from sklearn.linear_model import LinearRegression
import statsmodels.formula.api as sm
import statsmodels.formula.api as sm
result = sm.ols(formula="y ~ treated + time + did", data=mydata).fit()
print(result.summary())
##                             OLS Regression Results                            
## ==============================================================================
## Dep. Variable:                      y   R-squared:                       0.083
## Model:                            OLS   Adj. R-squared:                  0.041
## Method:                 Least Squares   F-statistic:                     1.984
## Date:                Mon, 29 Mar 2021   Prob (F-statistic):              0.125
## Time:                        17:46:13   Log-Likelihood:                -1623.7
## No. Observations:                  70   AIC:                             3255.
## Df Residuals:                      66   BIC:                             3264.
## Df Model:                           3                                         
## Covariance Type:            nonrobust                                         
## ==============================================================================
##                  coef    std err          t      P>|t|      [0.025      0.975]
## ------------------------------------------------------------------------------
## Intercept   3.581e+08   7.38e+08      0.485      0.629   -1.12e+09    1.83e+09
## treated     1.776e+09   1.13e+09      1.575      0.120   -4.75e+08    4.03e+09
## time        2.289e+09   9.53e+08      2.402      0.019    3.87e+08    4.19e+09
## did         -2.52e+09   1.46e+09     -1.731      0.088   -5.43e+09    3.87e+08
## ==============================================================================
## Omnibus:                        6.182   Durbin-Watson:                   1.615
## Prob(Omnibus):                  0.045   Jarque-Bera (JB):                7.779
## Skew:                          -0.304   Prob(JB):                       0.0205
## Kurtosis:                       4.516   Cond. No.                         7.26
## ==============================================================================
## 
## Notes:
## [1] Standard Errors assume that the covariance matrix of the errors is correctly specified.

5 Referências

FIGUEIREDO, Adriano Marcos Rodrigues. Econometria: exemplo de dados em painel em R - Torres-Reyna data. Campo Grande-MS,Brasil: RStudio/Rpubs, 2019. Disponível em http://rpubs.com/amrofi/Econometrics_panel_torres_reyna e em https://adrianofigueiredo.netlify.app/post/Econometrics_panel_torres_reyna/.

TORRES-REYNA, Oscar. Differences-in-Differences (using R). Princeton,NJ(USA): Princeton University, 2015. Disponível em: https://www.princeton.edu/~otorres/DID101R.pdf.

WALDINGER, Fabian. Lectures 3 - Differences-in-Differences. Munich (sd). Disponível em: https://www.fabianwaldinger.com/applied-econometrics. Acesso: March, 27th, 2021.

WOOLDRIDGE, J. Difference-in-Differences Estimation. NBER, 2007. Disponível em: https://www.nber.org/lecture/difference-differences-estimation e video em: https://vimeo.com/27776614. Acesso: March, 27th, 2021.

LS0tDQp0aXRsZTogIkV4ZW1wbG8gRGlmZXJlbsOnYXMgZW0gZGlmZXJlbsOnYXMgZG8gVG9ycmVzLVJleW5hICgyMDE1KTogUiBlIFB5dGhvbiINCmF1dGhvcjogIkFkcmlhbm8gTWFyY29zIFJvZHJpZ3VlcyBGaWd1ZWlyZWRvLCAqZS1tYWlsOiBhZHJpYW5vLmZpZ3VlaXJlZG9AdWZtcy5icioiDQpkYXRlOiAiYHIgZm9ybWF0KFN5cy5EYXRlKCksICclZCAlQiAlWScpYCINCm91dHB1dDoNCiAgaHRtbF9kb2N1bWVudDoNCiAgICBjb2RlX2Rvd25sb2FkOiB5ZXMNCiAgICB0aGVtZTogZGVmYXVsdA0KICAgIG51bWJlcl9zZWN0aW9uczogeWVzDQogICAgdG9jOiB5ZXMNCiAgICB0b2NfZmxvYXQ6IG5vDQogICAgZGZfcHJpbnQ6IHBhZ2VkDQogICAgZmlnX2NhcHRpb246IHllcw0KLS0tDQoNCmBgYHtyIHNldHVwLCBpbmNsdWRlPUZBTFNFfQ0Ka25pdHI6Om9wdHNfY2h1bmskc2V0KGVjaG8gPSBUUlVFKQ0KYGBgDQoNCiMgTGljZW7Dp2EgeyNMaWNlbsOnYSAudW5udW1iZXJlZH0NCg0KVGhpcyB3b3JrIGlzIGxpY2Vuc2VkIHVuZGVyIHRoZSBDcmVhdGl2ZSBDb21tb25zIEF0dHJpYnV0aW9uLVNoYXJlQWxpa2UgNC4wIEludGVybmF0aW9uYWwgTGljZW5zZS4gVG8gdmlldyBhIGNvcHkgb2YgdGhpcyBsaWNlbnNlLCB2aXNpdCA8aHR0cDovL2NyZWF0aXZlY29tbW9ucy5vcmcvbGljZW5zZXMvYnktc2EvNC4wLz4gb3Igc2VuZCBhIGxldHRlciB0byBDcmVhdGl2ZSBDb21tb25zLCBQTyBCb3ggMTg2NiwgTW91bnRhaW4gVmlldywgQ0EgOTQwNDIsIFVTQS4NCg0KIVtMaWNlbnNlOiBDQyBCWS1TQSA0LjBdKGh0dHBzOi8vbWlycm9ycy5jcmVhdGl2ZWNvbW1vbnMub3JnL3ByZXNza2l0L2J1dHRvbnMvODh4MzEvcG5nL2J5LXNhLnBuZyl7d2lkdGg9IjI1JSJ9DQoNCg0KIyBDaXRhw6fDo28geyNDaXRhw6fDo28gLnVubnVtYmVyZWR9DQoNClN1Z2VzdMOjbyBkZSBjaXRhw6fDo286IEZJR1VFSVJFRE8sIEFkcmlhbm8gTWFyY29zIFJvZHJpZ3Vlcy4gRXhlbXBsbyBEaWZlcmVuw6dhcyBlbSBkaWZlcmVuw6dhcyBkbyBUb3JyZXMtUmV5bmEgKDIwMTUpOiBSIGUgUHl0aG9uLiBDYW1wbyBHcmFuZGUtTVMsIEJyYXNpbDogUlN0dWRpby9ScHVicywgMjAyMS4gRGlzcG9uw612ZWwgZW0gPGh0dHBzOi8vcnB1YnMuY29tL2Ftcm9maS9kaWZmX2luX2RpZmZfdG9ycmVzX3JleW5hPi4NCg0KIyBJbnRyb2R1w6fDo28NCg0KTmVzdGUgZXhlbXBsbywgcmVwcm9kdXppbW9zIG8gc2NyaXB0IGRlIGF1bGEgZG8gUHJvZi4gVG9ycmVzLVJleW5hIChUT1JSRVMtUkVZTkEsIDIwMTUpLiBPcyBkYWRvcyB2ZW0gZGFzIG5vdGFzIGRlIGF1bGEgKGh0dHBzOi8vd3d3LnByaW5jZXRvbi5lZHUvfm90b3JyZXMvRElEMTAxUi5wZGYpLiBVbSBleGVtcGxvIHBhcmVjaWRvIHV0aWxpemFuZG8gZGFkb3MgZW0gcGFpbmVsIHBvZGUgc2VyIHZpc3RvIGVtIEZpZ3VlaXJlZG8gKDIwMTkpLg0KDQpgYGB7ciwgZWNobz1UUlVFLCBtZXNzYWdlPUZBTFNFLHdhcm5pbmc9RkFMU0UsIHRpZHk9VFJVRSwgcmVzdWx0cz0nYXNpcyd9DQojIEV4ZW1wbG8gZGUgRGFkb3MgZGUgQ29ydGUgVHJhbnN2ZXJzYWwNCmxpYnJhcnkodGlkeXIpO2xpYnJhcnkoZHBseXIpO2xpYnJhcnkoRFQpO2xpYnJhcnkobWFncml0dHIpDQpgYGANCg0KT3MgZGFkb3MgZW0gcGFpbmVsLCBvdSBkZSBjb21iaW5hw6fDo28gZGUgc2XDp8OjbyBjcnV6YWRhIGUgc8OpcmllIHRlbXBvcmFsIChTQ1NUKSwgb3UgdGFtYsOpbSBjaGFtYWRvcyBkZSBkYWRvcyBsb25naXR1ZGluYWlzLCBhc3NvY2lhbSBkYWRvcyBkZSBkaWZlcmVudGVzIHVuaWRhZGVzIG91IGluZGl2w61kdW9zIHBhcmEgZGlmZXJlbnRlcyBwZXLDrW9kb3MgZGUgdGVtcG8uICAgICANCg0KU2VndWUgbyBleGVtcGxvIGRlIFRvcnJlcy1SZXluYSAoMjAxNSkgcGFyYSBvcyBkYWRvcyBlbSBgUGFuZWwxMDEuZHRhYCwgcGFyYSAxMCBhbm9zICgxOTkwLTk5KSBlIDcgcGHDrXNlcyAoYXF1aSBkZXNpZ25hZG9zIHBlbGFzIGxldHJhcyBBIGF0w6kgRykuIFByaW1laXJvIGNoYW1hcmVtb3Mgb3MgZGFkb3MuDQoNCmBgYHtyfQ0KbGlicmFyeShmb3JlaWduKQ0KbXlkYXRhIDwtcmVhZC5kdGEoImh0dHA6Ly9kc3MucHJpbmNldG9uLmVkdS90cmFpbmluZy9QYW5lbDEwMS5kdGEiKQ0KZGF0YXRhYmxlKG15ZGF0YSkNCg0KYGBgDQoNCkFnb3JhLCBjcmlhcmVtb3MgYSB2YXJpw6F2ZWwgZHVtbXkgaW5kaWNhZG9yYSBkbyBhbm8gZW0gcXVlIG8gdHJhdGFtZW50byBmb2kgaW5pY2lhZG8uIEVuY29yYWphbW9zIG9zIGxlaXRvcmVzIGEgbGVyIFdhbGRpbmdlciAoc2QpIGUgV29vbGRyaWRnZSAoMjAwNykgcGFyYSBvIHRyYXRhbWVudG8gdGXDs3JpY28uICAgIA0KDQpOZXN0ZSBjYXNvLCBhc3N1bWlyZW1vcywgY29tbyBlbSBUb3JyZXMtUmV5bmEgKDIwMTUpLCBxdWUgbyB0cmF0YW1lbnRvIGluaWNpb3UgZW0gMTk5NC4gQSB2YXJpw6F2ZWwgZHVtbXkgdGVyw6EgbyBub21lIGB0aW1lYCBlIHNlcsOhIGNyaWFkYSBlbSBgbXlkYXRhYCBjb20gdmFsb3JlcyAwIHBhcmEgYHllYXJgIG1lbm9yIHF1ZSAxOTk0IGUgdmFsb3JlcyAxIHBhcmEgYHllYXIgPj0gMTk5NGAuIA0KDQpgYGB7cn0NCm15ZGF0YSR0aW1lID0gaWZlbHNlKG15ZGF0YSR5ZWFyID49IDE5OTQsIDEsIDApDQpgYGANCg0KUG9ydGFudG8sIGFnb3JhIG8gYG15ZGF0YWB0ZW0gMTAgdmFyacOhdmVpcywgZSBhIHZhcmnDoXZlbCB0aW1lIGNvbW8gc2UgdsOqIGFiYWl4bzoNCg0KYGBge3J9DQpkYXRhdGFibGUobXlkYXRhWyxjKDE6MiwxMCldKQ0KYGBgDQoNCkFnb3JhIGZhcmVtb3MgdW1hIHZhcmnDoXZlbCBwYXJhIGlkZW50aWZpY2FyIG9zIGdydXBvcyAidHJhdGFkb3MiLCBjaGFtYWRhIGB0cmVhdGVkYC4gRWxhIMOpIGNyaWFkYSB0YW1iw6ltIGNvbW8gZHVtbXkgY29tIHZhbG9yZXMgaWd1YWlzIGEgMSBwYXJhIG9zIGdydXBvcyBkZSBwYcOtc2VzIEUsIEYgZSBHLCBlIHplcm8gcGFyYSBvcyBkZW1haXMuIEFzc2ltLCB0ZW1vcyBvIGPDs2RpZ28uDQoNCmBgYHtyfQ0KbXlkYXRhJHRyZWF0ZWQgPSBpZmVsc2UobXlkYXRhJGNvdW50cnkgPT0gIkUiIHwgDQogICAgICAgICAgICAgICAgICAgICAgICBteWRhdGEkY291bnRyeSA9PSAiRiIgfCANCiAgICAgICAgICAgICAgICAgICAgICAgIG15ZGF0YSRjb3VudHJ5ID09ICJHIiwgMSwgMCkNCmBgYA0KDQpPIGxlaXRvciBwb2RlIHZlcmlmaWNhciBxdWUgYWdvcmEgZXhpc3RlbSAxMSB2YXJpw6F2ZWlzIGVtIGBteWRhdGFgLCBlIHF1ZSBvcyBwYcOtc2VzIEUsIEYgZSBHIHRlbSB2YWxvcmVzIGRlIGB0cmVhdGVkYCBpZ3VhaXMgYSAxLiAgICANCg0KYGBge3J9DQpkYXRhdGFibGUobXlkYXRhWyxjKDE6MiwxMDoxMSldKQ0KYGBgDQoNCkNyaWFyZW1vcyBhZ29yYSB1bWEgdmFyacOhdmVsIHF1ZSBmYXLDoSBhIGludGVyYcOnw6NvIGVudHJlIGB0aW1lYCBlIGB0cmVhdGVkYCwgb3Ugc2VqYSwgb3MgYW5vcyBlbSBxdWUgaG91dmUgdHJhdGFtZW50byBlIG9zIHBhw61zZXMgcXVlIHNvZnJlcmFtIG8gdHJhdGFtZW50by4gRXN0YSBzZXLDoSBhIHZhcmnDoXZlbCBjaGF2ZSBwYXJhIG8gbW9kZWxvIGRpZmYtaW4tZGlmZiwgYSB2YXJpw6F2ZWwgYGRpZGAuICAgIA0KDQpgYGB7cn0NCm15ZGF0YSRkaWQgPSBteWRhdGEkdGltZSAqIG15ZGF0YSR0cmVhdGVkDQp3cml0ZS5jc3YobXlkYXRhLCdteWRhdGEuY3N2JykNCmBgYA0KDQpUaXBpY2FtZW50ZSwgbyBtb2RlbG8gZGlmZi1pbi1kaWZmIHNlcsOhIHVtYSByZWdyZXNzw6NvIGRvIHRpcG8gKFdBTERJTkdFUiwgc2QpOg0KDQokJA0KUmVzdWx0YWRvX3tpdH0gPSBcYmV0YV8xICsgXGJldGFfMi5UcmF0YW1lbnRvX2kgKyBcYmV0YV8zLlBlcmlvZG9fdCArICsgXGJldGFfNC4oVHJhdGFtZW50by5QZXJpb2RvKV97aXR9ICsgXHZhcmVwc2lsb24NCiQkDQplbSBxdWU6ICAgICANCiogJFJlc3VsdGFkbyQgw6kgYSBub3NzYSB2YXJpw6F2ZWwgZGVwZW5kZW50ZTsgICAgIA0KKiAkVHJhdGFtZW50byQgw6kgYSBkdW1teSBxdWUgaW5kaWNhIGEgb2JzZXJ2YcOnw6NvIGRvIGdydXBvIGRlIHRyYXRhbWVudG87ICAgIA0KKiAkUGVyaW9kbyQgw6kgYSBkdW1teSBxdWUgaW5kaWNhIG8gcGVyw61vZG8gcMOzcy10cmF0YW1lbnRvOyAgICANCiogJFxiZXRhJCBzw6NvIHBhcsOibWV0cm9zIGEgc2VyZW0gZXN0aW1hZG9zOyAgICANCiogJGkkIGUgJHQkIHPDo28gc3Vic2NyaXRvcyBwYXJhIGEgb2JzZXJ2YcOnw6NvIGUgbyBhbm87ICAgIA0KKiAkXHZhcmVwc2lsb24kIMOpIG8gcmVzw61kdW8gcnXDrWRvIGJyYW5jby4NCg0KIyBFc3RpbWFuZG8gbyBESUQgKGRpZmYtaW4tZGlmZikNCg0KYGBge3J9DQoNCmRpZHJlZyA9IGxtKHkgfiB0cmVhdGVkICsgdGltZSArIGRpZCwgZGF0YSA9IG15ZGF0YSkNCnN1bW1hcnkoZGlkcmVnKQ0KYGBgDQojIEVzdGltYcOnw6NvIGRvIERJRCBwZWxvIG3DqXRvZG8gbXVsdGlwbGljYXRpdm8NCg0KTmVzdGUgY2FzbywgbsOjbyBwcmVjaXNhbW9zIGZhemVyIGEgaW50ZXJhw6fDo28sIG8gcHJvZ3JhbWEgZmF6IGF1dG9tYXRpY2FtZW50ZS4NCg0KYGBge3J9DQpkaWRyZWcxID0gbG0oeSB+IHRyZWF0ZWQqdGltZSwgZGF0YSA9IG15ZGF0YSkNCnN1bW1hcnkoZGlkcmVnMSkNCmBgYA0KDQpPIGNvZWZpY2llbnRlIGRlIOKAmHRyZWF0ZWQ6dGltZeKAmSDDqSBhIGVzdGltYXRpdmEgZG8gZGlmZmVyZW5jZXMtaW4tZGlmZmVyZW5jZXMgKGBkaWRgIG5vIGV4ZW1wbG8gYW50ZXJpb3IpLiBPIGVmZWl0byBmb2kgc2lnbmlmaWNhdGl2byBhIDEwJSBjb20gbyB0cmF0YW1lbnRvIHRlbmRvIGVmZWl0byBuZWdhdGl2by4NCg0KIyBFeGVtcGxvIGVtIFB5dGhvbg0KDQpDaGFtYXJlaSBvIGFycXVpdm8gY3N2IHNhbHZvIG5vIGluw61jaW8gZG8gZXhlbXBsby4NCg0KYGBge3B5dGhvbn0NCmltcG9ydCBudW1weSBhcyBucA0KaW1wb3J0IGNzdg0KaW1wb3J0IHBhbmRhcw0KDQpgYGANCmBgYHtweXRob259DQpteWRhdGEgPSBwYW5kYXMucmVhZF9jc3YoIm15ZGF0YS5jc3YiKQ0KcHJpbnQobXlkYXRhLmhlYWQoKSkNCmBgYA0KQ29tbyBnZXJlaSBvIGNzdiBhcMOzcyBjcmlhciBhcyB2YXJpw6F2ZWlzIGVtIFIsIGrDoSB0ZW5obyB0b2RhcyBhcyB2YXJpw6F2ZWlzIHBhcmEgYSByZWdyZXNzw6NvIGxpbmVhciBubyBQeXRob24uDQoNCkPDs2RpZ28gcGFyYSBpbnN0YWxhciBvIHNrbGVhcm46DQoNCmBgYA0KbGlicmFyeShyZXRpY3VsYXRlKQ0KcHlfaW5zdGFsbCgic2Npa2l0LWxlYXJuIikNCnB5X2luc3RhbGwoInN0YXRzbW9kZWxzIikNCnB5X2luc3RhbGwoInN0YXRzbW9kZWxzLmZvcm11bGEuYXBpIikNCg0KYGBgDQoNCg0KYGBge3B5dGhvbiwgZXZhbD1UfQ0KaW1wb3J0IG51bXB5IGFzIG5wDQpmcm9tIHNrbGVhcm4ubGluZWFyX21vZGVsIGltcG9ydCBMaW5lYXJSZWdyZXNzaW9uDQppbXBvcnQgc3RhdHNtb2RlbHMuZm9ybXVsYS5hcGkgYXMgc20NCmBgYA0KYGBge3B5dGhvbiwgZXZhbD1UfQ0KaW1wb3J0IHN0YXRzbW9kZWxzLmZvcm11bGEuYXBpIGFzIHNtDQpyZXN1bHQgPSBzbS5vbHMoZm9ybXVsYT0ieSB+IHRyZWF0ZWQgKyB0aW1lICsgZGlkIiwgZGF0YT1teWRhdGEpLmZpdCgpDQpgYGANCmBgYHtweXRob259DQpwcmludChyZXN1bHQuc3VtbWFyeSgpKQ0KYGBgDQoNCg0KIyBSZWZlcsOqbmNpYXMNCg0KRklHVUVJUkVETywgQWRyaWFubyBNYXJjb3MgUm9kcmlndWVzLiBFY29ub21ldHJpYTogZXhlbXBsbyBkZSBkYWRvcyBlbSBwYWluZWwgZW0gUiAtIFRvcnJlcy1SZXluYSBkYXRhLiBDYW1wbyBHcmFuZGUtTVMsQnJhc2lsOiBSU3R1ZGlvL1JwdWJzLCAyMDE5LiBEaXNwb27DrXZlbCBlbSA8aHR0cDovL3JwdWJzLmNvbS9hbXJvZmkvRWNvbm9tZXRyaWNzX3BhbmVsX3RvcnJlc19yZXluYT4gZSBlbSA8aHR0cHM6Ly9hZHJpYW5vZmlndWVpcmVkby5uZXRsaWZ5LmFwcC9wb3N0L0Vjb25vbWV0cmljc19wYW5lbF90b3JyZXNfcmV5bmEvPi4NCg0KVE9SUkVTLVJFWU5BLCBPc2Nhci4gRGlmZmVyZW5jZXMtaW4tRGlmZmVyZW5jZXMgKHVzaW5nIFIpLiBQcmluY2V0b24sTkooVVNBKTogUHJpbmNldG9uIFVuaXZlcnNpdHksIDIwMTUuIERpc3BvbsOtdmVsIGVtOiA8aHR0cHM6Ly93d3cucHJpbmNldG9uLmVkdS9+b3RvcnJlcy9ESUQxMDFSLnBkZj4uICAgIA0KDQpXQUxESU5HRVIsIEZhYmlhbi4gTGVjdHVyZXMgMyAtIERpZmZlcmVuY2VzLWluLURpZmZlcmVuY2VzLiBNdW5pY2ggKHNkKS4gRGlzcG9uw612ZWwgZW06IDxodHRwczovL3d3dy5mYWJpYW53YWxkaW5nZXIuY29tL2FwcGxpZWQtZWNvbm9tZXRyaWNzPi4gQWNlc3NvOiBNYXJjaCwgMjd0aCwgMjAyMS4gICAgDQoNCldPT0xEUklER0UsIEouIERpZmZlcmVuY2UtaW4tRGlmZmVyZW5jZXMgRXN0aW1hdGlvbi4gTkJFUiwgMjAwNy4gRGlzcG9uw612ZWwgZW06IDxodHRwczovL3d3dy5uYmVyLm9yZy9sZWN0dXJlL2RpZmZlcmVuY2UtZGlmZmVyZW5jZXMtZXN0aW1hdGlvbj4gZSB2aWRlbyBlbTogPGh0dHBzOi8vdmltZW8uY29tLzI3Nzc2NjE0Pi4gQWNlc3NvOiBNYXJjaCwgMjd0aCwgMjAyMS4NCg==