setwd("~/Downloads/sociology lab")
library(ggplot2)
library(dplyr)##
## Attaching package: 'dplyr'## The following objects are masked from 'package:stats':
##
## filter, lag## The following objects are masked from 'package:base':
##
## intersect, setdiff, setequal, unionlibrary(knitr)
library(tidyverse)## ── Attaching packages ─────────────────────────────────────── tidyverse 1.3.1 ──## ✓ tibble 3.1.6 ✓ purrr 0.3.4
## ✓ tidyr 1.1.4 ✓ stringr 1.4.0
## ✓ readr 2.1.1 ✓ forcats 0.5.1## ── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
## x dplyr::filter() masks stats::filter()
## x dplyr::lag() masks stats::lag()library(haven)
library(forcats)
library(sjPlot)
d <- read_dta("mto_sci_puf_pseudo_20130206 (1).dta")
d <- d %>% select(ra_group, ra_site,
ps_rad_c9010t_perpov_yr1, ps_rad_c9010t_perpov_yr5) %>%
mutate(ra_group = as_factor(ra_group),
ra_site = as_factor(ra_site))
d2 <- rename(d, group = ra_group, city = ra_site, year_one_poverty = ps_rad_c9010t_perpov_yr1,
year_five_poverty = ps_rad_c9010t_perpov_yr5)
d3 <- d2 %>%
select(group, city, year_one_poverty, year_five_poverty) %>%
mutate(group = case_when(group == '1=Low-Poverty Voucher/Experimental' ~ 'treatment',
str_detect(group, '2=Traditional Voucher/Section 8') ~ 'section_8',
str_detect(group, '3=Control') ~ 'control')) %>%
mutate(city = case_when(city == '2=Boston' ~ 'Boston',
str_detect(city, '1=Baltimore') ~ 'Baltimore',
str_detect(city, '3=Chicago') ~ 'Chicago',
str_detect(city, '4=Los Angeles') ~ 'Los Angeles',
str_detect(city, '5=New York') ~ 'New York'))
d4 <- d3 %>%
select(group, city, year_one_poverty, year_five_poverty) %>%
mutate(group = fct_relevel(group, c('control')))
year_one_pov_groups.lm<-lm(year_one_poverty ~ group, data = d4)
summary(year_one_pov_groups.lm)##
## Call:
## lm(formula = year_one_poverty ~ group, data = d4)
##
## Residuals:
## Min 1Q Median 3Q Max
## -1.06003 -0.11589 -0.01697 0.10935 1.15126
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 0.489860 0.005833 83.98 <2e-16 ***
## groupsection_8 -0.128731 0.009527 -13.51 <2e-16 ***
## grouptreatment -0.156074 0.007789 -20.04 <2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.1953 on 3221 degrees of freedom
## (49 observations deleted due to missingness)
## Multiple R-squared: 0.1166, Adjusted R-squared: 0.1161
## F-statistic: 212.6 on 2 and 3221 DF, p-value: < 2.2e-16year_one_poverty_groups_and_city.lm<-lm(year_one_poverty ~ group + city, data = d4)
summary(year_one_poverty_groups_and_city.lm)##
## Call:
## lm(formula = year_one_poverty ~ group + city, data = d4)
##
## Residuals:
## Min 1Q Median 3Q Max
## -1.19797 -0.07784 -0.00811 0.09908 1.01333
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 0.466837 0.009477 49.262 < 2e-16 ***
## groupsection_8 -0.129216 0.008707 -14.840 < 2e-16 ***
## grouptreatment -0.174041 0.007155 -24.324 < 2e-16 ***
## cityBoston -0.063718 0.010883 -5.855 5.25e-09 ***
## cityChicago 0.178926 0.010946 16.346 < 2e-16 ***
## cityLos Angeles 0.020211 0.010749 1.880 0.0602 .
## cityNew York 0.014269 0.010712 1.332 0.1829
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.178 on 3217 degrees of freedom
## (49 observations deleted due to missingness)
## Multiple R-squared: 0.2671, Adjusted R-squared: 0.2657
## F-statistic: 195.4 on 6 and 3217 DF, p-value: < 2.2e-16year_one_poverty_groups_city_interaction.lm<-lm(year_one_poverty ~ group + city + group * city, data = d4)
summary(year_one_poverty_groups_city_interaction.lm)##
## Call:
## lm(formula = year_one_poverty ~ group + city + group * city,
## data = d4)
##
## Residuals:
## Min 1Q Median 3Q Max
## -1.20799 -0.08706 0.00130 0.08187 1.00331
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 0.474134 0.014227 33.325 < 2e-16 ***
## groupsection_8 -0.153354 0.023156 -6.623 4.12e-11 ***
## grouptreatment -0.179033 0.019039 -9.403 < 2e-16 ***
## cityBoston -0.094586 0.018067 -5.235 1.75e-07 ***
## cityChicago 0.168672 0.019360 8.712 < 2e-16 ***
## cityLos Angeles 0.045536 0.017512 2.600 0.00936 **
## cityNew York -0.012371 0.018407 -0.672 0.50158
## groupsection_8:cityBoston 0.054992 0.029848 1.842 0.06551 .
## grouptreatment:cityBoston 0.046197 0.024449 1.890 0.05891 .
## groupsection_8:cityChicago -0.003365 0.031739 -0.106 0.91557
## grouptreatment:cityChicago 0.017964 0.024880 0.722 0.47032
## groupsection_8:cityLos Angeles -0.017594 0.029246 -0.602 0.54749
## grouptreatment:cityLos Angeles -0.058957 0.024135 -2.443 0.01463 *
## groupsection_8:cityNew York 0.070290 0.028929 2.430 0.01516 *
## grouptreatment:cityNew York 0.022161 0.024504 0.904 0.36585
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.1771 on 3209 degrees of freedom
## (49 observations deleted due to missingness)
## Multiple R-squared: 0.276, Adjusted R-squared: 0.2728
## F-statistic: 87.36 on 14 and 3209 DF, p-value: < 2.2e-16year_five_pov_groups.lm<-lm(year_five_poverty ~ group, data = d4)
summary(year_five_pov_groups.lm)##
## Call:
## lm(formula = year_five_poverty ~ group, data = d4)
##
## Residuals:
## Min 1Q Median 3Q Max
## -0.80640 -0.07227 -0.00469 0.07445 0.96079
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 0.398443 0.004938 80.681 <2e-16 ***
## groupsection_8 -0.068661 0.008093 -8.484 <2e-16 ***
## grouptreatment -0.098501 0.006603 -14.917 <2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.1653 on 3206 degrees of freedom
## (64 observations deleted due to missingness)
## Multiple R-squared: 0.06587, Adjusted R-squared: 0.06529
## F-statistic: 113 on 2 and 3206 DF, p-value: < 2.2e-16year_five_poverty_groups_and_city.lm<-lm(year_five_poverty ~ group + city, data = d4)
summary(year_five_poverty_groups_and_city.lm)##
## Call:
## lm(formula = year_five_poverty ~ group + city, data = d4)
##
## Residuals:
## Min 1Q Median 3Q Max
## -0.86227 -0.06374 -0.00538 0.06062 0.90492
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 0.367014 0.008531 43.022 < 2e-16 ***
## groupsection_8 -0.070443 0.007858 -8.965 < 2e-16 ***
## grouptreatment -0.104406 0.006445 -16.199 < 2e-16 ***
## cityBoston -0.031242 0.009805 -3.186 0.00146 **
## cityChicago 0.087298 0.009886 8.830 < 2e-16 ***
## cityLos Angeles 0.051909 0.009662 5.372 8.32e-08 ***
## cityNew York 0.051147 0.009648 5.301 1.23e-07 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.1601 on 3202 degrees of freedom
## (64 observations deleted due to missingness)
## Multiple R-squared: 0.1257, Adjusted R-squared: 0.1241
## F-statistic: 76.74 on 6 and 3202 DF, p-value: < 2.2e-16year_five_poverty_groups_city_interaction.lm<-lm(year_five_poverty ~ group + city + group * city, data = d4)
summary(year_five_poverty_groups_city_interaction.lm)##
## Call:
## lm(formula = year_five_poverty ~ group + city + group * city,
## data = d4)
##
## Residuals:
## Min 1Q Median 3Q Max
## -0.84003 -0.05933 -0.00203 0.05169 0.92716
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 0.353710 0.012823 27.584 < 2e-16 ***
## groupsection_8 -0.053559 0.020940 -2.558 0.010580 *
## grouptreatment -0.082281 0.017160 -4.795 1.70e-06 ***
## cityBoston -0.025500 0.016284 -1.566 0.117449
## cityChicago 0.078356 0.017539 4.468 8.18e-06 ***
## cityLos Angeles 0.092600 0.015730 5.887 4.35e-09 ***
## cityNew York 0.062376 0.016619 3.753 0.000178 ***
## groupsection_8:cityBoston -0.006942 0.027007 -0.257 0.797175
## grouptreatment:cityBoston -0.008869 0.022050 -0.402 0.687552
## groupsection_8:cityChicago 0.019905 0.028753 0.692 0.488804
## grouptreatment:cityChicago 0.006807 0.022523 0.302 0.762488
## groupsection_8:cityLos Angeles -0.072136 0.026406 -2.732 0.006333 **
## grouptreatment:cityLos Angeles -0.065629 0.021714 -3.022 0.002528 **
## groupsection_8:cityNew York -0.004887 0.026174 -0.187 0.851906
## grouptreatment:cityNew York -0.024576 0.022101 -1.112 0.266234
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.1596 on 3194 degrees of freedom
## (64 observations deleted due to missingness)
## Multiple R-squared: 0.1325, Adjusted R-squared: 0.1287
## F-statistic: 34.84 on 14 and 3194 DF, p-value: < 2.2e-16tab_model(year_one_pov_groups.lm, year_one_poverty_groups_and_city.lm, year_one_poverty_groups_city_interaction.lm)|
pseudo obs rad_c9010t_perpov_yr1 - Tract poverty 1 yr post-randomization |
pseudo obs rad_c9010t_perpov_yr1 - Tract poverty 1 yr post-randomization |
pseudo obs rad_c9010t_perpov_yr1 - Tract poverty 1 yr post-randomization |
|||||||
|---|---|---|---|---|---|---|---|---|---|
| Predictors | Estimates | CI | p | Estimates | CI | p | Estimates | CI | p |
| (Intercept) | 0.49 | 0.48 – 0.50 | <0.001 | 0.47 | 0.45 – 0.49 | <0.001 | 0.47 | 0.45 – 0.50 | <0.001 |
| group [section 8] | -0.13 | -0.15 – -0.11 | <0.001 | -0.13 | -0.15 – -0.11 | <0.001 | -0.15 | -0.20 – -0.11 | <0.001 |
| group [treatment] | -0.16 | -0.17 – -0.14 | <0.001 | -0.17 | -0.19 – -0.16 | <0.001 | -0.18 | -0.22 – -0.14 | <0.001 |
| city [Boston] | -0.06 | -0.09 – -0.04 | <0.001 | -0.09 | -0.13 – -0.06 | <0.001 | |||
| city [Chicago] | 0.18 | 0.16 – 0.20 | <0.001 | 0.17 | 0.13 – 0.21 | <0.001 | |||
| city [Los Angeles] | 0.02 | -0.00 – 0.04 | 0.060 | 0.05 | 0.01 – 0.08 | 0.009 | |||
| city [New York] | 0.01 | -0.01 – 0.04 | 0.183 | -0.01 | -0.05 – 0.02 | 0.502 | |||
|
group [section 8] * city [Boston] |
0.05 | -0.00 – 0.11 | 0.066 | ||||||
|
group [treatment] * city [Boston] |
0.05 | -0.00 – 0.09 | 0.059 | ||||||
|
group [section 8] * city [Chicago] |
-0.00 | -0.07 – 0.06 | 0.916 | ||||||
|
group [treatment] * city [Chicago] |
0.02 | -0.03 – 0.07 | 0.470 | ||||||
|
group [section 8] * city [Los Angeles] |
-0.02 | -0.07 – 0.04 | 0.547 | ||||||
|
group [treatment] * city [Los Angeles] |
-0.06 | -0.11 – -0.01 | 0.015 | ||||||
|
group [section 8] * city [New York] |
0.07 | 0.01 – 0.13 | 0.015 | ||||||
|
group [treatment] * city [New York] |
0.02 | -0.03 – 0.07 | 0.366 | ||||||
| Observations | 3224 | 3224 | 3224 | ||||||
| R2 / R2 adjusted | 0.117 / 0.116 | 0.267 / 0.266 | 0.276 / 0.273 | ||||||
tab_model(year_five_pov_groups.lm, year_five_poverty_groups_and_city.lm, year_five_poverty_groups_city_interaction.lm)|
pseudo obs rad_c9010t_perpov_yr5 - Tract poverty 5 yrs post-randomization |
pseudo obs rad_c9010t_perpov_yr5 - Tract poverty 5 yrs post-randomization |
pseudo obs rad_c9010t_perpov_yr5 - Tract poverty 5 yrs post-randomization |
|||||||
|---|---|---|---|---|---|---|---|---|---|
| Predictors | Estimates | CI | p | Estimates | CI | p | Estimates | CI | p |
| (Intercept) | 0.40 | 0.39 – 0.41 | <0.001 | 0.37 | 0.35 – 0.38 | <0.001 | 0.35 | 0.33 – 0.38 | <0.001 |
| group [section 8] | -0.07 | -0.08 – -0.05 | <0.001 | -0.07 | -0.09 – -0.06 | <0.001 | -0.05 | -0.09 – -0.01 | 0.011 |
| group [treatment] | -0.10 | -0.11 – -0.09 | <0.001 | -0.10 | -0.12 – -0.09 | <0.001 | -0.08 | -0.12 – -0.05 | <0.001 |
| city [Boston] | -0.03 | -0.05 – -0.01 | 0.001 | -0.03 | -0.06 – 0.01 | 0.117 | |||
| city [Chicago] | 0.09 | 0.07 – 0.11 | <0.001 | 0.08 | 0.04 – 0.11 | <0.001 | |||
| city [Los Angeles] | 0.05 | 0.03 – 0.07 | <0.001 | 0.09 | 0.06 – 0.12 | <0.001 | |||
| city [New York] | 0.05 | 0.03 – 0.07 | <0.001 | 0.06 | 0.03 – 0.09 | <0.001 | |||
|
group [section 8] * city [Boston] |
-0.01 | -0.06 – 0.05 | 0.797 | ||||||
|
group [treatment] * city [Boston] |
-0.01 | -0.05 – 0.03 | 0.688 | ||||||
|
group [section 8] * city [Chicago] |
0.02 | -0.04 – 0.08 | 0.489 | ||||||
|
group [treatment] * city [Chicago] |
0.01 | -0.04 – 0.05 | 0.762 | ||||||
|
group [section 8] * city [Los Angeles] |
-0.07 | -0.12 – -0.02 | 0.006 | ||||||
|
group [treatment] * city [Los Angeles] |
-0.07 | -0.11 – -0.02 | 0.003 | ||||||
|
group [section 8] * city [New York] |
-0.00 | -0.06 – 0.05 | 0.852 | ||||||
|
group [treatment] * city [New York] |
-0.02 | -0.07 – 0.02 | 0.266 | ||||||
| Observations | 3209 | 3209 | 3209 | ||||||
| R2 / R2 adjusted | 0.066 / 0.065 | 0.126 / 0.124 | 0.132 / 0.129 | ||||||
QUESTIONS
• What do you notice about the difference in the intercepts of models 1 and 4? What could this indicate? Between models 1 and 4, the estimate of the intercept decreases from .49 to .40. Because we set the control group as the base level, this intercept reflects poverty levels for those who did not have access to the section 8 or experimental treatments. Model one might have a higher intercept, therefore, because of declining poverty rates within the control group from year one to year five. Perhaps the economy is stronger than it was a few years before, lowering the rates of poverty throughout the country, or there were other outside variables responsible for lowering poverty levels that were not accounted for in this study. Furthermore, it could be that over time people’s wealth tends to increase as they accumulate income and gain more work experience. More study is needed to determine exactly what could be responsible for decreasing poverty levels from year one to year five, but this is suggests that there is a decrease in poverty levels.
• Is there evidence here that, one year out, the neighborhood poverty rates of the treatment and section 8 groups are lower than those of the control group? If so, what are your best estimates of the magnitude of the relevant differences? Yes, the estimates in our regression tables provide support for the idea that poverty rates are lower for participants in the section 8 and experimental treatment groups. The best estimates of the magnitude of the relevant differences is one which accounts for the variables we believe could impact the results, so the estimates from model 3, which accounts for treatment as well as the potential confounder of city and the potential interaction between group and city. In model three, the estimated intercept, which represents the control group, was .47 while the section 8 estimate was .32 and the experimental estimate was .29. The small p values provide strong evidence that these values are statistically significant.
• Is there evidence here that, five years out, the neighborhood poverty rates of the treatment and section 8 groups are lower than those of the control group? If so, what are the best estimates of the magnitude of the relevant differences? Just as with the one year measurements, the five year measurements provided compelling, statistically significant (based on p-values) evidence that poverty rates are lower within the section 8 and experimental treatment groups compared to the control group. Once again, the best estimates would probably come from model 6, because it takes into account the city variable and the possible effects of the interaction between city and treatment. The estimated intercept of the model, a.k.a. the control group, was .35, while the section 8 estimate was .30 and the experimental treatment estimate was .27.
• What do you notice when comparing your answers to questions 2 and 3? In comparing the results from the one year and five year model, I noticed that, as stated in question one, overall poverty levels seemed to decrease between years one and five for the control group. Also noteworthy is that the size of the gaps between the control and the treatment groups shrunk for both the section 8 and the experimental group. In the section 8 treatment group, the difference between the estimated value and the control shrunk from -.15 to -.05, though overall the estimate in year five (.30) indicated a slightly lower poverty rate than the year one estimate for section 8 (.32). For the experimental group, the difference between the experimental group and the control shrank from -.18 to -.08, even as overall poverty rates in the experimental group decreased slightly, from .29 to .27. This could indicate that the effects of treatment are stronger in the short run, which would typically be considered a negative result as most poverty programs target long-term success.
• Is there any evidence here of “city effects” on poverty? Which coefficients from which models are relevant to this question? We can see evidence of “city effects” on poverty when looking at models two, three, five, and six, which account for the city variable. This is evident by looking at the changes in the coefficients of the control, section 8, and experimental groups when the city variable as well as the interaction between city and treatment group are accounted for in our model. Generally, we find that after account for the city variable, the estimate of the coefficients shrinks, implying city in some way influences the rate of poverty and possibly how effective the poverty treatments are on the population in each city.
• Do you think there are any differences in how we should interpret the coefficients on the treatment groups in models 2 and 5, as compared to those in models 1 and 4? Why or why not? Yes, when interpreting the coefficients in models 1 and 4, we need to be aware that the city variable, a possible confounder, is not taken into account. When looking at the coefficients in models 2 and 5, we need to keep in mind that the model is not looking solely at the correlation between treatment group and poverty levels, but also at the impact city can have on poverty levels. As a result of looking at this extra variable, we might expect to see a changes in the coefficients of the treatment groups.
• Is there any evidence in these models that the magnitude of any effect of treatment assignment on poverty rates depends on the city in which someone lives? Explain your answer, describing the implications of the coefficients and models that are relevant to this question. In the interaction term between treatment group and state, represented in models 3 and 6, we see that there is often some slight relationship between the two variables. The coefficients range from -.07 to .02 in model 6, and from -.06 to .07 in model 3. It is important to note that many of these values would not be considered statistically significant, with some high p values and confidence intervals that range from negative to positive intervals. Given these circumstances, it is hard to draw very strong conclusions about the effect of the interaction term, but we do have some evidence of variance in coefficients between different cities and treatment groups, suggesting the magnitude of the effect of treatment on poverty rates may be influenced by their city of origin, though further research is needed.
• Is there anything else interesting that you note about your regression results? Something I found interesting in these results was how incredibly small many of the p values were, especially compared to what I’ve seen in previous statistics classes. Many of values had p values of <.001 or very similar, with the exception of several of the interaction terms which higher p values. This implies that the variables we were looking at really do have some correlation with poverty levels, and therefore this data is useful in the real world. Being statistically significant at such high levels is a good sign that the variables we have isolated are important ones to look at when considering poverty levels.