Analyse IHDS ASER data

Summary

This notebook analyses IHDS ASER data to determine whether absent students could affect in-school assessments like the NAS. To do this we run a regression of the ASER score on a variable for the number of days absent from school in the previous month, state dummies, and interactions between the state dummies and the absence variable. We then inspect the coefficients for the interaction terms and test their joint significance. Lastly, we compare estimates of state averages on the full sample with estimates of state averages where we take into account possible missingness due to absence.

Install packages and import data.

library(tidyverse)
library(haven)

ihds_ind_dir <- "C:/Users/dougj/Documents/Data/IHDS/IHDS 2012/DS0001"
ind_file <- file.path(ihds_ind_dir, "36151-0001-Data.dta")
# read in just those variables that i need
# this is much faster than reading in everything and then selecting
df <- read_dta(ind_file, col_select = c(STATEID, PSUID, URBAN2011, HHID, HHSPLITID, PERSONID, IDPSU, WT, RO3, RO7, RO5, starts_with("CS"), starts_with("TA"), starts_with("ED")) )

Inspect data

Inspect the CS13 variable which is self-reported number of days absent from school each month. Only look at observations for which we have ASER score and the child attends a govt or govt aided school.

attributes(df$CS13)

$label
[1] "EQ4 2.13 Days/month absent"

$format.stata
[1] "%2.0f"

# simple tab of CS13 --> note that the max value is 30.
df %>% filter(!is.na(TA8B)) %>%
  filter(CS4 == 2 | CS4 == 3) %>%
  group_by(CS13) %>% 
  count()

# check that this is not missing for kids with ASER results --> looks like there are only a few NAs
df %>% filter(!is.na(TA8B)) %>% summarise(non_na_count = sum(!is.na(CS13)), na_count = sum(is.na(CS13)) )

Graphical analysis of ASER scores by absence

Compare average ASER score for different levels of absence for kids attending govt schools. First, graph average score vs absence. Note that there are relatively few kids with absence over 10 so the area of this graph with absence > 10 should probably be ignored. Second, create histograms of score for each value of absence up to 10.

temp <- df %>% filter(!is.na(TA8B) & !is.na(CS13)) %>% 
  filter(CS4 == 2 | CS4 == 3) %>%
  group_by(CS13) %>% 
  summarise(ASER_score = weighted.mean(TA8B, WT)) %>% 
  ungroup() %>%
  rename(days_absent = CS13)
ggplot(temp, aes(x = days_absent, y = ASER_score)) + geom_line()

temp <- df %>% mutate(days_absent_capped = ifelse(CS13 > 10, 10, CS13)) %>% 
  filter(!is.na(TA8B) & !is.na(days_absent_capped))
# check that this worked
temp %>% group_by(days_absent_capped) %>% count()

# the graph below shows the relative frequency of different ASER scores by # days absent
# note that 10 could be 10 or greater as I have replaced all values of CS13>10 with 10
ggplot(temp, aes(factor(TA8B), group = factor(days_absent_capped))) + 
  geom_bar(aes(y = ..prop.., fill = factor(..x..)), stat="count", fill = "gray") + 
  scale_y_continuous(labels=scales::percent) +
  ylab("relative frequencies") +
  xlab("ASER reading score") +
  facet_grid(~factor(days_absent_capped))

Regression of ASER score on absence and state vars

Regress ASER score on age, absence, state, and state x absence. Then test the joint significance of all the state x absence terms.

# display the label for each of variables
vars <- list(df$CS13, df$RO5)
lapply(vars, FUN = function(x) attributes(x)$label)

[[1]]
[1] "EQ4 2.13 Days/month absent"

[[2]]
[1] "HQ4 2.5 Age"

model <- lm(TA8B ~ CS13 + factor(STATEID) + CS13*factor(STATEID) + RO5, data = df)
library(car)

Loading required package: carData
Registered S3 method overwritten by 'data.table':
  method           from
  print.data.table     

Attaching package: 㤼㸱car㤼㸲

The following object is masked from 㤼㸱package:dplyr㤼㸲:

    recode

The following object is masked from 㤼㸱package:purrr㤼㸲:

    some

linearHypothesis(model, matchCoefs(model, "CS13:"), white.adjust = "hc1")

Linear hypothesis test

Hypothesis:
CS13:factor(STATEID)2 = 0
CS13:factor(STATEID)3 = 0
CS13:factor(STATEID)4 = 0
CS13:factor(STATEID)5 = 0
CS13:factor(STATEID)6 = 0
CS13:factor(STATEID)7 = 0
CS13:factor(STATEID)8 = 0
CS13:factor(STATEID)9 = 0
CS13:factor(STATEID)10 = 0
CS13:factor(STATEID)11 = 0
CS13:factor(STATEID)12 = 0
CS13:factor(STATEID)13 = 0
CS13:factor(STATEID)14 = 0
CS13:factor(STATEID)15 = 0
CS13:factor(STATEID)16 = 0
CS13:factor(STATEID)17 = 0
CS13:factor(STATEID)18 = 0
CS13:factor(STATEID)19 = 0
CS13:factor(STATEID)20 = 0
CS13:factor(STATEID)21 = 0
CS13:factor(STATEID)22 = 0
CS13:factor(STATEID)23 = 0
CS13:factor(STATEID)24 = 0
CS13:factor(STATEID)25 = 0
CS13:factor(STATEID)26 = 0
CS13:factor(STATEID)27 = 0
CS13:factor(STATEID)28 = 0
CS13:factor(STATEID)29 = 0
CS13:factor(STATEID)30 = 0
CS13:factor(STATEID)32 = 0
CS13:factor(STATEID)33 = 0

Model 1: restricted model
Model 2: TA8B ~ CS13 + factor(STATEID) + CS13 * factor(STATEID) + RO5

Note: Coefficient covariance matrix supplied.

  Res.Df Df      F    Pr(>F)    
1  11129                        
2  11098 31 4.8327 < 2.2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

library(broom)
coefs <- tidy(model)
coefs

Compare state estimates assuming some students more likely to be absent

We can also more directly calculate the effect of absence on state scores by comparing estimates of state averages with estimates which take into account potential absence on the day of the assessment. To do this, we create a new weight variable which takes into account the probability that the student would be present on the day of the exam. Our new WT variable is…

\[ newweight = oldweight*\frac{30-daysabsentpermonth}{30} \] As in previous analyses, we only look at students in govt or private aided schools.

state_averages <- df %>% filter(!is.na(TA8B) & !is.na(CS13)) %>% 
  filter(CS4 == 2 | CS4 == 3) %>%
  mutate(new.weight = WT*(30-CS13)/30) %>%
  group_by(STATEID) %>% 
  summarise(full_sample = weighted.mean(TA8B, WT), absence_weighted = weighted.mean(TA8B, new.weight), 
            diff = full_sample-absence_weighted)

state_averages$full_rank[order(state_averages$full_sample)] <- 1:nrow(state_averages)

Unknown or uninitialised column: 'full_rank'.

state_averages$absence_rank[order(state_averages$absence_weighted)] <- 1:nrow(state_averages)

Unknown or uninitialised column: 'absence_rank'.

state_averages

NA