Data Preparation

Introduction

Does an increase in teacher salary improve student performance? If this is the case, it would indicate a viable intervention for addressing areas of need with respect to student performance.

In this analysis, we consider average SAT math score as a proxy for student performance and look at a dataset detailing characteristics of Massachusetts public schools in attempt to understand the effect of teacher salary on this metric, when controlling for other factors that were felt to be possible confounders: average class size, overall average expenditure per student, percent of economically disadvantaged students, and percent of high needs students.

Data

Limiting our Massachusetts public school data set to those who reported enrolling grade 12 students (the cohort who would be considering a transition to a secondary institution), we have a total of 394 schools.

62 of these schools lacked data on average SAT math scores. Average teacher salaries were not seen to be significantly different for this group compared to the entire cohort. Additionally, 32 rows did not have data on teacher salary. These 94 rows were removed from the data set. Finally, an additional 8 schools were missing covariate data, and were removed from the analysis on the remaining 292 public schools.

The data for this analysis was sourced from Department of Education reports (linked via Kaggle.com). In particular, the reports of interest were outlined in the associated data dictionary:

On this data, the Massachusetts DOE site (http://profiles.doe.mass.edu/help/data.aspx) notes,“Schools and Districts view, add, update and delete their own directory information to ensure that the information is as up-to-date and accurate as possible.” The current analysis represents an observational study as it utilizes this retrospectively collected data.

Given that this data reflects only responses from Massachusetts public schools, its results may not be generalizable to private schools or dissimilar geographical/socio-economic regions. As the results may shed light on public schools with similar demographic populations, we take this data set as a sample of public schools in the United States whose demographics and socio-economic characteristics are similar to those observed in Massachusetts.

We will be considering 8 fields (all numerical). The response variable, average SAT math scores, and 5 explanatory variables: average teacher salary, percent of high needs students, percent of economically disadvantaged students, and average expenditures per pupil.

Exploratory data analysis

Average SAT Math Scores

The average SAT Math score is 508.95, with a standard deviation of 16.18%. The data are approximately normally distributed, though slightly left skewed.

Average teacher salary

Minimum average teacher salary of $74,505, with a standard deviation of $8,441. This data is approximately normally distributed, with one notable outlier. In our analysis, we will compare the effect of removing these outlier salaries.

Plot of average teacher salary vs. average SAT Math scores

We can see a slightly positive associated between salary and SAT Math scores above. From this descriptive result, we expect that salary might be related to performance.

It seems to be average teacher salary increases with overall average expenditure per student, especially as salary increases, but there are differences we can observe between the two.

The idea for including this field was that if we could control for average expenditure per pupil, we can get a better sense of how teacher salary affects SAT Math results when controlling for variation in total amount spent. That is, some schools may have more money to spend, but when controlling for that factor, we are interested in how the difference in pay to teachers affects the outcome variable.

Histograms for the remaining two covariates are as follows. Both are approximately normally distributed, but with boundaries at 0 and 100, exhibit a pronounced right-skewness.

Inference

After using backwards model selection, we are left with three explanatory variables for average SAT Math score: average teacher salary, percent of economically distantaged students, percent of high needs students. These fields explain about 75% of the variability seen in SAT Math scores.

## 
## Call:
## lm(formula = df$avgSATMath ~ df$salary + df$econDisadv + df$highNeeds)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -61.886 -20.721  -2.316  19.252  86.559 
## 
## Coefficients:
##                 Estimate Std. Error t value Pr(>|t|)    
## (Intercept)    5.482e+02  1.540e+01  35.589  < 2e-16 ***
## df$salary      7.253e-04  2.187e-04   3.316  0.00103 ** 
## df$econDisadv -7.287e-01  3.957e-01  -1.842  0.06657 .  
## df$highNeeds  -1.957e+00  3.655e-01  -5.353 1.77e-07 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 29.31 on 288 degrees of freedom
## Multiple R-squared:  0.7501, Adjusted R-squared:  0.7475 
## F-statistic: 288.1 on 3 and 288 DF,  p-value: < 2.2e-16

We saw that an increase in average teacher salary was significantly and positively associated with average SAT math scores, both in the full and reduced models.

We noted that removing the outlier salary cases slightly increased the positive relationship between average teacher salary and average SAT Math score. To be conservative in our analysis, we did not remove these outlier cases in the present analysis.

Conclusion

When controlling for the relevant variables in our reduced model, we see that an increase in teacher salary of $10,000 dollars was associated with a statistically significant increase (p = 0.001) in the average SAT math scores by 7 points (with a confidence interval of about 3 to 11 points).

Given the limited effect size for the intervention, it is by no means conclusive that the investment would be worth the return.

It is possible that SAT Math score is not the most representative proxy of student performance that can be impacted by a teacher’s intervention. For example, perhaps college attendance rates would be a more holistic metric. The present data had limitations in what analysis could performed given this field’s distribution, so average SAT Math score was selected instead. Another study could investigate additional metrics of performance.

A further study could also randomize an intervention of increased salary for an experimental group of teachers against a control group of teachers to investigate a causal relationship between teacher pay and student performance.

Additionally, a later study might expand to difference geographic regions within the US to expand on the generalizability of the results.

Finally, a significant question that is left is how distribution of teacher salaries affects individual performance. Are merit bonuses more or less effective than a general pay increase? The current study does not shed light on these more detailed concerns.