1. Introduction

In this project I will be using the Texas High School AP and SAT data to analyze the potential relationship between AP Pass Rates for Economically Disadvantaged Students and the school’s Total Expenditures per Student. Thus, AP Pass Rates for Economically Disadvantaged Students will be my criterion variable, and my predictor variable will be the Total Expenditures per Student. For my category separation variable, I will use Total School Enrollment divided into 3 subgroups - Small, Medium and Large. Each subgroup possesses one-third of the sample size in their group, with the cutoffs between groups being 684 and 1955 students respectively.

The criterion variable of AP Pass Rates for Economically Disadvantaged Students is economically significant because economically disadvantaged students typically have lower scores on standardized tests due to the extra barriers they face in education. However, one would expect a positive correlation between AP Pass Rates for Economically Disadvantaged Students and Total School Expenditures per Student if our funding is being allocated wisely.

This is also why I created subgroups by School Size - I wanted to examine whether the potential correlation between an increase in funding and an increase in AP test performance for economically disadvantaged students holds for all schools or depends upon the size of the school as well. The basis for expecting differences in the results among each subgroup is that larger schools might overlook funding their underprivileged students in favor of other programs that might have a larger scope (in terms of impacting the higher class students). However, I believe that smaller schools would be more invested in each individual student, and thus push their total spending per student into programs that specifically target their economically disadvantaged population, resulting in a stronger correlation between Total Expenditures per Student and AP pass rates for smaller schools’ economically disadvantaged population. Thus, the null hypothesis is that AP Pass Rates for Economically Disadvantaged Students have no correlation with Total Expenditure per Student, and there are no statistically significant differences as the Total Enrollment subgroup changes. The alternative hypothesis is that if Total Expenditure per Student increases then the AP Pass Rate for Economically Disadvantaged Students increases, and there are statistically significant differences as we change Total Enrollment.

2. Literature Review

When we examine the literature on this topic, experts typically agree that an increase in spending correlates with an increase in educational attainment, and that this benefit should extend to economically disadvantaged students. For example, Daarel Burnette II from Education Week found in 2020 that when “politicians and taxpayers invested more money in teacher salaries, school construction, and schools with high populations of low-income students… students’ test scores [jumped].” This would support the alternative hypothesis written earlier in this proposal. C. Kirabo Jackson reaches a similar conclusion in his 2020 paper “Does School Spending Matter? the New Literature on an Old Question”, writing that “the recent quasi-experimental literature that relates school spending to student outcomes overwhelmingly support a causal relationship between increased school spending and student outcomes.” With this research in mind, it is expected that we should find a positive correlation between school’s Total Expenditure per Student and their AP Pass Rates for Economically Disadvantaged Students, and likely reject the stated null hypothesis.

3. Basic Descriptive Statistics & Frequency Distributions

4. Single Sample Confidence Intervals and Hypothesis Tests

5. Two Sample Confidence Intervals and Hypothesis Tests

6. Single Factor Analysis of Variance (ANOVA) Tests for Joint Equality of Means

7. Correlation Analysis

8. Conclusion

The bulk of our findings lie in Section #7, where we analyzed correlations between our predictor and criterion variable, even separating among subgroups and comparing these subgroups individually. Our findings seem quite troubling if they hold true.

Overall, we found no statistically significant correlation between Total Expenditures Per Student and AP Pass Rates for Economically Disadvantaged Students. We had the same findings for the “Small” subgroup. This would imply that an increase in per pupil spending has no impact on economically disadvantaged students’ AP test performance. However, when we look at our other subgroups, the findings become more troublesome.

When we tested the “Medium” and “Large” subgroups, we found a statistically significant negative correlation between Total Expenditures Per Student and AP Pass Rates for Economically Disadvantaged students. This goes directly against the original hypothesis posited in Section #1 of this paper, and against the previous literature on this topic. However, the deviance of the “Medium” and “Large” subgroups would agree with one part of our original hypothesis - that smaller schools focus their funding to help their economically disadvantaged students better than larger schools do. However, this finding is not much help considering that small schools were still found to have no benefit to their economically disadvantaged students when they increase spending.

9. Extension 1 - Imputing Missing Data

When analyzing our dataset, we had to cut out every observation that did not have a recorded value for any of “Total Enrollment”, “Pass_Rate_EconDis” or “Total Exp per Student”. While we were still left with many data points to analyze for this project, this is still not ideal. In the best case scenario, we are given a completed dataset that we do not have to delete entries from - but this is unrealistic. Thus, to get a (reasonably accurate) completed dataset, we have to impute for the missing values. We can do this in multiple ways - we can take the mean/median/mode of the dataset and plug it in for the missing values (not very accurate), or we can use a Machine Learning Model to predict these missing values based on where the other stats of the school lie. We can use the “caret” (Classification And REgression Training) package in R to make a simple model to impute these missing values, using Bagged Decision Trees.

There are some steps to do before jumping straight into the model, however. We must first choose which features we want our model to take in when attempting to predict our 3 variables. We obviously can cut out all the basic information for each High School, such as “Campus,” “CampName,” “District,” “DistName,” “County,” “CntyName,” “Region,” and “RegnName.” We also have to cut out all the different subgroups of spending, such as Instructional, Leadership, Guidance, and Extracurricular spending per student, as all of those combined are equivalent to “Total Exp per Student,” which means that if Total Expenditure Per Student is missing, these values must be missing as well. It is also important to note that we DO include the variables we are testing for as features in our model.

This leaves us with the features “Campus_Type”, “Part_Rate_All”, “Part_Rate_Female”, “Pass_Rate_Female”, “Part_Rate_Male”, “Pass_Rate_Male”, “Part_Rate_EconDis”, “SAT_All”, “SAT_EconDis”, “SAT_Female”, “SAT_Male”, “EconDis Enr”, “Pass_Rate_EconDis”, “TOTAL ENROLLMENT”, and “Total Exp per Student” that will be used to predict “Pass_Rate_EconDis”, “TOTAL ENROLLMENT”, and “Total Exp per Student”.

Next, we need to add new columns that track whether or not the value of our variables was missing in the original dataset. As such, we tack on 3 new binary columns to our dataset, which can read “Y” (Yes, was missing), or “N” (No, was not missing): “MissingTotalEnrollment”, “MissingPassRateEconDis”, and “MissingTotalExpPerStudent”. This is so that looking back on the dataset we don’t lose any information and can easily tell what was imputed and what was originally given. An example of what our dataset’s new columns look like is show below. Note that when there is an “NA” in our dataframe for “Pass_Rate_EconDis”, our “MissingPassRateEconDis” variable takes on a value of “Y” for yes. This happens throughout the dataset for “Total Exp Per Student” and “Total Enrollment” as well.

Once we have done all of this, we can run the model and replace our original dataset’s missing values with our imputed values, and compare their basic descriptive statistics.

10. Extension 2 - Creating & Training a Machine Learning Regression Model

Though we attempted to examine a relationship between Total Expenditures per Student and AP Pass Rates for Economically Disadvantaged students, we can take this one step further. We can now ask the question: how is “AP_Pass_Rate_EconDis” affected by every other variable we are given data for? We do this by creating an algorithm. First, we split the data into two sets: our Training set and our Testing set. Basically, we ask our Model to analyze a proportion of our data (the Training set) and determine the relationship between all of our factors (ex: SAT Scores, Instructional Expenditurem Per Student, etc.) and the AP Pass Rate for Economically Disadvantaged Students. Then, we use this Model on the remaining portion of our data (the Testing set) to evaluate how effective our created Model is when examining “new” data. The split between Train & Test used in this paper will be 70/30.

Below is the code used to create our Machine Learning model. I have set this code to not run, and to just display, with interspersed comments to explain exactly what is happening within each line.

# Creating our 70/30 Train/Test split of our data with createDataPartition()
indexes <- createDataPartition(hs$Pass_Rate_EconDis,
                               times = 1,
                               p = .7,
                               list = F
                               )

# Train gets the chosen training rows (70%), Test gets everything else (30%)
hs.train <- hs[indexes, ]
hs.test <- hs[-indexes, ]

# Training Our Model ####
# We need to set precise parameters for our trainControl(), and tuning grid which we will later
# pass to our final training function

# We tell our model to run with the "repeatedcv" method. We divide our training data set into
# 10 parts and then train each part based off of the other 9. We do this 3 times, and find the
# average error across all attempts. We then conduct a grid search, which means we exhaustively
# look through every single tested value to find the best one.
train.control <- trainControl(method = "repeatedcv",
                              number = 10,
                              repeats = 3,
                              search = "grid"
                              )
# Now we construct our tune grid. This consists of multiple hyperparameters which we have to
# tune to get our most efficient model. Hence, we have it try out multiple different 
# combinations (it will try eta = .025, nrounds = 75, etc. and then do the exact same thing
# except eta = .05).
tune.grid <- expand.grid(eta = c(0.025, .05),
                         nrounds = c(75, 100, 125),
                         max_depth = c(6, 7, 8),
                         min_child_weight = c(1.5, 2, 2.5),
                         colsample_bytree = c(0.6, 0.7, 0.8),
                         gamma = 0,
                         subsample = 1
                         )

# In broad terms, this opens up 3 sessions of RStudio to all run through our train() function
# at the same time to speed it up, because this will take a long time to run.
c1 <- makeCluster(3, type = "SOCK")
registerDoSNOW(c1)

# Training our model. We want to predict "Pass_Rate_EconDis" as a function of every other given
# variable. We pass it our tune grid, train.control, and we tell it to give us the model with
# the highest R squared value.
caret.cv <- train(Pass_Rate_EconDis ~ .,
                  data = hs,
                  method = "xgbTree",
                  metric = "Rsquared",
                  tuneGrid = tune.grid,
                  trControl = train.control,
                  na.action = na.pass,
                  verbose = F,
                  verbosity = 0
                  )

# Turning off our other instances of RStudio
stopCluster(c1)

Works Cited

Alboukadel Kassambara (2020). ggpubr: ‘ggplot2’ Based Publication Ready Plots. R package version 0.4.0. https://CRAN.R-project.org/package=ggpubr

Andri Signorell et mult. al. (2021). DescTools: Tools for descriptive statistics. R package version 0.99.44. https://cran.r-project.org/web/packages/DescTools/index.html

Burnette, Daarel. “Student Outcomes: Does More Money Really Matter?” Education Week, Education Week, 8 Dec. 2020, www.edweek.org/policy-politics/student-outcomes-does-more-money-really-matter/2019/06.

Hadley Wickham and Jennifer Bryan (2022). readxl: Read Excel Files. R package version 1.4.0. https://CRAN.R-project.org/package=readxl

Hadley Wickham (2019). stringr: Simple, Consistent Wrappers for Common String Operations. R package version 1.4.0. https://CRAN.R-project.org/package=stringr

H. Wickham. ggplot2: Elegant Graphics for Data Analysis. Springer-Verlag New York, 2016.

Jackson, C. Kirabo. “Does School Spending Matter? the New Literature on an Old Question.”Confronting Inequality: How Policies and Practices Shape Children’s Opportunities., 2020, pp. 165–186., https://doi.org/10.1037/0000187-008.

Jonathan M. Lees (2020). PEIP: Geophysical Inverse Theory and Optimization. R package version 2.2-3. https://CRAN.R-project.org/package=PEIP

Julien Barnier (2021). rmdformats: HTML Output Formats and Templates for ‘rmarkdown’ Documents. R package version 1.0.3. https://CRAN.R-project.org/package=rmdformats

Kun Ren and Kenton Russell (2021). formattable: Create ‘Formattable’ Data Structures. R package version 0.2.1. https://CRAN.R-project.org/package=formattable

Max Kuhn (2022). caret: Classification and Regression Training. R package version 6.0-91. https://CRAN.R-project.org/package=caret

Microsoft Corporation and Stephen Weston (2022). doSNOW: Foreach Parallel Adaptor for the ‘snow’ Package. R package version 1.0.20. https://CRAN.R-project.org/package=doSNOW

Millard SP (2013). _EnvStats: An R Package for Environmental Statistics_. Springer, New York. ISBN 978-1-4614-8455-4, <URL: https://www.springer.com>.

R Core Team (2021). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. URL https://www.R-project.org/.

Rinker, T. W. & Kurkiewicz, D. (2017). pacman: Package Management for R. version 0.5.0. Buffalo, New York. http://github.com/trinker/pacman

RStudio Team (2020). RStudio: Integrated Development for R. RStudio, PBC, Boston, MA URL http://www.rstudio.com/.

Schulman, Craig T. “Texas High School AP and SAT Data.” Econometrics 461, http://people.tamu.edu/~cschulman/ECMT461/TermPr461.html.

Wickham et al., (2019). Welcome to the tidyverse. Journal of Open Source Software, 4(43), 1686, https://doi.org/10.21105/joss.01686

Yihui Xie (2022). knitr: A General-Purpose Package for Dynamic Report Generation in R. R package version 1.38.