BCPS Performance Task
Sarah Bluher
January 28, 2019
Performance Task
Performance on the PARCC assessment is measured by a scale score (SCALE_SCORE) that ranges from 650 (poorest performance) to 850 (highest performance). Student demographic information is also included in the test file in the columns RACE, SWD (Student With Disability), ELL (English Language Learner), and FARMS (Free and Reduced Meals).
Please examine the test result file and answer the following questions below in a Word document. Please also provide any syntax or code used to complete the task. 1. Construct a table that summarizes the average scale score and the number of test takers by each of the identified racial or ethnic groups. Do Hispanic students show a significant difference in test performance from African-American students? (Please provide a p-value) 2. What are the rates of students in each racial/ethnic group that are flagged as having free or reduced meal status (FARMS)? Please include these poverty rates in the table from question 1. 3. Construct a linear model that will independently assess the effect of FARMS status and race/ethnicity on test performance. What is the effect size of each racial/ethnic status on scale score performance? (Use African-American students who are not flagged as FARMS as the baseline for comparison; include a p-value for comparisons). Do Hispanic students have significantly different performance from African-American students once FARMS is taken into account? Please summarize these results in few sentences.
Summary Statistics
The table below shows the mean (mean_score) and standard deviation (stdev) of scale scores, the total number of students in each racial/ethnic group (count), and the percent of each racial/ethnic group that qualifies for free or reduced meals (perc_FARMS). African American and Hispanic students’ average scale scores differ by just 3.7 points, which statistical analysis confirms is not significant (see statistical analysis section below).
## # A tibble: 7 x 5
## RACE mean_score stdev count perc_FARMS
## <fct> <dbl> <dbl> <int> <dbl>
## 1 African American/Black 716. 32.6 5292 69.5
## 2 American Indian/Alaska Native 711. 39.6 14 50.0
## 3 Asian 761. 51.5 48 22.9
## 4 Hispanic 720. 34.7 735 50.2
## 5 Native Hawaiian/Pacific Islander 739. 64.8 12 50.0
## 6 Two or more races 733. 35.5 60 56.7
## 7 White 742. 43.5 557 38.4In order to further explore the full spread of data, I visualized histograms of student scale scores grouped by race/ethnicity and FARM status.
To check for normality, I generated qqplots for each group. The left tail of each plot shows deviation from the expected normal distribution. The lower bound on scores of 650 causes left-censoring in the dataset due to a significant portion of students performing at or below the limit of detection.
Statistical Analysis
I performed statistical tests to determine whether there is a significant effect of race, FARM status and the interaction between race and FARM status on scale scores for African American and Hispanic students. To compensate for violation of the normality assumption inherent in OLS regression, I conducted additional regression analysis specialized to handle censored data.
ANOVA
To begin, I conducted an ANOVA using a subset of the data with only African American and Hispanic students included. The boxplot below shows the spread of scale score values for the four groups of students considered in this analysis, categorized by race/ethnicity and FARM status.
The results of the ANOVA below show that there is no significant difference between African American and Hispanic scale scores (p = 0.35). There is, however, a large and highly significant difference between scale scores of students with and without FARM status (p = 0.00016), as well as a moderate and significant effect of the interaction between race and FARM status on scale scores (p = 0.015).
## Df Sum Sq Mean Sq F value Pr(>F)
## RACE 1 976 976 0.883 0.347754
## FARMS 1 15889 15889 14.370 0.000161 ***
## RACE:FARMS 1 6518 6518 5.894 0.015396 *
## Residuals 852 942083 1106
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1Multiple Linear Regression
As a follow-up to ANOVA, I ran multiple linear regressions to determine the size and direction of additive and interaction effects of race/ethnicity and FARM status on student scale scores. To begin, I ran a standard OLS additive regression with race/ethnicity and FARM status as the independent variables and scale score as the dependent variable. The results below confirm that there is no effect of race on overall performance. There is a significant negative effect of FARM status, meaning that students who qualify for free and reduced meals have overall lower test scores.
##
## Call:
## lm(formula = SCALE_SCORE ~ RACE + FARMS, data = data_stats_subset)
##
## Residuals:
## Min 1Q Median 3Q Max
## -76.217 -22.929 0.612 22.612 136.324
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 724.9285 0.7652 947.359 <2e-16 ***
## RACEHispanic 1.2886 1.2860 1.002 0.316
## FARMSY -12.5408 0.8959 -13.998 <2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 32.37 on 6024 degrees of freedom
## Multiple R-squared: 0.03282, Adjusted R-squared: 0.0325
## F-statistic: 102.2 on 2 and 6024 DF, p-value: < 2.2e-16Next, I adjusted my model to build in interaction between race/ethnicity and FARM status. In contrast to the additive model, the model with interaction shows a small but significant negative effect of Hispanic race on performance. This effect is driven by the difference in performance between Hispanic and African American students who do not have FARM status. As in the additive model, there is a highly significant negative effect of FARM status on performance. The interaction term is large, positive and highly significant, meaning that FARM status has a much weaker negative effect mong Hispanic students than African American students. In other words, the performance of African American students seems to be more sensitive to FARM status than that of Hispanic students.
##
## Call:
## lm(formula = SCALE_SCORE ~ RACE * FARMS, data = data_stats_subset)
##
## Residuals:
## Min 1Q Median 3Q Max
## -76.119 -23.865 0.135 23.135 131.114
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 726.1189 0.8042 902.936 < 2e-16 ***
## RACEHispanic -5.1544 1.8709 -2.755 0.00589 **
## FARMSY -14.2540 0.9647 -14.775 < 2e-16 ***
## RACEHispanic:FARMSY 12.1757 2.5719 4.734 2.25e-06 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 32.32 on 6023 degrees of freedom
## Multiple R-squared: 0.0364, Adjusted R-squared: 0.03592
## F-statistic: 75.85 on 3 and 6023 DF, p-value: < 2.2e-16Multiple Linear Regression for Censored Data
Finally, I conducted two more linear regression analyses using the R package censReg specially suited to handle censored data. The models confirm that the data is left-censored, due to the large portion of students performing at the lowest possible test score of 650. The additive censored regression is shown below. The additive censored regression results do not meaningfully differ from the standard additive OLS.
## $maximType
## [1] "Newton-Raphson maximisation"
##
## $iterations
## [1] 7
##
## $returnCode
## [1] 2
##
## $returnMessage
## [1] "successive function values within tolerance limit"
##
## $loglik
## [1] -29045.8
##
## $estimate
## Estimate Std. error t value Pr(> t)
## (Intercept) 724.705227 0.785946379 922.0797318 0.000000e+00
## RACEHispanic 1.306353 1.321641442 0.9884319 3.229412e-01
## FARMSY -12.754981 0.920525826 -13.8561901 1.167064e-43
## logSigma 3.503258 0.009328601 375.5395035 0.000000e+00
##
## $fixed
## (Intercept) RACEHispanic FARMSY logSigma
## FALSE FALSE FALSE FALSE
##
## $NActivePar
## [1] 4
##
## $constraints
## NULL
##
## $call
## censReg(formula = SCALE_SCORE ~ RACE + FARMS, left = 650, right = 850,
## data = data_stats_subset)
##
## $nObs
## Total Left-censored Uncensored Right-censored
## 6027 175 5851 1
##
## attr(,"class")
## [1] "summary.censReg" "summary.maxLik"The final model, censored regression with interaction, is shown below. As with the additive model, the censored results do not differ substantially from the standard OLS with interaction.
## $maximType
## [1] "Newton-Raphson maximisation"
##
## $iterations
## [1] 8
##
## $returnCode
## [1] 1
##
## $returnMessage
## [1] "gradient close to zero"
##
## $loglik
## [1] -29034.33
##
## $estimate
## Estimate Std. error t value Pr(> t)
## (Intercept) 725.943117 0.825808398 879.069671 0.000000e+00
## RACEHispanic -5.401657 1.923018468 -2.808947 4.970384e-03
## FARMSY -14.536497 0.991134845 -14.666518 1.056282e-48
## RACEHispanic:FARMSY 12.671890 2.642950185 4.794600 1.629995e-06
## logSigma 3.501383 0.009328395 375.346818 0.000000e+00
##
## $fixed
## (Intercept) RACEHispanic FARMSY
## FALSE FALSE FALSE
## RACEHispanic:FARMSY logSigma
## FALSE FALSE
##
## $NActivePar
## [1] 5
##
## $constraints
## NULL
##
## $call
## censReg(formula = SCALE_SCORE ~ RACE * FARMS, left = 650, right = 850,
## data = data_stats_subset)
##
## $nObs
## Total Left-censored Uncensored Right-censored
## 6027 175 5851 1
##
## attr(,"class")
## [1] "summary.censReg" "summary.maxLik"Conclusion
Based on this regression analysis of BCPS performance data, I conclude that Hispanic students do not have significantly different performance from African American students once FARMS is taken into account. While the regression model with interaction showed a small significant difference between Hispanic and African American students in the non-FARMS subset, there is no overall effect of race on performance. Rather, the largest differences in performance are driven purely by FARM status. Lastly, the results of the model with interaction suggest that FARM status may have a greater impact on African American versus Hispanic student test scores. This finding warrants further investigation.