A recent report by the Detroit Free Press looked at the portfolio of Michigan charter schools operated by National Heritage Academies and used a percentile rank to try to show that 50% of schools in the state of Michigan “do better” than 26 of NHA’s 47 schools.

The graphic can be viewed here:

http://www.freep.com/article/20140622/NEWS06/140621007/Graphic-look-National-Heritage-Academy-s-47-charter-schools

However, this type of percentile ranking is largely meaningless and not the best metric to use for evaluating school performance. This is because 50% of schools must fall into the bottom 50 percentile. However, if all schools were hitting state benchmarks for test scores and college preparedness then we could hardly conclude that the bottom 50% were performing poorly.

Charter schools, in particular, provide a choice for a community. One example of an evaluation that would be more meaningful in this case would be to compare a given charter school with other schools in the geographic area to see if the charter school adds value for the members of the community.

In the analysis below, we will examine 3rd grade MEAP test scores for elementary schools in the ZIP code 49546 which includes the NHA school Ridge Park Charter Academy which was in the 32nd Percentile during School Year 12-13.

The data files required to replicate this analysis are available here:

https://www.mischooldata.org/DistrictSchoolProfiles/EntitySummary/SchoolDataFile.aspx

(The data files can be requested via a web form but the link to the file expires after 72 hours so those wishing to reproduce the research below will have to request their own copies)

The files that I use are:

All files from School Year 13-14

## read in school entity data

setwd("C:/Users/pawlusm/Desktop/schlData")
schlEntity <- read.csv("a702e75b-4b57-405a-b0da-771669857152.csv")

The Entity File contains address information. This is used to find the ZIP code for the school that will be compared with others from the same geographic region. In this case, the ZIP code for Ridge Park Charter Academy is identified and used to select other schools from the same 49546 ZIP code.

## look for Ridge Park Charter Academy in the data set

grep("Ridge Park", schlEntity$BuildingName, value=TRUE)
## [1] "Ridge Park Charter Academy" "Ridge Park Charter Academy"
## isolate the Ridge Park row to find the ZIP code to use to find nearby schools

subset(schlEntity, schlEntity$BuildingName=="Ridge Park Charter Academy")
##               SchoolYear BuildingCode               BuildingName
## 1961 13 - 14 School Year            0 Ridge Park Charter Academy
## 1962 13 - 14 School Year         8652 Ridge Park Charter Academy
##      DistrictCode DistrictName ISDCode  ISDName   EntityType      Status
## 1961        41919   Ridge Park      41 Kent ISD PSA District Open-Active
## 1962        41919   Ridge Park      41 Kent ISD   PSA School Open-Active
##        SCHOOL_EMPHASIS           SETTING
## 1961                                    
## 1962 General Education Multiple Settings
##                              EMAIL_ADDRESS PhoneNumber
## 1961     18.EWYCKOFF@HERITAGEACADEMIES.COM  6162220093
## 1962 18.BMORGENSTEIN@HERITAGEACADEMIES.COM  6162220093
##                ADDRESS_LINE_1         CITY STATE ZIP_CODE
## 1961 4120 CAMELOT RIDGE DR SE GRAND RAPIDS    MI    49546
## 1962 4120 CAMELOT RIDGE DR SE GRAND RAPIDS    MI    49546
##            SCHOOL_TYPE       SCHOOL_LEVEL   LOCALE_NAME COUNTY_NAME
## 1961                   Elem_Middle School City: Midsize        Kent
## 1962 General Education Elem_Middle School Suburb: Large        Kent
## subset schools from the same ZIP code

schlRP <- subset(schlEntity, schlEntity$ZIP_CODE=="49546" & schlEntity$BuildingCode!=0)

The Combined File dataset includes enrollment details. This file will be merged with the Entity file to further refine the school set used for comparison. Using enrollment data, the merged file will be narrowed down to only elementary schools.

## read in the Combined File with enrollment details

schlCF <- read.csv("d991c29c-1b2b-45b5-80a7-f17e93a68fd5.csv")
## subset the rows from the Combined File data to match those selected from the Entity file

schlCFrp <- subset(schlCF, schlCF$BuildingName %in% schlRP$BuildingName & schlCF$ISDCode==41)
## select out relevant columns to make a more compact data frame for analysis

schlRPsub <- subset(schlRP, select=c(SchoolYear,BuildingCode,BuildingName,ISDCode,ZIP_CODE))

schlCFrpSub <- subset(schlCFrp, select=c(SchoolYear,BuildingCode,BuildingName,ISDCode,TOTAL_ENROLLMENT,GRADE_3_ENROLLMENT,ECONOMIC_DISADVANTAGED_ENROLLMENT))
## merge the Entity and Combined File data frames

schlRPcomp <- merge(schlRPsub, schlCFrpSub, by="BuildingName")
## keep only elementary schools
## in particular, we are keeping those with 3rd grade enrollment since we will compare 3rd grade MEAP scores

schlRPelem <- subset(schlRPcomp, schlRPcomp$GRADE_3_ENROLLMENT!=0)

Looking at the Detroit Free Press graphic, one value that jumps out is that a significant portion of Ridge Park Charter Academy’s student population qualify for Free or Reduced Lunch. This group is also referred to as Economically Disadvantaged in the Combined File. The code below compares the Economically Disadvantaged population among the 5 elementary schools in the same ZIP code.

## add a column for percentage of students Economically Disadvantaged

schlRPelem$percentEDE <- (schlRPelem$ECONOMIC_DISADVANTAGED_ENROLLMENT/schlRPelem$TOTAL_ENROLLMENT) * 100
## plot the data

library(ggplot2)

ggplot(data=schlRPelem, aes(x=BuildingName, y=percentEDE)) + geom_bar(aes(fill=BuildingName), stat="identity") + theme(axis.text.x = element_blank(), axis.title.x = element_blank()) + ylab("% Economically Disadvantaged") + labs(title="Economical Disadvantaged student population \nas a proportion of total enrollment \namong 5 elementary schools from the same ZIP code")

The MEAP file contains MEAP scores for the state, districts, and schools for Reading and Math and for every grade level (it is a large file). This will be used to compare test scores. In particular, following on from the plot above, we will look at the test scores among those classified as Economically Disadvantaged.

## read in MEAP data 

schlM <- read.csv("d5340f52-1d0a-4092-b287-821320a6c44b.csv")

First, the Math Scores for 3rd Graders classified as Economically Disadvantaged will be compared

## subset the data to only include 3rd Grade Math scores for those classified as Economically Disadvantaged from the schools identified

schlMedeMath <- subset(schlM, schlM$Subject.Name=="Mathematics" & schlM$Subgroup=="Economically Disadvantaged" & schlM$BuildingCode %in% schlRPelem$BuildingCode.x & schlM$ISDCode==41 & schlM$BuildingCode!=0 & schlM$Grade==3)
## remove schools with very small Econimically Disadvantaged populations

schlMedeMath <- subset(schlMedeMath, schlMedeMath$Number.Tested!="< 10")
## convert the values in the percent proficient column from factors to numbers

schlMedeMath$Percent.Proficient.n <- as.numeric(as.character(schlMedeMath$Percent.Proficient))
## plot the data

ggplot(data=schlMedeMath, aes(x=BuildingName, y=Percent.Proficient.n)) + geom_bar(aes(fill=BuildingName), stat="identity") + theme(axis.text.x = element_blank(), axis.title.x = element_blank()) + ylab("% Proficient") + labs(title="Comparison of 3rd Grade MEAP Math scores \namong ED population at 3 elementary schools \nfrom the same ZIP code")

The following comparison will be done for 3rd Grade Reading scores

## repeat for reading scores

schlMedeReading <- subset(schlM, schlM$Subject.Name=="Reading" & schlM$Subgroup=="Economically Disadvantaged" & schlM$BuildingCode %in% schlRPelem$BuildingCode.x & schlM$ISDCode==41 & schlM$BuildingCode!=0 & schlM$Grade==3)

schlMedeReading <- subset(schlMedeReading, schlMedeReading$Number.Tested!="< 10")

schlMedeReading$Percent.Proficient.n <- as.numeric(as.character(schlMedeReading$Percent.Proficient))
ggplot(data=schlMedeReading, aes(x=BuildingName, y=Percent.Proficient.n)) + geom_bar(aes(fill=BuildingName), stat="identity") + theme(axis.text.x = element_blank(), axis.title.x = element_blank()) + ylab("% Proficient") + labs(title="Comparison of 3rd Grade MEAP Reading scores \namong ED population at 3 elementary schools \nfrom the same ZIP code")

Ridge Park serves a population that is largely Economically Disadvantaged. The analysis shows that test scores for this population are the best at Ridge Park. Offering the children of Economically Disadvantaged families the opportunity to attend Ridge Park Charter Academy leads to better outcomes when compared with public schools in the same geographic area demonstrating a value for the community not reflected in the 32nd percentile statewide ranking. For this reason, it is more meaningful to look at a “basket” of schools with some similar characteristics rather than just relying on statewide percentiles when evaluating school performance.