Summary

The goal of this document is to internally clarify and discuss solutions for our work with Queens Public Library (QPL). The current agreement with QPL requires that we provide the following:

1. A clear empirical process for selecting neighborhoods and partner middle schools

2. Built-in flexibility that allows our partners to voice their expertise and preferences when making decisions

3. Complete analyses that highlight clear recommendations based on QPL’s stated preferences

Framework to Prioritize Service by Neighborhood Profile

QPL wishes to provide program services to middle school students across Queens. QPL cannot offer these services at every location, so we created a process to optimize targeting of finite resources. This document prvides a description of this process and allows QPL to make a final determination based on their preferences.

To select neighborhoods that will ultimately host QPL programming in local branches, we built a tool to classify each neighborhood on the basis of two factors:

1. An economic need index score developed by NYC

We first sought to assess local need. QPL has expressed an interest in providing program services in ways that promote equity and we understand that children in economically disadvantaged neighborhoods are less likely to have access to afterschool programs and services than their more affluent peers. To assess local economic need, we aggregated school-level economic index scores to the neighborhood level.

As described on the NYC Open Data website, a higher economic index score indicates higher local need (this is highly correlated with local poverty, in general). We standardized need index scores across neighborhoods to determine local need relative to other neighborhoods. These scores are presented in standard deviation (SD) units (where the mean is 0 and the standard deviation is 1).

2. Academic achievement relative to other neighborhoods across Queens (after accounting for local demographic and school characteristics)

We aggregated school-level academic performance data, which is available on the NYC Open Data website, to the neighborhood level. We excluded high school achievement scores because these scores are calculated differently than elementary and middle school scores and because high schools are only present in 25 neighborhoods. We then standardized local achievement measures to account for differences across grade levels and assess local achievement relative to other neighborhoods in NYC. We followed the same process to obtain relative measures of various neighborhood characteristics (poverty rate, percentage of nonwhite students, gender composition, ell and disability composition, rates of chronic absenteeism, and school-level instructional quality index scores).

We built a model to predict neighborhood achievement that accounted for each of the above characteristics.

The preferred model predicts local achievement (in standardized units) in neighborhood i, and reads as follows:

\[Achieve_i = \alpha + \beta_1(Pov_i) + \beta_2(Nonwhite_i) + \beta_3(Absent_i) + \beta_4(ELL_i) + \beta_5(Disabled_i) + \beta_6(Instr_i) + \epsilon_i\]

Results from this model (model II), and from a simpler model accounting only for racial and socioeconomic characteristics of the neighborhoods, are provided below:

Summary of Predictive Model for Relative Neighborhood Achievement
Achievement
Model I Model II
Poverty -0.287 (0.195) -0.325* (0.179)
Nonwhite -0.039 (0.195) 0.087 (0.125)
Chron. Absent 0.015 (0.152)
ELL 0.439*** (0.145)
Disability -0.011 (0.101)
Instruction Q 0.620*** (0.105)
Constant -0.000 (0.129) -0.000 (0.080)
N 56 56
R2 0.101 0.685
Adjusted R2 0.067 0.646
Residual Std. Error 0.966 (df = 53) 0.595 (df = 49)
F Statistic 2.967* (df = 2; 53) 17.731*** (df = 6; 49)
Note: p < .01; p < .05; p < .1
Coefficients are presented in standardized units.

The preferred model, presented in the second column, captured 65% of the variance in local achievement, indicating a strong fit with the data. We found that racial and socioeconomic characteristics alone do very little to explain variation in neighborhood performance (7%). We selected model II for its superior fit and predictive quality. We used residuals from this model to determine whether neighborhoods demonstrated achievement outcomes above or below prediction.

Summary
neighborhoods demonstrating achievement outcomes above prediction were classified as “high-achieving;”
neighborhoods below prediction were classified as “low-achieving.”

Classifying Neighborhoods Based on These Criteria

To visualize neighborhood classification on these criteria, we presented the relationship between local economic need and academic performance relative to prediction. As we present in the graphic below, we split neighborhoods into four broad categories. We also highlighted the highest-need neighborhoods for clarity.

Visualizing the Relationship Between Achievement and Local Economic Need

Mapping QPL Branches Across Neighborhoods by Category

We then mapped QPL locations across the borough and highlighted the neighborhood category where each branch was located. The map tool is available here.

A table of neighborhood-level characteristics, including residuals, achievement, need, afterschool program access, school choice education, and enrollment size, is available here.

Prioritizing High- or Low-Achieving Neighborhoods

We classified neighborhoods by need and achievement to provide QPL with a clear framework to guide their selections. They may, for example, find it most effective to invest program resources in neighborhoods where the economic need is particularly high and local academic achievement exceeds predictions. This may indicate that additional resources could have a large impact on local students.

An alternative view is that resources should go to the neighborhoods where the need is high and achievement falls below predictions. These neighborhoods are clearly facing meaningful challenges and QPL may believe it best to target resources in these contexts. The following steps provide support for QPL to determine the optimal choices based on their preference after considering these options.

“Choice A” is a preference for selecting neighborhoods where the economic need is high and the academic achievement relative to prediction is high (high need, high achievement). “Choice B” is a preference for selecting neighborhoods where the economic need is high and the academic achievement relative to prediction is low (high need, low achievement).

Choice A: Avoiding Service Overlap in High-Need, High-Achieving Neighborhoods

It is likely that QPL wishes to prioritize services where they are not currently available to local students. Using a standardized measure of access to afterschool programs, we compared high-need, high-achieving neighborhoods on the basis of local access to these programs. From this analysis, we determined which neighborhoods had an outsized need for afterschool programs in their communities.

Specifically, the neighborhoods with below average access included Briarwood-Jamaica Hills, Springfield Gardens North, Hunters Point-Sunnyside-West Maspeth, College Point, Elmhurst, and South Ozone Park.

Afterschool Program Access in High-Need, High-Achieving Neighborhoods

Selecting Middle School Partners

QPL has indicated that they would like to partner their services with local middle schools. After selecting neighborhoods on the basis of economic need, achievement, and pre-existing access to afterschool programs, we highlight schools in these neighborhoods for potential partnerships. Similar to their selections for neighborhoods, QPL will have the option to prioritize high-achieving or low-achieving schools. The rationale for either selection is valid and depends on their preferences as well as their theory of change for the program. QPL may also prioritize schools where parents’ preferences for more enrichment opportunities are relatively high, as these parents may encourage their children to participate.

The graphs below do the following:

1. compare academic achievement across all schools enrolling middle school-aged students (k-8, k-12, and middle schools) in the neighborhoods classified as high-need and high-performing. Each school’s value is color-coded to indicate its neighborhood. Using the graphic tools presented above, we can search for schools in the neighborhoods with lower relative access to afterschool programs.

2. compare parents’ preferences for more enrichment opportunities in their schools in the neighborhoods classified as high-need and high-achieving. Each school’s value, as in the graph above, is color-coded to indicate its neighborhood.

Graph 1.

Graph 2.

Choice B: Avoiding Service Overlap in High-Need, Low-Achieving Neighborhoods

QPL may also view those neighborhoods where academic performance is low as optimal locations to provide their program services. Using a standardized measure of access to afterschool programs, we compared high-need, academically low-achieving neighborhoods. From this analysis, we determined which neighborhoods had an outsized need for more after-school programs in their communities.
Specifically, the neighborhoods with below average access included Ozone Park, Murray Hill, East Elmhurst, Corona, Ridgewood, Queensbridge-Ravenswood-Long Island City, Hollis, and North Corona.

Afterschool Program Access in High-Need, Low-Achieving Neighborhoods

Selecting Middle School Partners

QPL has indicated that they would like to partner their services with local middle schools. After selecting neighborhoods on the basis of economic need, achievement, and pre-existing access to afterschool programs, we highlight schools in these neighborhoods for potential partnerships. Similar to their selections for neighborhoods, QPL will have the option to prioritize high-achieving or low-achieving schools. The rationale for either selection is valid and depends on their preferences as well as their theory of change for the program. QPL may also prioritize schools where parents’ preferences for more enrichment opportunities are relatively high, as these parents may encourage their children to participate.

The graphs below do the following:

1. compare academic achievement across all schools enrolling middle school-aged students (k-8, k-12, and middle schools) in the neighborhoods classified as high-need and low-achieving. Each school’s value is color-coded to indicate its neighborhood. Using the graphic tools presented above, we can search for schools in the neighborhoods with lower relative access to afterschool programs.

2. compare parents’ preferences for more enrichment opportunities in their schools in the neighborhoods classified as high-need and low-achieving. Each school’s value, as in the graph above, is color-coded to indicate its neighborhood.

Graph 1.

Graph 2.

The preceeding description and analyses were intended to serve as a start for internal discussions about best steps as we move forward in this work with QPL. As stated, the goal here is to provide QPL with clear paths forward while ensuring that they feel the freedom to make decisions based on their contextual expertise and organizational values. This draft is an early attempt to provide QPL with both clarity and freedom in their decision-making.

Thank you very much for reading! As you review and consider what you saw here, please feel free to send along your thoughts, ask any clarifying questions, or request time to speak further via Zoom or Teams! I really look forward to upcoming conversation and am excited about working with you all!

PLM