Identifying Opportunity Zones with ML Algorithms
Kabir Jain
2024-07-05
Note: All R code to run this analysis can be found here (please right-click the link and open it in a new tab).
1 Abstract
In early 2018, all U.S. States, territories, and the District of Columbia nominated census tracts as Qualified Opportunity Zones (QOZs), out of which 8,764 tracts were selected. In this paper, I split the database of census tracts into training and testing sets, and attempt to train machine learning models to identify census tracts as QOZs based on census data. The best performing model was the random forest model. However, even the best performing model was unable to identify QOZs with a reasonable degree of specificity. This raises questions about not just the data and the models, but also about the process of QOZ nomination and selection.
2 Introduction and Policy Impacts
Opportunity zones are a place-based economic development program. They were first conceptualized in 2015, in a white paper title Unlocking Private Capital to Facilitate Economic Growth in Distressed Areas (Bernstein and Hassett 2015). This suggestion was prompted by rising concerns surrounding growing income inequality; in Q1 of 2015, 30.7% of the nation’s net worth was held by the top 1% (99th to 100th wealth percentiles) (“Share of Total Net Worth Held by the Top 1 Percent (99th to 100th Wealth Percentiles)” 2024). This was an all-time high. Policymakers were searching for ways to incentivize wealthy individuals to invest wealth in low-income areas.
Opportunity zones, created as part of the Tax Cuts and Jobs Act of 2017 (“An Act to Provide for Reconciliation Pursuant to Titles II and v of the Concurrent Resolution on the Budget for Fiscal Year 2018” 2017), served as a solution to this challenge. According to the IRS, Qualified Opportunity Zones (QOZs) spur economic growth in the following ways:
First, an investor can defer tax on any prior eligible gain to the extent that a corresponding amount is timely invested in a Qualified Opportunity Fund (QOF). The deferral lasts until the earlier of the date on which the investment in the QOF is sold or exchanged, or December 31, 2026. If the QOF investment is held for at least 5 years, there is a 10% exclusion of the deferred gain. If held for at least 7 years, the 10% exclusion becomes 15%.
Second, if the investor holds the investment in the QOF for at least 10 years, the investor is eligible for an adjustment in the basis of the QOF investment to its fair market value on the date that the QOF investment is sold or exchanged. As a result of this basis adjustment, the appreciation in the QOF investment is never taxed. A similar rule applies to exclude the QOF investor’s share of gain and loss from sales of QOF assets.
Recently, the Economic Innovation Group published another paper title Are Opportunity Zones Working? What the literature tells us, looking at post-implementation studies that attempt to quantify the impact of QOZ designation on low-income communities (Fikri and Glasner 2023). This paper highlighted the main pitfalls of current OZ research: variable specification (“price or transaction volume data for commercial or residential real estate will be poor estimators for the near-term activity induced by OZs”), model selection (the parallel trends assumption may not hold for DID models, whereas “zooming” into two sides of a threshold means RD models are not well-suited to detecting changes elsewhere in the samples), and window of analysis (the main variables of interest, poverty, employment, and incomes, are long-run by nature; it is not plausible to expect poverty rates to fall simply because an area was designated as a QOZ). Overall, the conclusions were that Opportunity Zones are a relatively young policy, and not enough time has elapsed since implementation for researchers to observe a significant effect on low-income communities.
This study, however, is concerned not with the impacts of OZ designation, but the factors that led to it. Each U.S. State nominated eligible census tracts which were then shortlisted by the U.S. Treasury. Several states provided information on the selection process, while many did not. Largely, most states that did provide the public with a rationale for QOZ nomination quoted poverty, unemployment, and low incomes as the driving factors. This analysis seeks to verify this - can we use machine learning methods to differentiate between QOZ-nominated and non-nominated census tracts?
3 Data Sources
Table 1 shows the data sources used in this study.
| Data.Title | Data.Source |
|---|---|
| Opportunity Zone Census Tracts | U.S. Department of Housing and Urban Development |
| CDC/ATSDR Social Vulnerability Index (SVI) | Center for Disease Control, Agency for Toxic Substances and Disease Registry |
| American Community Survey (ACS) 5-year Estimates | U.S. Census Bureau |
Data on ACS 5-year estimates were obtained from the U.S. Census Bureau through the Census Bureau API (using the package tidycensus). These are the predictive indicators used by the models in this study. ACS estimates were pulled for the year 2015 (5 year estimates for 2011-2015) since this was the year the Economic Innovation Group published the paper introducing the idea of opportunity zones.
4 Methodology
4.1 Models
In this study, three machine learning algorithms are used to predict the likelihood of a given census tract being classified as a QOZ based on available data.
For each algorithm, the dataset of census tract data and QOZ classification was segmented in a ratio of 70:30 for training and testing purposes respectively.
4.1.1 Logistic Regression Classification
Logistic regression is a parametric classification model that uses a logistic function to estimate binary output model. It assumes a linear relationship between independent and dependent variables and homoskedasticity of the training data. Also, independent variables are not allowed to be collinear. This model can be used for multi-class classifications. However, it requires proper selection of features and cannot be applied on non-linear classification problems (Yoo 2023).
4.1.2 K-Nearest Neighbors Classification
The k-nearest neighbors (KNN) algorithm is a non-parametric, supervised learning classifier, which uses proximity to make classifications or predictions about the grouping of an individual data point. Good practice dictates an investigation of the optimal \(k\) to be used for the classification model, which will be shown later in this study.
4.1.3 Random Forest Classification
Random forest is a machine learning algorithm that combines the output of multiple decision trees to reach a single result. Its ease of use and flexibility have fueled its adoption, as it handles both classification and regression problems. Good practice dictates an investigation of the optimal number of trees and the optimal number of simultaneous variables, which will be shown later in this study.
4.2 Robustness
To check for robustness, three levels of data were used in each model:
4.2.1 Level 1
The first level of data includes estimates of median household income (at the tract and state level) and of the population living in poverty. Table A-1 (in the Appendix) shows a detailed list of all the variables acquired from the American Community Survey 5-year estimates for this level.
\[ Opp.Zone = \beta_0 + \beta_1(Poverty.Below.150\%) + \beta_2(MHHI.TractMinusState) + \epsilon \] Where:
- Opp.Zone is a binary variable for whether a tract belongs to an opportunity zone (1) or not (0).
- Poverty.Below.150% is an estimate of the percent of the census tract’s population that has an income equal to or less than 150% of the federal poverty level.
- MHHI.TractMinusState is an estimate of the difference between a tract’s median household income and the respective state’s median household income.
4.2.2 Level 2
The second level of data includes estimates on 16 indicators of social vulnerability. The Center for Disease Control and the Agency for Toxic Substances and Disease Registry maintain a Social Vulnerability Index (SVI) that ranks census tracts by performace across 16 different metrics. Additional information on the SVI can be accessed here. This level of data pulls all the ACS estimates required to recreate this index. Table A-2 (in the Appendix) shows a detailed list of all the variables acquired from the ACS 5-year estimate for this level.
\[ \begin{gather} Opp.Zone = \beta_0 + \beta_1(Poverty.Below.150\%) + \beta_2(Unemployment.Rate)\ + \\ \beta_3(Percent.CostBurdened) + \beta_4(Percent.NoHSD) + \beta_5(Percent.NoInsurance)\ + \\ \beta_6(Percent.AgeAbove65) + \beta_7(Percent.AgeUnder18) + \beta_8(Percent.Disabled)\ + \\ \beta_9(Percent.SPH) + \beta_{10}(Percent.LimitedEnglishHH) + \beta_{11}(Percent.Minority)\ + \\ \beta_{12}(Percent.MSU) + \beta_{13}(Percent.MobileHome) + \beta_{14}(Percent.Overcrowded)\ + \\ \beta_{15}(Percent.NoVehicle) + \beta_{16}(Percent.GroupQuarters) + \epsilon \end{gather} \] Where:
- Opp.Zone is a binary variable for whether a tract belongs to an opportunity zone (1) or not (0).
- Poverty.Below.150% is an estimate of the percent of the census tract’s population that has an income equal to or less than 150% of the federal poverty level.
- Unemployment.Rate is the percent of the labor force that is classified as unemployed.
- Percent.CostBurdened is the percent of households where the income is below $75,000 and over 30% of income is spent on housing costs.
- Percent.NoHSD is the percent of the population above the age of 16 that does not have a high school diploma.
- Percent.NoInsurance is the percent of the population that has no health insurance coverage.
- Percent.AgeAbove65 is the percent of the population above the age of 65.
- Percent.AgeUnder18 is the percent of the population under the age of 18.
- Percent.Disabled is the percent of the population that identifies as having a disability.
- Percent.SPH is the percent of all households that have a single householder (no partner present) and children under the age of 18.
- Percent.LimitedEnglishHH is the percent of all households that report a limited proficiency of English.
- Percent.Minorty is the percent of the population that identifies as any race/ethnicity except “White alone, not Hispanic”.
- Percent.MSU is the percent of all housing units that are in structures that have over 10 units.
- Percent.MobileHome is the percent of all housing units that are classified as mobile homes.
- Percent.Overcrowded is the percent of all housing units that have over 1.0 occupants per room.
- Percent.NoVehicle is the percent of all households that report having no access to a personal vehicle.
- Percent.GroupQuarters is the percent of the population that reports living in group quarters (dormitories, shelters, etc).
4.2.3 Level 3
The third and final level of data combines the Level 1 and Level 2 variables. While Level 2 incorporates the entirety of the Social Vulnerability Index, it does not explicitly include median household income. In level 3, this variable is added into the mix.
\[ \begin{gather} Opp.Zone = \beta_0 + \beta_1(Poverty.Below.150\%) + \beta_2(Unemployment.Rate)\ + \\ \beta_3(Percent.CostBurdened) + \beta_4(Percent.NoHSD) + \beta_5(Percent.NoInsurance)\ + \\ \beta_6(Percent.AgeAbove65) + \beta_7(Percent.AgeUnder18) + \beta_8(Percent.Disabled)\ + \\ \beta_9(Percent.SPH) + \beta_{10}(Percent.LimitedEnglishHH) + \beta_{11}(Percent.Minority)\ + \\ \beta_{12}(Percent.MSU) + \beta_{13}(Percent.MobileHome) + \beta_{14}(Percent.Overcrowded)\ + \\ \beta_{15}(Percent.NoVehicle) + \beta_{16}(Percent.GroupQuarters) + \beta_{17}(MHHI.TractMinusState) + \epsilon \end{gather} \] Please refer to Levels 1 & 2 for the definitions of each variable.
5 Summary Statistics
Table 2 shows the number of census tracts in each category, as defined by the variable Opp.Zone, which is a binary variable indicating whether a census tract belongs to a QOZ.
| Opp.Zone | N |
|---|---|
| 0 | 64,256 |
| 1 | 7,796 |
For most (if not all) states, the process of shortlisting census tracts for QOZ nomination was largely influenced by estimates of poverty, unemployment, and income. The table below shows the group-wise means for these variables.
| Opp.Zone | N | Poverty.Below.150Percent | Unemployment.Rate | MHHI.TractMinusState |
|---|---|---|---|---|
| 0 | 64,256 | 0.1950 | 0.08246 | 5,768 |
| 1 | 7,796 | 0.3754 | 0.13670 | -20,694 |
Census tracts that were selected as QOZs had higher poverty and unemployment rates, and lower median incomes relative to their state. A series of unpaired, two-sample t-tests validates that the means are statistically significantly different.
| Variable | T.statistic | P.value | Test.Mean.for.QOZ | Test.Mean.for.non.QOZ |
|---|---|---|---|---|
| Poverty.Below.150Percent | 109.96278 | 0 | 0.3754251 | 0.1949662 |
| Unemployment.Rate | 62.49945 | 0 | 0.1366763 | 0.0824558 |
| MHHI.TractMinusState | -151.05604 | 0 | -20694.2385839 | 5767.6496825 |
Table 5 shows the groupwise mean values for all variables.
| Variable | Opp.Zone_0 | Opp.Zone_1 |
|---|---|---|
| Poverty.Below.150Percent | 0.195000 | 0.375400 |
| Unemployment.Rate | 0.082460 | 0.136700 |
| Percent.CostBurdened | 0.304100 | 0.412900 |
| Percent.NoHSD | 0.129100 | 0.229400 |
| Percent.NoInsurance | 0.001243 | 0.001569 |
| Percent.AgeAbove65 | 0.150400 | 0.128300 |
| Percent.AgeUnder18 | 0.225200 | 0.240800 |
| Percent.Disabled | 0.124900 | 0.160200 |
| Percent.SPH | 0.109100 | 0.177100 |
| Percent.LimitedEnglishHH | 0.044050 | 0.072420 |
| Percent.Minority | 0.348600 | 0.560500 |
| Percent.MSU | 0.114900 | 0.174800 |
| Percent.MobileHome | 0.073780 | 0.091370 |
| Percent.Overcrowded | 0.032760 | 0.061410 |
| Percent.NoVehicle | 0.044440 | 0.103300 |
| Percent.GroupQuarters | 0.020820 | 0.044640 |
| MHHI.TractMinusState | 5768.000000 | -20690.000000 |
6 Model Paramaters
6.1 Logistic Regression Classification
The model specifications for the logistic regression are fairly simple. It doesn’t require additional parameters, so the generalized linear model (GLM) follow the equations outlined in the Methodology section. Probability values were rounded to the nearest integers to convert predictions to a binary level.
6.2 K-Nearest Neighbors
The model specification for KNN classification does require additional parameters, and it’s in our best interest to investigate the optimal values of these. A KNN classification model requires a specified \(k\), which is the number of nearest neighbors that will be used to classify a new data point. Fortunately, in R, the train() function, with method=“knn” specified, automatically selects the optimal value of \(k\) (the value for which the predicted accuracy is the highest). The graphs below show the elbow-graphs for each of the three levels of data.
For all three levels of data, accuracy increases significantly as \(k\) approaches \(5\), and plateaus out around \(k = 15\). The optimal \(k\) values for the three levels of data were \(20, 19\), and \(20\).
6.3 Random Forest Classification
The model specification for random forest classification required two additional parameters: \(ntree\) (the number of decision trees to use) and \(mtry\) (the number of variables that are randomly selected at each split). \(ntree\) was set to 1,000, while \(mtry\) was set to 3 (the default value).
7 Results
The table below shows the sensitivity and specificity values for each model at each data level. The way the models are set up, sensitivity can be defined as the probability that a given model will correctly classify a new observation as “not an Opportunity Zone” (Opp.Zone == 0), and specificity can be defined as the probability that a given model will correctly classify a new observation as “belonging to an Opportunity Zone” (Opp.Zone == 1).
|
Level 1
|
Level 2
|
Level 3
|
||||
|---|---|---|---|---|---|---|
| Model | Sensitivity | Specificity | Sensitivity | Specificity | Sensitivity | Specificity |
| Logistic Regression | 0.9826 | 0.1306 | 0.9818 | 0.1248 | 0.9828 | 0.1341 |
| KNN Classification | 0.9841 | 0.1068 | 0.9903 | 0.1003 | 0.9898 | 0.1036 |
| Random Forest | 0.9605 | 0.1727 | 0.9910 | 0.1143 | 0.9905 | 0.1298 |
While nearly every model provided a high degree of sensitivity (\(> 0.98\)) at every level, the highest specificity achieved by any model was exceedingly low - \(~0.17\), achieved by the random forest model at the first data level.
While sensitivity usually rose from the first level to the third level, specificity fell (except in the logistic regression model, where there was a marginal increase). This suggests that model specifications lacked a balance between having a sufficient number of predictors and having good predictors.
8 Next Steps
Since each State was technically free to use their own criteria to nominate census tracts for QOZs, a per-state model, or a model that incorporates state-level fixed effects (such as a fixed effects logit model) might be able to achieve a higher degree of specificity.
Fine-tuning the list of predictors used would also be a significant improvement. There are a number of variable selection strategies that would likely improve the model specifications.
Shifting from 2015 ACS data to 2018 ACS data might be useful. Initially, data for the year 2015 was used due to it being the year the EIG introduced the concept of opportunity zones. However, since QOZs were implemented in 2018, using the “most recent” data might prove beneficial. However, it’s unlikely that there will be significant shifts in ACS data between 2015 and 2018.
9 Bibliography
10 Appendix
| Variable.Code.for.ACS.2015 | Variable.Description | Geography |
|---|---|---|
| B01001_001E | Total Population | Census Tract |
| B19013_001E | Median Household Income | Census Tract & State |
| C17002_001E | Income to Poverty Ratio: Universe | Census Tract |
| C17002_002E | Income to Poverty Ratio: Less than 0.50 | Census Tract |
| C17002_003E | Income to Poverty Ratio: Less than 0.99 | Census Tract |
| C17002_004E | Income to Poverty Ratio: Less than 1.25 | Census Tract |
| C17002_005E | Income to Poverty Ratio: Less than 1.50 | Census Tract |
| SVI.Domain | SVI.Indicator | Variable.Code.for.ACS.2015 | Variable.Description | Geography |
|---|---|---|---|---|
| NA | NA | B01001_001E | Total Population | Census Tract |
| NA | NA | B11005_001E | Total Households | Census Tract |
| NA | NA | B25106_001E | Occupied Housing Units | Census Tract |
| Socioeconomic Status | Below 150% Poverty | C17002_001E | Income to Povert Ratio: Universe | Census Tract |
| Socioeconomic Status | Below 150% Poverty | C17002_002E | Income to Poverty Ratio: Less than 0.50 | Census Tract |
| Socioeconomic Status | Below 150% Poverty | C17002_003E | Income to Poverty Ratio: Less than 0.99 | Census Tract |
| Socioeconomic Status | Below 150% Poverty | C17002_004E | Income to Poverty Ratio: Less than 1.25 | Census Tract |
| Socioeconomic Status | Below 150% Poverty | C17002_005E | Income to Poverty Ratio: Less than 1.50 | Census Tract |
| Socioeconomic Status | Unemployed | B23025_002E | Workers: Labor Force | Census Tract |
| Socioeconomic Status | Unemployed | B23025_005E | Workers: Unemployed | Census Tract |
| Socioeconomic Status | Housing Cost Burdened | B25106_006E | Housing Costs: Owner-occupied, income below $20,000, costs above 30% of income | Census Tract |
| Socioeconomic Status | Housing Cost Burdened | B25106_010E | Housing Costs: Owner-occupied, income below $35,000, costs above 30% of income | Census Tract |
| Socioeconomic Status | Housing Cost Burdened | B25106_014E | Housing Costs: Owner-occupied, income below $50,000, costs above 30% of income | Census Tract |
| Socioeconomic Status | Housing Cost Burdened | B25106_018E | Housing Costs: Owner-occupied, income below $75,000, costs above 30% of income | Census Tract |
| Socioeconomic Status | Housing Cost Burdened | B25106_028E | Housing Costs: Renter-occupied, income below $20,000, costs above 30% of income | Census Tract |
| Socioeconomic Status | Housing Cost Burdened | B25106_032E | Housing Costs: Renter-occupied, income below $35,000, costs above 30% of income | Census Tract |
| Socioeconomic Status | Housing Cost Burdened | B25106_036E | Housing Costs: Renter-occupied, income below $50,000, costs above 30% of income | Census Tract |
| Socioeconomic Status | Housing Cost Burdened | B25106_040E | Housing Costs: Renter-occupied, income below $75,000, costs above 30% of income | Census Tract |
| Socioeconomic Status | No High School Diploma | B06009_001E | Educational Attainment: Universe | Census Tract |
| Socioeconomic Status | No High School Diploma | B06009_002E | Educational Attainment: Less than High School | Census Tract |
| Socioeconomic Status | No Health Insurance | B27010_001E | Insurance Coverage: Universe | Census Tract |
| Socioeconomic Status | No Health Insurance | B27010_066E | Insurance Coverage: No Coverage | Census Tract |
| Household Characteristics | Aged 65 and Over | B09020_001E | Age: Over 65 | Census Tract |
| Household Characteristics | Aged 17 and Under | B09001_001E | Age: Under 18 | Census Tract |
| Household Characteristics | Civilian with a Disability | B18135_003E | Disability by Age: 18 and under | Census Tract |
| Household Characteristics | Civilian with a Disability | B18135_014E | Disability by Age: 19 to 64 | Census Tract |
| Household Characteristics | Civilian with a Disability | B18135_025E | Disability by Age: 65 and over | Census Tract |
| Household Characteristics | Single Parent Households | B11005_007E | Single Parent Households: Female Householder | Census Tract |
| Household Characteristics | Single Parent Households | B11005_006E | Single Parent Households: Male Householder | Census Tract |
| Household Characteristics | English Language Proficiency | B16002_004E | Limited English Proficiency: Spanish-speaking Household | Census Tract |
| Household Characteristics | English Language Proficiency | B16002_007E | Limited English Proficiency: Indo-European language-speaking Household | Census Tract |
| Household Characteristics | English Language Proficiency | B16002_010E | Limited English Proficiency: AAPI language-speaking Household | Census Tract |
| Household Characteristics | English Language Proficiency | B16002_013E | Limited English Proficienct: Other language-speaking Household | Census Tract |
| Racial & Ethnic Minority Status | Population not belonging to White Non-Hispanic group | B03002_003E | Race & Ethnicity: Population identifying as White alone, Non-Hispanic | Census Tract |
| Housing Type and Transportation | Multi-Unit Structures | B25024_001E | Units in Structure: Universe | Census Tract |
| Housing Type and Transportation | Multi-Unit Structures | B25024_007E | Units in Strucutre: 10 to 19 Units | Census Tract |
| Housing Type and Transportation | Multi-Unit Structures | B25024_008E | Units in Structure: 20 to 49 Units | Census Tract |
| Housing Type and Transportation | Multi-Unit Structures | B25024_009E | Units in Structure: 50 plus Units | Census Tract |
| Housing Type and Transportation | Mobile Homes | B25024_010E | Units in Structure: Mobile Homes | Census Tract |
| Housing Type and Transportation | Overcrowding | B25014_001E | Occupants per Room: Universe | Census Tract |
| Housing Type and Transportation | Overcrowding | B25014_005E | Occupants per Room: Owner-occupied, 1.0 to 1.49 Occupants per room | Census Tract |
| Housing Type and Transportation | Overcrowding | B25014_006E | Occupants per Room: Owner-occupied, 1.50 to 1.99 Occupants per room | Census Tract |
| Housing Type and Transportation | Overcrowding | B25014_007E | Occupants per Room: Owner-occupied, over 2.0 Occupants per room | Census Tract |
| Housing Type and Transportation | Overcrowding | B25014_011E | Occupants per Room: Renter-occupied, 1.0 to 1.49 Occupants per room | Census Tract |
| Housing Type and Transportation | Overcrowding | B25014_012E | Occupants per Room: Renter-occupied, 1.50 to 1.99 Occupants per room | Census Tract |
| Housing Type and Transportation | Overcrowding | B25014_013E | Occupants per Room: Renter-occupied, over 2.0 Occupants per room | Census Tract |
| Housing Type and Transportation | No Vehicle | B08014_001E | Vehicle Access: Universe | Census Tract |
| Housing Type and Transportation | No Vehicle | B08014_002E | Vehicle Access: No vehicles | Census Tract |
| Housing Type and Transportation | Group Quarters | B26001_001E | Population Living in Group Quarters: Universe | Census Tract |