NYS Policing Project
ARiefer, KOliverio
3/18/2021
Background:
In the last few years, videos of traffic stops have created a national debate about the way law enforcement treats minorities (4). Cases have spurred calls for more reliable information; from police video to data logged every time someone is pulled over, since traffic stops are one of the most common ways members of the public interact with the police (4). The Fourth Amendment requires that before stopping the suspect, the police must have a reasonable suspicion that a crime has been, is being, or is about to be committed by the suspect. If the police reasonably suspect that the suspect is armed and dangerous, the police may stop the suspect. A reasonable stop is one “in which a reasonably prudent officer is warranted in the circumstances of a given case in believing that his safety or that of others is endangered, he may make a reasonable search for weapons of the person believed by him to be armed and dangerous.” Stops fall under criminal law, as opposed to civil law (1). Our data is taken from The Stanford Open Policing Project which is a unique partnership between the Stanford Computational Journalism Lab and the Stanford Computational Policy Lab. Starting in 2015, the Open Policing Project began requesting such data from state after state. To date, the project has collected and standardized over 200 million records of traffic stop and search data from across the country. Some states do not collect the demographic information of the drivers that police pull over. States that do collect the information do not always release the data. Even when states do provide the information, the way they track and then process the data varies widely across the country, creating challenges for standardizing the information. Data from 21 state patrol agencies and 29 municipal police departments, comprising nearly 100 million traffic stops, are sufficiently detailed to facilitate rigorous statistical analysis. The project has found significant racial disparities in policing. These disparities can occur for many reasons: differences in driving behavior, to name one. But, in some cases, we find evidence that bias also plays a role.
Benchmark analysis is the most common statistical method for assessing racial bias in police stops and searches. The key methodological challenge with this approach is estimating the race distribution of the at-risk, or benchmark, population. Traditional benchmarks include the residential population, licensed drivers, arrestees, and reported crime suspects [Engel and Calnon (2004)]. McConnell and Scheidegger (2001) have looked at stops initiated by aerial patrols and those based on radar and cameras [Lange, Blackman and Johnson (2001)], arguing that such stops are less prone to potential bias, and thus more likely to reflect the true population of traffic violators. Gelman, Fagan and Kiss (2007) use a hierarchical Bayesian model to construct a benchmark based on neighborhood and race specific crime rates. Ridgeway (2006) studies post-stop police actions by creating benchmarks based on propensity scores, with minority and white drivers matched using demographics and the time, location, and purpose of the stops. Grogger and Ridgeway (2006) construct benchmarks by considering stops at night, when a “veil of darkness” masks race. Antonovics and Knight (2009) use officer-level demographics in a variation of the standard benchmark test: they argue that search rates that are higher when the officer’s race differs from that of the suspect is evidence of discrimination. Finally, “internal benchmarks” have been used to flag potentially biased officers by comparing each officer’s stop decisions to those made by others patrolling the same area at the same time [Ridgeway and MacDonald (2009), Walker (2003)].
In benchmarking, one compares the rate at which whites and minorities are treated favorably. For example, in the case of stopping decisions, if white drivers are stopped less often than non-white drivers, that may be the result of bias against minorities. However, if minorities in reality are less creditworthy, possibly due to more minority population than whites, then such disparities in lending rates (stop-and-frisk) may simply reflect reasonable practices rather than discrimination. This limitation of benchmarking is referred to in the literature as the qualified pool or denominator problem [Ayres (2002)] and is a specific instance of omitted variable bias. Ideally, one would like to compare similarly qualified white and minority applicants, but such a comparison requires detailed individual-level data and is often infeasible to carry out in practice.
Methods:
We used a popularly used statistical methods, i.e., benchmarking, to analyze the data for detecting disparities in policing decisions, specifically, a benchmark comparison for stop rates. We use descriptive and inferential statistics to explore and analyze the New York State Patrol data, along with quantitative observations about the proportion of stops for each race. We define proportion of stops to be the number of stops within a specific race divided by the total number of stops. We are using the population of each race as our baseline, defining stop rate to be the proportion of stopped drivers of a specific race. The proportion is defined by the number of stops for a specific race divided by the population of that specific race. Furthermore, we made quantitative claims about disparities in stop rates by dividing the white stop rate by each of the other stop rates for each specific race. Lastly, we computed Cohen’s H value which measures the distance between the proportions of stops for drivers of each specific race compared to proportion of stops of White drivers. We use Cohen’s H value to describe the difference between the two proportions as either small, medium, or large. Thus, Cohen’s H value allows us to determine if the difference between the two proportions is meaningful if the difference is large.
The Data:
We consider a comprehensive dataset of 7.96 million traffic stops conducted in the state of New York between January 2010 and December 2017 that was obtained through the Stanford Open Policing Project website (2). Several variables are recorded for each stop, including the location of the stop, the race of the driver (White, Black, Hispanic, Asian/Pacific Islander, Other, or NA), the subject’s sex (Male or Female), the subject’s age, the violation, the county the stop happened in, and several characteristics of the vehicle that was being driven, including vehicle make, model, color, and type. Our data set does not contain data on whether a search was conducted, if a frisk was performed, if an arrest was made, nor a reason for the stop. In our primary analysis, we include all stops. We have acquired population data of New York State from the Census Bureau website (5). This data gives us the population of each race within New York State during 2017. We also acquired the population data of Monroe County, New York from the Census Bureau website as well (6). This data gives us the population of each race within Monroe County, New York during 2017.
Results:
There are 63 counties in our dataset (62 counties and one NA “county”). We find that the number of stops per year from 2010 to 2017 has stayed relatively the same. The year 2010 consisted of the highest number of stops with 1,120,192 stops.
Table 1:
| Year | Number of Stops |
|---|---|
| 2010 | 1120192 |
| 2011 | 994119 |
| 2012 | 943322 |
| 2013 | 955896 |
| 2014 | 994232 |
| 2015 | 982626 |
| 2016 | 984923 |
| 2017 | 986859 |
The NYS Patrol data consists of 7,962,169 number of stops from 2010-2017, with an average number of stops per year around 995, 271 stops. We then look at the number of stops per race to conclude if there is any specific race being pulled over consistently more than the others.
Table 2:
| Race | Number of Stopped Drivers | Proportion of Stopped Drivers |
|---|---|---|
| asian/pacific islander | 278075 | 0.0349245 |
| black | 888696 | 0.1116148 |
| hispanic | 553552 | 0.0695228 |
| other | 315590 | 0.0396362 |
| white | 5926237 | 0.7442993 |
| NA | 19 | 0.0000024 |
Among this set of stops, we find that 74.43% of drivers are white, 11.16% are black, 6.95% are Hispanic, 3.49% are Asian/Pacific Islander, and 3.96% are unknown/other. Notice how the highest percentage of drivers being stopped are of White drivers and the second highest percentage of drivers being stopped is of Black drivers.
Table 3:
We computed Cohen’s H value which measures the distance between the proportions of stops for drivers of each specific race compared to proportion of stops of White drivers.| Cohen’s H Value | |
|---|---|
| White vs. Non-White | 1.02 |
| White vs. Asian/Pacific Islander | 1.71 |
| White vs. Black | 1.40 |
| White vs. Hispanic | 1.55 |
| White vs. Other | 1.68 |
| White vs. NA | 2.08 |
Here, we find that when each race is compared to the White race, the Cohen’s H value is above 1. Since each comparison is larger than 1, we can conclude that each difference in proportions of stops between Whites and every other race is large. Thus, each difference has a large effect size. Hence, we can conclude that each difference in proportions of stops between races is meaningful, and therefore statistically significant.
Table 4:
| Year | Number of Stops |
|---|---|
| 2010 | 47604 |
| 2011 | 42504 |
| 2012 | 42244 |
| 2013 | 41063 |
| 2014 | 48664 |
| 2015 | 43416 |
| 2016 | 38867 |
| 2017 | 32517 |
We further restrict our stops data set to look at Monroe County specifically. We find that the average of stops per year from 2010-2017 in Monroe County is 42,109 stops per year. We will use these numbers for our primary analysis for Monroe County.
Table 5:
| Subject Race | Number of Stopped Drivers | Proportion of Stopped Drivers |
|---|---|---|
| asian/pacific islander | 9696 | 0.0287818 |
| black | 88083 | 0.2614678 |
| hispanic | 27444 | 0.0814655 |
| other | 3744 | 0.0111138 |
| white | 207912 | 0.6171712 |
Among this set of stops for Monroe County, we find that 61.72% of drivers are white, 26.15% are black, 8.15% are Hispanic, 2.88% are Asian/Pacific Islander, and 1.11% are unknown/other. Notice how the results for Monroe County are the same as they were for NYS. The highest percentage of stopped drivers are of White drivers and the second highest percentage of stopped drivers are of Black drivers. We then want to graphically look at the number of stops per year subjected by race to see the trend of stops over time for each race.
Figure 1:
Graphically, we can see that the number of stops of White drivers is extremely higher than stops of any other race throughout the years. From this plot we see that, at least for Black and White drivers, the annual trends are very different by race. It is difficult to tell from this plot for drivers of other races because the counts are comparatively so much smaller. All races experienced a spike in stops in 2014, but thereafter, there were fewer White drivers stopped from 2015-2017, whereas there continued to be an increase in the number of Black and Hispanic drivers stopped over those two years.
This is already a potential lead. We investigated further to see what the trend looks like when adjusting for gender within each specific race. Thus, we can see if there was a difference between male drivers and female drivers within each specific race over time.
Figure 2:
Figure 2 depicts the same trends for both male and female drivers for each race over the years from 2010-2017. Regardless of gender, White drivers seem to be stopped at a much higher rate than compared to any other race. However, notice how in the graph of male drivers, the number of male drivers being stopped is drastically higher than the number of female drivers being stopped, regardless of race. This could be due to there being more males in NYS than females, more males driving than females, etc.
Next, we look at the number of stops for each race over time specifically for Monroe County.
Figure 3:
Graphically, we can see that in Monroe County, the number of stops of White and Black drivers is extremely higher than stops of drivers of any other race throughout the years. From this plot we see that, at least for Black and White drivers, the annual trends are very different by race. It is difficult to tell from this plot for drivers of other races because the counts are drastically smaller. Note how all races experienced a spike in stops in 2013, but then a drop in stops starting in 2014. Thereafter, there were fewer drivers stopped from 2014-2017 for all races except there continued to be an increase in the number of Hispanic drivers stopped within the last year.
In order to conclude any racial discrimination of police stops, we need to conduct a benchmark analysis. With a benchmark analysis, we are able to compare the number of stops of each race with the population of that race as a whole. We are now able to determine if any specific race is overrepresented in the number of stops compared to its respective population. First, we retrieve the population numbers of each specific race within NYS.
Table 6:
| Race | Population Number of People | Proportion of Population |
|---|---|---|
| Asian/Pacific Islander | 1770274 | 0.0866667 |
| Black | 3423826 | 0.1676190 |
| Hispanic | 3754537 | 0.1838095 |
| Other/Unknown | 719781 | 0.0352381 |
| White | 10757819 | 0.5266667 |
Among the population data for each race in NYS, we find that Whites make up 52.67% of the population while all other races make up far less of the population. We find that NYS population consists of 8.67% Asian/Pacific Islander, 16.76% Black, 18.38% Hispanic, and 3.52% other/unknown. Notice how NYS population consists of mostly people who are White and then the second largest proportion are of people who are Black.
We then want to investigate the stop rates for each race to determine if there is any racial bias within police stops. `
Table 7:
| Race | Number of Stops | Population Number of People | Stop Rate |
|---|---|---|---|
| asian/pacific islander | 278075 | NA | NA |
| black | 888696 | NA | NA |
| hispanic | 553552 | NA | NA |
| other | 315590 | NA | NA |
| white | 5926237 | NA | NA |
| NA | 19 | NA | NA |
By conducting a benchmark analysis and comparing the number of stops for each race with the population of each race in NYS, we can determine if a certain race is overrepresented. Therefore, proving the existence of racial bias within police stops. From Table 6, we find that the highest stop rate in NYS is of White drivers who are stopped 55.10% of the time. The next highest percentage of stops are of Black drivers who are stopped 26% of the time. Here, we can conclude that White drivers are stopped the most out of any race, however, this is relative to such a high population of Whites within NYS. We see that White drivers are stopped at a rate 2.12 times higher than Black drivers and 3.74 times higher than Hispanic drivers.
Table 8:
| Race | Population Number of People | Proportion Population of People |
|---|---|---|
| Asian/Pacific Islander | 28187 | 0.0371092 |
| Black | 120166 | 0.1582029 |
| Hispanic | 68242 | 0.0898431 |
| Other/Unknown | 22994 | 0.0302724 |
| White | 519980 | 0.6845724 |
Among the population data for each race in Monroe County, we find that Whites make up 68.46% of the population while all other races make up far less of the population. We find that Monroe County population consists of 3.71% Asian/Pacific Islander, 15.82% Black, 8.98% Hispanic, and 3.02% other/unknown. Notice how NYS population consists of mostly people who are White and then the second largest proportion are of people who are Black, just like NYS.
Table 9:
| Race | Number of Stops | Population Number of People | Stop Rate |
|---|---|---|---|
| asian/pacific islander | 9696 | NA | NA |
| black | 88083 | NA | NA |
| hispanic | 27444 | NA | NA |
| other | 3744 | NA | NA |
| white | 207912 | NA | NA |
From Table 8, we find that we have different results than of NYS. We see that the highest stop rate in Monroe County is of Black drivers who are stopped 73.30 % of the time. We see the next highest percentage of stops are of Hispanic drivers who are stopped 40.22% of the time. Here, we can conclude that Black and Hispanic drivers are stopped the most out of any race. We see that Black drivers are stopped at a rate 1.83 times higher than White drivers. We also find that Hispanic drivers are stopped at a rate 1.01 times higher than White drivers.
Benchmark Results:
We saw before that about 11.16% of stops in NYS were of Black drivers. However, the NYS population was about 16.76% Black and having a stop rate of .26. Due to this difference in population size, 11.16% of stops being of Black drivers is not all that surprising. We also saw that 6.95% of stops in NYS were of Hispanic drivers, while the NYS population was about 18.38% Hispanic and having a stop rate of only .1474. Thus, 6.95% of stops being of Hispanic drivers is not that surprising as well. Lastly, we saw that 74.43% of stops in NYS were of White drivers. It is important to note that the NYS population was about 52.67% White and having a stop rate of .5509. Therefore, 74.43% of stops being of White drivers also makes sense. White drivers are stopped at a rate 2.12 times higher than Black drivers and 3.74 times higher than Hispanic drivers. Here, we see that the number of stops for each specific race relates to the population percentage of each specific race. Thus, there is no racial discrimination of police stops within NYS.
For Monroe County, we retrieve different results. We saw before that about 26.15% of total stops in Monroe County were of Black drivers. However, the Monroe County population was about 15.82% Black but has a stop rate of .733. Now note that 26.15% of stops being of Black drivers is very surprising since Black drivers only make up 15.82% of the population but are being stopped at a very high rate of 0.733. We also saw that 8.15% of stops in Monroe County were of Hispanic drivers, but the Monroe County population was about 8.98% Hispanic and has a stop rate of .4022. Thus, 8.15% of stops being of Hispanic drivers is very shocking as well, considering that Hispanic drivers only make up 8.98% of the Monroe County population. However, Hispanic drivers are being stopped at a somewhat high rate of 0.4022. Lastly, we saw that 61.72% of stops in Monroe County were of White drivers. It is important to note that the Monroe County population was about 68.46% White and has a stop rate of only .3998. Therefore, 61.72% of stops being of White drivers is somewhat surprising. White drivers make up quite a big percentage of the population in Monroe County; but White drivers are only stopped at a rate of .3998. In Monroe County, Black drivers are stopped at a rate 1.83 times higher than White drivers and Hispanic drivers are stopped at a rate 1.01 times higher than White drivers. Here, we see that even though Whites make up most of the population and stops in Monroe County, we find that Black and Hispanic drivers are stopped at a rate way higher than Whites. Thus, there is evidence of racial discrimination of police stops within Monroe County.
Discussion:
While these baseline stats give us a sense that there are racial disparities in policing practices in NYS, but they are not evidence of discrimination. The argument against the benchmark test is that we haven’t identified the correct baseline to compare to.
Now, these same baseline stats showed us that there are racial disparities in policing practices in Monroe County, and they are evidence of discrimination. In Monroe County, Black and Hispanic drivers were overrepresented and disproportionately stopped relative to their population numbers. Specifically, the population of White drivers was 52.64% higher than Black drivers and 59.48% higher than Hispanic drivers, making it obvious that with stop rates being almost double for Black drivers, racial bias is present in Monroe County.
Limitations:
We ran into a few limitations within our data sets. Within the NYS Patrol data, not all the variables were collected, i.e. search conducted, contraband found, citation issued, warning issued, frisk performed, arrest made, and reason for stop all were missing from our database, which includes Monroe County. The most important variables for our exploration of racial disparities were, search conducted, and frisk performed. These variables would have given us a better understanding of what exactly these drivers are being stopped for and if/how they are being racially discriminated against. Hence our investigation into the Stop-and-Frisk data, was just narrowly focused onto stops due to missing information. Thus, there could be the issue of racial discrimination within these factors but since we’re not given the data, we could not perform any test.
Outcome tests, however, are known to suffer from the problem of infra-marginality: even absent discrimination, the repayment rates for minority and white loan recipients might differ if the two groups have different risk distributions. Thus, at least in theory, outcome tests can fail to accurately detect discrimination.
Nevertheless, despite these shortcomings, these tests provide a useful if imperfect measure of bias in stop decisions.