HateCrimes
So now we know that there is possible bias in the dataset, what can we do with it?
library(tidyverse)## ── Attaching packages ─────────────────────────────────────── tidyverse 1.3.2 ──
## ✔ ggplot2 3.4.0 ✔ purrr 1.0.1
## ✔ tibble 3.1.8 ✔ dplyr 1.1.0
## ✔ tidyr 1.3.0 ✔ stringr 1.5.0
## ✔ readr 2.1.3 ✔ forcats 1.0.0
## ── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
## ✖ dplyr::filter() masks stats::filter()
## ✖ dplyr::lag() masks stats::lag()#tinytex::install_tinytex()
library(tinytex)setwd("D:/data/files")
hatecrimes <- read_csv("hateCrimes2010.csv")## Rows: 423 Columns: 44
## ── Column specification ────────────────────────────────────────────────────────
## Delimiter: ","
## chr (2): County, Crime Type
## dbl (42): Year, Anti-Male, Anti-Female, Anti-Transgender, Anti-Gender Identi...
##
## ℹ Use `spec()` to retrieve the full column specification for this data.
## ℹ Specify the column types or set `show_col_types = FALSE` to quiet this message.Clean up the data
names(hatecrimes) <- tolower(names(hatecrimes))
names(hatecrimes) <- gsub(" ","",names(hatecrimes))
str(hatecrimes)## spc_tbl_ [423 × 44] (S3: spec_tbl_df/tbl_df/tbl/data.frame)
## $ county : chr [1:423] "Albany" "Albany" "Allegany" "Bronx" ...
## $ year : num [1:423] 2016 2016 2016 2016 2016 ...
## $ crimetype : chr [1:423] "Crimes Against Persons" "Property Crimes" "Property Crimes" "Crimes Against Persons" ...
## $ anti-male : num [1:423] 0 0 0 0 0 0 0 0 0 0 ...
## $ anti-female : num [1:423] 0 0 0 0 0 0 0 0 0 0 ...
## $ anti-transgender : num [1:423] 0 0 0 4 0 0 0 0 0 0 ...
## $ anti-genderidentityexpression : num [1:423] 0 0 0 0 0 0 0 0 0 0 ...
## $ anti-age* : num [1:423] 0 0 0 0 0 0 0 0 0 0 ...
## $ anti-white : num [1:423] 0 0 0 1 1 0 0 0 0 0 ...
## $ anti-black : num [1:423] 1 2 1 0 0 1 0 1 0 2 ...
## $ anti-americanindian/alaskannative : num [1:423] 0 0 0 0 0 0 0 0 0 0 ...
## $ anti-asian : num [1:423] 0 0 0 0 0 1 0 0 0 0 ...
## $ anti-nativehawaiian/pacificislander : num [1:423] 0 0 0 0 0 0 0 0 0 0 ...
## $ anti-multi-racialgroups : num [1:423] 0 0 0 0 0 0 0 0 0 0 ...
## $ anti-otherrace : num [1:423] 0 0 0 0 0 0 0 0 0 0 ...
## $ anti-jewish : num [1:423] 0 0 0 0 1 0 1 0 0 0 ...
## $ anti-catholic : num [1:423] 0 0 0 0 0 0 0 0 0 0 ...
## $ anti-protestant : num [1:423] 0 0 0 0 0 0 0 0 0 0 ...
## $ anti-islamic(muslim) : num [1:423] 1 0 0 6 0 0 0 0 1 0 ...
## $ anti-multi-religiousgroups : num [1:423] 0 1 0 0 0 0 0 0 0 0 ...
## $ anti-atheism/agnosticism : num [1:423] 0 0 0 0 0 0 0 0 0 0 ...
## $ anti-religiouspracticegenerally : num [1:423] 0 0 0 0 0 0 0 0 0 0 ...
## $ anti-otherreligion : num [1:423] 0 0 0 0 0 0 0 0 0 0 ...
## $ anti-buddhist : num [1:423] 0 0 0 0 0 0 0 0 0 0 ...
## $ anti-easternorthodox(greek,russian,etc.): num [1:423] 0 0 0 0 0 0 0 0 0 0 ...
## $ anti-hindu : num [1:423] 0 0 0 0 0 0 0 0 0 0 ...
## $ anti-jehovahswitness : num [1:423] 0 0 0 0 0 0 0 0 0 0 ...
## $ anti-mormon : num [1:423] 0 0 0 0 0 0 0 0 0 0 ...
## $ anti-otherchristian : num [1:423] 0 0 0 0 0 0 0 0 0 0 ...
## $ anti-sikh : num [1:423] 0 0 0 0 0 0 0 0 0 0 ...
## $ anti-hispanic : num [1:423] 0 0 0 0 0 0 0 0 0 0 ...
## $ anti-arab : num [1:423] 0 0 0 0 0 0 0 0 0 0 ...
## $ anti-otherethnicity/nationalorigin : num [1:423] 0 0 0 0 0 0 0 0 0 0 ...
## $ anti-non-hispanic* : num [1:423] 0 0 0 0 0 0 0 0 0 0 ...
## $ anti-gaymale : num [1:423] 1 0 0 8 0 1 0 0 0 0 ...
## $ anti-gayfemale : num [1:423] 0 0 0 1 0 0 0 0 0 0 ...
## $ anti-gay(maleandfemale) : num [1:423] 0 0 0 0 0 0 0 0 0 0 ...
## $ anti-heterosexual : num [1:423] 0 0 0 0 0 0 0 0 0 0 ...
## $ anti-bisexual : num [1:423] 0 0 0 0 0 0 0 0 0 0 ...
## $ anti-physicaldisability : num [1:423] 0 0 0 0 0 0 0 0 0 0 ...
## $ anti-mentaldisability : num [1:423] 0 0 0 0 0 0 0 0 0 0 ...
## $ totalincidents : num [1:423] 3 3 1 20 2 3 1 1 1 2 ...
## $ totalvictims : num [1:423] 4 3 1 20 2 3 1 1 1 2 ...
## $ totaloffenders : num [1:423] 3 3 1 25 2 3 1 1 1 2 ...
## - attr(*, "spec")=
## .. cols(
## .. County = col_character(),
## .. Year = col_double(),
## .. `Crime Type` = col_character(),
## .. `Anti-Male` = col_double(),
## .. `Anti-Female` = col_double(),
## .. `Anti-Transgender` = col_double(),
## .. `Anti-Gender Identity Expression` = col_double(),
## .. `Anti-Age*` = col_double(),
## .. `Anti-White` = col_double(),
## .. `Anti-Black` = col_double(),
## .. `Anti-American Indian/Alaskan Native` = col_double(),
## .. `Anti-Asian` = col_double(),
## .. `Anti-Native Hawaiian/Pacific Islander` = col_double(),
## .. `Anti-Multi-Racial Groups` = col_double(),
## .. `Anti-Other Race` = col_double(),
## .. `Anti-Jewish` = col_double(),
## .. `Anti-Catholic` = col_double(),
## .. `Anti-Protestant` = col_double(),
## .. `Anti-Islamic (Muslim)` = col_double(),
## .. `Anti-Multi-Religious Groups` = col_double(),
## .. `Anti-Atheism/Agnosticism` = col_double(),
## .. `Anti-Religious Practice Generally` = col_double(),
## .. `Anti-Other Religion` = col_double(),
## .. `Anti-Buddhist` = col_double(),
## .. `Anti-Eastern Orthodox (Greek, Russian, etc.)` = col_double(),
## .. `Anti-Hindu` = col_double(),
## .. `Anti-Jehovahs Witness` = col_double(),
## .. `Anti-Mormon` = col_double(),
## .. `Anti-Other Christian` = col_double(),
## .. `Anti-Sikh` = col_double(),
## .. `Anti-Hispanic` = col_double(),
## .. `Anti-Arab` = col_double(),
## .. `Anti-Other Ethnicity/National Origin` = col_double(),
## .. `Anti-Non-Hispanic*` = col_double(),
## .. `Anti-Gay Male` = col_double(),
## .. `Anti-Gay Female` = col_double(),
## .. `Anti-Gay (Male and Female)` = col_double(),
## .. `Anti-Heterosexual` = col_double(),
## .. `Anti-Bisexual` = col_double(),
## .. `Anti-Physical Disability` = col_double(),
## .. `Anti-Mental Disability` = col_double(),
## .. `Total Incidents` = col_double(),
## .. `Total Victims` = col_double(),
## .. `Total Offenders` = col_double()
## .. )
## - attr(*, "problems")=<externalptr>Select only certain hate-crimes
hatecrimes2 <- hatecrimes %>%
select(county, year, 'anti-black', 'anti-white', 'anti-jewish', 'anti-catholic','anti-age*','anti-islamic(muslim)', 'anti-gaymale', 'anti-hispanic') %>%
group_by(county, year)
head(hatecrimes2)Check the dimensions and the summary to make sure no missing values
dim(hatecrimes2)## [1] 423 10# There are currently 13 variables with 423 rows.
summary(hatecrimes2)## county year anti-black anti-white
## Length:423 Min. :2010 Min. : 0.000 Min. : 0.0000
## Class :character 1st Qu.:2011 1st Qu.: 0.000 1st Qu.: 0.0000
## Mode :character Median :2013 Median : 1.000 Median : 0.0000
## Mean :2013 Mean : 1.761 Mean : 0.3357
## 3rd Qu.:2015 3rd Qu.: 2.000 3rd Qu.: 0.0000
## Max. :2016 Max. :18.000 Max. :11.0000
## anti-jewish anti-catholic anti-age* anti-islamic(muslim)
## Min. : 0.000 Min. : 0.0000 Min. :0.00000 Min. : 0.0000
## 1st Qu.: 0.000 1st Qu.: 0.0000 1st Qu.:0.00000 1st Qu.: 0.0000
## Median : 0.000 Median : 0.0000 Median :0.00000 Median : 0.0000
## Mean : 3.981 Mean : 0.2695 Mean :0.05201 Mean : 0.4704
## 3rd Qu.: 3.000 3rd Qu.: 0.0000 3rd Qu.:0.00000 3rd Qu.: 0.0000
## Max. :82.000 Max. :12.0000 Max. :9.00000 Max. :10.0000
## anti-gaymale anti-hispanic
## Min. : 0.000 Min. : 0.0000
## 1st Qu.: 0.000 1st Qu.: 0.0000
## Median : 0.000 Median : 0.0000
## Mean : 1.499 Mean : 0.3735
## 3rd Qu.: 1.000 3rd Qu.: 0.0000
## Max. :36.000 Max. :17.0000Use Facet_Wrap
hatecrimeslong <- hatecrimes2 %>%
tidyr::gather("id", "crimecount", 3:10)
hatecrimesplot <-hatecrimeslong %>%
ggplot(., aes(year, crimecount))+
geom_point()+
aes(color = id)+
facet_wrap(~id)
hatecrimesplot
Look deeper into crimes against blacks, gay males, and jews
hatenew <- hatecrimeslong %>%
filter( id== "anti-black" | id == "anti-jewish" | id == "anti-gaymale")%>%
group_by(year, county) %>%
arrange(desc(crimecount))
hatenewPlot these three types of hate crimes together
plot2 <- hatenew %>%
ggplot() +
geom_bar(aes(x=year, y=crimecount, fill = id),
position = "dodge", stat = "identity") +
ggtitle("Hate Crime Type in NY Counties Between 2010-2016") +
ylab("Number of Hate Crime Incidents") +
labs(fill = "Hate Crime Type")
plot2
What about the counties?
plot3 <- hatenew %>%
ggplot() +
geom_bar(aes(x=county, y=crimecount, fill = id),
position = "dodge", stat = "identity") +
ggtitle("Hate Crime Type in NY Counties Between 2010-2016") +
ylab("Number of Hate Crime Incidents") +
labs(fill = "Hate Crime Type")
plot3
So many counties
counties <- hatenew %>%
group_by(county, year)%>%
summarize(sum = sum(crimecount)) %>%
arrange(desc(sum)) ## `summarise()` has grouped output by 'county'. You can override using the
## `.groups` argument.countiesplot4 <- hatenew %>%
filter(county =="Kings" | county =="New York" | county == "Suffolk" | county == "Nassau" | county == "Queens") %>%
ggplot() +
geom_bar(aes(x=county, y=crimecount, fill = id),
position = "dodge", stat = "identity") +
labs(ylab = "Number of Hate Crime Incidents",
title = "5 Counties in NY with Highest Incidents of Hate Crimes",
subtitle = "Between 2010-2016",
fill = "Hate Crime Type")
plot4
How would calculations be affected by looking at hate crimes in counties per year by population densities?
nypop <- read_csv("newyorkpopulation.csv")## Rows: 62 Columns: 8
## ── Column specification ────────────────────────────────────────────────────────
## Delimiter: ","
## chr (1): Geography
## dbl (7): 2010, 2011, 2012, 2013, 2014, 2015, 2016
##
## ℹ Use `spec()` to retrieve the full column specification for this data.
## ℹ Specify the column types or set `show_col_types = FALSE` to quiet this message.Clean the county name to match the other dataset
nypop$Geography <- gsub(" , New York", "", nypop$Geography)
nypop$Geography <- gsub("County", "", nypop$Geography)
nypoplong <- nypop %>%
rename(county = Geography) %>%
gather("year", "population", 2:8)
nypoplong$year <- as.double(nypoplong$year)
head(nypoplong)Focus on 2012
nypoplong12 <- nypoplong %>%
filter(year == 2012) %>%
arrange(desc(population)) %>%
head(10)
nypoplong12$county<-gsub(" , New York","",nypoplong12$county)
nypoplong12Filter hate crimes just for 2012 as well
counties12 <- counties %>%
filter(year == 2012) %>%
arrange(desc(sum))
counties12Join the Hate Crimes data with NY population data for 2012
datajoin <- counties12 %>%
full_join(nypoplong12, by=c("county", "year"))
datajoinCalculate the rate of incidents per 100,000. Then arrange in descending order
datajoinrate <- datajoin %>%
mutate(rate = sum/population*100000) %>%
arrange(desc(rate))
datajoinratedt <- datajoinrate[,c("county","rate")]
dtFollow Up
Aggregating some of the categories
aggregategroups <- hatecrimes %>%
tidyr::gather("id", "crimecount", 4:44)
unique(aggregategroups$id)## [1] "anti-male"
## [2] "anti-female"
## [3] "anti-transgender"
## [4] "anti-genderidentityexpression"
## [5] "anti-age*"
## [6] "anti-white"
## [7] "anti-black"
## [8] "anti-americanindian/alaskannative"
## [9] "anti-asian"
## [10] "anti-nativehawaiian/pacificislander"
## [11] "anti-multi-racialgroups"
## [12] "anti-otherrace"
## [13] "anti-jewish"
## [14] "anti-catholic"
## [15] "anti-protestant"
## [16] "anti-islamic(muslim)"
## [17] "anti-multi-religiousgroups"
## [18] "anti-atheism/agnosticism"
## [19] "anti-religiouspracticegenerally"
## [20] "anti-otherreligion"
## [21] "anti-buddhist"
## [22] "anti-easternorthodox(greek,russian,etc.)"
## [23] "anti-hindu"
## [24] "anti-jehovahswitness"
## [25] "anti-mormon"
## [26] "anti-otherchristian"
## [27] "anti-sikh"
## [28] "anti-hispanic"
## [29] "anti-arab"
## [30] "anti-otherethnicity/nationalorigin"
## [31] "anti-non-hispanic*"
## [32] "anti-gaymale"
## [33] "anti-gayfemale"
## [34] "anti-gay(maleandfemale)"
## [35] "anti-heterosexual"
## [36] "anti-bisexual"
## [37] "anti-physicaldisability"
## [38] "anti-mentaldisability"
## [39] "totalincidents"
## [40] "totalvictims"
## [41] "totaloffenders"aggregategroups <- aggregategroups %>%
mutate(group = case_when(
id %in% c("anti-transgender", "anti-gayfemale", "anti-genderidendityexpression", "anti-gaymale", "anti-gay(maleandfemale", "anti-bisexual") ~ "anti-lgbtq",
id %in% c("anti-multi-racialgroups", "anti-jewish", "anti-protestant", "anti-multi-religousgroups", "anti-religiouspracticegenerally", "anti-buddhist", "anti-hindu", "anti-mormon", "anti-sikh", "anti-catholic", "anti-islamic(muslim)", "anti-atheism/agnosticism", "anti-otherreligion", "anti-easternorthodox(greek,russian,etc.)", "anti-jehovahswitness", "anti-otherchristian") ~ "anti-religion",
id %in% c("anti-asian", "anti-arab", "anti-non-hispanic", "anti-white", "anti-americanindian/alaskannative", "anti-nativehawaiian/pacificislander", "anti-otherrace", "anti-hispanic", "anti-otherethnicity/nationalorigin") ~ "anti-ethnicity",
id %in% c("anti-physicaldisability", "anti-mentaldisability") ~ "anti-disability",
id %in% c("anti-female", "anti-male") ~ "anti-gender",
TRUE ~ "others"))
aggregategroupsor create subset with just lgbtq
lgbtq <- hatecrimes %>%
tidyr::gather("id", "crimecount", 4:44) %>%
filter(id %in% c("anti-transgender", "anti-gayfemale", "anti-genderidendityexpression", "anti-gaymale", "anti-gay(maleandfemale", "anti-bisexual"))
lgbtqEassy
Based on the preliminary analysis and understanding of the entire dataset and the results of the above R output, I think the strengths and weaknesses of the dataset are roughly as follows.First of all, from the data point of view, the data volume of this dataset is sufficient and contains a lot of information, which has high value of data mining. However, on the other hand, the dataset also has more redundant data, and the purity of information is not high enough, so it needs to spend more data operation means to refine and normalize the data. In terms of the practical significance behind the data, the social and criminal issues that the data set focuses on have strong social guiding significance. By analyzing the data set, we can help judge the occurrence of hate crime from multiple perspectives, such as geographical factors and gender factors, so as to better solve the continuing deterioration of the problem.In addition, the data set contains a large number of dummy variables, which is not conducive to the construction of the linear model, and is easy to cause problems such as the collinearity of the linear model.
Further, if I analyze the data set, I may conduct further analysis from the following two possible ways. First of all, the first way is to collect more granular data from regions such as King, so as to further explore the reasons for the differences in hate crimes in different regions and the differences of hate crimes in different regions. Secondly, it is necessary to analyze in more detail the changes of hate crimes in a wider range of time, as well as the nodes of important time changes, so as to analyze whether the introduction of policies such as the hate crimes act has a lasting and profound impact on hate crimes in time.
As for the above output results, I think the two important aspects that can continue are the reasons for the differences between hate crimes against Jews and hate crimes against LGBT groups. As can be seen from the above results, hate crimes against black groups have gradually decreased with the development of time, but hate crimes against Jews have a slight upward trend, so whether there is any hidden reason behind this cause is worth exploring. On the other hand, with the gradual increase of social tolerance, the tolerance of LGBT groups has also increased significantly. What groups are the main perpetrators of hate crimes against LGBT groups and what are the reasons for them to commit hate crimes? These are all worthy of further exploration.