Hotspot Analysis Using Chicago crime data

Introduction

Policing in recent times in regarded as a multi-dimensional problem, where law enforcement agencies initiate efforts to utilize scarce resources to minimize crime occurrence across various geographies.This study tend to utilize Chicago crime data set which is publicly made available via criminal data of Chicago. The study employs various statistical techniques to visual occurrence of crime which include looking at the data set from spatial point of view. The primary interest in this study is to identify hot spots areas of assault crime that happened within the period of Aug 1st 2020- Aug 1st 2021.

Required Libraries

library(sp)
library(spdep)
library(tidyr)
library(tidyverse)
library(lubridate)
library(ggplot2)
library(dplyr)
library(plotly)
library(TSstudio)
library(forecast)
library(MLmetrics)
library(viridisLite)
library(viridis)
library(padr)
library(rmdformats)
library(recipes)
library(rgdal)
library(raster)
library(rgeos)

Data Extraction For the purpose of this study , we utilize the crime data for the city of Chicago which is publicly available on the city’s open data portal. The dataset is loaded using the R statistical software. —

setwd("C:/Users/charl/OneDrive/Desktop/Data Science Projects")

Warning: The working directory was changed to C:/Users/charl/OneDrive/Desktop/Data Science Projects inside a notebook chunk. The working directory will be reset when the chunk is finished running. Use the knitr root.dir option in the setup chunk to change the working directory for notebook chunks.

library(readr)
Threeyears_18_19_20_21Aug_1 <- read_csv("R Projects/Threeyears_18_19_20_21Aug-1.csv")

Rows: 700810 Columns: 30── Column specification ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Delimiter: ","
chr   (9): Case.Number, Block, IUCR, Primary.Type, Description, Location.Description, FBI.Code, Updated.On, Location
dbl  (18): ï..ID, Beat, District, Ward, Community.Area, X.Coordinate, Y.Coordinate, Year, Latitude, Longitude, Historical.Wards.2003.2015, Zip.Codes, Com...
lgl   (2): Arrest, Domestic
dttm  (1): Date
ℹ Use `spec()` to retrieve the full column specification for this data.
ℹ Specify the column types or set `show_col_types = FALSE` to quiet this message.

#View(Threeyears_18_19_20_21Aug_1)
data<-Threeyears_18_19_20_21Aug_1

Data Description

The data for this study as labelled Crime data is from the Chicago Data Portal. For the purpose of this study, the initial dataset employed in R statistical software is from 2018-2021 which consisted of 700810 observations and 30 columns. However a subset dataset from Aug 01 2020 to Aug 01 2021 is taken for the final analysis on the new crime data. The new subset data however consisted of 200810 observations and 30 variables.The dataset however consisted of both qualitative and quantitative variables.Below shows all the process enaged in R to describe the dataset and subset the new data from the required dataset for further analysis.

names(data)

 [1] "ï..ID"                      "Case.Number"                "Date"                       "Block"                      "IUCR"                      
 [6] "Primary.Type"               "Description"                "Location.Description"       "Arrest"                     "Domestic"                  
[11] "Beat"                       "District"                   "Ward"                       "Community.Area"             "FBI.Code"                  
[16] "X.Coordinate"               "Y.Coordinate"               "Year"                       "Updated.On"                 "Latitude"                  
[21] "Longitude"                  "Location"                   "Historical.Wards.2003.2015" "Zip.Codes"                  "Community.Areas"           
[26] "Census.Tracts"              "Wards"                      "Boundaries...ZIP.Codes"     "Police.Districts"           "Police.Beats"              

skim(data)

── Data Summary ────────────────────────
                           Values
Name                       data  
Number of rows             700810
Number of columns          30    
_______________________          
Column type frequency:           
  character                9     
  logical                  2     
  numeric                  18    
  POSIXct                  1     
________________________         
Group variables            None  

Data Cleaning

The subset dataset for this study is taken from the initial dataset which covers the period of Aug 01 2020 to Aug 01 2021.

##subset dataset
newdata<- data %>%
filter(Date > "2020-08-01" & Date< "2021-08-01")
head(newdata)

Converting to Numbers

For this purpose of this lab work variables longitude and latitude are both coverted into numeric variables using the function “as.numeric”.

newdata$Latitude = as.numeric (newdata$Latitude)
newdata$Longitude = as.numeric(newdata$Longitude)
class(newdata$Latitude)

[1] "numeric"

Removing Missing Values

The dataset is preprocessed to remove all incidents without any latitude and longitude

newdata = newdata[!is.na(newdata$Latitude),]

Analysis Methods

The subset data is preprocessed for the purpose of removing all incidents with no latitude and longitude from the data.Spatial points object is further created from the subset dataset, which is over layed with the neighborhood polygons data using world geodetic system. The shape file is further read into R. Aggregation of assault by area within the neighborhood is further conducted to enhance our understanding on crime patterns in neighborhoods. The points are converted into raster which helps in generating a heat map using the focal function in R. The Getis-Ord GI is also employed conduct the hot spots analysis, which provides us with z-values to illustrate the presence of clustering in the used dataset. Open Street Map is also utilized in generating the hot maps.Below details the analysis conducted on the crime data.

###Convert to an object##
wgs84 = CRS("+proj=longlat +ellps=WGS84 +datum=WGS84 +no_defs")
crime_spt= SpatialPointsDataFrame(newdata[,c('Longitude','Latitude')], newdata, proj4string = wgs84)
plot(crime_spt)

## SHAPEFILE LOADING
neighborhoods = readOGR(dsn='.', layer='geo_export_b4cc3c31-7809-42e5-a88f-292b52d17cd9', stringsAsFactors = F)

Warning: OGR support is provided by the sf and terra packages among othersWarning: OGR support is provided by the sf and terra packages among othersWarning: OGR support is provided by the sf and terra packages among othersWarning: OGR support is provided by the sf and terra packages among othersWarning: OGR support is provided by the sf and terra packages among othersWarning: OGR support is provided by the sf and terra packages among others

OGR data source with driver: ESRI Shapefile 
Source: "C:\Users\charl\OneDrive\Desktop\Data Science Projects\R Projects", layer: "geo_export_b4cc3c31-7809-42e5-a88f-292b52d17cd9"
with 61 features
It has 4 fields

neighborhoods = spTransform(neighborhoods, wgs84)
plot(neighborhoods, col=terrain.colors(nrow(neighborhoods)))

####overlaying the points######
crime_spt$zip= over(crime_spt, neighborhoods)$zip
crime_spt= crime_spt[!is.na(crime_spt$zip),]
plot(crime_spt,col= '#00000010')
plot(neighborhoods, border= 'gray', add= T)

chicago <- readOGR(dsn='.', layer= 'cityboundary', stringsAsFactors = F)

Warning: OGR support is provided by the sf and terra packages among othersWarning: OGR support is provided by the sf and terra packages among othersWarning: OGR support is provided by the sf and terra packages among othersWarning: OGR support is provided by the sf and terra packages among othersWarning: OGR support is provided by the sf and terra packages among othersWarning: OGR support is provided by the sf and terra packages among others

OGR data source with driver: ESRI Shapefile 
Source: "C:\Users\charl\OneDrive\Desktop\Data Science Projects\R Projects", layer: "cityboundary"
with 1 features
It has 4 fields
Integer64 fields read as strings:  OBJECTID 

##Chicago_prj<-st_crs(chicago)
chicago_prj <- crs(chicago)
chicago_prj

Coordinate Reference System:
Deprecated Proj.4 representation:
 +proj=tmerc +lat_0=36.6666666666667 +lon_0=-88.3333333333333 +k=0.999975 +x_0=300000 +y_0=0 +datum=NAD83 +units=us-ft +no_defs 
WKT2 2019 representation:
PROJCRS["NAD83 / Illinois East (ftUS)",
    BASEGEOGCRS["NAD83",
        DATUM["North American Datum 1983",
            ELLIPSOID["GRS 1980",6378137,298.257222101,
                LENGTHUNIT["metre",1]]],
        PRIMEM["Greenwich",0,
            ANGLEUNIT["degree",0.0174532925199433]],
        ID["EPSG",4269]],
    CONVERSION["SPCS83 Illinois East zone (US Survey feet)",
        METHOD["Transverse Mercator",
            ID["EPSG",9807]],
        PARAMETER["Latitude of natural origin",36.6666666666667,
            ANGLEUNIT["degree",0.0174532925199433],
            ID["EPSG",8801]],
        PARAMETER["Longitude of natural origin",-88.3333333333333,
            ANGLEUNIT["degree",0.0174532925199433],
            ID["EPSG",8802]],
        PARAMETER["Scale factor at natural origin",0.999975,
            SCALEUNIT["unity",1],
            ID["EPSG",8805]],
        PARAMETER["False easting",984250,
            LENGTHUNIT["US survey foot",0.304800609601219],
            ID["EPSG",8806]],
        PARAMETER["False northing",0,
            LENGTHUNIT["US survey foot",0.304800609601219],
            ID["EPSG",8807]]],
    CS[Cartesian,2],
        AXIS["easting (X)",east,
            ORDER[1],
            LENGTHUNIT["US survey foot",0.304800609601219]],
        AXIS["northing (Y)",north,
            ORDER[2],
            LENGTHUNIT["US survey foot",0.304800609601219]],
    USAGE[
        SCOPE["Engineering survey, topographic mapping."],
        AREA["United States (USA) - Illinois - counties of Boone; Champaign; Clark; Clay; Coles; Cook; Crawford; Cumberland; De Kalb; De Witt; Douglas; Du Page; Edgar; Edwards; Effingham; Fayette; Ford; Franklin; Gallatin; Grundy; Hamilton; Hardin; Iroquois; Jasper; Jefferson; Johnson; Kane; Kankakee; Kendall; La Salle; Lake; Lawrence; Livingston; Macon; Marion; Massac; McHenry; McLean; Moultrie; Piatt; Pope; Richland; Saline; Shelby; Vermilion; Wabash; Wayne; White; Will; Williamson."],
        BBOX[37.06,-89.28,42.5,-87.02]],
    ID["EPSG",3435]] 

crime1 = spTransform(crime_spt, chicago_prj)
neighborhoods1 = spTransform(neighborhoods, chicago_prj)
assault2020= crime1[crime1$Primary.Type == 'ASSAULT',]
plot(assault2020, col= "#00000040")
plot(neighborhoods1, border= 'gray',add= T)

Neighborhood Analysis

##Aggregating by Area
names(assault2020)

 [1] "ï..ID"                      "Case.Number"                "Date"                       "Block"                      "IUCR"                      
 [6] "Primary.Type"               "Description"                "Location.Description"       "Arrest"                     "Domestic"                  
[11] "Beat"                       "District"                   "Ward"                       "Community.Area"             "FBI.Code"                  
[16] "X.Coordinate"               "Y.Coordinate"               "Year"                       "Updated.On"                 "Latitude"                  
[21] "Longitude"                  "Location"                   "Historical.Wards.2003.2015" "Zip.Codes"                  "Community.Areas"           
[26] "Census.Tracts"              "Wards"                      "Boundaries...ZIP.Codes"     "Police.Districts"           "Police.Beats"              
[31] "zip"                       

neighborhoods1$ASSAULT2020 = over(neighborhoods1, assault2020, fn= length)$ï..ID
spplot(neighborhoods1, zcol= 'ASSAULT2020',main=list(label= "2020 Assault crime by Neighborhood in the City of Chicago", cex= 1))

Heat Map

##To create a heat map you have to convert the points to a raster
pixelsize= 1000
box = round(extent(neighborhoods1) / pixelsize) * pixelsize
template= raster(box,crs = chicago_prj,
nrows = (box@ymax - box@ymin) / pixelsize,
ncols = (box@xmax - box@xmin) / pixelsize)
assault2020$PRESENT = 1
names(assault2020)

 [1] "ï..ID"                      "Case.Number"                "Date"                       "Block"                      "IUCR"                      
 [6] "Primary.Type"               "Description"                "Location.Description"       "Arrest"                     "Domestic"                  
[11] "Beat"                       "District"                   "Ward"                       "Community.Area"             "FBI.Code"                  
[16] "X.Coordinate"               "Y.Coordinate"               "Year"                       "Updated.On"                 "Latitude"                  
[21] "Longitude"                  "Location"                   "Historical.Wards.2003.2015" "Zip.Codes"                  "Community.Areas"           
[26] "Census.Tracts"              "Wards"                      "Boundaries...ZIP.Codes"     "Police.Districts"           "Police.Beats"              
[31] "zip"                        "PRESENT"                   

raster2020 = rasterize(assault2020, template, field = 'PRESENT', fun = sum)
plot(raster2020, xlim=c(1091131, 1205198), ylim=c(1813892, 1951669))
plot(neighborhoods1, border = '#00000040', add= T)

#####Next we use the focal function to run a gaussian smoothing kernel
kernel= focalWeight(raster2020, d=264, type ='Gauss')
heat2020= focal(raster2020, kernel, fun = sum, na.rm= T)
plot(heat2020, xlim=c(1091131, 1205199), ylim=c(1813892, 1951669))

###Getis-Ord GI
##one major issue with knrnel density estimation is that the choice of a threshold for what it is
##create a regular grid of 1/8-mile-square crime areas via a raster
pixelsize= 500
box = round(extent(neighborhoods1) / pixelsize) * pixelsize
template= raster(box,crs = chicago_prj,
nrows = (box@ymax - box@ymin) / pixelsize,
ncols = (box@xmax - box@xmin) / pixelsize)
assault2020$PRESENT = 1
#names(assault2020)

getisraster= rasterize(assault2020, template, field = 'PRESENT', fun = sum)
getisgrid = rasterToPolygons(getisraster)

##create the list of neighbors
#install.packages("spdep")
#library(spdep)
neighbors = poly2nb(getisgrid) ##construct neighbours list from polygon list
weighted_neighbors = nb2listw(neighbors, zero.policy = T) ## spatial weights for neighbours list

##perform the local G analysis
getisgrid$HOTSPOT = as.vector(localG(getisgrid$layer, weighted_neighbors))

Warning: NAs in lagged values

names(getisgrid)

[1] "layer"   "HOTSPOT"

### color the grid cells based on the z-score
min(getisgrid$HOTSPOT,na.rm= TRUE)

[1] -1.651486

max(getisgrid$HOTSPOT,na.rm= TRUE)

[1] 11.05128

breaks= c(-20, -1.96, -1, 1, 1.96, 20)
palette= c("#0000FF80","#8080FF80", "#FFFFFF80","#FF808080", "#FF000080")
col= palette[cut(getisgrid$HOTSPOT, breaks)]
###plot
plot(getisgrid, col=col, border= NA, xlim=c(1091131,1205199), ylim=c(1813892,1951669) )
plot(neighborhoods1, border= "gray", add= T)

#############OPEN STREET MAP#########
#install.packages("OpenStreetMap",dependencies = TRUE)
library(OpenStreetMap)

Warning: package ‘OpenStreetMap’ was built under R version 4.2.3

extent(neighborhoods)

class      : Extent 
xmin       : -87.94011 
xmax       : -87.52414 
ymin       : 41.64454 
ymax       : 42.02304 

basemap = openmap(upperLeft = c(42.02304 , -87.94011), lowerRight = c(41.64454 , -87.52414 ), type="osm")
basemap = openproj(basemap, chicago_prj)
plot(basemap)
plot(getisgrid, col=col, border= NA, add= T)

basemap = openmap(upperLeft = c(42.02304 , -87.94011), lowerRight = c(41.64454 , -87.52414 ), type="osm")
basemap = openproj(basemap, chicago_prj)
plot(basemap)
plot(getisgrid, col=col, border= NA, add= T)

legend(x= 'bottomleft', inset= 0.05, bg= "white", col=palette, pch=15,
legend = c('cold spot >99% confidence','cold spot 68-95% confidence',
'Not Significant', 'Hot Spot 68-95% confidence', 'Hot spot >95% confidence'))

Findings

From the above crime data analysis, there is the development of spatial plot on Assault crime in the city of Chicago from the period of August 01,2020 to August,01 2021. The plots as displayed above showed spatial locations of assault data in the city of Chicago. There is however the identification of both hot spot and cold spot areas of assault crime in the city of Chicago. As shown, there is high occurrence of assault crime (hot spot) in the central part of Chicago city. These are areas centered around the central business areas of the city which constitute more than 95% significance confidence level whiles 65-95% confidence hot-spot traces can be seen in the north and south part of the city. The significant cold spot of assault crime of more than 95% confidence level can be found in north and eastern part of Chicago, with suburbs of the city displaying cold spots of assault crime as defined with 65-95% confidence level.

Conclusion

In conclusion, it can be suggested that the streets of Chicago during the period of 2020-2021 was considered unsafe for its residents, with high occurrence of assault crime. This however does not rule out the occurrence of other crimes in the city. Most of these crimes(assault) occurred in the downtown area of the city. However with more precautionary measures and collaborative efforts by the city police, community leaders and other state agencies, the streets of Chicago will become more safer for its residents.