library(tidyr)
library(ggplot2)
library(skimr)
library(readr)
library(dplyr)
library(stringr)
library(lubridate)
library(gt)
library(gtExtras)
library(plotly)
library(maps)
library(gridExtra)
library(archive)
library(readr)
library(shiny)
library(rmarkdown)
library(vcd)
library(ggmap)
library(wordcloud2)
library(tidytext)
library(boot)
library(naniar)
crime_stat <- read_csv(
archive_read("DATA_01-2013_03-2018.tar.gz", file = 1),
col_types = cols())
Here we have an extract of the gun violence related incidents data
set from March 2013 to March 2018. It records each incident by a primary
key called incident id. Most of the data has been scraped from website
therefore there are records that are missing, when we perform the
analysis we will be removing all incidents that have missing values. We
will also split the some of the fields that are delimited by
|| and store in a key value format separated by
::.
The purpose of the analysis is to get a good understanding of the crime that occurs in each state and see which areas have more crime vs which areas have less. We also want to answer the following questions when it comes to crime related activity.
Here we have the data dictionary, specifying the field name, data-type, brief description and if the field is required.
| Field | Data Type | Description | Required? |
|---|---|---|---|
incident_id |
int | gunviolencearchive.org ID for incident | Yes |
date |
str | date of occurrence | Yes |
state |
str | State in which the incident occurred | Yes |
city_or_county |
str | City or county in which the incident occurred | Yes |
address |
str | address where incident took place | Yes |
n_killed |
int | number of people killed | Yes |
n_injured |
int | number of people injured | Yes |
incident_url |
str | link to gunviolencearchive.org webpage containing details of incident | Yes |
source_url |
str | link to online news story concerning incident | No |
incident_url_fields_missing |
bool | ignore, always False | Yes |
congressional_district |
int | Number of districts | No |
gun_stolen |
dict[int, str] | key: gun ID, value: ‘Unknown’ or ‘Stolen’ | No |
gun_type |
dict[int, str] | key: gun ID, value: description of gun type | No |
incident_characteristics |
list[str] | list of incident characteristics | No |
latitude |
float | geolocation | No |
location_description |
str | description of location where incident took place | No |
longitude |
float | geolocation | No |
n_guns_involved |
int | number of guns involved | No |
notes |
str | additional notes about the incident | No |
participant_age |
dict[int, int] | key: participant ID | No |
participant_age_group |
dict[int, str] | key: participant ID, value: description of age group, e.g. ‘Adult 18+’ | No |
participant_gender |
dict[int, str] | key: participant ID, value: ‘Male’ or ‘Female’ | No |
participant_name |
dict[int, str] | key: participant ID | No |
participant_relationship |
dict[int, str] | key: participant ID, value: relationship of participant to others | No |
participant_status |
dict[int, str] | key: participant ID, value: ‘Arrested’, ‘Killed’, ‘Injured’, or ‘Unharmed’ | No |
participant_type |
dict[int, str] | key: participant ID, value: ‘Victim’ or ‘Subject-Suspect’ | No |
sources |
list[str] | links to online news stories concerning incident | No |
state_house_district |
int | Number of state house districts | No |
state_senate_district |
int | Number of state senate districts | No |
gg_miss_var(crime_stat, show_pct = T)
According to the graph we do have some missing data present in our dataset. The following variables seem to have the most missing data:
Even though we have some missing records in our dataset. It will not really impact our analysis as we may not even use some of the data. Therefore lets move onto the data cleansing portion!
Some of the columns are concatenated by || and are
stored in a key-value pair format. Lets create a custom function that
runs on the concatenated columns in order to perform some meaningful
analysis.
Generate sub tables from the current data set.
columns_to_process = c("gun_stolen", "gun_type", "participant_age",
"participant_age_group", "participant_gender",
"participant_name", "participant_relationship",
"participant_status", "participant_type")
split_tables_func <- function(df, column_name_list) {
# Initialize a list to store the data frames
list_of_dfs <- list()
# Check if all column names in the list are actually present in the data frame
valid_columns <- column_name_list[column_name_list %in% names(df)]
for (i in valid_columns) {
# Process each column and store in the list
list_of_dfs[[i]] <- df |>
select(incident_id, all_of(i)) |>
separate_rows(all_of(i), sep = "\\|\\|") |>
separate(
all_of(i),
into = c("person_id",
paste0(i, "_status")),
sep = "::",
convert = TRUE) # Ensure person_id is an integer
}
# Return the list of data frames
return(list_of_dfs)
}
Now that we have the function defined and the list of columns to
split defined we will go ahead and create the different tables to
perform specific analysis. The idea is to split the columns into
different tables using a primary key “incident_id” and a
surrogate key “person_id” to correctly merge data-sets.
# run the function to split all columns into sub tables
processed_dfs <- split_tables_func(crime_stat,columns_to_process)
# Taking a look at the output of one of the generated tables.
head(processed_dfs$participant_age) |>
gt() |>
gt_theme_guardian() |>
tab_header(title = "First 6 records from Participant Age Table")