Introduction

For this module the data set I chose to work with was Baltimore Crime. The data set contained information on crimes documented throughout the city of Baltimore.

Dataset

The data set has 664,738 rows and 22 columns. The most important columns I considered were CrimeDateTime, CrimeCode, Neighborhood, Latitude, and Longitude. However it also contained other data like Age, Race, Weapon, Gender, Premise Type, and Description. Each crime also had a CCNumber which was a unique code under which the case was filed. So each row was a different case with it’s respective information tied to the case.

I didn’t run into many limitations with this data as it had a lot of useful data groups and also already had the Longitude and Latitude of the crime broken up to be separate columns which was very convenient. One thing was that there were a lot of boxes that were empty or NULL. When creating my Folium Map I had to make sure to strip down the data by one, making all the words lowercase and get rid of the whitespace, and two, put in 0 (zero) for all NAs found. Still though the map ended up working! Overall, this data was very interesting and allowed me to dive into different visualizations that compared similar ideas in different contexts like the location of crimes, the frequency through out a year, the frequency throughout a day, the frequency of different types of crimes and Just a general count of crimes committed over the years.

Findings

Something general about what I found

Graph 1: Scatter plot

This Scatter plot is showing the Crime Count for each month in the years in the data set. This allows us to see if there is a trend in when crime may increase or decrease throughout a calendar year.

import wget


URL = "https://data.baltimorecity.gov/maps/e0992dddbbf64231976d5d57763ec4f5"
path = "/Users/mollykyle/Dropbox/Baltimore Crime/"

wget.download(URL, path + 'BaltimoreCrime.csv')
import os

size = os.stat(path + 'BaltimoreCrime.csv').st_size

print("File size:", round(size/10**9,2), "GB")
import pandas as pd

filename = "BaltimoreCrime.csv"

df = pd.read_csv(path + filename, nrows = 5, skiprows = 0)
df = pd.read_csv(path + filename, usecols = ['CrimeDateTime'], skiprows = lambda i: i % 100 != 0)
df['CrimeDateTime'] = pd.to_datetime(df['CrimeDateTime'],format = '%m/%d/%Y %I:%M:%S %p')

df['Year'] = df['CrimeDateTime'].dt.year
df['Quarter'] = df['CrimeDateTime'].dt.quarter

df['Month'] = df['CrimeDateTime'].dt.month
df['Day'] = df['CrimeDateTime'].dt.day
df['Hour'] = df['CrimeDateTime'].dt.hour

df['DayOfTheWeek'] = df['CrimeDateTime'].dt.dayofweek

df['MonthName'] = df['CrimeDateTime'].dt.strftime('%B')
df['MonthNameAbbrev'] = df['CrimeDateTime'].dt.strftime('%b')

df['DayName'] = df['CrimeDateTime'].dt.strftime('%A')
df['DayNameAbbrev'] = df['CrimeDateTime'].dt.strftime('%a')

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import warnings

print(df.columns)
df.shape #check shape
df.isna().sum() #check for NAs
df = df[df['Month'] != 0]

df['Year']
np.unique(df['Year']) #check how many unique years
x = df.groupby(['Year', 'Month'])['Year'].count().reset_index(name = 'Count')
x = pd.DataFrame(x)

x['Count'].max()
x['Count'].min()
x['Count'].max()
x['Count_hundreds'] = round(x['Count'] / 100, 0)

x2 = x.groupby(['Year', 'Month'])['Count'].sum().reset_index()
x2 = pd.DataFrame(x2)

x2 = x2.drop([0,1]) #dropped the first two columns as they are incomplete years

x2['Count'].max()
x2['Count'].min()
plt.figure(figsize = (18,10))


plt.scatter(x2['Month'], x2['Year'], marker = '8', cmap = 'viridis', 
            c = x2['Count'], s = x2['Count']*10, edgecolor = 'black') #multiplied by 10 to increase visability

plt.title('Count of Crime by Year and Month', fontsize = 18)
plt.xlabel ('Months of the Year', fontsize = 14)
plt.ylabel ('Year', fontsize = 14)

cbar = plt.colorbar()
cbar.set_label('Crime Count', rotation = 270, fontsize = 14, color = 'black', labelpad = 30)

my_colorbar_ticks = [*range(20, int(x2['Count'].max()), 1)]
cbar.set_ticks(my_colorbar_ticks)

my_x_ticks = [*range(x2['Month'].min(), x2['Month'].max()+1, 1)]
plt.xticks(my_x_ticks, fontsize = 14, color = 'black')
my_y_ticks = [*range(x2['Year'].min(), x2['Year'].max()+1, 1)]
plt.yticks(my_y_ticks, fontsize = 14, color = 'black')
plt.show()

Graph 2: Vertical Bar Charts

These two bar charts give us a look into the type of crimes being committed and the frequency. By looking at the top 50, and then the top 10, we can see how much more the top 10 crimes are committed. Also by comparing the mean to the bar charts, we can look at if the data is skewed by a certain crime happening a lot, or not.

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import warnings

warnings.filterwarnings("ignore")

path = "/Users/mollykyle/Dropbox/Baltimore Crime/"
filename = "BaltimoreCrime.csv"


df = pd.read_csv(path + filename,nrows = 5)
print(df.columns)
df = pd.read_csv(path + filename , usecols = ['CrimeCode', 'CrimeDateTime'], engine = 'python')

df.isna().sum()
df['CrimeDateTime'] = pd.to_datetime(df['CrimeDateTime'], format = '%m/%d/%Y %I:%M:%S %p' )

df['Hour'] = df.CrimeDateTime.dt.hour
df['Day'] = df.CrimeDateTime.dt.day
df['Month'] = df.CrimeDateTime.dt.month
df['Year'] = df.CrimeDateTime.dt.year
df['WeekDay'] = df.CrimeDateTime.dt.strftime('%a')
df['MonthName'] = df.CrimeDateTime.dt.strftime('%b')
df
x3 = df.groupby(['CrimeCode']).agg({'CrimeCode':['count']}).reset_index()

#Rename columns
x3.columns = ['CrimeCode' , 'Count']

#Sort the values
x3 = x3.sort_values('Count', ascending = False)


x3.reset_index(inplace = True, drop = True)


def pick_colors_according_to_mean_age(this_data):
    colors = []
    avg = this_data.Count.mean()
    for each in this_data.Count:
        if each > avg*1.01:
            colors.append('lightcoral')
        elif each < avg*0.99:
            colors.append('green')
        else:
            colors.append('black')
    return colors
            

bottom1 = 0
top1 = 50
d1 = x3.loc[bottom1:top1]
my_colors1 = pick_colors_according_to_mean_age(d1)

bottom2 = 0
top2 = 10
d2 = x3.loc[bottom2:top2]
my_colors2 = pick_colors_according_to_mean_age(d2)

fig = plt.figure(figsize = (18,16))
fig.suptitle('Frequency of Crimes by Crime Code:\n Top ' + str(top1) + ' and Top ' + str(top2), fontsize = 18, fontweight = 'bold')

ax1 = fig.add_subplot(2, 1, 1)
ax1.bar(d1.CrimeCode, d1.Count, label = 'Count', color = my_colors1)
ax1.legend(fontsize = 14)
plt.axhline(d1.Count.mean(), color = 'black', linestyle = 'dashed')

ax1.spines['right'].set_visible(False)
ax1.spines['top'].set_visible(False)
ax1.axes.xaxis.set_visible(False)

ax1.set_title('Top ' + str(top1) + ' Crimes' , size = 20)
ax1.text(top1-10, d1.Count.mean()+ 1000, 'Mean = ' + str(d1.Count.mean()), rotation = 0, fontsize = 14)





ax2 = fig.add_subplot(2, 1, 2)
ax2.bar(d2.CrimeCode, d2.Count, label = 'Count', color = my_colors1)
ax2.legend(fontsize = 14)
plt.axhline(d2.Count.mean(), color = 'black', linestyle = 'solid')

ax2.spines['right'].set_visible(False)
ax2.spines['top'].set_visible(False)
#ax2.axes.xaxis.set_visible(False)

ax2.set_title('Top ' + str(top2) + ' Crimes' , size = 20)
ax2.text(top2-1, d2.Count.mean()+ 1000, 'Mean = ' + str(d2.Count.mean()), rotation = 0, fontsize = 14)


plt.show()

Graph 3: Multiple Line Plots

This line graph shows how many crimes happen in a given hour each day of the week. This allows people to evaluate if there are times where there is more crime, and therefore if there is more protection necessary at those times.

CrimeCount_df = df.groupby(['Hour','WeekDay'])['CrimeCode'].count().reset_index(name = "CrimeCount")
CrimeCount_df[CrimeCount_df.WeekDay == 'Fri']
fig = plt.figure(figsize = (18,10))
ax = fig.add_subplot(1,1,1)

my_colors = {'Mon':'blue',
            'Tue':'red',
            'Wed' : 'green',
            'Thu' : 'gray',
            'Fri' : 'purple',
            'Sat' : 'gold',
            'Sun' :'brown'}

for key, grp, in CrimeCount_df.groupby('WeekDay'):
    grp.plot(ax=ax, kind = 'line', x = 'Hour', y = 'CrimeCount', color = my_colors[key], label = key, marker = '8')

plt.title('Total Crime Count by Hour', fontsize = 18)
ax.set_xlabel('Hour (24 Hour Interval)', fontsize = 18)
ax.set_ylabel('Crime Count', fontsize = 18, labelpad = 20)
ax.tick_params(axis = 'x', labelsize  = 14, rotation = 0)
ax.tick_params(axis = 'y', labelsize = 14, rotation = 0)

ax.set_xticks(np.arange(24))

handles, labels = ax.get_legend_handles_labels()
handles = [handles[1], handles[5], handles[6], handles[4], handles[0], handles[2], handles[3] ]
labels = [  labels[1], labels[5], labels[6], labels[4], labels[0], labels[2], labels[3] ]
plt.legend(handles, labels, loc = 'best', fontsize = 14, ncol = 1)


plt.show()

Graph 4: Nested Pie Chart

The nested pie chart shows the percentage of crime that happens each quarter and each month. This is different than the scatterplot as rather than simply a count of crimes committed, the percentage allows a more comparative look at the amount of crime throughout a year.

df['Quarter'] = 'Quarter ' + df['CrimeDateTime'].dt.quarter.astype('string')

pie_df = df.groupby(['Quarter','MonthName', 'Month'])['CrimeCode'].count().reset_index(name = "CrimeCount")

pie_df.sort_values(by=['Month'], inplace = True)

pie_df.reset_index(inplace = True, drop = True)

del pie_df['Month']

#Set up inside and outside reference numbers for colors

number_outside_colors = len(pie_df.Quarter.unique())
outside_color_ref_number = np.arange(number_outside_colors)*4


number_inside_colors = len(pie_df.MonthName.unique())
all_color_ref_number = np.arange(number_outside_colors + number_inside_colors)

inside_color_ref_number = []
for each in all_color_ref_number:
    if each not in outside_color_ref_number:
        inside_color_ref_number.append(each)

print(outside_color_ref_number)
print(inside_color_ref_number)
#Set up pie chart

fig = plt.figure(figsize = (10,10))
ax = fig.add_subplot(1,1,1)

colormap = plt.get_cmap("tab20c")
outer_colors = colormap(outside_color_ref_number)

all_crime = pie_df.CrimeCount.sum()


pie_df.groupby(['Quarter'])['CrimeCount'].sum().plot(
    kind = 'pie' , radius = 1, colors = outer_colors, pctdistance = 0.8, labeldistance = 1.1,
    wedgeprops = dict(edgecolor = 'w'), textprops = {'fontsize' :18},
    autopct = lambda p: '{:.2f}%\n({:.1f})'.format(p,(p*all_crime/100)),
    startangle = 90)


inner_colors = colormap(inside_color_ref_number)
pie_df.CrimeCount.plot(
    kind = 'pie' , radius = 0.6, colors = inner_colors, pctdistance = 0.65, labeldistance = 0.8,
    wedgeprops = dict(edgecolor = 'w'), textprops = {'fontsize' :13},
    labels = pie_df.MonthName, 
    autopct = '%1.2f%%',
    startangle = 90)


hole = plt.Circle((0,0), 0.3, fc = 'white')
fig1 = plt.gcf()
fig1.gca().add_artist(hole)

ax.yaxis.set_visible(False)
plt.title('Total Crime Count by Quarter and Month', fontsize = 18)

ax.text(0,0, 'Total Crime\n' + str(all_crime), ha = 'center', va = 'center', fontsize = 15)

ax.axis('equal')
plt.tight_layout()

plt.show()

Graph 5: Folium Map

This map shows how much crime occurs in the different neighborhoods in Baltimore. While hovering over the map you’re able to see the neighborhoods name, with each neighborhood having a color corresponding to the number of crimes committed within the neighborhood. Again, this gives more information about if different areas have more crime, and allows people to evaluate how to possibly prevent that.

path = "/Users/mollykyle/Dropbox/Baltimore Crime/"
filename = "BaltimoreCrime.csv"

df = pd.read_csv(path + filename,nrows = 5)

df = pd.read_csv(path + filename , usecols = ['GeoLocation','CrimeCode', 'CrimeDateTime', 'Neighborhood', 'Latitude','Longitude' ], engine = 'python')

map_df = df

import folium

neigh_df = map_df.groupby(['Neighborhood']).size().reset_index(name = "Count")

path = "/Users/mollykyle/Dropbox/"
balto_geo = path + 'baltimore.txt'
center_of_map = [39.3024273,-76.6195023]

#Make data in df lowercase and strip spacing
neigh_df['Neighborhood'] = neigh_df['Neighborhood'].str.strip().str.title()
#Fill in any Nas with 0
neigh_df['Count'] = neigh_df['Count'].fillna(0)

my_map = folium.Map(location = center_of_map,
                    zoom_start = 12,
                    tiles = 'cartodbpositron', 
                    width = '90%',
                    height = '100%',
                    left = '5%',
                    right = '5%',
                    top = '0%')

tiles = ['cartodbpositron','openstreetmap']
for tile in tiles:
    folium.TileLayer(tile).add_to(my_map)
ch_map = folium.Choropleth(geo_data = balto_geo,
                          name = 'choropleth',
                          data = neigh_df,
                          columns = ['Neighborhood', 'Count'],
                          key_on = 'feature.properties.name',
                          fill_color = 'RdPu',
                          fill_opacity = 0.9,
                          line_opacity = 0.4,
                          legend_name = 'Neighborhood Based on Crime',
                          highlight = True).add_to(my_map)



#Display region/neigh label
ch_map.geojson.add_child(
    folium.features.GeoJsonTooltip(fields = ['name'], aliases = ['Neighborhood: '],
                                  labels = True, style = ('background-color : black; color : white;'))
)
ch_map.add_to(my_map)
folium.LayerControl().add_to(my_map)
my_map.save(path + 'Choropleth_Crime_Baltimore.html')
my_map
Make this Notebook Trusted to load map: File -> Trust Notebook

Conclusion

Overall, these visualizations allowed for examination of how often crimes are committed, the frequency of different crimes committed and where different crimes are committed. This can be used in real life situations as it allows for a better understanding of patterns and trends that can be possibly prevented.