Project Overview

The purpose of this project is to analyse and discover patterns in crimes reported by the Metropolian Police Service in the January of 2024. The association rules techniques will be used in order to find the relationships between the type of crime, last outcome of the investigation and location of the crime. The data was collected from DATA.POLICE.UK.

Dataset

Dataset contains information on the crimes that fall within Metropolitan Police Service. Dataset consists of 12 columns:

  • Crime ID
  • Month - this is the same for each crime (01.2024)
  • Reported by - the police department which reported the crime
  • Falls Within - the same as reported by
  • Lognitude - anonymysed coordinates
  • Latitude - anonymysed coordinates
  • LSOA code - Lower Layer Super Output Area code
  • LSOA name - Lower Layer Super Output Area name
  • Location
  • Crime type
  • Last outcome category - Most recent outcome of crime
  • Context - additional data about a specific crime (empty for newly added data)

Reading Data

data <- read.csv("C:/Users/karol/OneDrive/Pulpit/association rules/2024-01-metropolitan-street.csv")

Exploratory Data Analysis

Exploratory Data Analysis will help to better understand the dataset.

Initially there were 91411 observations and 12 columns. After deleting duplicates 82225 observations are left.

library(dplyr)
## 
## Attaching package: 'dplyr'
## The following objects are masked from 'package:stats':
## 
##     filter, lag
## The following objects are masked from 'package:base':
## 
##     intersect, setdiff, setequal, union
dim(data)
## [1] 91411    12
length(unique(data$Crime.ID))
## [1] 71872
data <- distinct(data)
dim(data)
## [1] 82225    12
head(data)
##                                                           Crime.ID   Month
## 1 e5b3ed0375b796067eb668f37a18094cce300bdbd5446c340a2a6b1993eb2d8c 2024-01
## 2 d5d46efe4a74b2fdf87f28f7c89c4d7534c80d4150f58f3309dc055f4205b19c 2024-01
## 3 8ee491f5ebeda54bda879fed602d2f308bfd8abf4b59f749d94a2a8570324a0f 2024-01
## 4 318e91d9bd9ad081bb348b8c743236365c9178eb60229d55692d8581852edd77 2024-01
## 5 8ec949bf76f02215ad73b489ff502ab32101b0834b49fc7b634f27b3daab9550 2024-01
## 6 dcad5a76bdab37d9b0d0911856e31534441634741084dcd52367bc85a5f0756c 2024-01
##                   Reported.by                Falls.within Longitude Latitude
## 1 Metropolitan Police Service Metropolitan Police Service -0.552328 50.85316
## 2 Metropolitan Police Service Metropolitan Police Service -0.666558 50.84894
## 3 Metropolitan Police Service Metropolitan Police Service -0.657460 50.78727
## 4 Metropolitan Police Service Metropolitan Police Service  0.876815 51.14766
## 5 Metropolitan Police Service Metropolitan Police Service  0.884060 51.11065
## 6 Metropolitan Police Service Metropolitan Police Service  0.833354 51.13943
##                      Location LSOA.code    LSOA.name
## 1     On or near Queen Street E01031392    Arun 001C
## 2 On or near Northfields Lane E01031396    Arun 003D
## 3        On or near Kingsmead E01031419    Arun 015C
## 4     On or near Parking Area E01034986 Ashford 005G
## 5  On or near Southdown Close E01034989 Ashford 010G
## 6       On or near Deyley Way E01032813 Ashford 012E
##                     Crime.type Last.outcome.category Context
## 1                  Other theft   Under investigation      NA
## 2 Violence and sexual offences   Under investigation      NA
## 3 Violence and sexual offences   Under investigation      NA
## 4 Violence and sexual offences   Under investigation      NA
## 5 Violence and sexual offences   Under investigation      NA
## 6 Violence and sexual offences   Under investigation      NA
str(data)
## 'data.frame':    82225 obs. of  12 variables:
##  $ Crime.ID             : chr  "e5b3ed0375b796067eb668f37a18094cce300bdbd5446c340a2a6b1993eb2d8c" "d5d46efe4a74b2fdf87f28f7c89c4d7534c80d4150f58f3309dc055f4205b19c" "8ee491f5ebeda54bda879fed602d2f308bfd8abf4b59f749d94a2a8570324a0f" "318e91d9bd9ad081bb348b8c743236365c9178eb60229d55692d8581852edd77" ...
##  $ Month                : chr  "2024-01" "2024-01" "2024-01" "2024-01" ...
##  $ Reported.by          : chr  "Metropolitan Police Service" "Metropolitan Police Service" "Metropolitan Police Service" "Metropolitan Police Service" ...
##  $ Falls.within         : chr  "Metropolitan Police Service" "Metropolitan Police Service" "Metropolitan Police Service" "Metropolitan Police Service" ...
##  $ Longitude            : num  -0.552 -0.667 -0.657 0.877 0.884 ...
##  $ Latitude             : num  50.9 50.8 50.8 51.1 51.1 ...
##  $ Location             : chr  "On or near Queen Street" "On or near Northfields Lane" "On or near Kingsmead" "On or near Parking Area" ...
##  $ LSOA.code            : chr  "E01031392" "E01031396" "E01031419" "E01034986" ...
##  $ LSOA.name            : chr  "Arun 001C" "Arun 003D" "Arun 015C" "Ashford 005G" ...
##  $ Crime.type           : chr  "Other theft" "Violence and sexual offences" "Violence and sexual offences" "Violence and sexual offences" ...
##  $ Last.outcome.category: chr  "Under investigation" "Under investigation" "Under investigation" "Under investigation" ...
##  $ Context              : logi  NA NA NA NA NA NA ...

The column Context is empty. There are some missing observations in the other columns as well. Since these columns will not be important for association rules mining, the empty values will not be replaced or deleted.

anyNA(data)
## [1] TRUE
colSums(is.na(data))
##              Crime.ID                 Month           Reported.by 
##                     0                     0                     0 
##          Falls.within             Longitude              Latitude 
##                     0                   646                   646 
##              Location             LSOA.code             LSOA.name 
##                     0                     0                     0 
##            Crime.type Last.outcome.category               Context 
##                     0                     0                 82225

Now, let’s analyse the observations for which coordinates are empty. If the coordinates are empty, then the location and LSOA are empty as well. The missing coordinates won’t be handled in any way, because these variables will not be important for association rules mining.

data %>%
  group_by(Location, Crime.type, Last.outcome.category) %>%
  filter(is.na(Longitude) | is.na(Latitude)) %>%
  ungroup() %>%
  distinct(Location, Crime.type, Last.outcome.category, LSOA.code, LSOA.name)
## # A tibble: 32 × 5
##    Location    Crime.type              Last.outcome.category LSOA.code LSOA.name
##    <chr>       <chr>                   <chr>                 <chr>     <chr>    
##  1 No Location Bicycle theft           Investigation comple… ""        ""       
##  2 No Location Bicycle theft           Under investigation   ""        ""       
##  3 No Location Burglary                Investigation comple… ""        ""       
##  4 No Location Burglary                Under investigation   ""        ""       
##  5 No Location Criminal damage and ar… Under investigation   ""        ""       
##  6 No Location Criminal damage and ar… Investigation comple… ""        ""       
##  7 No Location Criminal damage and ar… Awaiting court outco… ""        ""       
##  8 No Location Criminal damage and ar… Offender given a cau… ""        ""       
##  9 No Location Drugs                   Offender given penal… ""        ""       
## 10 No Location Drugs                   Under investigation   ""        ""       
## # ℹ 22 more rows

Let’s review the most common crimes’ location & types. There are more that 22000 different locations, so only a few most common ones will be presented. The most common location is On or near Supermarket with more than 3000 crimes commited in this location in January.

library(ggplot2)
length(unique(data$Location))
## [1] 22080
top_locations <- data %>%
  count(Location) %>%
  top_n(10)
## Selecting by n
data_subset <- data %>%
  filter(Location %in% top_locations$Location)

ggplot(data_subset, aes(x = Location)) +
  geom_bar() +
  labs(title = "Top 10 Most Common Locations") +
  theme(axis.text.x = element_text(angle = 50, hjust = 1))

The number of different locations is very large. In order to improve performance, the locations which are appearing only 3 or less times will be deleted and they will not be considered for association rules mining.

location_freq <- table(data$Location)
rare_locations <- names(location_freq[location_freq <= 3])
length(unique(rare_locations))
## [1] 17008
data <- data[!data$Location %in% rare_locations, ]

The most common Lower Layer Super Output Area is Westminster 013G.

top_LSOA <- data %>%
  count(LSOA.name) %>%
  top_n(10)
## Selecting by n
data_subset <- data %>%
  filter(LSOA.name %in% top_LSOA$LSOA.name)

ggplot(data_subset, aes(x = LSOA.name)) +
  geom_bar() +
  labs(title = "Top 10 Lower Layer Super Output Areas") +
  theme(axis.text.x = element_text(angle = 50, hjust = 1))

Majority of the crimes are classified as “Violence and sexual offences”. Different types of thefts are very common as well.

length(unique(data$Crime.type))
## [1] 14
ggplot(data, aes(x = Crime.type)) +
  geom_bar() +
  labs(title = "Most Common Crime Types") +
  theme(axis.text.x = element_text(angle = 50, hjust = 1))

Most of the crimes are either under investigation or the investigation was completed with no suspect identified.

length(unique(data$Last.outcome.category))
## [1] 8
ggplot(data, aes(x = Last.outcome.category)) +
  geom_bar() +
  labs(title = "Count of Outcome Categories") +
  theme(axis.text.x = element_text(angle = 50, hjust = 1))

There are some duplicated Crime IDs. This might be caused by the fact that multiple crimes were commited by the same group of people in the same type or there are just some mistakes in the data. Anyway, this data won’t be deleted as these are probably valuable observations.

head(data[duplicated(data$Crime.ID) | duplicated(data$Crime.ID, fromLast = TRUE), ])
##                                                            Crime.ID   Month
## 18 03acba1d3a26c70f1038ef56827b2ab947b0fbc926edc0221360bd6d00115de0 2024-01
## 24 03acba1d3a26c70f1038ef56827b2ab947b0fbc926edc0221360bd6d00115de0 2024-01
## 44                                                                  2024-01
## 45                                                                  2024-01
## 47                                                                  2024-01
## 51 8fdbb739d17378fe8117ec0bb696808efe7a3643f4a167e36064b7a26d0653fd 2024-01
##                    Reported.by                Falls.within Longitude Latitude
## 18 Metropolitan Police Service Metropolitan Police Service  0.135924 51.58735
## 24 Metropolitan Police Service Metropolitan Police Service  0.135924 51.58735
## 44 Metropolitan Police Service Metropolitan Police Service  0.135537 51.58433
## 45 Metropolitan Police Service Metropolitan Police Service  0.132700 51.58610
## 47 Metropolitan Police Service Metropolitan Police Service  0.135611 51.58282
## 51 Metropolitan Police Service Metropolitan Police Service  0.135611 51.58282
##                             Location LSOA.code                 LSOA.name
## 18        On or near Gibbfield Close E01000027 Barking and Dagenham 001A
## 24        On or near Gibbfield Close E01000027 Barking and Dagenham 001A
## 44            On or near Supermarket E01000030 Barking and Dagenham 001D
## 45 On or near Sports/Recreation Area E01000030 Barking and Dagenham 001D
## 47        On or near Sheepcotes Road E01000030 Barking and Dagenham 001D
## 51        On or near Sheepcotes Road E01000030 Barking and Dagenham 001D
##                      Crime.type                         Last.outcome.category
## 18 Violence and sexual offences Investigation complete; no suspect identified
## 24 Violence and sexual offences                           Under investigation
## 44        Anti-social behaviour                                              
## 45        Anti-social behaviour                                              
## 47        Anti-social behaviour                                              
## 51    Criminal damage and arson                           Under investigation
##    Context
## 18      NA
## 24      NA
## 44      NA
## 45      NA
## 47      NA
## 51      NA

Lastly,let’s see count of crimes per coordinates. For some specific coordinates more than one crime was commited.

data %>%
  group_by(Latitude, Longitude) %>%
  summarise(crime_count = n_distinct(Crime.ID)) %>%
  ungroup()
## `summarise()` has grouped output by 'Latitude'. You can override using the
## `.groups` argument.
## # A tibble: 13,105 × 3
##    Latitude Longitude crime_count
##       <dbl>     <dbl>       <int>
##  1     50.8   -1.07             1
##  2     50.8   -0.134            1
##  3     50.9    0.535            1
##  4     50.9   -0.552            1
##  5     51.1    0.159            2
##  6     51.1    0.877            1
##  7     51.3   -0.0786           1
##  8     51.3   -0.109            1
##  9     51.3   -0.0973           1
## 10     51.3   -0.112            3
## # ℹ 13,095 more rows

Association Rules

In order to proceed with association rules mining the dataset needs to be modified. New object that contains only the relevant columns as transactions will be prepared. Apriori algorithm was able to distinguish 14 different rules (14 different frequent itemsets). Minimum support is equal to 1% (so the itemset needs to appear at least in the 1% of transactions). Minimum confidence is equal to 0.5 (the consequent appears at least in the 50% of transactions that antecedent is in).

library(arules)
## Loading required package: Matrix
## 
## Attaching package: 'arules'
## The following object is masked from 'package:dplyr':
## 
##     recode
## The following objects are masked from 'package:base':
## 
##     abbreviate, write
transactions <- as(data[,c("Location", "Crime.type", "Last.outcome.category")], "transactions")
## Warning: Column(s) 1, 2, 3 not logical or factor. Applying default
## discretization (see '? discretizeDF').
arules <- apriori(transactions, supp = 0.01, conf = 0.5)
## Apriori
## 
## Parameter specification:
##  confidence minval smax arem  aval originalSupport maxtime support minlen
##         0.5    0.1    1 none FALSE            TRUE       5    0.01      1
##  maxlen target  ext
##      10  rules TRUE
## 
## Algorithmic control:
##  filter tree heap memopt load sort verbose
##     0.1 TRUE TRUE  FALSE TRUE    2    TRUE
## 
## Absolute minimum support count: 546 
## 
## set item appearances ...[0 item(s)] done [0.00s].
## set transactions ...[5094 item(s), 54629 transaction(s)] done [0.04s].
## sorting and recoding items ... [27 item(s)] done [0.00s].
## creating transaction tree ... done [0.01s].
## checking subsets of size 1 2 3 done [0.00s].
## writing ... [14 rule(s)] done [0.00s].
## creating S4 object  ... done [0.00s].

The rules were arranged by their support. The first itemset occurs most frequently in the dataset out of all the itemsets. The last itemset occurs least frequently given specified minimum support and confidence. As can be seen, even the last itemset occurs quite frequently, it is present 1591 times.

rules <- as(arules, "data.frame")
rules %>% arrange(desc(support))
##                                                                                                                                rules
## 1                                           {Crime.type=Violence and sexual offences} => {Last.outcome.category=Under investigation}
## 2                                  {Crime.type=Other theft} => {Last.outcome.category=Investigation complete; no suspect identified}
## 3                                                                     {Crime.type=Anti-social behaviour} => {Last.outcome.category=}
## 4                                                                     {Last.outcome.category=} => {Crime.type=Anti-social behaviour}
## 5                        {Crime.type=Theft from the person} => {Last.outcome.category=Investigation complete; no suspect identified}
## 6                                {Crime.type=Vehicle crime} => {Last.outcome.category=Investigation complete; no suspect identified}
## 7                                  {Crime.type=Shoplifting} => {Last.outcome.category=Investigation complete; no suspect identified}
## 8                                                           {Crime.type=Public order} => {Last.outcome.category=Under investigation}
## 9                         {Location=On or near Supermarket} => {Last.outcome.category=Investigation complete; no suspect identified}
## 10                   {Crime.type=Criminal damage and arson} => {Last.outcome.category=Investigation complete; no suspect identified}
## 11                                                              {Crime.type=Burglary} => {Last.outcome.category=Under investigation}
## 12                                                               {Crime.type=Robbery} => {Last.outcome.category=Under investigation}
## 13 {Location=On or near Supermarket,Crime.type=Shoplifting} => {Last.outcome.category=Investigation complete; no suspect identified}
## 14                          {Location=On or near Nightclub} => {Last.outcome.category=Investigation complete; no suspect identified}
##       support confidence   coverage      lift count
## 1  0.15848725  0.6688813 0.23694375  1.509681  8658
## 2  0.08477182  0.6052804 0.14005382  1.369924  4631
## 3  0.08208095  1.0000000 0.08208095 12.183095  4484
## 4  0.08208095  1.0000000 0.08208095 12.183095  4484
## 5  0.06853503  0.6245204 0.10974025  1.413470  3744
## 6  0.06119460  0.7049768 0.08680371  1.595566  3343
## 7  0.05815592  0.6665967 0.08724304  1.508701  3177
## 8  0.03188783  0.6061239 0.05260942  1.368036  1742
## 9  0.03033188  0.5107891 0.05938238  1.156063  1657
## 10 0.02912373  0.5362319 0.05431181  1.213648  1591
## 11 0.02859287  0.5469188 0.05227992  1.234409  1562
## 12 0.02634132  0.6968523 0.03780044  1.572812  1439
## 13 0.01398525  0.6678322 0.02094126  1.511497   764
## 14 0.01177031  0.5023437 0.02343078  1.136949   643

Each time the crime type is the anti-social behaviour, the last outcome category is empty.

rules %>% arrange(desc(confidence))
##                                                                                                                                rules
## 1                                                                     {Crime.type=Anti-social behaviour} => {Last.outcome.category=}
## 2                                                                     {Last.outcome.category=} => {Crime.type=Anti-social behaviour}
## 3                                {Crime.type=Vehicle crime} => {Last.outcome.category=Investigation complete; no suspect identified}
## 4                                                                {Crime.type=Robbery} => {Last.outcome.category=Under investigation}
## 5                                           {Crime.type=Violence and sexual offences} => {Last.outcome.category=Under investigation}
## 6  {Location=On or near Supermarket,Crime.type=Shoplifting} => {Last.outcome.category=Investigation complete; no suspect identified}
## 7                                  {Crime.type=Shoplifting} => {Last.outcome.category=Investigation complete; no suspect identified}
## 8                        {Crime.type=Theft from the person} => {Last.outcome.category=Investigation complete; no suspect identified}
## 9                                                           {Crime.type=Public order} => {Last.outcome.category=Under investigation}
## 10                                 {Crime.type=Other theft} => {Last.outcome.category=Investigation complete; no suspect identified}
## 11                                                              {Crime.type=Burglary} => {Last.outcome.category=Under investigation}
## 12                   {Crime.type=Criminal damage and arson} => {Last.outcome.category=Investigation complete; no suspect identified}
## 13                        {Location=On or near Supermarket} => {Last.outcome.category=Investigation complete; no suspect identified}
## 14                          {Location=On or near Nightclub} => {Last.outcome.category=Investigation complete; no suspect identified}
##       support confidence   coverage      lift count
## 1  0.08208095  1.0000000 0.08208095 12.183095  4484
## 2  0.08208095  1.0000000 0.08208095 12.183095  4484
## 3  0.06119460  0.7049768 0.08680371  1.595566  3343
## 4  0.02634132  0.6968523 0.03780044  1.572812  1439
## 5  0.15848725  0.6688813 0.23694375  1.509681  8658
## 6  0.01398525  0.6678322 0.02094126  1.511497   764
## 7  0.05815592  0.6665967 0.08724304  1.508701  3177
## 8  0.06853503  0.6245204 0.10974025  1.413470  3744
## 9  0.03188783  0.6061239 0.05260942  1.368036  1742
## 10 0.08477182  0.6052804 0.14005382  1.369924  4631
## 11 0.02859287  0.5469188 0.05227992  1.234409  1562
## 12 0.02912373  0.5362319 0.05431181  1.213648  1591
## 13 0.03033188  0.5107891 0.05938238  1.156063  1657
## 14 0.01177031  0.5023437 0.02343078  1.136949   643

It seems that the items in the itemsets are positively associated with each other. Some of the consequents are much more likely to be present with the antecedents than on their own.

library(arulesViz)
plot(arules, measure=c("support","lift"), shading="confidence")
## To reduce overplotting, jitter is added! Use jitter = 0 to prevent jitter.

The antecedents differ, so this graph is not very readable. It can be seen that itemsets consist of different items.

plot(arules, method="graph")

It is possible to check which crimes have the last outcome of local resolution. It seems that crimes associated with drugs often have this outcome.

rules.newcomers<-apriori(transactions, parameter=list(supp=0.001, conf=0.06), appearance=list(default="lhs", rhs="Last.outcome.category=Local resolution"), control=list(verbose=F)) 
rules.newcomers.byconf<-sort(rules.newcomers, by="confidence", decreasing=TRUE)
inspect(head(rules.newcomers.byconf))
##     lhs                   rhs                                          support confidence coverage     lift count
## [1] {Crime.type=Drugs} => {Last.outcome.category=Local resolution} 0.008457047  0.2523211 0.033517 22.01925   462

Crimes associated with drugs often await court outcome, however, they are much more likely to have the outcome of local resolution.

rules.newcomers<-apriori(transactions, parameter=list(supp=0.001, conf=0.06), appearance=list(default="lhs", rhs="Last.outcome.category=Awaiting court outcome"), control=list(verbose=F)) 
rules.newcomers.byconf<-sort(rules.newcomers, by="confidence", decreasing=TRUE)
inspect(head(rules.newcomers.byconf))
##     lhs                   rhs                                                support confidence coverage     lift count
## [1] {Crime.type=Drugs} => {Last.outcome.category=Awaiting court outcome} 0.002928847 0.08738394 0.033517 5.217156   160

Conclusion

The purpose of this project was to find the association rules in the dataset containing crime information using the apriori algorithm. 14 frequent itemsets were found (with minimum support of 0.01 and minimum confidence of 0.6).

Bibliography

Police UK. (2022). Police Data. Retrieved from https://data.police.uk/data/