Libraries

library(tidyverse)
library(caret)
library(earth)
library(knitr)
library(caret)
library(pls)
library(glmnet)
library(Amelia)
library(knitr)
library(mice)
library(psych)
library(rattle)
library(arules)
library(igraph)

Instruction

Imagine 10000 receipts sitting on your table. Each receipt represents a transaction with items that were purchased. The receipt is a representation of stuff that went into a customer’s basket - and therefore ‘Market Basket Analysis’.

That is exactly what the Groceries Data Set contains: a collection of receipts with each line representing 1 receipt and the items purchased. Each line is called a transaction and each column in a row represents an item. The data set is attached.

Your assignment is to use R to mine the data for association rules. You should report support, confidence and lift and your top 10 rules by lift.

Extra credit: do a simple cluster analysis on the data as well. Use whichever packages you like. Due Nov 29 before midnight.

Data Exploration

I will load the data using read.transactions() and look at the top 20 items purchased.

grocery <- read.transactions('GroceryDataSet.csv', sep=',')
itemFrequencyPlot(grocery, topN=20, type="absolute", main="Top 10 Items", col = "Blue")

We can see from above that whole milk is the most frequently purchased item and that seems obvious as we know whole milk is a destination item. Our top 20 list matches to recent data i found on grocery items here: https://www.statista.com/statistics/962873/product-category-grocery-bought-online-most-popular-us/

Association rules

I will use apriori() to get top 10 rules with support, confidence and lift.

apriori(grocery, parameter=list(supp=0.001, conf=0.5) , control=list(verbose=FALSE)) %>%
  DATAFRAME() %>%
  arrange(desc(lift)) %>%
  top_n(10) %>%
  kable()

## Selecting by count

Cluster Analysis

I will look for item groupings to do cluster analysis. First step is to create a network graph from the grocery transaction data. Then i will use cluster_louvain() to detect communities.

grocery_data <- read.csv("GroceryDataSet.csv", header = FALSE) %>%
  mutate(shoper_id = row_number()) %>%
  pivot_longer(-shoper_id) %>%
  filter(value != "") %>%
  select(-name)

communities <- grocery_data %>%
  rename(to = value, from = shoper_id) %>%
  graph_from_data_frame(directed = FALSE) %>%
  cluster_louvain() %>%
  communities()

I have all all customers and items are assigned to 19 clusters. Now, I can summarize them with item names and count of shoppers in each cluster.

products <- as.character(unique(grocery_data$value)) # Get unique item list

df <- data.frame(name = c(NA), members = c(NA)) %>% na.omit() # create data frame

for (i in 1:length(communities)){
  cluster_name <- paste0(i,": ")
  cluster_members <- 0
  for (member in communities[[i]]){
    if (member %in% products){
      cluster_name <- paste0(cluster_name, member, " + ")
    } else {
      cluster_members <- cluster_members + 1
    }
  }
  cluster_name <- substr(cluster_name,1,nchar(cluster_name)-3)
  df <- rbind(df, data.frame(name = cluster_name, members = cluster_members))
}
  df %>%
  arrange(desc(members)) %>%
  kable()