Associative Analysis
Dynasty
2022-07-27
Associative Analysis
In this session, we will go through an example of association rules using the arules package. The documentation of this package can be found by visiting the following link: https://www.rdocumentation.org/packages/arules/versions/1.6-4. Below is an extract from its documentation:
“It provides the infrastructure for representing, manipulating and analyzing transaction data and patterns (frequent itemsets and association rules). It also provides interfaces to C implementations of the association mining algorithms Apriori and Eclat.”
We first we install the required arules library and loading the library
library(arules)## Loading required package: Matrix##
## Attaching package: 'arules'## The following objects are masked from 'package:base':
##
## abbreviate, writeLoading our transactions dataset from our csv file
path <-"http://bit.ly/GroceriesDataset"
Transactions<-read.transactions(path, sep = ",")
Transactions## transactions in sparse format with
## 9835 transactions (rows) and
## 169 items (columns)Verifying the object’s class
class(Transactions)## [1] "transactions"
## attr(,"package")
## [1] "arules"Previewing the first 5 transactions
inspect(Transactions[1:5])## items
## [1] {citrus fruit,
## margarine,
## ready soups,
## semi-finished bread}
## [2] {coffee,
## tropical fruit,
## yogurt}
## [3] {whole milk}
## [4] {cream cheese,
## meat spreads,
## pip fruit,
## yogurt}
## [5] {condensed milk,
## long life bakery product,
## other vegetables,
## whole milk}If we wanted to preview the items that make up our dataset, alternatively we can do the following
items<-as.data.frame(itemLabels(Transactions))
colnames(items) <- "Item"
head(items, 10) ## Item
## 1 abrasive cleaner
## 2 artif. sweetener
## 3 baby cosmetics
## 4 baby food
## 5 bags
## 6 baking powder
## 7 bathroom cleaner
## 8 beef
## 9 berries
## 10 beveragesGenerating a summary of the transaction dataset
summary(Transactions)## transactions as itemMatrix in sparse format with
## 9835 rows (elements/itemsets/transactions) and
## 169 columns (items) and a density of 0.02609146
##
## most frequent items:
## whole milk other vegetables rolls/buns soda
## 2513 1903 1809 1715
## yogurt (Other)
## 1372 34055
##
## element (itemset/transaction) length distribution:
## sizes
## 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
## 2159 1643 1299 1005 855 645 545 438 350 246 182 117 78 77 55 46
## 17 18 19 20 21 22 23 24 26 27 28 29 32
## 29 14 14 9 11 4 6 1 1 1 1 3 1
##
## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 1.000 2.000 3.000 4.409 6.000 32.000
##
## includes extended item information - examples:
## labels
## 1 abrasive cleaner
## 2 artif. sweetener
## 3 baby cosmeticsExploring the frequency of some articles i.e. transacations ranging from 8 to 10 and performing some operation in percentage terms of the total transactions
itemFrequency(Transactions[, 8:10],type = "absolute")## beef berries beverages
## 516 327 256round(itemFrequency(Transactions[, 8:10],type = "relative")*100,2)## beef berries beverages
## 5.25 3.32 2.60Displaying top 10 most common items in the transactions dataset and the items whose relative importance is at least 10%
par(mfrow = c(1, 2))plot the frequency of items
itemFrequencyPlot(Transactions, topN = 10,col="darkgreen")
itemFrequencyPlot(Transactions, support = 0.1,col="darkred")
Building a model based on association rules using the apriori function
rules <- apriori (Transactions, parameter = list(supp = 0.001, conf = 0.8))## Apriori
##
## Parameter specification:
## confidence minval smax arem aval originalSupport maxtime support minlen
## 0.8 0.1 1 none FALSE TRUE 5 0.001 1
## maxlen target ext
## 10 rules TRUE
##
## Algorithmic control:
## filter tree heap memopt load sort verbose
## 0.1 TRUE TRUE FALSE TRUE 2 TRUE
##
## Absolute minimum support count: 9
##
## set item appearances ...[0 item(s)] done [0.00s].
## set transactions ...[169 item(s), 9835 transaction(s)] done [0.00s].
## sorting and recoding items ... [157 item(s)] done [0.00s].
## creating transaction tree ... done [0.00s].
## checking subsets of size 1 2 3 4 5 6 done [0.02s].
## writing ... [410 rule(s)] done [0.00s].
## creating S4 object ... done [0.00s].rules## set of 410 rulesWe use measures of significance and interest on the rules, determining which ones are interesting and which to discard. However since we built the model using 0.001 Min support and confidence as 0.8 we obtained 410 rules. However, in order to illustrate the sensitivity of the model to these two parameters, we will see what happens if we increase the support or lower the confidence level
Building a apriori model with Min Support as 0.002 and confidence as 0.8.
rules2 <- apriori (Transactions,parameter = list(supp = 0.002, conf = 0.8)) ## Apriori
##
## Parameter specification:
## confidence minval smax arem aval originalSupport maxtime support minlen
## 0.8 0.1 1 none FALSE TRUE 5 0.002 1
## maxlen target ext
## 10 rules TRUE
##
## Algorithmic control:
## filter tree heap memopt load sort verbose
## 0.1 TRUE TRUE FALSE TRUE 2 TRUE
##
## Absolute minimum support count: 19
##
## set item appearances ...[0 item(s)] done [0.00s].
## set transactions ...[169 item(s), 9835 transaction(s)] done [0.01s].
## sorting and recoding items ... [147 item(s)] done [0.00s].
## creating transaction tree ... done [0.00s].
## checking subsets of size 1 2 3 4 5 done [0.01s].
## writing ... [11 rule(s)] done [0.00s].
## creating S4 object ... done [0.00s].rules2## set of 11 rulesBuilding apriori model with Min Support as 0.002 and confidence as 0.6.
rules3 <- apriori (Transactions, parameter = list(supp = 0.001, conf = 0.6)) ## Apriori
##
## Parameter specification:
## confidence minval smax arem aval originalSupport maxtime support minlen
## 0.6 0.1 1 none FALSE TRUE 5 0.001 1
## maxlen target ext
## 10 rules TRUE
##
## Algorithmic control:
## filter tree heap memopt load sort verbose
## 0.1 TRUE TRUE FALSE TRUE 2 TRUE
##
## Absolute minimum support count: 9
##
## set item appearances ...[0 item(s)] done [0.00s].
## set transactions ...[169 item(s), 9835 transaction(s)] done [0.01s].
## sorting and recoding items ... [157 item(s)] done [0.00s].
## creating transaction tree ... done [0.00s].
## checking subsets of size 1 2 3 4 5 6 done [0.02s].
## writing ... [2918 rule(s)] done [0.00s].
## creating S4 object ... done [0.01s].rules3## set of 2918 rulesIn our first example, we increased the minimum support of 0.001 to 0.002 and model rules went from 410 to only 11. This would lead us to understand that using a high level of support can make the model lose interesting rules. In the second example, we decreased the minimum confidence level to 0.6 and the number of model rules went from 410 to 2918. This would mean that using a low confidence level increases the number of rules to quite an extent and many will not be useful.
We can perform an exploration of our model through the use of the summary function as shown
summary(rules)## set of 410 rules
##
## rule length distribution (lhs + rhs):sizes
## 3 4 5 6
## 29 229 140 12
##
## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 3.000 4.000 4.000 4.329 5.000 6.000
##
## summary of quality measures:
## support confidence coverage lift
## Min. :0.001017 Min. :0.8000 Min. :0.001017 Min. : 3.131
## 1st Qu.:0.001017 1st Qu.:0.8333 1st Qu.:0.001220 1st Qu.: 3.312
## Median :0.001220 Median :0.8462 Median :0.001322 Median : 3.588
## Mean :0.001247 Mean :0.8663 Mean :0.001449 Mean : 3.951
## 3rd Qu.:0.001322 3rd Qu.:0.9091 3rd Qu.:0.001627 3rd Qu.: 4.341
## Max. :0.003152 Max. :1.0000 Max. :0.003559 Max. :11.235
## count
## Min. :10.00
## 1st Qu.:10.00
## Median :12.00
## Mean :12.27
## 3rd Qu.:13.00
## Max. :31.00
##
## mining info:
## data ntransactions support confidence
## Transactions 9835 0.001 0.8
## call
## apriori(data = Transactions, parameter = list(supp = 0.001, conf = 0.8))Observing rules built in our model i.e. first 5 model rules
inspect(rules[1:5])## lhs rhs support confidence
## [1] {liquor, red/blush wine} => {bottled beer} 0.001931876 0.9047619
## [2] {cereals, curd} => {whole milk} 0.001016777 0.9090909
## [3] {cereals, yogurt} => {whole milk} 0.001728521 0.8095238
## [4] {butter, jam} => {whole milk} 0.001016777 0.8333333
## [5] {bottled beer, soups} => {whole milk} 0.001118454 0.9166667
## coverage lift count
## [1] 0.002135231 11.235269 19
## [2] 0.001118454 3.557863 10
## [3] 0.002135231 3.168192 17
## [4] 0.001220132 3.261374 10
## [5] 0.001220132 3.587512 11Interpretation of the first rule:
- If someone buys liquor and red/blush wine, they are 90% likely to buy bottled beer too
We can also use different criteria such as: (by = “lift” or by = “support”)
rules<-sort(rules, by="confidence", decreasing=TRUE)
inspect(rules[1:5])## lhs rhs support confidence coverage lift count
## [1] {rice,
## sugar} => {whole milk} 0.001220132 1 0.001220132 3.913649 12
## [2] {canned fish,
## hygiene articles} => {whole milk} 0.001118454 1 0.001118454 3.913649 11
## [3] {butter,
## rice,
## root vegetables} => {whole milk} 0.001016777 1 0.001016777 3.913649 10
## [4] {flour,
## root vegetables,
## whipped/sour cream} => {whole milk} 0.001728521 1 0.001728521 3.913649 17
## [5] {butter,
## domestic eggs,
## soft cheese} => {whole milk} 0.001016777 1 0.001016777 3.913649 10We can see that the given five rules have a confidence of 100
If we’re interested in making a promotion relating to the sale of yogurt, we could create a subset of rules concerning these products This would tell us the items that the customers bought before purchasing yogurt
yogurt <- subset(rules, subset = rhs %pin% "yogurt")Then order by confidence
yogurt<-sort(yogurt, by="confidence", decreasing=TRUE)
inspect(yogurt[1:5])## lhs rhs support confidence coverage lift count
## [1] {butter,
## cream cheese,
## root vegetables} => {yogurt} 0.001016777 0.9090909 0.001118454 6.516698 10
## [2] {butter,
## sliced cheese,
## tropical fruit,
## whole milk} => {yogurt} 0.001016777 0.9090909 0.001118454 6.516698 10
## [3] {cream cheese,
## curd,
## other vegetables,
## whipped/sour cream} => {yogurt} 0.001016777 0.9090909 0.001118454 6.516698 10
## [4] {butter,
## other vegetables,
## tropical fruit,
## white bread} => {yogurt} 0.001016777 0.9090909 0.001118454 6.516698 10
## [5] {pip fruit,
## sausage,
## sliced cheese} => {yogurt} 0.001220132 0.8571429 0.001423488 6.144315 12We can see that the given five rules have a confidence of 90
What if we wanted to determine items that customers might buy who have previously bought yogurt?
Subset the rules
yogurt <- subset(rules, subset = lhs %pin% "yogurt")Order by confidence
yogurt<-sort(yogurt, by="confidence", decreasing=TRUE)
inspect(yogurt[15:19])## lhs rhs support confidence coverage lift count
## [1] {butter,
## domestic eggs,
## tropical fruit,
## yogurt} => {whole milk} 0.001220132 0.9230769 0.001321810 3.612599 12
## [2] {cream cheese,
## other vegetables,
## pip fruit,
## yogurt} => {whole milk} 0.001118454 0.9166667 0.001220132 3.587512 11
## [3] {curd,
## domestic eggs,
## tropical fruit,
## yogurt} => {whole milk} 0.001118454 0.9166667 0.001220132 3.587512 11
## [4] {butter,
## domestic eggs,
## root vegetables,
## yogurt} => {whole milk} 0.001118454 0.9166667 0.001220132 3.587512 11
## [5] {domestic eggs,
## tropical fruit,
## whipped/sour cream,
## yogurt} => {whole milk} 0.001118454 0.9166667 0.001220132 3.587512 11We can see that the given five rules have a confidence of 92
Challenge 1
Build an apriori model previewing the rules with the highest confidence interval given the following interval. Dataset url = http://bit.ly/AssociativeAnalysisDataset
path <- "http://bit.ly/AssociativeAnalysisDataset"
aad <- read.transactions(path, sep = ",")## Warning in asMethod(object): removing duplicated items in transactionsaad## transactions in sparse format with
## 7501 transactions (rows) and
## 119 items (columns)Verifying class
class(aad)## [1] "transactions"
## attr(,"package")
## [1] "arules"Previewing our first 10 transactions
inspect(aad[1:10])## items
## [1] {almonds,
## antioxydant juice,
## avocado,
## cottage cheese,
## energy drink,
## frozen smoothie,
## green grapes,
## green tea,
## honey,
## low fat yogurt,
## mineral water,
## olive oil,
## salad,
## salmon,
## shrimp,
## spinach,
## tomato juice,
## vegetables mix,
## whole weat flour,
## yams}
## [2] {burgers,
## eggs,
## meatballs}
## [3] {chutney}
## [4] {avocado,
## turkey}
## [5] {energy bar,
## green tea,
## milk,
## mineral water,
## whole wheat rice}
## [6] {low fat yogurt}
## [7] {french fries,
## whole wheat pasta}
## [8] {light cream,
## shallot,
## soup}
## [9] {frozen vegetables,
## green tea,
## spaghetti}
## [10] {french fries}Viewing the items in our data set
items<-as.data.frame(itemLabels(aad))
colnames(items) <- "Item"
head(items, 10) ## Item
## 1 almonds
## 2 antioxydant juice
## 3 asparagus
## 4 avocado
## 5 babies food
## 6 bacon
## 7 barbecue sauce
## 8 black tea
## 9 blueberries
## 10 body spraySummary
summary(aad)## transactions as itemMatrix in sparse format with
## 7501 rows (elements/itemsets/transactions) and
## 119 columns (items) and a density of 0.03288973
##
## most frequent items:
## mineral water eggs spaghetti french fries chocolate
## 1788 1348 1306 1282 1229
## (Other)
## 22405
##
## element (itemset/transaction) length distribution:
## sizes
## 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
## 1754 1358 1044 816 667 493 391 324 259 139 102 67 40 22 17 4
## 18 19 20
## 1 2 1
##
## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 1.000 2.000 3.000 3.914 5.000 20.000
##
## includes extended item information - examples:
## labels
## 1 almonds
## 2 antioxydant juice
## 3 asparagusExploring the frequency
itemFrequency(aad[, 8:10],type = "absolute")## black tea blueberries body spray
## 107 69 86round(itemFrequency(aad[, 8:10],type = "relative")*100,2)## black tea blueberries body spray
## 1.43 0.92 1.15Plotting frequency
par(mfrow = c(1, 2))
itemFrequencyPlot(aad, topN = 10,col="blue")
itemFrequencyPlot(aad, support = 0.1,col="gold")
Building a model based on association rules a. Min Support as 0.001 and confidence as 0.8
rules <- apriori (aad, parameter = list(supp = 0.001, conf = 0.8))## Apriori
##
## Parameter specification:
## confidence minval smax arem aval originalSupport maxtime support minlen
## 0.8 0.1 1 none FALSE TRUE 5 0.001 1
## maxlen target ext
## 10 rules TRUE
##
## Algorithmic control:
## filter tree heap memopt load sort verbose
## 0.1 TRUE TRUE FALSE TRUE 2 TRUE
##
## Absolute minimum support count: 7
##
## set item appearances ...[0 item(s)] done [0.00s].
## set transactions ...[119 item(s), 7501 transaction(s)] done [0.00s].
## sorting and recoding items ... [116 item(s)] done [0.00s].
## creating transaction tree ... done [0.00s].
## checking subsets of size 1 2 3 4 5 6 done [0.01s].
## writing ... [74 rule(s)] done [0.00s].
## creating S4 object ... done [0.00s].- Min Support as 0.002 and confidence as 0.8.
rules2 <- apriori (aad,parameter = list(supp = 0.002, conf = 0.8)) ## Apriori
##
## Parameter specification:
## confidence minval smax arem aval originalSupport maxtime support minlen
## 0.8 0.1 1 none FALSE TRUE 5 0.002 1
## maxlen target ext
## 10 rules TRUE
##
## Algorithmic control:
## filter tree heap memopt load sort verbose
## 0.1 TRUE TRUE FALSE TRUE 2 TRUE
##
## Absolute minimum support count: 15
##
## set item appearances ...[0 item(s)] done [0.00s].
## set transactions ...[119 item(s), 7501 transaction(s)] done [0.00s].
## sorting and recoding items ... [115 item(s)] done [0.00s].
## creating transaction tree ... done [0.00s].
## checking subsets of size 1 2 3 4 5 done [0.00s].
## writing ... [2 rule(s)] done [0.00s].
## creating S4 object ... done [0.00s].- Min Support as 0.002 and confidence as 0.6.
rules3 <- apriori (aad, parameter = list(supp = 0.002, conf = 0.6)) ## Apriori
##
## Parameter specification:
## confidence minval smax arem aval originalSupport maxtime support minlen
## 0.6 0.1 1 none FALSE TRUE 5 0.002 1
## maxlen target ext
## 10 rules TRUE
##
## Algorithmic control:
## filter tree heap memopt load sort verbose
## 0.1 TRUE TRUE FALSE TRUE 2 TRUE
##
## Absolute minimum support count: 15
##
## set item appearances ...[0 item(s)] done [0.00s].
## set transactions ...[119 item(s), 7501 transaction(s)] done [0.00s].
## sorting and recoding items ... [115 item(s)] done [0.00s].
## creating transaction tree ... done [0.00s].
## checking subsets of size 1 2 3 4 5 done [0.00s].
## writing ... [43 rule(s)] done [0.00s].
## creating S4 object ... done [0.00s].summary(rules)## set of 74 rules
##
## rule length distribution (lhs + rhs):sizes
## 3 4 5 6
## 15 42 16 1
##
## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 3.000 4.000 4.000 4.041 4.000 6.000
##
## summary of quality measures:
## support confidence coverage lift
## Min. :0.001067 Min. :0.8000 Min. :0.001067 Min. : 3.356
## 1st Qu.:0.001067 1st Qu.:0.8000 1st Qu.:0.001333 1st Qu.: 3.432
## Median :0.001133 Median :0.8333 Median :0.001333 Median : 3.795
## Mean :0.001256 Mean :0.8504 Mean :0.001479 Mean : 4.823
## 3rd Qu.:0.001333 3rd Qu.:0.8889 3rd Qu.:0.001600 3rd Qu.: 4.877
## Max. :0.002533 Max. :1.0000 Max. :0.002666 Max. :12.722
## count
## Min. : 8.000
## 1st Qu.: 8.000
## Median : 8.500
## Mean : 9.419
## 3rd Qu.:10.000
## Max. :19.000
##
## mining info:
## data ntransactions support confidence
## aad 7501 0.001 0.8
## call
## apriori(data = aad, parameter = list(supp = 0.001, conf = 0.8))inspect(rules[1:5])## lhs rhs support confidence
## [1] {frozen smoothie, spinach} => {mineral water} 0.001066524 0.8888889
## [2] {bacon, pancakes} => {spaghetti} 0.001733102 0.8125000
## [3] {nonfat milk, turkey} => {mineral water} 0.001199840 0.8181818
## [4] {ground beef, nonfat milk} => {mineral water} 0.001599787 0.8571429
## [5] {mushroom cream sauce, pasta} => {escalope} 0.002532996 0.9500000
## coverage lift count
## [1] 0.001199840 3.729058 8
## [2] 0.002133049 4.666587 13
## [3] 0.001466471 3.432428 9
## [4] 0.001866418 3.595877 12
## [5] 0.002666311 11.976387 19If someone buys frozen smoothie, spinach, they are 88% likely to buy mineral water too
Let’s order using other criteria
rules<-sort(rules, by="confidence", decreasing=TRUE)
inspect(rules[1:5])## lhs rhs support confidence coverage lift count
## [1] {french fries,
## mushroom cream sauce,
## pasta} => {escalope} 0.001066524 1.00 0.001066524 12.606723 8
## [2] {ground beef,
## light cream,
## olive oil} => {mineral water} 0.001199840 1.00 0.001199840 4.195190 9
## [3] {cake,
## meatballs,
## mineral water} => {milk} 0.001066524 1.00 0.001066524 7.717078 8
## [4] {cake,
## olive oil,
## shrimp} => {mineral water} 0.001199840 1.00 0.001199840 4.195190 9
## [5] {mushroom cream sauce,
## pasta} => {escalope} 0.002532996 0.95 0.002666311 11.976387 19If someone buys french fries, mushroom cream sauce, pasta, they are 100% likely to buy escalope too
Let’s say we’re interested in making a promotion relating to the sale of milk, we could create a subset of rules concerning these products
milk<- subset(rules, subset = rhs %pin% "milk")Then order by confidence
milk<-sort(milk, by="confidence", decreasing=TRUE)
inspect(milk[1:5])## lhs rhs support confidence
## [1] {cake, meatballs, mineral water} => {milk} 0.001066524 1.0000000
## [2] {escalope, hot dogs, mineral water} => {milk} 0.001066524 0.8888889
## [3] {meatballs, whole wheat pasta} => {milk} 0.001333156 0.8333333
## [4] {black tea, frozen smoothie} => {milk} 0.001199840 0.8181818
## [5] {burgers, ground beef, olive oil} => {milk} 0.001066524 0.8000000
## coverage lift count
## [1] 0.001066524 7.717078 8
## [2] 0.001199840 6.859625 8
## [3] 0.001599787 6.430898 10
## [4] 0.001466471 6.313973 9
## [5] 0.001333156 6.173663 8We can clearly see that if someone buys cake, meatballs, mineral water, they are 100% likely to buy milk so we might focus our promotion on them
What if we wanted to determine items that customers might buy who have previously bought milk?
# Subset the rules
milk <- subset(rules, subset = lhs %pin% "milk")
# Order by confidence
milk<-sort(milk, by="confidence", decreasing=TRUE)
# inspect top 5
inspect(milk[1:5])## lhs rhs support confidence coverage lift count
## [1] {frozen vegetables,
## milk,
## spaghetti,
## turkey} => {mineral water} 0.001199840 0.9000000 0.001333156 3.775671 9
## [2] {cake,
## meatballs,
## milk} => {mineral water} 0.001066524 0.8888889 0.001199840 3.729058 8
## [3] {burgers,
## milk,
## salmon} => {spaghetti} 0.001066524 0.8888889 0.001199840 5.105326 8
## [4] {chocolate,
## ground beef,
## milk,
## mineral water,
## spaghetti} => {frozen vegetables} 0.001066524 0.8888889 0.001199840 9.325253 8
## [5] {ground beef,
## nonfat milk} => {mineral water} 0.001599787 0.8571429 0.001866418 3.595877 12Among the items bought together with milk are frozen vegetables, spaghetti, turkey, cake, meatballs, burgers