Introduction

Dataset comes from UC Irvine Machine Learning Repository. It contains the data about shopping baskets of customers from different countries around the world. In this excercise the data will be analyzed to find recommendations for customers in future shopping.

Reading the data from excel file

data_raw <- read_excel('Data/online_retail_II.xlsx')

Description of the data

Data:
Invoice - Invoice number. Nominal. A 6-digit integral number uniquely assigned to each transaction. If this code starts with the letter ‘C’, it indicates a cancellation
StockCode - Product (item) code. Nominal. A 5-digit integral number uniquely assigned to each distinct product
Description - Product (item) name. Nominal
Quantity - The quantities of each product (item) per transaction. Numeric
InvoiceDate - Invoice date and time. Numeric. The day and time when a transaction was generated
UnitPrice - Unit price. Numeric. Product price per unit in sterling (£)
CustomerID - Customer number. Nominal. A 5-digit integral number uniquely assigned to each customer
Country - Country name. Nominal. The name of the country where a customer resides

Exploring the data

head(data_raw)
## # A tibble: 6 × 8
##   Invoice StockCode Description Quantity InvoiceDate         Price `Customer ID`
##   <chr>   <chr>     <chr>          <dbl> <dttm>              <dbl>         <dbl>
## 1 489434  85048     "15CM CHRI…       12 2009-12-01 07:45:00  6.95         13085
## 2 489434  79323P    "PINK CHER…       12 2009-12-01 07:45:00  6.75         13085
## 3 489434  79323W    "WHITE CHE…       12 2009-12-01 07:45:00  6.75         13085
## 4 489434  22041     "RECORD FR…       48 2009-12-01 07:45:00  2.1          13085
## 5 489434  21232     "STRAWBERR…       24 2009-12-01 07:45:00  1.25         13085
## 6 489434  22064     "PINK DOUG…       24 2009-12-01 07:45:00  1.65         13085
## # ℹ 1 more variable: Country <chr>

Data preparation

data_to_transform <- data_raw
# Convert StockCode column to factor 
data_to_transform$StockCode <- as.factor(data_to_transform$StockCode)

# Create a new column with value 1 indicating the existence of the 
# combination of Invoice and StockCode
data_to_transform$Exist <- 1

# Check how many unique stock codes there are 
length <- length(unique(as.character(data_to_transform$StockCode)))
# 4631 - too many columns 

Exploring countries

# Display all possible countries 
print(unique(data_to_transform$Country))
##  [1] "United Kingdom"       "France"               "USA"                 
##  [4] "Belgium"              "Australia"            "EIRE"                
##  [7] "Germany"              "Portugal"             "Japan"               
## [10] "Denmark"              "Nigeria"              "Netherlands"         
## [13] "Poland"               "Spain"                "Channel Islands"     
## [16] "Italy"                "Cyprus"               "Greece"              
## [19] "Norway"               "Austria"              "Sweden"              
## [22] "United Arab Emirates" "Finland"              "Switzerland"         
## [25] "Unspecified"          "Malta"                "Bahrain"             
## [28] "RSA"                  "Bermuda"              "Hong Kong"           
## [31] "Singapore"            "Thailand"             "Israel"              
## [34] "Lithuania"            "West Indies"          "Lebanon"             
## [37] "Korea"                "Brazil"               "Canada"              
## [40] "Iceland"
# Count products per invoice in each country 
products_per_invoice <- data_raw %>% 
  group_by(Country, Invoice) %>% 
  summarise(total_count=n(),
            .groups = 'drop')

# Calculate average products per invoice in each country 
products_per_invoice_avg <- products_per_invoice %>% 
  group_by(Country) %>% 
  summarise_at(vars(total_count), list(avg_prod = mean))

# Displaying countries with the highest average numbers of products per invoice 
products_per_invoice_avg[order(products_per_invoice_avg$avg_prod, 
                               decreasing = T), ]
## # A tibble: 40 × 2
##    Country     avg_prod
##    <chr>          <dbl>
##  1 Brazil          62  
##  2 West Indies     54  
##  3 Canada          38.5
##  4 Thailand        38  
##  5 Israel          37  
##  6 Iceland         35.5
##  7 Greece          34.5
##  8 Bermuda         34  
##  9 Singapore       29.2
## 10 Malta           28.7
## # ℹ 30 more rows
# Count invoices per country 
invoices_per_country <- data_raw %>% 
  group_by(Country) %>% 
  summarise(distinct_invoices = n_distinct(Invoice))

# Displaying countries with the highest numbers of invoices 
invoices_per_country[order(invoices_per_country$distinct_invoices, 
                           decreasing = T), ]
## # A tibble: 40 × 2
##    Country        distinct_invoices
##    <chr>                      <int>
##  1 United Kingdom             26633
##  2 Germany                      513
##  3 EIRE                         457
##  4 France                       297
##  5 Netherlands                  150
##  6 Spain                         85
##  7 Sweden                        83
##  8 Belgium                       65
##  9 Portugal                      57
## 10 Australia                     50
## # ℹ 30 more rows
# Count StockCodes per country 
stock_per_country <- data_raw %>% 
  group_by(Country) %>% 
  summarise(distinct_codes = n_distinct(StockCode))

# Displaying countries with the lowest numbers of stock codes  
stock_per_country[order(stock_per_country$distinct_codes, 
                           decreasing = F), ]
## # A tibble: 40 × 2
##    Country     distinct_codes
##    <chr>                <int>
##  1 Lebanon                 13
##  2 Nigeria                 32
##  3 Bermuda                 34
##  4 West Indies             49
##  5 Korea                   52
##  6 Hong Kong               60
##  7 Brazil                  61
##  8 Iceland                 70
##  9 Israel                  73
## 10 Thailand                76
## # ℹ 30 more rows

Germany seem to be country with sufficient number of observations.

Germany data preparation

data_ger <- data_to_transform[data_to_transform$Country == 'Germany', ]

# Check how many StockCodes there are in Germany 
unique(as.character(data_ger[['StockCode']]))
##    [1] "85049E"  "21976"   "21498"   "22077"   "84946"   "84948"   "21537"  
##    [8] "21733"   "85123A"  "20718"   "21539"   "16169F"  "84836"   "21527"  
##   [15] "21533"   "21531"   "22180"   "22333"   "22319"   "85232B"  "20676"  
##   [22] "20914"   "21217"   "21238"   "21242"   "22232"   "21535"   "21844"  
##   [29] "22073"   "22072"   "22321"   "15056BL" "15056N"  "21731"   "22254"  
##   [36] "22255"   "46000S"  "20964"   "21788"   "21789"   "20682"   "85014C" 
##   [43] "84832"   "POST"    "22198"   "85232A"  "21491"   "21432"   "21672"  
##   [50] "21671"   "21658"   "22253"   "22348"   "21559"   "21121"   "21212"  
##   [57] "22139"   "22352"   "22114"   "22353"   "21576"   "21577"   "21578"  
##   [64] "21429"   "21481"   "21485"   "21700"   "21973"   "20719"   "21232"  
##   [71] "21231"   "22313"   "22314"   "22200"   "22301"   "22135"   "22331"  
##   [78] "21697"   "22138"   "21080"   "22202"   "22150"   "21210"   "21039"  
##   [85] "21674"   "85038"   "22178"   "22189"   "21693"   "22278"   "21500"  
##   [92] "51014L"  "22299"   "20692"   "20702"   "20703"   "20696"   "21284"  
##   [99] "21286"   "21843"   "21916"   "22354"   "20665"   "20750"   "20681"  
##  [106] "85049A"  "22192"   "22233"   "21258"   "84997A"  "84247L"  "85014B" 
##  [113] "20679"   "22034"   "22042"   "22044"   "16169C"  "21206"   "22090"  
##  [120] "85199L"  "85199S"  "22219"   "22179"   "21913"   "21556"   "21542"  
##  [127] "16169K"  "16169N"  "16169P"  "22045"   "22024"   "22025"   "22027"  
##  [134] "22029"   "22037"   "22075"   "22076"   "22078"   "85049B"  "85049F" 
##  [141] "85049C"  "22242"   "22243"   "22244"   "22245"   "21241"   "21245"  
##  [148] "20674"   "84978"   "21669"   "21223"   "22212"   "21482"   "21945"  
##  [155] "22355"   "22302"   "22303"   "21993"   "21580"   "21124"   "84247C" 
##  [162] "84247K"  "21892"   "22028"   "22064"   "37495"   "22318"   "22131"  
##  [169] "22344"   "21944"   "21785"   "21027"   "21842"   "85099F"  "21928"  
##  [176] "21929"   "21051"   "21249"   "21154"   "21216"   "85099B"  "84997D" 
##  [183] "84997C"  "84997B"  "20754"   "21673"   "21668"   "21285"   "21675"  
##  [190] "84692"   "21077"   "22332"   "21380"   "47590A"  "21126"   "21125"  
##  [197] "20713"   "20711"   "21440"   "84568"   "21680"   "21918"   "21791"  
##  [204] "21525"   "22203"   "22199"   "84029G"  "22193"   "21717"   "21164"  
##  [211] "22349"   "22350"   "85174"   "21115"   "22283"   "21980"   "90167"  
##  [218] "20675"   "20677"   "22236"   "22231"   "84464"   "35922"   "22252"  
##  [225] "84879"   "21157"   "20871"   "20726"   "21276"   "21275"   "20728"  
##  [232] "79191C"  "21936"   "20981"   "21426"   "21424"   "21431"   "84569C" 
##  [239] "22276"   "21935"   "22277"   "84592"   "20724"   "20723"   "21931"  
##  [246] "47594A"  "20658"   "20684"   "21484"   "22320"   "85206A"  "21439"  
##  [253] "22181"   "21156"   "47592A"  "47591D"  "20712"   "21383"   "22021"  
##  [260] "84437A"  "85205B"  "85206B"  "22326"   "22328"   "22329"   "22343"  
##  [267] "81952V"  "22082"   "21937"   "85059"   "35809A"  "35811A"  "84968A" 
##  [274] "81950B"  "79072"   "85162C"  "84968B"  "47559B"  "85162B"  "84279B" 
##  [281] "84279P"  "35241"   "81952B"  "20979"   "21035"   "21752"   "21667"  
##  [288] "85093"   "84531B"  "84691"   "21155"   "22218"   "22215"   "21670"  
##  [295] "20652"   "20653"   "21706"   "47367B"  "21652"   "22083"   "22230"  
##  [302] "22251"   "84466"   "21496"   "35970"   "47594B"  "22358"   "21240"  
##  [309] "21239"   "21243"   "21244"   "84270"   "84968D"  "84968C"  "84968F" 
##  [316] "84969"   "20685"   "20717"   "20725"   "20983"   "21041"   "21058"  
##  [323] "21059"   "21078"   "21085"   "21086"   "21091"   "21092"   "21094"  
##  [330] "21122"   "21153"   "21218"   "21329"   "21330"   "21541"   "21655"  
##  [337] "21768"   "21880"   "21894"   "21896"   "21972"   "21975"   "22079"  
##  [344] "22334"   "22356"   "84970S"  "85015"   "85016"   "85032A"  "85032B" 
##  [351] "85055"   "47590B"  "20857"   "20864"   "21226"   "16237"   "20751"  
##  [358] "21563"   "M"       "21394"   "21395"   "21392"   "21393"   "85221"  
##  [365] "22367"   "21381"   "21130"   "47471"   "20738"   "21592"   "21588"  
##  [372] "22197"   "ADJUST"  "22201"   "21881"   "21884"   "46126A"  "84854"  
##  [379] "85216"   "85217"   "21144"   "47570B"  "47587A"  "84510B"  "22133"  
##  [386] "79320"   "20716"   "20729"   "20730"   "20966"   "20963"   "21942"  
##  [393] "21943"   "21946"   "21165"   "22117"   "84676"   "22204"   "21042"  
##  [400] "21246"   "20672"   "22058"   "22059"   "22062"   "22063"   "37448"  
##  [407] "37450"   "47556B"  "21754"   "22127"   "22273"   "21746"   "21749"  
##  [414] "16256B"  "20977"   "22080"   "16052"   "21204"   "85049H"  "20990"  
##  [421] "21949"   "35810A"  "48194"   "21837"   "22031"   "22033"   "22040"  
##  [428] "22038"   "21610"   "21612"   "84932D"  "84932C"  "84932A"  "84932B" 
##  [435] "84199"   "21401"   "22500"   "84937"   "21445"   "21446"   "21447"  
##  [442] "21529"   "22366"   "22497"   "22495"   "22071"   "22070"   "82581"  
##  [449] "84927A"  "85184D"  "85007"   "21414"   "21415"   "21416"   "22325"  
##  [456] "20734"   "22417"   "21428"   "85231B"  "85231E"  "85231L"  "21763"  
##  [463] "21891"   "21709"   "21708"   "21561"   "21555"   "48185"   "84580"  
##  [470] "40046A"  "21100"   "22385"   "21930"   "84228"   "84461"   "84218"  
##  [477] "85215"   "10002"   "84012"   "22377"   "22378"   "22379"   "22381"  
##  [484] "22384"   "22386"   "20972"   "22149"   "22274"   "22272"   "22271"  
##  [491] "84625C"  "15060B"  "47585A"  "84991"   "22055"   "20828"   "21099"  
##  [498] "21499"   "16169M"  "85049D"  "22128"   "22129"   "22382"   "21883"  
##  [505] "22322"   "21534"   "22260"   "21397"   "21396"   "21528"   "22107"  
##  [512] "84564"   "22300"   "21398"   "21399"   "21906"   "71403"   "84670"  
##  [519] "21064"   "84575B"  "21579"   "22264"   "22256"   "84247G"  "22032"  
##  [526] "22411"   "85226A"  "85136A"  "22431"   "21219"   "22432"   "22371"  
##  [533] "22374"   "22375"   "22376"   "21715"   "35910A"  "84219"   "85200"  
##  [540] "21648"   "84905"   "84909A"  "84657"   "47586A"  "84467"   "84469"  
##  [547] "21524"   "47570"   "16161P"  "20758"   "20772"   "20768"   "20767"  
##  [554] "20755"   "20978"   "20975"   "85175"   "21889"   "37370"   "84934"  
##  [561] "21637"   "20759"   "20769"   "20770"   "22433"   "22136"   "21213"  
##  [568] "22327"   "47579"   "22067"   "84509G"  "84509B"  "37333"   "37500"  
##  [575] "21211"   "84987"   "37447"   "22134"   "47502"   "20689"   "37501"  
##  [582] "37464"   "82613A"  "82613D"  "21137"   "84709B"  "21278"   "37482P" 
##  [589] "21873"   "21872"   "84597C"  "22094"   "84534B"  "21720"   "21721"  
##  [596] "21722"   "21725"   "16156S"  "16161G"  "21495"   "22046"   "84533B" 
##  [603] "84522"   "37503"   "20661"   "20799"   "20798"   "21207"   "21374"  
##  [610] "22383"   "20749"   "47505"   "21082"   "22467"   "22466"   "22468"  
##  [617] "22609"   "21723"   "22509"   "22505"   "22478"   "22477"   "22472"  
##  [624] "22457"   "84988"   "21653"   "84970L"  "21679"   "21830"   "15036"  
##  [631] "40016"   "85227"   "21640"   "21716"   "84529C"  "47591B"  "22507"  
##  [638] "22479"   "47504J"  "21411"   "22315"   "21127"   "21724"   "16238"  
##  [645] "20971"   "22568"   "22569"   "22570"   "22450"   "22451"   "22449"  
##  [652] "20619"   "22473"   "22475"   "22471"   "20998"   "22228"   "22229"  
##  [659] "85169C"  "85169D"  "21319"   "21320"   "21661"   "21662"   "22423"  
##  [666] "22585"   "21981"   "16168M"  "22476"   "22583"   "22584"   "22445"  
##  [673] "22560"   "22556"   "22555"   "48189"   "48195"   "84815"   "22446"  
##  [680] "84536A"  "21471"   "21461"   "21348"   "85049G"  "21564"   "21452"  
##  [687] "21110"   "21281"   "22279"   "20866"   "82486"   "22508"   "22511"  
##  [694] "85129A"  "22317"   "85164B"  "20794"   "21272"   "35810B"  "84050"  
##  [701] "20771"   "22111"   "21523"   "48116"   "48129"   "48187"   "48188"  
##  [708] "48173C"  "48184"   "85162A"  "22122"   "22389"   "22521"   "22522"  
##  [715] "22523"   "22444"   "22559"   "22561"   "21718"   "22540"   "22544"  
##  [722] "22551"   "22554"   "22549"   "22515"   "22516"   "22517"   "22518"  
##  [729] "22519"   "22520"   "22454"   "22453"   "22531"   "22534"   "21914"  
##  [736] "21915"   "84715"   "22036"   "21507"   "21251"   "85232D"  "22557"  
##  [743] "21002"   "21001"   "16161U"  "21621"   "21162"   "21114"   "22403"  
##  [750] "22398"   "22402"   "21696"   "22480"   "22482"   "21488"   "22309"  
##  [757] "22582"   "21690"   "85040B"  "21222"   "21220"   "22536"   "22532"  
##  [764] "17091A"  "21866"   "21202"   "21203"   "84247N"  "16236"   "10125"  
##  [771] "78124"   "22530"   "22562"   "22563"   "22564"   "22567"   "22399"  
##  [778] "22405"   "22617"   "22618"   "47589"   "21614"   "22447"   "22452"  
##  [785] "21983"   "21984"   "21136"   "21932"   "21990"   "21992"   "22336"  
##  [792] "21786"   "21787"   "22611"   "22023"   "21509"   "22047"   "21922"  
##  [799] "21925"   "48138"   "21586"   "21590"   "21877"   "21871"   "21069"  
##  [806] "21067"   "21609"   "21620"   "21615"   "PADS"    "46000M"  "22406"  
##  [813] "22409"   "22360"   "22359"   "21875"   "22612"   "21878"   "22084"  
##  [820] "22357"   "22499"   "21622"   "21623"   "22494"   "22442"   "22443"  
##  [827] "22316"   "21987"   "21988"   "21989"   "21224"   "21703"   "15039"  
##  [834] "22169"   "72741"   "22393"   "22390"   "22448"   "35600D"  "35598D" 
##  [841] "84341B"  "21977"   "22416"   "21974"   "21829"   "79029"   "22606"  
##  [848] "22605"   "22400"   "22410"   "22496"   "84029E"  "47566"   "21741"  
##  [855] "22345"   "22194"   "21354"   "40018F"  "22081"   "84797B"  "84797A" 
##  [862] "21197"   "20961"   "22436"   "22439"   "22211"   "22526"   "22525"  
##  [869] "22524"   "22487"   "22460"   "85017A"  "21135"   "84581"   "21735"  
##  [876] "79030G"  "21307"   "21474"   "22069"   "21833"   "22627"   "22625"  
##  [883] "22504"   "22503"   "22489"   "22491"   "22626"   "21719"   "21562"  
##  [890] "84899E"  "15044B"  "15044D"  "22498"   "22088"   "47566B"  "21096"  
##  [897] "21088"   "79321"   "20973"   "22275"   "22624"   "85099C"  "21882"  
##  [904] "21327"   "84828"   "46118"   "85119"   "21117"   "22492"   "22621"  
##  [911] "22622"   "15056P"  "35015"   "22420"   "84949"   "85062"   "22514"  
##  [918] "15058A"  "21714"   "17109D"  "72750B"  "47567B"  "85090"   "22427"  
##  [925] "22493"   "22424"   "22429"   "22426"   "22425"   "22430"   "84086C" 
##  [932] "10135"   "16045"   "22634"   "22635"   "22607"   "21832"   "84675"  
##  [939] "21923"   "22089"   "22533"   "21328"   "21986"   "21933"   "20974"  
##  [946] "20752"   "84598"   "22440"   "22435"   "22613"   "22649"   "22636"  
##  [953] "22646"   "22644"   "21558"   "84546"   "47421"   "22502"   "22501"  
##  [960] "22324"   "22704"   "22705"   "22711"   "22716"   "21508"   "22437"  
##  [967] "22614"   "22615"   "22620"   "21205"   "21704"   "47591A"  "85178"  
##  [974] "22637"   "21033"   "85132C"  "20985"   "21503"   "85098B"  "84212"  
##  [981] "22419"   "22068"   "84520B"  "72140F"  "72351A"  "72351B"  "21215"  
##  [988] "84989A"  "21291"   "47504K"  "79030A"  "22558"   "84840"   "21742"  
##  [995] "72760B"  "21982"   "22418"   "37474"   "21108"   "21190"   "82578"  
## [1002] "84380"   "47568"   "15034"   "84839"   "85068"   "35637A"  "44230"  
## [1009] "84356"   "20829"   "21836"   "84968E"  "22262"   "84509A"  "84913A" 
## [1016] "84347"   "84247E"  "35833G"  "22086"   "35964"   "21494"   "35400"  
## [1023] "35597B"  "35598B"  "35599B"  "22365"   "22414"   "48197"   "22465"  
## [1030] "20654"   "20727"   "21084"   "21947"   "21967"   "22035"   "22171"  
## [1037] "22650"   "72728"   "72819"   "84929"   "85006"   "85017B"  "85017C" 
## [1044] "85031A"  "85031B"  "85054"   "21090"   "21260"   "21358"   "21506"  
## [1051] "22553"   "22628"   "84912A"  "84819"   "84818"   "22630"   "21410"  
## [1058] "79030D"  "85159B"  "21775"   "84755"   "21687"   "21691"   "85064"  
## [1065] "85067"   "21625"   "85121C"  "21259"   "20997"   "84658"   "21544"  
## [1072] "22052"   "21519"   "21581"   "22546"   "22529"   "22535"   "22537"  
## [1079] "22547"   "85226C"  "22545"   "22629"   "22631"   "22669"   "22596"  
## [1086] "22602"   "22600"   "22603"   "22595"   "22593"   "22130"   "21802"  
## [1093] "22588"   "22589"   "22591"   "21188"   "71459"   "84992"   "22030"  
## [1100] "21497"   "22438"   "85132A"  "85132B"  "79190B"  "79190D"  "21221"  
## [1107] "22667"   "22665"   "22666"   "22488"   "21917"   "22619"   "21106"  
## [1114] "22539"   "22364"   "20886"   "21282"   "21611"   "79000"   "21324"  
## [1121] "22485"   "21827"   "22616"   "21890"   "21845"   "84792"   "82494L" 
## [1128] "21361"   "82484"   "22312"   "84849B"  "84847"   "21116"   "84006"  
## [1135] "22652"   "22659"   "22668"   "20982"   "11001"   "22176"   "22571"  
## [1142] "22581"   "22578"   "22752"   "22757"   "22753"   "22754"   "84859A" 
## [1149] "22759"   "22743"   "22654"   "21430"   "22718"   "22742"   "22744"  
## [1156] "84899F"  "51008"   "47593A"  "22661"   "22712"   "22483"   "22692"  
## [1163] "84661C"  "84661B"  "84661A"  "22380"   "22601"   "22116"   "22413"  
## [1170] "21163"   "21248"   "22391"   "21357"   "21355"   "22132"   "22108"  
## [1177] "22124"   "21025"   "21026"   "21705"   "22221"   "21011"   "22155"  
## [1184] "22653"   "21263"   "82616C"  "22702"   "22700"   "22697"   "22698"  
## [1191] "21269"   "21270"   "22691"   "21707"   "84510A"  "21651"   "20984"  
## [1198] "85087"   "35951"   "21000"   "84912B"  "84685"   "82580"   "35832"  
## [1205] "84945"   "47422"   "84507C"  "22835"   "22865"   "22867"   "37340"  
## [1212] "22813"   "22820"   "16012"   "22749"   "22951"   "22949"   "22908"  
## [1219] "22907"   "22894"   "22897"   "22806"   "22816"   "22809"   "22812"  
## [1226] "22945"   "22734"   "22739"   "22952"   "22950"   "22909"   "22733"  
## [1233] "22731"   "21056"   "22464"   "21739"   "22456"   "22725"   "22726"  
## [1240] "22727"   "22730"   "22191"   "21247"   "22170"   "22719"   "22895"  
## [1247] "22898"   "22814"   "22815"   "22821"   "21486"   "84030E"  "22074"  
## [1254] "22207"   "79191B"  "21070"   "21464"   "22756"   "22834"   "70007"  
## [1261] "22939"   "22113"   "21098"   "21645"   "22857"   "22860"   "22859"  
## [1268] "22538"   "84031A"  "22910"   "90200E"  "20671"   "22807"   "47518F" 
## [1275] "22755"   "82613B"  "22323"   "62094B"  "21166"   "21903"   "22746"  
## [1282] "22745"   "22747"   "22748"   "22741"   "22728"   "22938"   "22572"  
## [1289] "21186"   "22346"   "47599B"  "16259"   "84077"   "16033"   "84826"  
## [1296] "47599A"  "22896"   "22900"   "35599D"  "21618"   "21619"   "84378"  
## [1303] "22707"   "82482"   "22294"   "22173"   "22714"   "22819"   "22984"  
## [1310] "35095A"  "21790"   "35471D"  "35471A"  "85032D"  "72816"   "22946"  
## [1317] "22651"   "22758"   "22043"   "22959"   "22295"   "22304"   "21191"  
## [1324] "84947"   "22766"   "22632"   "22764"   "22849"   "22847"   "22788"  
## [1331] "22740"   "22843"   "22842"   "22904"   "22750"   "22732"   "22940"  
## [1338] "22168"   "22892"   "82551"   "22717"   "22671"   "22675"   "84508B" 
## [1345] "22175"   "84078A"  "20816"   "22735"   "22737"   "22594"   "22145"  
## [1352] "22706"   "15044A"  "22837"   "22049"   "22099"   "22699"   "22781"  
## [1359] "22799"   "71053"   "22690"   "22845"   "22844"   "22840"   "22841"  
## [1366] "22968"   "22110"   "20764"   "20766"   "20760"   "20762"   "20756"  
## [1373] "20761"   "20622"   "20617"   "84406B"  "21467"   "21472"   "22846"  
## [1380] "22396"   "21870"   "22891"   "17003"   "90018A"  "90019A"  "21755"  
## [1387] "22664"   "22663"   "22902"   "22801"   "22798"   "22818"   "22736"  
## [1394] "22738"   "22573"   "22574"   "22576"   "22577"   "22579"   "21801"  
## [1401] "21803"   "22687"   "22580"   "20957"   "22779"   "84569B"  "22307"  
## [1408] "71477"   "21730"   "22751"   "22941"   "22890"   "22776"   "21179"  
## [1415] "84429A"  "22729"   "22137"   "22633"   "22953"   "22906"   "22810"  
## [1422] "22852"   "22853"   "22141"   "21479"   "90104"   "22929"   "22964"  
## [1429] "22966"   "20967"   "22956"   "20867"   "20868"   "22794"   "22913"  
## [1436] "22923"   "22969"   "22962"   "22961"   "21756"   "22922"   "22924"  
## [1443] "22973"   "22767"   "22963"   "22926"   "22914"   "84849D"  "16016"  
## [1450] "22960"   "22972"   "22975"   "22976"   "84375"   "20668"   "21817"  
## [1457] "22337"   "22144"   "84920"   "22943"   "22550"
# 1400 ok 

# Check numbers of each stockcode in Germany 
num_of_stocks <- data_ger %>% 
  group_by(StockCode) %>% 
  summarise(distinct_invoices = n_distinct(Invoice))

# Display products with high number of purchases 
num_of_stocks[order(num_of_stocks$distinct_invoices, decreasing = T), ]
## # A tibble: 1,461 × 2
##    StockCode distinct_invoices
##    <fct>                 <int>
##  1 POST                    280
##  2 22326                   103
##  3 20719                    77
##  4 22328                    69
##  5 21731                    66
##  6 20914                    58
##  7 21238                    57
##  8 22243                    52
##  9 22242                    51
## 10 21578                    50
## # ℹ 1,451 more rows
# I'll select products that appear at least 50 times in the data 
products <- num_of_stocks[num_of_stocks[['distinct_invoices']] >= 50 & 
                            num_of_stocks[['StockCode']] != 'POST', ]
# Filtering the data to only most popular products 
data_filtered <- data_ger[data_ger[['StockCode']] %in% products[['StockCode']], 
                          c('Invoice', 'Description')]

for(i in colnames(data_filtered)) {
  data_filtered[[i]] = as.factor(data_filtered[[i]])
}

# Transforming the data to transaction 
transactions <- transactions(data_filtered, format = 'long')

Displaying information about purchased products

summary(transactions)
## transactions as itemMatrix in sparse format with
##  242 rows (elements/itemsets/transactions) and
##  15 columns (items) and a density of 0.1606061 
## 
## most frequent items:
## ROUND SNACK BOXES SET OF4 WOODLAND             WOODLAND CHARLOTTE BAG 
##                                102                                 77 
##  ROUND SNACK BOXES SET OF 4 FRUITS      RED TOADSTOOL LED NIGHT LIGHT 
##                                 67                                 66 
##   WOODLAND DESIGN  COTTON TOTE BAG                            (Other) 
##                                 50                                221 
## 
## element (itemset/transaction) length distribution:
## sizes
##  1  2  3  4  5  6 
## 79 65 52 23 12 11 
## 
##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
##   1.000   1.000   2.000   2.409   3.000   6.000 
## 
## includes extended item information - examples:
##                              labels
## 1     5 HOOK HANGER MAGIC TOADSTOOL
## 2 5 HOOK HANGER RED MAGIC TOADSTOOL
## 3  HOOK, 5 HANGER , MAGIC TOADSTOOL
## 
## includes extended transaction information - examples:
##   transactionID
## 1        489526
## 2        490395
## 3        490563
# Displaying simple statistics 
itemFrequency(transactions, type="relative")
##       5 HOOK HANGER MAGIC TOADSTOOL   5 HOOK HANGER RED MAGIC TOADSTOOL 
##                         0.061983471                         0.078512397 
##    HOOK, 5 HANGER , MAGIC TOADSTOOL HOOK, 5 HANGER ,MAGIC TOADSTOOL RED 
##                         0.148760331                         0.136363636 
##                   RED RETROSPOT CUP                      RED SPOTTY CUP 
##                         0.041322314                         0.194214876 
##       RED TOADSTOOL LED NIGHT LIGHT    ROUND SNACK BOXES ,SET 4, FRUITS 
##                         0.272727273                         0.008264463 
##   ROUND SNACK BOXES ,SET4, WOODLAND   ROUND SNACK BOXES SET OF 4 FRUITS 
##                         0.004132231                         0.276859504 
##  ROUND SNACK BOXES SET OF4 WOODLAND SET/5 RED RETROSPOT LID GLASS BOWLS 
##                         0.421487603                         0.053719008 
##    SET/5 RED SPOTTY LID GLASS BOWLS              WOODLAND CHARLOTTE BAG 
##                         0.185950413                         0.318181818 
##    WOODLAND DESIGN  COTTON TOTE BAG 
##                         0.206611570
itemFrequency(transactions, type="absolute")
##       5 HOOK HANGER MAGIC TOADSTOOL   5 HOOK HANGER RED MAGIC TOADSTOOL 
##                                  15                                  19 
##    HOOK, 5 HANGER , MAGIC TOADSTOOL HOOK, 5 HANGER ,MAGIC TOADSTOOL RED 
##                                  36                                  33 
##                   RED RETROSPOT CUP                      RED SPOTTY CUP 
##                                  10                                  47 
##       RED TOADSTOOL LED NIGHT LIGHT    ROUND SNACK BOXES ,SET 4, FRUITS 
##                                  66                                   2 
##   ROUND SNACK BOXES ,SET4, WOODLAND   ROUND SNACK BOXES SET OF 4 FRUITS 
##                                   1                                  67 
##  ROUND SNACK BOXES SET OF4 WOODLAND SET/5 RED RETROSPOT LID GLASS BOWLS 
##                                 102                                  13 
##    SET/5 RED SPOTTY LID GLASS BOWLS              WOODLAND CHARLOTTE BAG 
##                                  45                                  77 
##    WOODLAND DESIGN  COTTON TOTE BAG 
##                                  50
# Check item frequency 
itemFrequency(transactions)
##       5 HOOK HANGER MAGIC TOADSTOOL   5 HOOK HANGER RED MAGIC TOADSTOOL 
##                         0.061983471                         0.078512397 
##    HOOK, 5 HANGER , MAGIC TOADSTOOL HOOK, 5 HANGER ,MAGIC TOADSTOOL RED 
##                         0.148760331                         0.136363636 
##                   RED RETROSPOT CUP                      RED SPOTTY CUP 
##                         0.041322314                         0.194214876 
##       RED TOADSTOOL LED NIGHT LIGHT    ROUND SNACK BOXES ,SET 4, FRUITS 
##                         0.272727273                         0.008264463 
##   ROUND SNACK BOXES ,SET4, WOODLAND   ROUND SNACK BOXES SET OF 4 FRUITS 
##                         0.004132231                         0.276859504 
##  ROUND SNACK BOXES SET OF4 WOODLAND SET/5 RED RETROSPOT LID GLASS BOWLS 
##                         0.421487603                         0.053719008 
##    SET/5 RED SPOTTY LID GLASS BOWLS              WOODLAND CHARLOTTE BAG 
##                         0.185950413                         0.318181818 
##    WOODLAND DESIGN  COTTON TOTE BAG 
##                         0.206611570

Plotting frequency of purchased items

# Plot frequency 
itemFrequencyPlot(transactions, support = 0.1)

# Plot top frequency items 
itemFrequencyPlot(transactions, topN = 15)

Association rules mining

# Calculate AR 
ar <- apriori(transactions, 
              parameter = list(support = 0.006, 
                               confidence = 0.25, 
                               minlen = 2))
## Apriori
## 
## Parameter specification:
##  confidence minval smax arem  aval originalSupport maxtime support minlen
##        0.25    0.1    1 none FALSE            TRUE       5   0.006      2
##  maxlen target  ext
##      10  rules TRUE
## 
## Algorithmic control:
##  filter tree heap memopt load sort verbose
##     0.1 TRUE TRUE  FALSE TRUE    2    TRUE
## 
## Absolute minimum support count: 1 
## 
## set item appearances ...[0 item(s)] done [0.00s].
## set transactions ...[15 item(s), 242 transaction(s)] done [0.00s].
## sorting and recoding items ... [14 item(s)] done [0.00s].
## creating transaction tree ... done [0.00s].
## checking subsets of size 1 2 3 4 5 done [0.00s].
## writing ... [572 rule(s)] done [0.00s].
## creating S4 object  ... done [0.00s].
summary(ar)
## set of 572 rules
## 
## rule length distribution (lhs + rhs):sizes
##   2   3   4   5 
##  52 225 245  50 
## 
##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
##   2.000   3.000   4.000   3.512   4.000   5.000 
## 
## summary of quality measures:
##     support           confidence        coverage             lift        
##  Min.   :0.008264   Min.   :0.2500   Min.   :0.008264   Min.   : 0.7857  
##  1st Qu.:0.008264   1st Qu.:0.3383   1st Qu.:0.016529   1st Qu.: 1.4235  
##  Median :0.012397   Median :0.5000   Median :0.024793   Median : 1.8857  
##  Mean   :0.019426   Mean   :0.5365   Mean   :0.045917   Mean   : 2.7151  
##  3rd Qu.:0.020661   3rd Qu.:0.6667   3rd Qu.:0.046488   3rd Qu.: 2.7500  
##  Max.   :0.210744   Max.   :1.0000   Max.   :0.421488   Max.   :16.1333  
##      count       
##  Min.   : 2.000  
##  1st Qu.: 2.000  
##  Median : 3.000  
##  Mean   : 4.701  
##  3rd Qu.: 5.000  
##  Max.   :51.000  
## 
## mining info:
##          data ntransactions support confidence
##  transactions           242   0.006       0.25
##                                                                                            call
##  apriori(data = transactions, parameter = list(support = 0.006, confidence = 0.25, minlen = 2))
# Inspect the rules 
inspect(ar[1:10])
##      lhs                                      rhs                                     support confidence   coverage      lift count
## [1]  {RED RETROSPOT CUP}                   => {5 HOOK HANGER RED MAGIC TOADSTOOL}  0.02066116  0.5000000 0.04132231 6.3684211     5
## [2]  {5 HOOK HANGER RED MAGIC TOADSTOOL}   => {RED RETROSPOT CUP}                  0.02066116  0.2631579 0.07851240 6.3684211     5
## [3]  {RED RETROSPOT CUP}                   => {5 HOOK HANGER MAGIC TOADSTOOL}      0.02066116  0.5000000 0.04132231 8.0666667     5
## [4]  {5 HOOK HANGER MAGIC TOADSTOOL}       => {RED RETROSPOT CUP}                  0.02066116  0.3333333 0.06198347 8.0666667     5
## [5]  {RED RETROSPOT CUP}                   => {WOODLAND CHARLOTTE BAG}             0.01652893  0.4000000 0.04132231 1.2571429     4
## [6]  {RED RETROSPOT CUP}                   => {ROUND SNACK BOXES SET OF4 WOODLAND} 0.02066116  0.5000000 0.04132231 1.1862745     5
## [7]  {SET/5 RED RETROSPOT LID GLASS BOWLS} => {WOODLAND DESIGN  COTTON TOTE BAG}   0.01652893  0.3076923 0.05371901 1.4892308     4
## [8]  {SET/5 RED RETROSPOT LID GLASS BOWLS} => {WOODLAND CHARLOTTE BAG}             0.02479339  0.4615385 0.05371901 1.4505495     6
## [9]  {SET/5 RED RETROSPOT LID GLASS BOWLS} => {ROUND SNACK BOXES SET OF4 WOODLAND} 0.02066116  0.3846154 0.05371901 0.9125189     5
## [10] {5 HOOK HANGER RED MAGIC TOADSTOOL}   => {5 HOOK HANGER MAGIC TOADSTOOL}      0.02892562  0.3684211 0.07851240 5.9438596     7
# Order them by lift 
inspect(sort(ar, by = "lift")[1:5])
##     lhs                                       rhs                                 support confidence    coverage     lift count
## [1] {5 HOOK HANGER RED MAGIC TOADSTOOL,                                                                                        
##      WOODLAND CHARLOTTE BAG}               => {RED RETROSPOT CUP}             0.008264463  0.6666667 0.012396694 16.13333     2
## [2] {5 HOOK HANGER RED MAGIC TOADSTOOL,                                                                                        
##      WOODLAND CHARLOTTE BAG}               => {5 HOOK HANGER MAGIC TOADSTOOL} 0.012396694  1.0000000 0.012396694 16.13333     3
## [3] {5 HOOK HANGER RED MAGIC TOADSTOOL,                                                                                        
##      RED RETROSPOT CUP,                                                                                                        
##      WOODLAND CHARLOTTE BAG}               => {5 HOOK HANGER MAGIC TOADSTOOL} 0.008264463  1.0000000 0.008264463 16.13333     2
## [4] {5 HOOK HANGER MAGIC TOADSTOOL,                                                                                            
##      5 HOOK HANGER RED MAGIC TOADSTOOL,                                                                                        
##      WOODLAND CHARLOTTE BAG}               => {RED RETROSPOT CUP}             0.008264463  0.6666667 0.012396694 16.13333     2
## [5] {ROUND SNACK BOXES SET OF 4 FRUITS,                                                                                        
##      SET/5 RED RETROSPOT LID GLASS BOWLS,                                                                                      
##      WOODLAND CHARLOTTE BAG}               => {5 HOOK HANGER MAGIC TOADSTOOL} 0.008264463  1.0000000 0.008264463 16.13333     2

Recommendations

For the sake of plotting selecting less products is better (more visible), for recommendations let’s take more products.
I’ll select products that appear at least 10 time in the data.

products <- num_of_stocks[num_of_stocks[['distinct_invoices']] >= 10 & 
                            num_of_stocks[['StockCode']] != 'POST', ]

# Filtering the data to only 25 most popular products 
data_filtered <- data_ger[data_ger[['StockCode']] %in% products[['StockCode']], 
                          c('Invoice', 'Description')]

for(i in colnames(data_filtered)) {
  data_filtered[[i]] = as.factor(data_filtered[[i]])
}

# Transforming the data to transaction 
transactions <- transactions(data_filtered, format = 'long')

# Calculate AR 
ar <- apriori(transactions, 
              parameter = list(support = 0.006, 
                               confidence = 0.25, 
                               minlen = 2, 
                               maxlen = 100, 
                               maxtime = 20))
## Apriori
## 
## Parameter specification:
##  confidence minval smax arem  aval originalSupport maxtime support minlen
##        0.25    0.1    1 none FALSE            TRUE      20   0.006      2
##  maxlen target  ext
##     100  rules TRUE
## 
## Algorithmic control:
##  filter tree heap memopt load sort verbose
##     0.1 TRUE TRUE  FALSE TRUE    2    TRUE
## 
## Absolute minimum support count: 2 
## 
## set item appearances ...[0 item(s)] done [0.00s].
## set transactions ...[308 item(s), 439 transaction(s)] done [0.00s].
## sorting and recoding items ... [278 item(s)] done [0.00s].
## creating transaction tree ... done [0.00s].
## checking subsets of size 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 done [0.05s].
## writing ... [1860292 rule(s)] done [0.10s].
## creating S4 object  ... done [0.24s].
summary(ar)
## set of 1860292 rules
## 
## rule length distribution (lhs + rhs):sizes
##      2      3      4      5      6      7      8      9     10     11     12 
##   4331  51083 129652 220734 300972 336889 311360 238149 149700  75944  30228 
##     13     14     15     16 
##   9061   1918    255     16 
## 
##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
##   2.000   6.000   7.000   7.217   9.000  16.000 
## 
## summary of quality measures:
##     support           confidence        coverage             lift        
##  Min.   :0.006834   Min.   :0.2500   Min.   :0.006834   Min.   :  1.076  
##  1st Qu.:0.006834   1st Qu.:1.0000   1st Qu.:0.006834   1st Qu.: 13.303  
##  Median :0.006834   Median :1.0000   Median :0.006834   Median : 17.560  
##  Mean   :0.007023   Mean   :0.9523   Mean   :0.007721   Mean   : 17.599  
##  3rd Qu.:0.006834   3rd Qu.:1.0000   3rd Qu.:0.006834   3rd Qu.: 21.950  
##  Max.   :0.116173   Max.   :1.0000   Max.   :0.232346   Max.   :109.750  
##      count       
##  Min.   : 3.000  
##  1st Qu.: 3.000  
##  Median : 3.000  
##  Mean   : 3.083  
##  3rd Qu.: 3.000  
##  Max.   :51.000  
## 
## mining info:
##          data ntransactions support confidence
##  transactions           439   0.006       0.25
##                                                                                                                        call
##  apriori(data = transactions, parameter = list(support = 0.006, confidence = 0.25, minlen = 2, maxlen = 100, maxtime = 20))

Results of recommendations

# Inspect the rules 
inspect(subset(ar, count >= 10)[1:10])
##      lhs                                 rhs                                 support confidence   coverage      lift count
## [1]  {FELT TOADSTOOL  SMALL}          => {FELT TOADSTOOL LARGE}           0.02733485  0.9230769 0.02961276 31.171598    12
## [2]  {FELT TOADSTOOL LARGE}           => {FELT TOADSTOOL  SMALL}          0.02733485  0.9230769 0.02961276 31.171598    12
## [3]  {SWEETHEART CERAMIC TRINKET BOX} => {STRAWBERRY CERAMIC TRINKET BOX} 0.02733485  0.8000000 0.03416856 10.034286    12
## [4]  {STRAWBERRY CERAMIC TRINKET BOX} => {SWEETHEART CERAMIC TRINKET BOX} 0.02733485  0.3428571 0.07972665 10.034286    12
## [5]  {TV DINNER TRAY DOLLY GIRL}      => {BIG DOUGHNUT FRIDGE MAGNETS}    0.02277904  0.7142857 0.03189066  9.502165    10
## [6]  {BIG DOUGHNUT FRIDGE MAGNETS}    => {TV DINNER TRAY DOLLY GIRL}      0.02277904  0.3030303 0.07517084  9.502165    10
## [7]  {TV DINNER TRAY DOLLY GIRL}      => {MEMO BOARD COTTAGE DESIGN}      0.02277904  0.7142857 0.03189066  9.222689    10
## [8]  {MEMO BOARD COTTAGE DESIGN}      => {TV DINNER TRAY DOLLY GIRL}      0.02277904  0.2941176 0.07744875  9.222689    10
## [9]  {GREEN SPOTTY BOWL}              => {GREEN SPOTTY CUP}               0.02277904  0.7142857 0.03189066 18.445378    10
## [10] {GREEN SPOTTY CUP}               => {GREEN SPOTTY BOWL}              0.02277904  0.5882353 0.03872437 18.445378    10
# Order them by lift 
inspect(sort(subset(ar, count >= 10), by = "lift")[1:5])
##     lhs                        rhs                        support confidence   coverage     lift count
## [1] {FELT TOADSTOOL  SMALL} => {FELT TOADSTOOL LARGE}  0.02733485  0.9230769 0.02961276 31.17160    12
## [2] {FELT TOADSTOOL LARGE}  => {FELT TOADSTOOL  SMALL} 0.02733485  0.9230769 0.02961276 31.17160    12
## [3] {GREEN SPOTTY CUP,                                                                                
##      RED SPOTTY BOWL}       => {GREEN SPOTTY BOWL}     0.02277904  0.9090909 0.02505695 28.50649    10
## [4] {BLUE SPOTTY BOWL,                                                                                
##      PINK  SPOTTY PLATE}    => {BLUE  SPOTTY PLATE}    0.02733485  1.0000000 0.02733485 25.82353    12
## [5] {PINK  SPOTTY PLATE,                                                                              
##      RED SPOTTY BOWL}       => {BLUE  SPOTTY PLATE}    0.02505695  1.0000000 0.02505695 25.82353    11

Recommendations seem reasonable. Most of them are similar items which bring more value if purchased together, because they make a set.