Overview

Market segmentation refers to the process of dividing a consumer market of existing and/or potential customers into groups (or segments) based on shared attributes, interests, and behaviours.

For this mini-project I will use the popular K-Means clustering algorithm to segment customers based on their response to a series of marketing campaigns. The basic concept is that consumers who share common traits would respond to marketing communication in a similar way so that companies can reach out for each group in a relevant and effective way.

library(tidyverse)
library(lubridate)
library(knitr)
library(readxl)
library(broom)
library(umap)
library(ggrepel)
library(plotly)

The Data

The dataset comes from John Foreman’s book, Data Smart. It contains sales promotion data for a fictional wine retailer and includes details of 32 promotions (including wine variety, minimum purchase quantity, percentage discount, and country of origin) and a list of 100 customers and the promotions they responded to.

offers_tbl <- read_excel('../00_data/WineKMC.xlsx', sheet = 'OfferInformation')

offers_tbl <- offers_tbl %>% 
        set_names(c('offer', 'campaign', 'varietal', 'min_qty_kg', 
                    'disc_pct','origin', 'past_peak'))

kable(head(offers_tbl))

transac_tbl <- read_excel('../00_data/WineKMC.xlsx', sheet = 'Transactions')

transac_tbl <- transac_tbl %>% 
        set_names(c('customer', 'offer'))

kable(head(transac_tbl))

The data needs to be converted to a User-Item format(a.k.a. Customer-Product matrix), featuring customers across the top and offers down the side. The cells are populated with 0’s and 1’s, where 1’s indicate if a customer responded to a specific offer.

This type of matrix is also known as binary rating matrix and does NOT require normalisation.

wine_tbl <- transac_tbl %>% 
    left_join(offers_tbl) %>% 
    mutate(value = 1) %>%
    spread(customer,value, fill = 0) 

kable(head(wine_tbl))

Clustering the Customers

The K-Means algorithm comes with the stats package, one of the Core System Libraries in R, and is fairly straightforward to use. I just need to pass a few parameters to the kmeans() function.

user_item_tbl <- wine_tbl[,8:107]

set.seed(196)             # for reproducibility of clusters visualisation with UMAP
kmeans_obj <- user_item_tbl %>%  
    kmeans(centers = 5,   # number of clusters to divide customer list into
           nstart = 100,  # specify number of random sets to be chosen
           iter.max = 50) # maximum number of iterations allowed - 

I can quickly inspect the model withglance()from the broompackage, which provides an summary of model-level statistics

glance(kmeans_obj) %>% glimpse()
## Observations: 1
## Variables: 4
## $ totss        <dbl> 283.1875
## $ tot.withinss <dbl> 189.7255
## $ betweenss    <dbl> 93.46201
## $ iter         <int> 3

The one metric to really keep an eye on is the total within-cluster sum of squares (or tot.withinss) as the optimal number of clusters is that which minimises the tot.withinss. So I want to fit the k-means model for different number of clusters and see where tot.withinss reaches its minimum.

First, I build a function for a set number of centers (4 in this case) and check that is working on glance().

kmeans_map <- function(centers = 4) {
    user_item_tbl %>%  
        kmeans(centers = centers, nstart = 100, iter.max = 50)
}

4 %>% kmeans_map() %>%  glance()
## # A tibble: 1 x 4
##   totss tot.withinss betweenss  iter
##   <dbl>        <dbl>     <dbl> <int>
## 1  283.         203.      80.0     2

Then, I create a nested tibble, which is a way of “nesting” columns inside a data frame. The great thing about nested data frames is that you can put essentially anything you want in them: lists, models, data frames, plots, etc!

kmeans_map_tbl <- tibble(centers = 1:15) %>%  # create column with centres 
          mutate(k_means = centers %>% 
                  map(kmeans_map)) %>%    # iterate `kmeans_map` row-wise to gather 
                                          # kmeans models for each centre in column 2
          mutate(glance = k_means %>%  
                  map(glance))            # apply `glance()` row-wise to gather each
                                          # model’s summary metrics in column 3

kmeans_map_tbl %>% glimpse()

## Observations: 15
## Variables: 3
## $ centers <int> 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15
## $ k_means <list> [<1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
## $ glance  <list> [<tbl_df[1 x 4]>, <tbl_df[1 x 4]>, <tbl_df[1 x 4]>, <...

Last, I can build a scree plot and look for the “elbow” on the graph, the point where the number of additional clusters seem to level off. In this case 5 appears to be an optimal number as the drop in tot.withinss for 6 is not as pronounced as the previous one.

kmeans_map_tbl %>% 
    unnest(glance) %>%                           # unnest the glance column
    select(centers, tot.withinss) %>%            # select centers and tot.withinss
    
    ggplot(aes(x = centers, y = tot.withinss)) + 
    geom_line(colour = 'grey30', size = .8) +
    geom_point(colour = 'green4', size = 3) +
    geom_label_repel(aes(label = centers), 
                     colour = 'grey30') +
    theme_bw() +
    labs(title = 'Scree Plot')

Visualising the Segments

Now that I have identified the optimal number of clusters I want to visualise them. To do so, I use the UMAP ( Uniform Manifold Approximation and Projection ), a dimentionality reduction technique that can be used for cluster visualisation in a similar way to Principal Component Analysis and t-SNE.

First, I create a umap object and pull out the layout argument (containing coordinates that can be used to visualize the dataset), change its format to a tibble and attach the offer column from the wine_tbl.

umap_obj <- user_item_tbl %>%  umap() 

umap_tbl <- umap_obj$layout %>% 
    as_tibble() %>%                       # change to a tibble
    set_names(c('x', 'y')) %>%            # remane columns
    bind_cols(wine_tbl %>% select(offer)) # attach offer reference

Then, I pluck the 5th kmeans model from the nested tibble, attach the cluster argument from the kmeans function to the output, and join offer and cluster to the umap_tbl.

umap_kmeans_5_tbl <- kmeans_map_tbl %>% 
    pull(k_means) %>%
    pluck(5) %>%                          # pluck element 5 
    broom::augment(wine_tbl) %>%          # attach .cluster to the tibble 
    select(offer, .cluster) %>% 
    left_join(umap_tbl, by = 'offer')     # join umap_tbl to clusters by offer

At last, I can visualise the UMAP’ed projections of the clusters. plotly adds some nice interactivity, which brings the chart to life!

g <- umap_kmeans_5_tbl %>% 
    mutate(label_text = str_glue('Offer: {offer}
                                  Cluster: {.cluster}')) %>%
    ggplot(aes(x,y, colour = .cluster)) +
    geom_point(aes(text = label_text)) +
    theme_light() +
    labs(title    = 'UMAP 2D Projections of K-Means Clusters',
         caption  = "") +
    theme(legend.position = 'none')

ggplotly(g, tooltip = "label_text")

Evaluating the Clusters

Now we can finally have a closer look at the single clusters to see what K-Means has identified.

But let’s first bring all information together in one data frame.

cluster_trends_tbl <- wine_tbl %>%
    left_join(umap_kmeans_5_tbl) %>%
    arrange(.cluster) %>%
    select(.cluster, offer:past_peak)

cluster_trends_tbl %>% 
    filter(.cluster ==1 | .cluster ==2) %>% 
    count(.cluster, varietal, origin, min_qty_kg, disc_pct) %>%
    select(-n) %>% 
    kable()

cluster_trends_tbl %>% 
    filter(.cluster ==3 | .cluster ==4 ) %>% 
    group_by() %>%
    count(.cluster, varietal, origin, min_qty_kg, disc_pct) %>%
    select(-n) %>%
    kable()

cluster_trends_tbl %>% 
    filter(.cluster ==5 ) %>% 
    count(.cluster, varietal, origin, min_qty_kg, disc_pct) %>%
    select(-n) %>% 
    kable()

Final thoughts

Although it’s not going to give you all the answers, clustering is a powerful exploratory exercise that can help you reveal patterns in your consumer base, especially when you have a brand new market to explore and do not have any prior knowledge of it.

It’s very easy to implement and even on a small dataset like the one I used here, you can unearth interesting patterns of behaviour in your customer base.

With little effort we have leaned that some of our customers favour certain varieties of wine whereas others prefer to buy high or low quantities. Such information can be used to tailor your pricing strategies and marketing campaings towards those customers that are more inclined to respond. Moreover, customer segmentation allows for a more efficient allocation of marketing resources and the maximization of cross- and up-selling opportunities.

Segmentation can also be enriched by overlaying information such as a customers’ demographics (age, race, religion, gender, family size, ethnicity, income, education level), geography (where they live and work), and psychographic (social class, lifestyle and personality characteristics) but these go beyond the scope of this mini-project.