Group Members

Aby Azid Abu Bakar S2127514
Chee Han Lin 23082140
Chong Kah Hoe 23070711
Izzul Ilham bIn YUsof 22107573
Sean Lee 23057957

Project Title

Predictive modeling and Regression analysis on food products bought by customers

1 Introduction

1.1 Description

For this study, an analysis of a dataset from a company in the retail food sector that sells products from 5 major categories: wines, rare meat products, exotic fruits, specially prepared fish and sweet products will be conducted. The customers can order and acquire products through 3 sales channels: physical stores, catalogs and company’s website. Then a predictive model will be built to support direct marketing initiatives and improve the company’s revenue.

1.2 Objectives

The primary objective of this study is to leverage this data to build predictive models using both classification and regression techniques. Specifically, this study aim to:

Classification Modeling: Determine if a customer would accept the next marketing campaign. This binary classification task will help us understand the factors that influence the effectiveness of marketing campaigns.
Regression Modeling: Predict the amount customer spend at physical stores. By analyzing the relationships between the location of food bought and other relevant features, more effective marketing campaigns can be generated by determine whether customers prefer online stores or physical stores.

This will be done by employing " " to help achieve the result.

2 Data Description

The dataset utilized for this study is a IFood Marketing dataset taken from https://www.kaggle.com/datasets/jackdaoud/marketing-data/data. The dataset contains 2206 observations (customers) with 39 variables related to marketing data. The dataset contains both numeric and categorical variables. The numeric variables are Year_Birth, Income, Kidhome, Teenhome, Recency, MntWines, MntFruits, MntMeatProducts, MntFishproducts, MntSweetProducts, MntGoldProds, NumDealsPurchases, NumWebPurchases, NumCatalogPurchases, NumStorePurchase and NumWebVisitsMont. The categorical variabls are ID, Education, Marital_Status, Dt_Customer, AcceptedCmp1, AcceptedCmp2, AcceptedCmp3, AcceptedCmp4, AcceptedCmp5, Response, Complain and Country.

install required packages dplyr, knitr, summarytools, VIM, import dataset to dataframe

library(dplyr)

## 
## Attaching package: 'dplyr'

## The following objects are masked from 'package:stats':
## 
##     filter, lag

## The following objects are masked from 'package:base':
## 
##     intersect, setdiff, setequal, union

library(knitr)
library(summarytools)
library(tidyr)
library(VIM)

## Loading required package: colorspace

## Loading required package: grid

## VIM is ready to use.

## Suggestions and bug-reports can be submitted at: https://github.com/statistikat/VIM/issues

## 
## Attaching package: 'VIM'

## The following object is masked from 'package:datasets':
## 
##     sleep

library(ggplot2)
library(htmltools)
library(htmlTable)
library(DT)

desc_df <- read.csv("Description.csv")
desc_df[1:29,1:2] %>%knitr::kable()

	Feature	Description
1	ID	Customer’s Unique Identifier
2	Year_Birth	Customer’s Birth Year
3	Education	Customer’s education level
4	Marital_Status	Customer’s marital status
5	Income	Customer’s yearly household income
6	Kidhome	Number of children in customer’s household
7	Teenhome	Number of teenagers in customer’s household
8	Dt_Customer	Date of customer’s enrollment with the company
9	Recency	Number of days since customer’s last purchase
10	MntWines	Amount spent on wine in the last 2 years
11	MntFruits	Amount spent on fruits in the last 2 years
12	MntMeatProducts	Amount spent on meat in the last 2 years
13	MntFishProducts	Amount spent on fish in the last 2 years
14	MntSweetProducts	Amount spent on sweets in the last 2 years
15	MntGoldProds	Amount spent on gold in the last 2 years
16	NumDealsPurchases	Number of purchases made with a discount
17	NumWebPurchases	Number of purchases made through the company’s web site
18	NumCatalogPurchases	Number of purchases made using a catalogue
19	NumStorePurchases	Number of purchases made directly in stores
20	NumWebVisitsMonth	Number of visits to company’s web site in the last month
21	AcceptedCmp1	1 if customer accepted the offer in the 1st campaign, 0 otherwise (Target variable)
22	AcceptedCmp2	1 if customer accepted the offer in the 2nd campaign, 0 otherwise (Target variable)
23	AcceptedCmp3	1 if customer accepted the offer in the 3rd campaign, 0 otherwise (Target variable)
24	AcceptedCmp4	1 if customer accepted the offer in the 4th campaign, 0 otherwise (Target variable)
25	AcceptedCmp5	1 if customer accepted the offer in the 5th campaign, 0 otherwise (Target variable)
26	Response	1 if customer accepted the offer in the last campaign, 0 otherwise (Target variable)
27	Complain	1 if customer complained in the last 2 years, 0 otherwise
28	Country	Customer’s location
NA	NA	NA

2.1 Overview

Using dplyr to get a glimpse of the dataset

df <- read.csv("Izzul_recommend_ml_project1_data.csv")
glimpse(df)

## Rows: 2,240
## Columns: 29
## $ ID                  <int> 5524, 2174, 4141, 6182, 5324, 7446, 965, 6177, 485…
## $ Year_Birth          <int> 1957, 1954, 1965, 1984, 1981, 1967, 1971, 1985, 19…
## $ Education           <chr> "Graduation", "Graduation", "Graduation", "Graduat…
## $ Marital_Status      <chr> "Single", "Single", "Together", "Together", "Marri…
## $ Income              <int> 58138, 46344, 71613, 26646, 58293, 62513, 55635, 3…
## $ Kidhome             <int> 0, 1, 0, 1, 1, 0, 0, 1, 1, 1, 1, 0, 0, 1, 0, 0, 1,…
## $ Teenhome            <int> 0, 1, 0, 0, 0, 1, 1, 0, 0, 1, 0, 0, 0, 1, 0, 0, 1,…
## $ Dt_Customer         <chr> "2012-09-04", "2014-03-08", "2013-08-21", "2014-02…
## $ Recency             <int> 58, 38, 26, 26, 94, 16, 34, 32, 19, 68, 11, 59, 82…
## $ MntWines            <int> 635, 11, 426, 11, 173, 520, 235, 76, 14, 28, 5, 6,…
## $ MntFruits           <int> 88, 1, 49, 4, 43, 42, 65, 10, 0, 0, 5, 16, 61, 2, …
## $ MntMeatProducts     <int> 546, 6, 127, 20, 118, 98, 164, 56, 24, 6, 6, 11, 4…
## $ MntFishProducts     <int> 172, 2, 111, 10, 46, 0, 50, 3, 3, 1, 0, 11, 225, 3…
## $ MntSweetProducts    <int> 88, 1, 21, 3, 27, 42, 49, 1, 3, 1, 2, 1, 112, 5, 1…
## $ MntGoldProds        <int> 88, 6, 42, 5, 15, 14, 27, 23, 2, 13, 1, 16, 30, 14…
## $ NumDealsPurchases   <int> 3, 2, 1, 2, 5, 2, 4, 2, 1, 1, 1, 1, 1, 3, 1, 1, 3,…
## $ NumWebPurchases     <int> 8, 1, 8, 2, 5, 6, 7, 4, 3, 1, 1, 2, 3, 6, 1, 7, 3,…
## $ NumCatalogPurchases <int> 10, 1, 2, 0, 3, 4, 3, 0, 0, 0, 0, 0, 4, 1, 0, 6, 0…
## $ NumStorePurchases   <int> 4, 2, 10, 4, 6, 10, 7, 4, 2, 0, 2, 3, 8, 5, 3, 12,…
## $ NumWebVisitsMonth   <int> 7, 5, 4, 6, 5, 6, 6, 8, 9, 20, 7, 8, 2, 6, 8, 3, 8…
## $ AcceptedCmp3        <int> 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0,…
## $ AcceptedCmp4        <int> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
## $ AcceptedCmp5        <int> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0,…
## $ AcceptedCmp1        <int> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0,…
## $ AcceptedCmp2        <int> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
## $ Complain            <int> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
## $ Z_CostContact       <int> 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3,…
## $ Z_Revenue           <int> 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11…
## $ Response            <int> 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 1, 0,…

Using DT to datatable

datatable(df)

Using base R to get a summary of the dataset

summary(df)

##        ID          Year_Birth    Education         Marital_Status    
##  Min.   :    0   Min.   :1893   Length:2240        Length:2240       
##  1st Qu.: 2828   1st Qu.:1959   Class :character   Class :character  
##  Median : 5458   Median :1970   Mode  :character   Mode  :character  
##  Mean   : 5592   Mean   :1969                                        
##  3rd Qu.: 8428   3rd Qu.:1977                                        
##  Max.   :11191   Max.   :1996                                        
##                                                                      
##      Income          Kidhome          Teenhome      Dt_Customer       
##  Min.   :  1730   Min.   :0.0000   Min.   :0.0000   Length:2240       
##  1st Qu.: 35303   1st Qu.:0.0000   1st Qu.:0.0000   Class :character  
##  Median : 51382   Median :0.0000   Median :0.0000   Mode  :character  
##  Mean   : 52247   Mean   :0.4442   Mean   :0.5062                     
##  3rd Qu.: 68522   3rd Qu.:1.0000   3rd Qu.:1.0000                     
##  Max.   :666666   Max.   :2.0000   Max.   :2.0000                     
##  NA's   :24                                                           
##     Recency         MntWines         MntFruits     MntMeatProducts 
##  Min.   : 0.00   Min.   :   0.00   Min.   :  0.0   Min.   :   0.0  
##  1st Qu.:24.00   1st Qu.:  23.75   1st Qu.:  1.0   1st Qu.:  16.0  
##  Median :49.00   Median : 173.50   Median :  8.0   Median :  67.0  
##  Mean   :49.11   Mean   : 303.94   Mean   : 26.3   Mean   : 166.9  
##  3rd Qu.:74.00   3rd Qu.: 504.25   3rd Qu.: 33.0   3rd Qu.: 232.0  
##  Max.   :99.00   Max.   :1493.00   Max.   :199.0   Max.   :1725.0  
##                                                                    
##  MntFishProducts  MntSweetProducts  MntGoldProds    NumDealsPurchases
##  Min.   :  0.00   Min.   :  0.00   Min.   :  0.00   Min.   : 0.000   
##  1st Qu.:  3.00   1st Qu.:  1.00   1st Qu.:  9.00   1st Qu.: 1.000   
##  Median : 12.00   Median :  8.00   Median : 24.00   Median : 2.000   
##  Mean   : 37.53   Mean   : 27.06   Mean   : 44.02   Mean   : 2.325   
##  3rd Qu.: 50.00   3rd Qu.: 33.00   3rd Qu.: 56.00   3rd Qu.: 3.000   
##  Max.   :259.00   Max.   :263.00   Max.   :362.00   Max.   :15.000   
##                                                                      
##  NumWebPurchases  NumCatalogPurchases NumStorePurchases NumWebVisitsMonth
##  Min.   : 0.000   Min.   : 0.000      Min.   : 0.00     Min.   : 0.000   
##  1st Qu.: 2.000   1st Qu.: 0.000      1st Qu.: 3.00     1st Qu.: 3.000   
##  Median : 4.000   Median : 2.000      Median : 5.00     Median : 6.000   
##  Mean   : 4.085   Mean   : 2.662      Mean   : 5.79     Mean   : 5.317   
##  3rd Qu.: 6.000   3rd Qu.: 4.000      3rd Qu.: 8.00     3rd Qu.: 7.000   
##  Max.   :27.000   Max.   :28.000      Max.   :13.00     Max.   :20.000   
##                                                                          
##   AcceptedCmp3      AcceptedCmp4      AcceptedCmp5      AcceptedCmp1    
##  Min.   :0.00000   Min.   :0.00000   Min.   :0.00000   Min.   :0.00000  
##  1st Qu.:0.00000   1st Qu.:0.00000   1st Qu.:0.00000   1st Qu.:0.00000  
##  Median :0.00000   Median :0.00000   Median :0.00000   Median :0.00000  
##  Mean   :0.07277   Mean   :0.07455   Mean   :0.07277   Mean   :0.06429  
##  3rd Qu.:0.00000   3rd Qu.:0.00000   3rd Qu.:0.00000   3rd Qu.:0.00000  
##  Max.   :1.00000   Max.   :1.00000   Max.   :1.00000   Max.   :1.00000  
##                                                                         
##   AcceptedCmp2        Complain        Z_CostContact   Z_Revenue 
##  Min.   :0.00000   Min.   :0.000000   Min.   :3     Min.   :11  
##  1st Qu.:0.00000   1st Qu.:0.000000   1st Qu.:3     1st Qu.:11  
##  Median :0.00000   Median :0.000000   Median :3     Median :11  
##  Mean   :0.01339   Mean   :0.009375   Mean   :3     Mean   :11  
##  3rd Qu.:0.00000   3rd Qu.:0.000000   3rd Qu.:3     3rd Qu.:11  
##  Max.   :1.00000   Max.   :1.000000   Max.   :3     Max.   :11  
##                                                                 
##     Response     
##  Min.   :0.0000  
##  1st Qu.:0.0000  
##  Median :0.0000  
##  Mean   :0.1491  
##  3rd Qu.:0.0000  
##  Max.   :1.0000  
##

Using summarytools for a more detailed description

#html_output <- htmlTable(dfSummary(df))
#html_output
print(dfSummary(df), method = "viewer")

## Switching method to 'browser'

## Output file written: C:\Users\chongkah\AppData\Local\Temp\RtmpqwIETM\fileb94c52283380.html

cat(readLines("Data Frame Summary1.html"), sep = "\n")

## Warning in readLines("Data Frame Summary1.html"): incomplete final line found
## on 'Data Frame Summary1.html'

<!DOCTYPE html> Data Frame Summary

Data Frame Summary

df

Dimensions: 2240 x 29
Duplicates: 0

No

Variable

Stats / Values

Freqs (% of Valid)

Graph

Valid

Missing

1

ID [integer]

Mean (sd) : 5592.2 (3246.7)

min ≤ med ≤ max:

0 ≤ 5458.5 ≤ 11191

IQR (CV) : 5599.5 (0.6)

2240 distinct values

2240 (100.0%)

0 (0.0%)

2

Year_Birth [integer]

Mean (sd) : 1968.8 (12)

min ≤ med ≤ max:

1893 ≤ 1970 ≤ 1996

IQR (CV) : 18 (0)

59 distinct values

2240 (100.0%)

0 (0.0%)

3

Education [character]

2n Cycle

Basic

Graduation

Master

PhD

203	(	9.1%	)
54	(	2.4%	)
1127	(	50.3%	)
370	(	16.5%	)
486	(	21.7%	)

2240 (100.0%)

0 (0.0%)

4

Marital_Status [character]

Absurd

Alone

Divorced

Married

Single

Together

Widow

YOLO

2	(	0.1%	)
3	(	0.1%	)
232	(	10.4%	)
864	(	38.6%	)
480	(	21.4%	)
580	(	25.9%	)
77	(	3.4%	)
2	(	0.1%	)

2240 (100.0%)

0 (0.0%)

5

Income [integer]

Mean (sd) : 52247.3 (25173.1)

min ≤ med ≤ max:

1730 ≤ 51381.5 ≤ 666666

IQR (CV) : 33219 (0.5)

1974 distinct values

2216 (98.9%)

24 (1.1%)

6

Kidhome [integer]

Mean (sd) : 0.4 (0.5)

min ≤ med ≤ max:

0 ≤ 0 ≤ 2

IQR (CV) : 1 (1.2)

0	:	1293	(	57.7%	)
1	:	899	(	40.1%	)
2	:	48	(	2.1%	)

2240 (100.0%)

0 (0.0%)

7

Teenhome [integer]

Mean (sd) : 0.5 (0.5)

min ≤ med ≤ max:

0 ≤ 0 ≤ 2

IQR (CV) : 1 (1.1)

0	:	1158	(	51.7%	)
1	:	1030	(	46.0%	)
2	:	52	(	2.3%	)

2240 (100.0%)

0 (0.0%)

8

Dt_Customer [character]

2012-08-31
2012-09-12
2013-02-14
2014-05-12
2013-08-20
2014-05-22
2012-10-29
2013-01-02
2014-03-01
2014-03-23
[ 653 others ]

12	(	0.5%	)
11	(	0.5%	)
11	(	0.5%	)
11	(	0.5%	)
10	(	0.4%	)
10	(	0.4%	)
9	(	0.4%	)
9	(	0.4%	)
9	(	0.4%	)
9	(	0.4%	)
2139	(	95.5%	)

2240 (100.0%)

0 (0.0%)

9

Recency [integer]

Mean (sd) : 49.1 (29)

min ≤ med ≤ max:

0 ≤ 49 ≤ 99

IQR (CV) : 50 (0.6)

100 distinct values

2240 (100.0%)

0 (0.0%)

10

MntWines [integer]

Mean (sd) : 303.9 (336.6)

min ≤ med ≤ max:

0 ≤ 173.5 ≤ 1493

IQR (CV) : 480.5 (1.1)

776 distinct values

2240 (100.0%)

0 (0.0%)

11

MntFruits [integer]

Mean (sd) : 26.3 (39.8)

min ≤ med ≤ max:

0 ≤ 8 ≤ 199

IQR (CV) : 32 (1.5)

158 distinct values

2240 (100.0%)

0 (0.0%)

12

MntMeatProducts [integer]

Mean (sd) : 166.9 (225.7)

min ≤ med ≤ max:

0 ≤ 67 ≤ 1725

IQR (CV) : 216 (1.4)

558 distinct values

2240 (100.0%)

0 (0.0%)

13

MntFishProducts [integer]

Mean (sd) : 37.5 (54.6)

min ≤ med ≤ max:

0 ≤ 12 ≤ 259

IQR (CV) : 47 (1.5)

182 distinct values

2240 (100.0%)

0 (0.0%)

14

MntSweetProducts [integer]

Mean (sd) : 27.1 (41.3)

min ≤ med ≤ max:

0 ≤ 8 ≤ 263

IQR (CV) : 32 (1.5)

177 distinct values

2240 (100.0%)

0 (0.0%)

15

MntGoldProds [integer]

Mean (sd) : 44 (52.2)

min ≤ med ≤ max:

0 ≤ 24 ≤ 362

IQR (CV) : 47 (1.2)

213 distinct values

2240 (100.0%)

0 (0.0%)

16

NumDealsPurchases [integer]

Mean (sd) : 2.3 (1.9)

min ≤ med ≤ max:

0 ≤ 2 ≤ 15

IQR (CV) : 2 (0.8)

15 distinct values

2240 (100.0%)

0 (0.0%)

17

NumWebPurchases [integer]

Mean (sd) : 4.1 (2.8)

min ≤ med ≤ max:

0 ≤ 4 ≤ 27

IQR (CV) : 4 (0.7)

15 distinct values

2240 (100.0%)

0 (0.0%)

18

NumCatalogPurchases [integer]

Mean (sd) : 2.7 (2.9)

min ≤ med ≤ max:

0 ≤ 2 ≤ 28

IQR (CV) : 4 (1.1)

14 distinct values

2240 (100.0%)

0 (0.0%)

19

NumStorePurchases [integer]

Mean (sd) : 5.8 (3.3)

min ≤ med ≤ max:

0 ≤ 5 ≤ 13

IQR (CV) : 5 (0.6)

14 distinct values

2240 (100.0%)

0 (0.0%)

20

NumWebVisitsMonth [integer]

Mean (sd) : 5.3 (2.4)

min ≤ med ≤ max:

0 ≤ 6 ≤ 20

IQR (CV) : 4 (0.5)

16 distinct values

2240 (100.0%)

0 (0.0%)

21

AcceptedCmp3 [integer]

Min : 0

Mean : 0.1

Max : 1

0	:	2077	(	92.7%	)
1	:	163	(	7.3%	)

2240 (100.0%)

0 (0.0%)

22

AcceptedCmp4 [integer]

Min : 0

Mean : 0.1

Max : 1

0	:	2073	(	92.5%	)
1	:	167	(	7.5%	)

2240 (100.0%)

0 (0.0%)

23

AcceptedCmp5 [integer]

Min : 0

Mean : 0.1

Max : 1

0	:	2077	(	92.7%	)
1	:	163	(	7.3%	)

2240 (100.0%)

0 (0.0%)

24

AcceptedCmp1 [integer]

Min : 0

Mean : 0.1

Max : 1

0	:	2096	(	93.6%	)
1	:	144	(	6.4%	)

2240 (100.0%)

0 (0.0%)

25

AcceptedCmp2 [integer]

Min : 0

Mean : 0

Max : 1

0	:	2210	(	98.7%	)
1	:	30	(	1.3%	)

2240 (100.0%)

0 (0.0%)

26

Complain [integer]

Min : 0

Mean : 0

Max : 1

0	:	2219	(	99.1%	)
1	:	21	(	0.9%	)

2240 (100.0%)

0 (0.0%)

27

Z_CostContact [integer]

1 distinct value

3

:

2240

(

100.0%

)

2240 (100.0%)

0 (0.0%)

28

Z_Revenue [integer]

1 distinct value

11

:

2240

(

100.0%

)

2240 (100.0%)

0 (0.0%)

29

Response [integer]

Min : 0

Mean : 0.1

Max : 1

0	:	1906	(	85.1%	)
1	:	334	(	14.9%	)

2240 (100.0%)

0 (0.0%)

Generated by summarytools 1.0.1 (R version 4.3.3)
2024-05-28

3 Data Preprocessing

3.1 Missing Data

3.1.1 Show all Missing Data

# Display the total missing values for each variable
data.frame("variable"=c(colnames(df)), 
           "missing values count"=sapply(df, function(x) sum(is.na(x))),
           row.names=NULL)

##               variable missing.values.count
## 1                   ID                    0
## 2           Year_Birth                    0
## 3            Education                    0
## 4       Marital_Status                    0
## 5               Income                   24
## 6              Kidhome                    0
## 7             Teenhome                    0
## 8          Dt_Customer                    0
## 9              Recency                    0
## 10            MntWines                    0
## 11           MntFruits                    0
## 12     MntMeatProducts                    0
## 13     MntFishProducts                    0
## 14    MntSweetProducts                    0
## 15        MntGoldProds                    0
## 16   NumDealsPurchases                    0
## 17     NumWebPurchases                    0
## 18 NumCatalogPurchases                    0
## 19   NumStorePurchases                    0
## 20   NumWebVisitsMonth                    0
## 21        AcceptedCmp3                    0
## 22        AcceptedCmp4                    0
## 23        AcceptedCmp5                    0
## 24        AcceptedCmp1                    0
## 25        AcceptedCmp2                    0
## 26            Complain                    0
## 27       Z_CostContact                    0
## 28           Z_Revenue                    0
## 29            Response                    0

3.1.2 Apply k-NN imputation to fill in missing data for [income] (only income having 24 missing value) since small dataset, therefore using k=5

df <- kNN(df, variable = "Income", k = 5, imp_var = FALSE)

3.1.3 Display all Missing Data after imputation

# Display the total missing values for each variable
data.frame("variable"=c(colnames(df)), 
           "missing values count"=sapply(df, function(x) sum(is.na(x))),
           row.names=NULL)

##               variable missing.values.count
## 1                   ID                    0
## 2           Year_Birth                    0
## 3            Education                    0
## 4       Marital_Status                    0
## 5               Income                    0
## 6              Kidhome                    0
## 7             Teenhome                    0
## 8          Dt_Customer                    0
## 9              Recency                    0
## 10            MntWines                    0
## 11           MntFruits                    0
## 12     MntMeatProducts                    0
## 13     MntFishProducts                    0
## 14    MntSweetProducts                    0
## 15        MntGoldProds                    0
## 16   NumDealsPurchases                    0
## 17     NumWebPurchases                    0
## 18 NumCatalogPurchases                    0
## 19   NumStorePurchases                    0
## 20   NumWebVisitsMonth                    0
## 21        AcceptedCmp3                    0
## 22        AcceptedCmp4                    0
## 23        AcceptedCmp5                    0
## 24        AcceptedCmp1                    0
## 25        AcceptedCmp2                    0
## 26            Complain                    0
## 27       Z_CostContact                    0
## 28           Z_Revenue                    0
## 29            Response                    0

3.2 Data Transformation

3.2.1 Remove the irrelevant ID column

df <- df %>%select(-ID)

3.2.2 Convert the Dt_Customer column to Date format, Income column to numeric

df <- df %>%mutate(Dt_Customer = as.Date(Dt_Customer, format = "%Y-%m-%d"))
df <- df %>%mutate(Income = as.numeric(Income))

3.2.3 Transform value for Marital_Status

custom_encoding <- c( "Divorced" = 1, "Single" = 1, "Married" = 2, "Together" = 2, "Widow" = 1, "YOLO" = 1, "Alone" = 1, "Absurd" = 1)
df <- df %>%mutate(Marital_Status = custom_encoding[Marital_Status])

3.2.4 Find out outlier with boxplot

# Create a function to plot boxplots for a specified number of numeric variables excluding specified variables
plot_boxplots_exclude <- function(data, exclude_vars = c(), num_per_page = 4) {
  # Get numeric variables excluding the ones to be excluded
  numeric_vars <- names(data)[sapply(data, is.numeric) & !names(data) %in% exclude_vars]
  
  # Calculate the number of pages needed
  num_plots <- length(numeric_vars)
  num_pages <- ceiling(num_plots / num_per_page)
  
  # Loop through each page
  for (page in 1:num_pages) {
    # Determine the variables to plot on this page
    start_index <- (page - 1) * num_per_page + 1
    end_index <- min(page * num_per_page, num_plots)
    vars_to_plot <- numeric_vars[start_index:end_index]
    
    # Set up the plotting layout with adjusted margins
    par(mfrow = c(ceiling(length(vars_to_plot) / 2), 2), mar = c(4, 4, 2, 1))
    
    # Loop through each numeric variable on this page and create a boxplot
    for (col in vars_to_plot) {
      boxplot(data[[col]], main = col)
    }
    
    # Reset plotting layout
    par(mfrow = c(1, 1))
  }
}

# Usage example excluding specific variables, with 4 variables per page
exclude_vars <- c('AcceptedCmp1', 'AcceptedCmp2', 'AcceptedCmp3', 'AcceptedCmp4', 
                  'AcceptedCmp5', 'Response', 'Complain', 'Education', 'Country')
plot_boxplots_exclude(df, exclude_vars, num_per_page = 4)

3.2.5 Handling outlier with IQR - Only handle Year_Birth while others outliers value is make sense data

# Create a function to clean specific variables using IQR method
clean_specific_variables <- function(data, vars_to_clean) {
  cleaned_data <- data  # Create a copy of the dataset
  
  # Loop through each variable to clean
  for (col in vars_to_clean) {
    # Calculate interquartile range (IQR)
    Q1 <- quantile(data[[col]], 0.25, na.rm = TRUE)
    Q3 <- quantile(data[[col]], 0.75, na.rm = TRUE)
    IQR <- Q3 - Q1
    
    # Define outlier thresholds
    lower_threshold <- Q1 - 1.5 * IQR
    upper_threshold <- Q3 + 1.5 * IQR
    
    # Identify outliers
    outliers <- data[[col]] < lower_threshold | data[[col]] > upper_threshold
    
    # Replace outliers with NA
    cleaned_data[[col]][outliers] <- NA
    
    # Replace NA values with median of the variable
    cleaned_data[[col]] <- ifelse(is.na(cleaned_data[[col]]), median(data[[col]], na.rm = TRUE), cleaned_data[[col]])
  }
  
  return(cleaned_data)
}

# Target variables to clean using IQR
vars_to_clean <- c("Year_Birth")
# Clean Outliner  variables using IQR
clean_data <- clean_specific_variables(df, vars_to_clean)

3.2.6 Remove outliers for income more than 20000

clean_data <- clean_data[clean_data$Income <= 200000, ]

3.3 Feature Engineering - New variable

# 1. Age variable instead of year of birth
# 2. Customer_Days variable instead of year of birth
# 3. Customer_Days variable instead of year of birth
# 4. FamilyMember variable the total number of family members
# 5. TotalPurchases variable Total places of purchase
# 6. Spending variable the sum of the amount spent on product categories
# 7. AcceptedCmpOverall variable Total amount spent on all the products
#Age of customer
clean_data$Age <- as.integer(format(Sys.Date(), "%Y")) - clean_data$Year_Birth

#number of days since registration as a customer
clean_data$Customer_Days <- as.integer(Sys.Date() - clean_data$Dt_Customer)

# Create the new variable TotalPurchases
clean_data$FamilyMember <- clean_data$Marital_Status + clean_data$Kidhome + clean_data$Teenhome

# Create the new variable TotalPurchases
clean_data$TotalPurchases <- clean_data$NumWebPurchases + clean_data$NumCatalogPurchases + clean_data$NumStorePurchases

#Total amount spent on all the products
clean_data$Spending <- rowSums(clean_data[, grep("^Mnt", names(clean_data))])


#Total overall number of accepted campaigns
clean_data$AcceptedCmpOverall <- rowSums(clean_data[, grep("^Accept|^Response", names(clean_data))])

3.4 Remove unwanted variable after tranformation

clean_data <- subset(clean_data, select = -c(Year_Birth, Dt_Customer))
summary(clean_data)

##   Education         Marital_Status      Income          Kidhome      
##  Length:2239        Min.   :1.000   Min.   :  1730   Min.   :0.0000  
##  Class :character   1st Qu.:1.000   1st Qu.: 35178   1st Qu.:0.0000  
##  Mode  :character   Median :2.000   Median : 51267   Median :0.0000  
##                     Mean   :1.644   Mean   : 51856   Mean   :0.4439  
##                     3rd Qu.:2.000   3rd Qu.: 68334   3rd Qu.:1.0000  
##                     Max.   :2.000   Max.   :162397   Max.   :2.0000  
##     Teenhome         Recency         MntWines        MntFruits     
##  Min.   :0.0000   Min.   : 0.00   Min.   :   0.0   Min.   :  0.00  
##  1st Qu.:0.0000   1st Qu.:24.00   1st Qu.:  24.0   1st Qu.:  1.00  
##  Median :0.0000   Median :49.00   Median : 174.0   Median :  8.00  
##  Mean   :0.5065   Mean   :49.12   Mean   : 304.1   Mean   : 26.31  
##  3rd Qu.:1.0000   3rd Qu.:74.00   3rd Qu.: 504.5   3rd Qu.: 33.00  
##  Max.   :2.0000   Max.   :99.00   Max.   :1493.0   Max.   :199.00  
##  MntMeatProducts MntFishProducts  MntSweetProducts  MntGoldProds   
##  Min.   :   0    Min.   :  0.00   Min.   :  0.00   Min.   :  0.00  
##  1st Qu.:  16    1st Qu.:  3.00   1st Qu.:  1.00   1st Qu.:  9.00  
##  Median :  67    Median : 12.00   Median :  8.00   Median : 24.00  
##  Mean   : 167    Mean   : 37.54   Mean   : 27.07   Mean   : 44.04  
##  3rd Qu.: 232    3rd Qu.: 50.00   3rd Qu.: 33.00   3rd Qu.: 56.00  
##  Max.   :1725    Max.   :259.00   Max.   :263.00   Max.   :362.00  
##  NumDealsPurchases NumWebPurchases  NumCatalogPurchases NumStorePurchases
##  Min.   : 0.000    Min.   : 0.000   Min.   : 0.000      Min.   : 0.000   
##  1st Qu.: 1.000    1st Qu.: 2.000   1st Qu.: 0.000      1st Qu.: 3.000   
##  Median : 2.000    Median : 4.000   Median : 2.000      Median : 5.000   
##  Mean   : 2.324    Mean   : 4.085   Mean   : 2.663      Mean   : 5.791   
##  3rd Qu.: 3.000    3rd Qu.: 6.000   3rd Qu.: 4.000      3rd Qu.: 8.000   
##  Max.   :15.000    Max.   :27.000   Max.   :28.000      Max.   :13.000   
##  NumWebVisitsMonth  AcceptedCmp3     AcceptedCmp4      AcceptedCmp5   
##  Min.   : 0.000    Min.   :0.0000   Min.   :0.00000   Min.   :0.0000  
##  1st Qu.: 3.000    1st Qu.:0.0000   1st Qu.:0.00000   1st Qu.:0.0000  
##  Median : 6.000    Median :0.0000   Median :0.00000   Median :0.0000  
##  Mean   : 5.316    Mean   :0.0728   Mean   :0.07459   Mean   :0.0728  
##  3rd Qu.: 7.000    3rd Qu.:0.0000   3rd Qu.:0.00000   3rd Qu.:0.0000  
##  Max.   :20.000    Max.   :1.0000   Max.   :1.00000   Max.   :1.0000  
##   AcceptedCmp1      AcceptedCmp2       Complain        Z_CostContact
##  Min.   :0.00000   Min.   :0.0000   Min.   :0.000000   Min.   :3    
##  1st Qu.:0.00000   1st Qu.:0.0000   1st Qu.:0.000000   1st Qu.:3    
##  Median :0.00000   Median :0.0000   Median :0.000000   Median :3    
##  Mean   :0.06431   Mean   :0.0134   Mean   :0.009379   Mean   :3    
##  3rd Qu.:0.00000   3rd Qu.:0.0000   3rd Qu.:0.000000   3rd Qu.:3    
##  Max.   :1.00000   Max.   :1.0000   Max.   :1.000000   Max.   :3    
##    Z_Revenue     Response           Age       Customer_Days   FamilyMember  
##  Min.   :11   Min.   :0.0000   Min.   :28.0   Min.   :3621   Min.   :1.000  
##  1st Qu.:11   1st Qu.:0.0000   1st Qu.:47.0   1st Qu.:3802   1st Qu.:2.000  
##  Median :11   Median :0.0000   Median :54.0   Median :3976   Median :3.000  
##  Mean   :11   Mean   :0.1492   Mean   :55.1   Mean   :3975   Mean   :2.595  
##  3rd Qu.:11   3rd Qu.:0.0000   3rd Qu.:65.0   3rd Qu.:4150   3rd Qu.:3.000  
##  Max.   :11   Max.   :1.0000   Max.   :84.0   Max.   :4320   Max.   :5.000  
##  TotalPurchases     Spending    AcceptedCmpOverall
##  Min.   : 0.00   Min.   :   5   Min.   :0.0000    
##  1st Qu.: 6.00   1st Qu.:  69   1st Qu.:0.0000    
##  Median :12.00   Median : 396   Median :0.0000    
##  Mean   :12.54   Mean   : 606   Mean   :0.4471    
##  3rd Qu.:18.00   3rd Qu.:1046   3rd Qu.:1.0000    
##  Max.   :32.00   Max.   :2525   Max.   :5.0000

# 6 new variables; removed 3 variables
glimpse(clean_data)

## Rows: 2,239
## Columns: 32
## $ Education           <chr> "Graduation", "Graduation", "Graduation", "Graduat…
## $ Marital_Status      <dbl> 1, 1, 2, 2, 2, 2, 1, 2, 2, 2, 2, 2, 1, 1, 2, 1, 2,…
## $ Income              <dbl> 58138, 46344, 71613, 26646, 58293, 62513, 55635, 3…
## $ Kidhome             <int> 0, 1, 0, 1, 1, 0, 0, 1, 1, 1, 1, 0, 0, 1, 0, 0, 1,…
## $ Teenhome            <int> 0, 1, 0, 0, 0, 1, 1, 0, 0, 1, 0, 0, 0, 1, 0, 0, 1,…
## $ Recency             <int> 58, 38, 26, 26, 94, 16, 34, 32, 19, 68, 11, 59, 82…
## $ MntWines            <int> 635, 11, 426, 11, 173, 520, 235, 76, 14, 28, 5, 6,…
## $ MntFruits           <int> 88, 1, 49, 4, 43, 42, 65, 10, 0, 0, 5, 16, 61, 2, …
## $ MntMeatProducts     <int> 546, 6, 127, 20, 118, 98, 164, 56, 24, 6, 6, 11, 4…
## $ MntFishProducts     <int> 172, 2, 111, 10, 46, 0, 50, 3, 3, 1, 0, 11, 225, 3…
## $ MntSweetProducts    <int> 88, 1, 21, 3, 27, 42, 49, 1, 3, 1, 2, 1, 112, 5, 1…
## $ MntGoldProds        <int> 88, 6, 42, 5, 15, 14, 27, 23, 2, 13, 1, 16, 30, 14…
## $ NumDealsPurchases   <int> 3, 2, 1, 2, 5, 2, 4, 2, 1, 1, 1, 1, 1, 3, 1, 1, 3,…
## $ NumWebPurchases     <int> 8, 1, 8, 2, 5, 6, 7, 4, 3, 1, 1, 2, 3, 6, 1, 7, 3,…
## $ NumCatalogPurchases <int> 10, 1, 2, 0, 3, 4, 3, 0, 0, 0, 0, 0, 4, 1, 0, 6, 0…
## $ NumStorePurchases   <int> 4, 2, 10, 4, 6, 10, 7, 4, 2, 0, 2, 3, 8, 5, 3, 12,…
## $ NumWebVisitsMonth   <int> 7, 5, 4, 6, 5, 6, 6, 8, 9, 20, 7, 8, 2, 6, 8, 3, 8…
## $ AcceptedCmp3        <int> 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0,…
## $ AcceptedCmp4        <int> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
## $ AcceptedCmp5        <int> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0,…
## $ AcceptedCmp1        <int> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0,…
## $ AcceptedCmp2        <int> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
## $ Complain            <int> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
## $ Z_CostContact       <int> 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3,…
## $ Z_Revenue           <int> 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11…
## $ Response            <int> 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 1, 0,…
## $ Age                 <dbl> 67, 70, 59, 40, 43, 57, 53, 39, 50, 74, 41, 48, 65…
## $ Customer_Days       <int> 4284, 3734, 3933, 3760, 3782, 3914, 4214, 4038, 40…
## $ FamilyMember        <dbl> 1, 3, 2, 3, 3, 3, 2, 3, 3, 4, 3, 2, 1, 3, 2, 1, 4,…
## $ TotalPurchases      <int> 22, 4, 20, 6, 14, 20, 17, 8, 5, 1, 3, 5, 15, 12, 4…
## $ Spending            <dbl> 1617, 27, 776, 53, 422, 716, 590, 169, 46, 49, 19,…
## $ AcceptedCmpOverall  <dbl> 1, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 3, 0,…

print(dfSummary(clean_data), method = "viewer")

## Switching method to 'browser'

## Output file written: C:\Users\chongkah\AppData\Local\Temp\RtmpqwIETM\fileb94c591366a.html

cat(readLines("Data Frame Summary2.html"), sep = "\n")

## Warning in readLines("Data Frame Summary2.html"): incomplete final line found
## on 'Data Frame Summary2.html'

<!DOCTYPE html> Data Frame Summary

Data Frame Summary

clean_data

Dimensions: 2239 x 32
Duplicates: 185

No

Variable

Stats / Values

Freqs (% of Valid)

Graph

Valid

Missing

1

Education [character]

2n Cycle

Basic

Graduation

Master

PhD

203	(	9.1%	)
54	(	2.4%	)
1126	(	50.3%	)
370	(	16.5%	)
486	(	21.7%	)

2239 (100.0%)

0 (0.0%)

2

Marital_Status [numeric]

Min : 1

Mean : 1.6

Max : 2

1	:	796	(	35.6%	)
2	:	1443	(	64.4%	)

2239 (100.0%)

0 (0.0%)

3

Income [numeric]

Mean (sd) : 51855.6 (21523.3)

min ≤ med ≤ max:

1730 ≤ 51267 ≤ 162397

IQR (CV) : 33156 (0.4)

1973 distinct values

2239 (100.0%)

0 (0.0%)

4

Kidhome [integer]

Mean (sd) : 0.4 (0.5)

min ≤ med ≤ max:

0 ≤ 0 ≤ 2

IQR (CV) : 1 (1.2)

0	:	1293	(	57.7%	)
1	:	898	(	40.1%	)
2	:	48	(	2.1%	)

2239 (100.0%)

0 (0.0%)

5

Teenhome [integer]

Mean (sd) : 0.5 (0.5)

min ≤ med ≤ max:

0 ≤ 0 ≤ 2

IQR (CV) : 1 (1.1)

0	:	1157	(	51.7%	)
1	:	1030	(	46.0%	)
2	:	52	(	2.3%	)

2239 (100.0%)

0 (0.0%)

6

Recency [integer]

Mean (sd) : 49.1 (29)

min ≤ med ≤ max:

0 ≤ 49 ≤ 99

IQR (CV) : 50 (0.6)

100 distinct values

2239 (100.0%)

0 (0.0%)

7

MntWines [integer]

Mean (sd) : 304.1 (336.6)

min ≤ med ≤ max:

0 ≤ 174 ≤ 1493

IQR (CV) : 480.5 (1.1)

776 distinct values

2239 (100.0%)

0 (0.0%)

8

MntFruits [integer]

Mean (sd) : 26.3 (39.8)

min ≤ med ≤ max:

0 ≤ 8 ≤ 199

IQR (CV) : 32 (1.5)

158 distinct values

2239 (100.0%)

0 (0.0%)

9

MntMeatProducts [integer]

Mean (sd) : 167 (225.7)

min ≤ med ≤ max:

0 ≤ 67 ≤ 1725

IQR (CV) : 216 (1.4)

558 distinct values

2239 (100.0%)

0 (0.0%)

10

MntFishProducts [integer]

Mean (sd) : 37.5 (54.6)

min ≤ med ≤ max:

0 ≤ 12 ≤ 259

IQR (CV) : 47 (1.5)

182 distinct values

2239 (100.0%)

0 (0.0%)

11

MntSweetProducts [integer]

Mean (sd) : 27.1 (41.3)

min ≤ med ≤ max:

0 ≤ 8 ≤ 263

IQR (CV) : 32 (1.5)

177 distinct values

2239 (100.0%)

0 (0.0%)

12

MntGoldProds [integer]

Mean (sd) : 44 (52.2)

min ≤ med ≤ max:

0 ≤ 24 ≤ 362

IQR (CV) : 47 (1.2)

213 distinct values

2239 (100.0%)

0 (0.0%)

13

NumDealsPurchases [integer]

Mean (sd) : 2.3 (1.9)

min ≤ med ≤ max:

0 ≤ 2 ≤ 15

IQR (CV) : 2 (0.8)

15 distinct values

2239 (100.0%)

0 (0.0%)

14

NumWebPurchases [integer]

Mean (sd) : 4.1 (2.8)

min ≤ med ≤ max:

0 ≤ 4 ≤ 27

IQR (CV) : 4 (0.7)

15 distinct values

2239 (100.0%)

0 (0.0%)

15

NumCatalogPurchases [integer]

Mean (sd) : 2.7 (2.9)

min ≤ med ≤ max:

0 ≤ 2 ≤ 28

IQR (CV) : 4 (1.1)

14 distinct values

2239 (100.0%)

0 (0.0%)

16

NumStorePurchases [integer]

Mean (sd) : 5.8 (3.3)

min ≤ med ≤ max:

0 ≤ 5 ≤ 13

IQR (CV) : 5 (0.6)

14 distinct values

2239 (100.0%)

0 (0.0%)

17

NumWebVisitsMonth [integer]

Mean (sd) : 5.3 (2.4)

min ≤ med ≤ max:

0 ≤ 6 ≤ 20

IQR (CV) : 4 (0.5)

16 distinct values

2239 (100.0%)

0 (0.0%)

18

AcceptedCmp3 [integer]

Min : 0

Mean : 0.1

Max : 1

0	:	2076	(	92.7%	)
1	:	163	(	7.3%	)

2239 (100.0%)

0 (0.0%)

19

AcceptedCmp4 [integer]

Min : 0

Mean : 0.1

Max : 1

0	:	2072	(	92.5%	)
1	:	167	(	7.5%	)

2239 (100.0%)

0 (0.0%)

20

AcceptedCmp5 [integer]

Min : 0

Mean : 0.1

Max : 1

0	:	2076	(	92.7%	)
1	:	163	(	7.3%	)

2239 (100.0%)

0 (0.0%)

21

AcceptedCmp1 [integer]

Min : 0

Mean : 0.1

Max : 1

0	:	2095	(	93.6%	)
1	:	144	(	6.4%	)

2239 (100.0%)

0 (0.0%)

22

AcceptedCmp2 [integer]

Min : 0

Mean : 0

Max : 1

0	:	2209	(	98.7%	)
1	:	30	(	1.3%	)

2239 (100.0%)

0 (0.0%)

23

Complain [integer]

Min : 0

Mean : 0

Max : 1

0	:	2218	(	99.1%	)
1	:	21	(	0.9%	)

2239 (100.0%)

0 (0.0%)

24

Z_CostContact [integer]

1 distinct value

3

:

2239

(

100.0%

)

2239 (100.0%)

0 (0.0%)

25

Z_Revenue [integer]

1 distinct value

11

:

2239

(

100.0%

)

2239 (100.0%)

0 (0.0%)

26

Response [integer]

Min : 0

Mean : 0.1

Max : 1

0	:	1905	(	85.1%	)
1	:	334	(	14.9%	)

2239 (100.0%)

0 (0.0%)

27

Age [numeric]

Mean (sd) : 55.1 (11.7)

min ≤ med ≤ max:

28 ≤ 54 ≤ 84

IQR (CV) : 18 (0.2)

56 distinct values

2239 (100.0%)

0 (0.0%)

28

Customer_Days [integer]

Mean (sd) : 3974.6 (202.2)

min ≤ med ≤ max:

3621 ≤ 3976 ≤ 4320

IQR (CV) : 348.5 (0.1)

663 distinct values

2239 (100.0%)

0 (0.0%)

29

FamilyMember [numeric]

Mean (sd) : 2.6 (0.9)

min ≤ med ≤ max:

1 ≤ 3 ≤ 5

IQR (CV) : 1 (0.3)

1	:	254	(	11.3%	)
2	:	764	(	34.1%	)
3	:	888	(	39.7%	)
4	:	301	(	13.4%	)
5	:	32	(	1.4%	)

2239 (100.0%)

0 (0.0%)

30

TotalPurchases [integer]

Mean (sd) : 12.5 (7.2)

min ≤ med ≤ max:

0 ≤ 12 ≤ 32

IQR (CV) : 12 (0.6)

33 distinct values

2239 (100.0%)

0 (0.0%)

31

Spending [numeric]

Mean (sd) : 606 (602.3)

min ≤ med ≤ max:

5 ≤ 396 ≤ 2525

IQR (CV) : 977 (1)

1054 distinct values

2239 (100.0%)

0 (0.0%)

32

AcceptedCmpOverall [numeric]

Mean (sd) : 0.4 (0.9)

min ≤ med ≤ max:

0 ≤ 0 ≤ 5

IQR (CV) : 1 (2)

0	:	1630	(	72.8%	)
1	:	370	(	16.5%	)
2	:	142	(	6.3%	)
3	:	51	(	2.3%	)
4	:	36	(	1.6%	)
5	:	10	(	0.4%	)

2239 (100.0%)

0 (0.0%)

Generated by summarytools 1.0.1 (R version 4.3.3)
2024-05-28

#html_output <- htmlTable(dfSummary(clean_data))
#html_output

G05_assignment_report

2024-05-12

Group Members

Project Title

1 Introduction

1.1 Description

1.2 Objectives

2 Data Description

install required packages dplyr, knitr, summarytools, VIM, import dataset to dataframe

2.1 Overview

Using dplyr to get a glimpse of the dataset

Using DT to datatable

Using base R to get a summary of the dataset

Using summarytools for a more detailed description

Data Frame Summary

df

3 Data Preprocessing

3.1 Missing Data

3.1.1 Show all Missing Data

3.1.2 Apply k-NN imputation to fill in missing data for [income] (only income having 24 missing value) since small dataset, therefore using k=5

3.1.3 Display all Missing Data after imputation

3.2 Data Transformation

3.2.1 Remove the irrelevant ID column

3.2.2 Convert the Dt_Customer column to Date format, Income column to numeric

3.2.3 Transform value for Marital_Status

3.2.4 Find out outlier with boxplot

3.2.5 Handling outlier with IQR - Only handle Year_Birth while others outliers value is make sense data

3.2.6 Remove outliers for income more than 20000

3.3 Feature Engineering - New variable

3.4 Remove unwanted variable after tranformation

Data Frame Summary

clean_data

4 Exploratory Data Analysis

4.1 Univariate Analysis

4.2 Bivariate Analysis

4.3 Multivariate Analysis

5 Modelling

5.1 Classification

5.1.1 Pre-Processing

5.1.2a Classification Model 1

5.1.2b Classification Model 2

5.1.2c Classification Model 3

5.2 Regression

5.2.1 Pre-Processing

5.2.2a Regression Model 1

5.2.2b Regression Model 2

5.2.2c Regression Model 3

6 Summary of Results

7 Conclusion