PCA (Credit Card Default)

R packages

# Install the two packages as follow
#install.packages(c("FactoMineR", "factoextra"))
library("FactoMineR")

## Warning: package 'FactoMineR' was built under R version 4.0.3

library("factoextra")

## Warning: package 'factoextra' was built under R version 4.0.3

Data format

We’ll use the MCI CreditCard Default Deta set For PCA

## reading data
Default.df <- read.csv("MCICreditCardDefault.csv")

We start by subsetting active variables for the principal component analysis

## taking subset of numeric variables
Default.active <- Default.df[,c(2,6,7,8)]
head(Default.active)

##   CreditLimit Age BillOutstanding LastPayment
## 1       20000  24            3913           0
## 2      120000  26            2682           0
## 3       90000  34           29239        1518
## 4       50000  37           46990        2000
## 5       50000  57            8617        2000
## 6       50000  37           64400        2500

Data standardization

In principal component analysis, variables are often scaled (i.e. standardized). This is particularly recommended when variables are measured in different scales (e.g: kilograms, kilometers, centimeters, …); otherwise, the PCA outputs obtained will be severely affected.

# The function PCA() [FactoMineR package] can be used :
#PCA(X, scale.unit = TRUE, ncp = 5, graph = TRUE)

Applying PCA Analysis

res.pca <- PCA(Default.active, scale.unit = TRUE, graph = FALSE)

The output of the function PCA() is a list, including the following components :

print(res.pca)

## **Results for the Principal Component Analysis (PCA)**
## The analysis was performed on 29601 individuals, described by 4 variables
## *The results are available in the following objects:
## 
##    name               description                          
## 1  "$eig"             "eigenvalues"                        
## 2  "$var"             "results for the variables"          
## 3  "$var$coord"       "coord. for the variables"           
## 4  "$var$cor"         "correlations variables - dimensions"
## 5  "$var$cos2"        "cos2 for the variables"             
## 6  "$var$contrib"     "contributions of the variables"     
## 7  "$ind"             "results for the individuals"        
## 8  "$ind$coord"       "coord. for the individuals"         
## 9  "$ind$cos2"        "cos2 for the individuals"           
## 10 "$ind$contrib"     "contributions of the individuals"   
## 11 "$call"            "summary statistics"                 
## 12 "$call$centre"     "mean of the variables"              
## 13 "$call$ecart.type" "standard error of the variables"    
## 14 "$call$row.w"      "weights for the individuals"        
## 15 "$call$col.w"      "weights for the variables"

Visualization and Interpretation

# We’ll use the factoextra R package to help in the interpretation of PCA. No matter what function you decide to use
# These functions include:

#get_eigenvalue(res.pca): Extract the eigenvalues/variances of principal components
#fviz_eig(res.pca): Visualize the eigenvalues
#get_pca_ind(res.pca), get_pca_var(res.pca): Extract the results for individuals and variables, respectively.
#fviz_pca_ind(res.pca), fviz_pca_var(res.pca): Visualize the results individuals and variables, respectively.
#fviz_pca_biplot(res.pca): Make a biplot of individuals and variables.
#In the next sections, we’ll illustrate each of these functions.

Eigenvalues / Variances

The eigenvalues measure the amount of variation retained by each principal component. Eigenvalues are large for the first PCs and small for the subsequent PCs. That is, the first PCs corresponds to the directions with the maximum amount of variation in the data set.

library("factoextra")
eig.val <- get_eigenvalue(res.pca)
eig.val

##       eigenvalue variance.percent cumulative.variance.percent
## Dim.1  1.4603748         36.50937                    36.50937
## Dim.2  0.9843115         24.60779                    61.11716
## Dim.3  0.8637435         21.59359                    82.71074
## Dim.4  0.6915702         17.28926                   100.00000

Scree Plot

fviz_eig(res.pca, addlabels = TRUE, ylim = c(0, 50))

Results

A simple method to extract the results, for variables, from a PCA output is to use the function get_pca_var() [factoextra package]. This function provides a list of matrices containing all the results for the active variables (coordinates, correlation between variables and axes, squared cosine and contributions)

var <- get_pca_var(res.pca)
var

## Principal Component Analysis Results for variables
##  ===================================================
##   Name       Description                                    
## 1 "$coord"   "Coordinates for the variables"                
## 2 "$cor"     "Correlations between variables and dimensions"
## 3 "$cos2"    "Cos2 for the variables"                       
## 4 "$contrib" "contributions of the variables"

Correlation circle

The correlation between a variable and a principal component (PC) is used as the coordinates of the variable on the PC. The representation of variables differs from the plot of the observations: The observations are represented by their projections, but the variables are represented by their correlations

fviz_pca_var(res.pca, col.var = "black")

Quality of representation

library("corrplot")

## Warning: package 'corrplot' was built under R version 4.0.3

## corrplot 0.84 loaded

corrplot(var$cos2, is.corr=FALSE)

Contributions of variables to PCs

library("corrplot")
corrplot(var$contrib, is.corr=FALSE)  

# Contributions of variables to PC1
fviz_contrib(res.pca, choice = "var", axes = 1, top = 10)

# Contributions of variables to PC2
fviz_contrib(res.pca, choice = "var", axes = 2, top = 10)

The most important (or, contributing) variables can be highlighted on the correlation plot as follow:

fviz_pca_var(res.pca, col.var = "contrib",
             gradient.cols = c("#00AFBB", "#E7B800", "#FC4E07")
             )

#library(devtools)
#install_github("vqv/ggbiplot")

library(ggbiplot)

## Loading required package: plyr

## Loading required package: scales

## Loading required package: grid

ggbiplot(res.pca,ellipse=TRUE,  labels=rownames(Default.active), 
         groups=as.factor(Default.df$Default))