INFO-H510 Week 5 Data Dive

The columns in the data set are - Geography, Gender, HasCrCard, IsActiveMember, Exited, Age, Balance, Estimated Salary, Credit Score, Tenure, No. Of Products

The column names that felt ambiguous before reading the documentation were - HasCrCard,

  1. HasCrCard

    • Although, there is high likelihood of the column meaning that it represents whether the customer Has a Credit Card, there are cases where the column could mean Hashed Credit Card. Since, the data set contains details of customers it is reasonable to assume it meant the column stored the hashed value of the customer’s credit card, as it would have been a decision made out of protecting the privacy of the customer.

    • As the column refers to whether a customer has a credit card, it is perfectly logical to name the column as HasCrCard.

    • Without the detailed explanation provided in the documentation, assuming the column meant Hashed Credit Card, every null value would be something other than null, since hash(null) would not be evaluated to null. Moreover another issue that would have been raised was that, often individual values are hashed, meaning what if the customers had multiple credit cards, in such cases how would have they been encoded?

  2. NumOfProducts

    • The name of the column does suggest that it indicated Number of Products the customer have been using, and they include options like savings account, checking account, credit card etc. But certain banks offer the products often as services such as with loans, mortgages etc. So a clear understanding of the which services/products are being accounted for Products is essential.

    • In most cases often Products mean entities such credit cards, different types of accounts, and services would mean entities such as mortgages, loans etc. And entities such as there are much more likely to indicated whether the customer have churned (Which essentially is the goal of the project, to understand underlying patterns).

  3. Exited
    • As stated earlier the goal of the project is to understand patterns of customer churning behavior. Churning is possible in numerous ways, Closing Accounts, Transferring Funds, Cancelling Services, Switching Banks. Lack of Engagement etc.
    • However, in this scenario churning translates to when a customer had closed their accounts (Of any type, checking, savings and investing etc.)
    • As most often the case with churning being closing the account completely, and essentially such actions translating to other forms of churning such as Cancelling Services etc., the name Exited would have been chosen as a umbrella term.
    • Without the clear explanation in the documentation the exact definition wouldn’t have been clear and there would have no clear idea regarding the kind of churning exited essentially meant.

Most of the columns within the data have meaning upon glancing the documentation, or from the column names. However, even upon inspecting the documentation the columns such as Balance, EstimatedSalary, CreditScore do not make proper sense. Although, the column names suggest the likely function of the column further important details about have not been mentioned.

  1. Balance and EstimatedSalary

    • For both these columns, the issue of currency in which the values have been stored is not specified.

    • Since the Geography column spans multiple geographies it is hard to gain an accurate understanding on the currency they represent.

    • Often, factors in real-life such as economic regression, market-crashes do influence customer churning, due to lack of information we cannot compare general trends in the market that would have affected the customer.

  2. CreditScore
    • Credit Score is often measured on different scales in each country. US employs a scale from 300 to 850, while UK uses a 0 to 999 scale, India with scale from 300 to 900, Germany on a scale of 100.
    • And most importantly certain countries prefer storing credit history rather credit score, such as Spain.
    • Without the exact specification it is not possible to understand the exact specification.
library(tidyverse)
## ── Attaching core tidyverse packages ──────────────────────── tidyverse 2.0.0 ──
## ✔ dplyr     1.1.4     ✔ readr     2.1.4
## ✔ forcats   1.0.0     ✔ stringr   1.5.1
## ✔ ggplot2   3.4.4     ✔ tibble    3.2.1
## ✔ lubridate 1.9.3     ✔ tidyr     1.3.0
## ✔ purrr     1.0.2     
## ── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
## ✖ dplyr::filter() masks stats::filter()
## ✖ dplyr::lag()    masks stats::lag()
## ℹ Use the conflicted package (<http://conflicted.r-lib.org/>) to force all conflicts to become errors
#Loading data
data <- read_delim("../Churn_Modelling.csv", delim = ",")
## Rows: 10002 Columns: 14
## ── Column specification ────────────────────────────────────────────────────────
## Delimiter: ","
## chr  (3): Surname, Geography, Gender
## dbl (11): RowNumber, CustomerId, CreditScore, Age, Tenure, Balance, NumOfPro...
## 
## ℹ Use `spec()` to retrieve the full column specification for this data.
## ℹ Specify the column types or set `show_col_types = FALSE` to quiet this message.

With the assumption that Germany credit score is on a scale of 100, and a reasonable way to perform analysis is to normalize to a different scale.

germany_credit_score <- filter(data, Geography == "Germany")["CreditScore"]

#Since Germany uses scale of 100, max value would be 100 and minimum would be 0
max_score <- 100
min_score <- 0

norm_germany_credit_score <- (germany_credit_score - min_score) / (max_score - min_score)

In similar fashion let’s normalize the credit score of customers from Spain, since the scale being is not known, max and min need to be calculated from given. Although it would not provide an ideal value, it would provide a reasonable value.

spain_credit_score <- filter(data, Geography == "Spain")["CreditScore"]

max_score <- max(spain_credit_score)
min_score <- min(spain_credit_score)

norm_spain_credit_score <- (spain_credit_score - min_score) / (max_score - min_score)
avg_germany <- mean(norm_germany_credit_score$CreditScore)
avg_spain <- mean(norm_spain_credit_score$CreditScore)

avg_data <- data.frame(Dataframe = c("Germany", "Spain"),
                       Average_CreditScore = c(avg_germany, avg_spain))

ggplot(avg_data, aes(x = Dataframe, y = Average_CreditScore)) +
  geom_bar(stat = "identity", fill = "skyblue", width = 0.5) +
  labs(x = "Geography",
       y = "Average Credit Score")

From the visualization we can infer that on average Germany customers have a significantly higher credit score than customers from Spain. But upon manually inspecting the values we would infer a different case.

Moreover, other analysis made from co-relation analysis of variables, would also provide a skewed picture, as data is transformed non-linearly, and it would result in wrong inferences.

Further inferences such ratio of CreditScore’s of different geographies would also provide incorrect inferences. For instance, “On average how many times is the average credit score of Spanish customers compared to that of Germans? For the given data the result would be -

avg_germany / avg_spain
## [1] 10.81029

More than 10 times, that would have been the value, but it’s incorrect due to the incorrect assumption on the Credit Score scale.