Data

In our example this week, we are going to use the fake data - about real estates in Wroclaw - prices by districts, size of apartments and many more.

apartments$district<-as.factor(apartments$district)
apartments$building_type<-as.factor(apartments$building_type)
apartments$rooms<-factor(apartments$rooms,ordered=TRUE)
attach(apartments)
apartments$price_PLN<-as.numeric(apartments$price_PLN)
apartments$price_EUR<-as.numeric(apartments$price_EUR)

Frequency Tables

In the first step of our analysis, we will group our data into a simple frequency table.

First, let’s look at the distribution of housing prices in our sample and verify tabular validity using the TAI measure:

##        # classes  Goodness of fit Tabular accuracy 
##       10.0000000        0.9780872        0.8508467

As we can see - the TAI index is quite high. 0.85 means that we can accept the proposed construction of the frequency table.

Basic plots

In this section, we should represent our data using basic (pre-installed in R) graphics. Select the most appropriate graphs depending on the scale of the selected variables. Explore the heterogeneity of the distribution by presenting the data by group (e.g., by neighborhood, building type, etc.). Don’t forget about main titles, labels and legends. Read more about graphical parameters here.

Note that the echo = FALSE parameter has been added to the code snippet to prevent printing the R code that generated the graph.

ggplot2 plots

Now, let’s use the ggplot2 and ggpubr libraries to plot.

Ggplot2 allows you to show the average value for each group using the stat_summary() function. You no longer need to calculate average values before creating a graph!

### RainCloud Plot

Univariate Statistics

Before automatically reporting the full summary table of descriptive statistics, this time your goal is to measure the central tendency of the price distribution. Compare the mean, median, and mode along with positional measures - quantiles - by district and building type or number of rooms in the apartment.

    mean(price_PLN)

## [1] 760035

    median(price_PLN)

## [1] 755719.5

    sd(price_PLN) #standard deviation

## [1] 186099.8

    var(price_PLN) #variance

## [1] 34633125960

    coeff_var<-sd(price_PLN)/mean(price_PLN) #coefficient of variability %
    coeff_var

## [1] 0.2448568

    IQR(price_PLN)# difference between quartiles =Q3-Q1 

##      75% 
## 282686.5

    sx<-IQR(price_PLN)/2  #interquartile deviation
    coeff_varx<-sx/median(price_PLN) #IQR coefficient of variability %
    coeff_varx

##       75% 
## 0.1870314

    min(price_PLN)

## [1] 359769

    max(price_PLN)

## [1] 1277691

    quantile(price_PLN,probs=c(0,0.1,0.25,0.5,0.75,0.95,1),na.rm=TRUE)

##        0%       10%       25%       50%       75%       95%      100% 
##  359769.0  518806.8  619073.8  755719.5  901760.2 1054250.8 1277691.0

apartments %>%
  select(price_PLN,rooms) %>%
  tbl_summary(label= price_PLN ~ "Price",digits=c(price_PLN)~2,by=rooms,type = all_continuous() ~ "continuous2", statistic = all_continuous() ~ c("{N_nonmiss}", "{median} ({p25}, {p75})", "{min}, {max}"),missing = "no")

dfSummary(apartments,
          plain.ascii  = FALSE, 
          style        = "grid", 
          graph.magnif = 0.75, 
          valid.col    = FALSE,
          tmp.img.dir  = "/tmp")

Data Frame Summary

apartments

Dimensions: 200 x 6
Duplicates: 0

No	Variable	Stats / Values	Freqs (% of Valid)	Graph	Missing
1	price_PLN [numeric]	Mean (sd) : 760035 (186099.8) min < med < max: 359769 < 755719.5 < 1277691 IQR (CV) : 282686.5 (0.2)	200 distinct values		0 (0.0%)
2	price_EUR [numeric]	Mean (sd) : 175934 (43078.6) min < med < max: 83280 < 174935 < 295762 IQR (CV) : 65436.2 (0.2)	200 distinct values		0 (0.0%)
3	rooms [ordered, factor]	1. 1 2. 2 3. 3 4. 4	44 (22.0%) 50 (25.0%) 58 (29.0%) 48 (24.0%)		0 (0.0%)
4	size [numeric]	Mean (sd) : 46.2 (20.1) min < med < max: 17 < 43.7 < 87.7 IQR (CV) : 30.2 (0.4)	162 distinct values		0 (0.0%)
5	district [factor]	1. Biskupin 2. Krzyki 3. Srodmiescie	65 (32.5%) 79 (39.5%) 56 (28.0%)		0 (0.0%)
6	building_type [factor]	1. kamienica 2. niski blok 3. wiezowiec	61 (30.5%) 63 (31.5%) 76 (38.0%)		0 (0.0%)

To produce optimal results, summarytools has its own version of the base by() function. It’s called stby(), and we use it exactly as we would by():

(stats_by_rooms <- stby(data      = apartments, INDICES   = apartments$rooms, FUN       = descr, stats     = "common", transpose = TRUE))

## Non-numerical variable(s) ignored: rooms, district, building_type

Descriptive Statistics
apartments
Group: rooms = 1
N: 44

	Mean	Std.Dev	Min	Median	Max	N.Valid	Pct.Valid
price_EUR	119332.95	15497.90	83280.00	120488.00	152117.00	44.00	100.00
price_PLN	515518.05	66951.03	359769.00	520507.00	657146.00	44.00	100.00
size	19.28	1.46	17.00	19.10	21.90	44.00	100.00

Group: rooms = 2
N: 50

	Mean	Std.Dev	Min	Median	Max	N.Valid	Pct.Valid
price_EUR	158233.22	15063.13	136640.00	156773.00	205702.00	50.00	100.00
price_PLN	683567.70	65072.66	590286.00	677260.00	888634.00	50.00	100.00
size	36.80	4.46	29.60	35.95	43.70	50.00	100.00

Group: rooms = 3
N: 58

	Mean	Std.Dev	Min	Median	Max	N.Valid	Pct.Valid
price_EUR	192987.55	20125.88	146475.00	195904.00	223572.00	58.00	100.00
price_PLN	833706.02	86943.90	632770.00	846303.50	965829.00	58.00	100.00
size	53.33	7.21	41.20	53.45	65.20	58.00	100.00

Group: rooms = 4
N: 48

	Mean	Std.Dev	Min	Median	Max	N.Valid	Pct.Valid
price_EUR	225650.42	26347.03	170525.00	223226.50	295762.00	48.00	100.00
price_PLN	974809.96	113819.21	736669.00	964338.50	1277691.00	48.00	100.00
size	72.05	10.18	53.30	70.85	87.70	48.00	100.00

apartments %>%
  descr(stats = "common") %>%
  tb()

## # A tibble: 3 × 8
##   variable      mean       sd    min      med       max n.valid pct.valid
##   <chr>        <dbl>    <dbl>  <dbl>    <dbl>     <dbl>   <dbl>     <dbl>
## 1 price_EUR 175934.   43079.   83280 174935    295762       200       100
## 2 price_PLN 760035.  186100.  359769 755720.  1277691       200       100
## 3 size          46.2     20.1     17     43.7      87.7     200       100

grouped_descr <- stby(data    = apartments,INDICES = apartments$rooms, FUN     = descr, stats   = "common")

grouped_descr %>% tb()

## # A tibble: 12 × 9
##    rooms variable      mean        sd      min      med    max n.valid pct.valid
##    <fct> <chr>        <dbl>     <dbl>    <dbl>    <dbl>  <dbl>   <dbl>     <dbl>
##  1 1     price_EUR 119333.   15498.    83280   120488   1.52e5      44       100
##  2 1     price_PLN 515518.   66951.   359769   520507   6.57e5      44       100
##  3 1     size          19.3      1.46     17       19.1 2.19e1      44       100
##  4 2     price_EUR 158233.   15063.   136640   156773   2.06e5      50       100
##  5 2     price_PLN 683568.   65073.   590286   677260   8.89e5      50       100
##  6 2     size          36.8      4.46     29.6     36.0 4.37e1      50       100
##  7 3     price_EUR 192988.   20126.   146475   195904   2.24e5      58       100
##  8 3     price_PLN 833706.   86944.   632770   846304.  9.66e5      58       100
##  9 3     size          53.3      7.21     41.2     53.4 6.52e1      58       100
## 10 4     price_EUR 225650.   26347.   170525   223226.  2.96e5      48       100
## 11 4     price_PLN 974810.  113819.   736669   964338.  1.28e6      48       100
## 12 4     size          72.0     10.2      53.3     70.8 8.77e1      48       100

stby(data    = apartments, 
     INDICES = apartments$rooms, 
     FUN     = descr, 
     stats   = "fivenum") %>%
  tb(order = 3) %>%
  kable(format = "html", digits = 2) %>%
  collapse_rows(columns = 1, valign = "top")

Task!

Our task for this week was to: prepare our own descriptive analysis for the “CreditCard” dataset (AER package). It is a cross-sectional dataframe on the credit history for a sample of applicants for a type of credit card.

Are the yearly incomes (in USD 10,000), credit card expenditures, age, ratio of monthly credit card expenditure to yearly income - significantly different for applicants for customers with different credit risk (“card” variable - factor)?

To start with the descriptive analysis of the “CreditCard” dataset, let’s first load the dataset and take a look at its structure:

library(AER)

# Load the CreditCard dataset
data(CreditCard)

# Display the structure of the dataset
str(CreditCard)

## 'data.frame':    1319 obs. of  12 variables:
##  $ card       : Factor w/ 2 levels "no","yes": 2 2 2 2 2 2 2 2 2 2 ...
##  $ reports    : num  0 0 0 0 0 0 0 0 0 0 ...
##  $ age        : num  37.7 33.2 33.7 30.5 32.2 ...
##  $ income     : num  4.52 2.42 4.5 2.54 9.79 ...
##  $ share      : num  0.03327 0.00522 0.00416 0.06521 0.06705 ...
##  $ expenditure: num  124.98 9.85 15 137.87 546.5 ...
##  $ owner      : Factor w/ 2 levels "no","yes": 2 1 2 1 2 1 1 2 2 1 ...
##  $ selfemp    : Factor w/ 2 levels "no","yes": 1 1 1 1 1 1 1 1 1 1 ...
##  $ dependents : num  3 3 4 0 2 0 2 0 0 0 ...
##  $ months     : num  54 34 58 25 64 54 7 77 97 65 ...
##  $ majorcards : num  1 1 1 1 1 1 1 1 1 1 ...
##  $ active     : num  12 13 5 7 5 1 5 3 6 18 ...

Now, let’s proceed with the descriptive analysis including data visualization and summary statistics for different variables based on the “card” variable (credit risk).

##Data Visualization: We’ll create visualizations to understand the distribution of yearly incomes, credit card expenditures, age, and the ratio of monthly credit card expenditure to yearly income among different credit risk groups.

# Visualizing yearly income by credit risk
income_plot <- ggplot(CreditCard, aes(x = card, y = income)) +
  geom_boxplot() +
  labs(
    title = "Yearly Income Distribution by Credit Risk",
    x = "Credit Risk",
    y = "Yearly Income (USD 10,000)"
  ) +
  theme_minimal()

income_plot

# Visualizing credit card expenditures by credit risk
expenditure_plot <- ggplot(CreditCard, aes(x = card, y = expenditure)) +
  geom_boxplot() +
  labs(
    title = "Credit Card Expenditure Distribution by Credit Risk",
    x = "Credit Risk",
    y = "Credit Card Expenditure"
  ) +
  theme_minimal()

expenditure_plot

# Visualizing age by credit risk
age_plot <- ggplot(CreditCard, aes(x = card, y = age)) +
  geom_boxplot() +
  labs(
    title = "Age Distribution by Credit Risk",
    x = "Credit Risk",
    y = "Age"
  ) +
  theme_minimal()

age_plot

##Descriptive Statistics:

Let’s calculate summary statistics for the variables of interest grouped by credit risk:

library(dplyr)

# Summary statistics by credit risk
summary_stats <- CreditCard %>%
  group_by(card) %>%
  summarize(
    mean_income = mean(income),
    sd_income = sd(income),
    mean_expenditure = mean(expenditure),
    sd_expenditure = sd(expenditure),
    mean_age = mean(age),
    sd_age = sd(age)
  )

summary_stats

## # A tibble: 2 × 7
##   card  mean_income sd_income mean_expenditure sd_expenditure mean_age sd_age
##   <fct>       <dbl>     <dbl>            <dbl>          <dbl>    <dbl>  <dbl>
## 1 no           3.07      1.62               0              0      33.2   9.92
## 2 yes          3.45      1.71             239.           288.     33.2  10.2

##Interpretation:

1.Yearly Income: The boxplot indicates that there might be differences in yearly income between different credit risk groups. The summary statistics confirm this, showing variations in mean income and standard deviation across groups.

2.Credit Card Expenditure: Similar to income, there seem to be differences in credit card expenditures across credit risk groups, as suggested by both the boxplot and summary statistics.

3.Age: While there might be slight variations in age distribution among different credit risk groups, the differences do not seem as pronounced as those observed for income and expenditure.

4.Ratio of Monthly Expenditure to Yearly Income: The boxplot suggests potential differences in this ratio among credit risk groups. However, further analysis, such as hypothesis testing, would be necessary to confirm whether these differences are statistically significant.

##Summary Overall, the visualizations and summary statistics provide insights into the relationships between credit risk and various financial attributes among credit card applicants. Further statistical analysis, such as hypothesis testing, can be conducted to determine the significance of these observed differences.