Exploratory Data Analysis

Load Libraries

library(dplyr)
library(tidyr)
library(readr)
library(ggplot2)

Import data

df <- read_csv('adult.csv.zip', na = c("", "NA", "?"))

Describe file format

str(df, max.level = 1)

Classes ‘tbl_df’, ‘tbl’ and 'data.frame':   32561 obs. of  15 variables:
 $ age           : int  90 82 66 54 41 34 38 74 68 41 ...
 $ workclass     : chr  "?" "Private" "?" "Private" ...
 $ fnlwgt        : int  77053 132870 186061 140359 264663 216864 150601 88638 422013 70037 ...
 $ education     : chr  "HS-grad" "HS-grad" "Some-college" "7th-8th" ...
 $ education.num : int  9 9 10 4 10 9 6 16 9 10 ...
 $ marital.status: chr  "Widowed" "Widowed" "Widowed" "Divorced" ...
 $ occupation    : chr  "?" "Exec-managerial" "?" "Machine-op-inspct" ...
 $ relationship  : chr  "Not-in-family" "Not-in-family" "Unmarried" "Unmarried" ...
 $ race          : chr  "White" "White" "Black" "White" ...
 $ sex           : chr  "Female" "Female" "Female" "Female" ...
 $ capital.gain  : int  0 0 0 0 0 0 0 0 0 0 ...
 $ capital.loss  : int  4356 4356 4356 3900 3900 3770 3770 3683 3683 3004 ...
 $ hours.per.week: int  40 18 40 40 40 45 40 20 40 60 ...
 $ native.country: chr  "United-States" "United-States" "United-States" "United-States" ...
 $ income        : chr  "<=50K" "<=50K" "<=50K" "<=50K" ...

Convert categorical variables

df$age <- as.factor(df$age)
df$workclass <- as.factor(df$workclass)
df$education <- as.factor(df$education)
df$marital.status <- as.factor(df$marital.status)
df$occupation <- as.factor(df$occupation)
df$relationship <- as.factor(df$relationship)
df$race <- as.factor(df$race)
df$sex <- as.factor(df$sex)
df$native.country <- as.factor(df$native.country)
df$income <- as.factor(df$income)

Remove duplicate variables

df <- df %>%
  select(-education.num)

VARIATION

Visualising the distribution of a categorical variable, use Bar-Chart

How you visualise the distribution of a variable will depend on whether the variable is categorical or continuous. To examine the distribution of a categorical variable, use a bar chart:

Visualising the distribution of a continuous variable, use a histogram:

ggplot(data = df) +
  geom_histogram(mapping = aes(x = hours.per.week, fill=..count..), binwidth = 5, col="red") +
  scale_fill_gradient("Count", low = "green", high = "red")

To make it easy to see the unusual values, we need to zoom into to small values of the y-axis with coord_cartesian():

ggplot(data = df) +
  geom_histogram(mapping = aes(x = hours.per.week)) +
  coord_cartesian(ylim = c(0, 100), xlim = c(90, 100))

COVARIATION

Variation describes the behavior within a variable, covariation describes the behavior between variables.

A categorical and continuous variable

to display the distribution of a continuous variable broken down by a categorical variable is the boxplot.

Two categorical variables

To visualise the covariation between categorical variables, you’ll need to count the number of observations for each combination. One way to do that is to rely on the built-in geom_count():

ggplot(data = df) +
  geom_count(mapping = aes(x = education, y = race)) +
   theme(axis.text.x=element_text(angle=90,hjust=1,vjust=0.5)) +
  coord_flip()

Then visualise with geom_tile() and the fill aesthetic:

df %>% 
  count(education, race) %>%  
  ggplot(mapping = aes(x = education, y = race)) +
    geom_tile(mapping = aes(fill = n)) +
  theme(axis.text.x=element_text(angle=90,hjust=1,vjust=0.5)) +
  coord_flip()