EDA Workbook

Note. The content is based on Wickham (2017), chapter 5.

knitr::opts_chunk$set(
    message = FALSE,
    warning = FALSE,
    include = FALSE
)
library(tidyverse)

To illustrate typical questions on data, we use the diamonds (to be described on a web page via a link from here) data set

Two very important question for any exploratory analysis are: 1. How do the values in one variable vary? (What is the distribution of values?)

2. How do two variables co-vary?

1. Variation of values in one variable

If a variable is categorical, the distribution of values can be visualised with a bar chart:

ggplot(diamonds, aes(x=cut)) + 
  geom_bar()

The numbers calculated with dplyr’s count() function as well:

diamonds %>% 
  count(cut)

For numerical variables, the distribution can be inspected with a histogram.

ggplot(data = diamonds) + 
  geom_histogram(mapping = aes(x = carat), binwidth = 0.5)

A histogram divides the x-axis into equally spaced bins and then uses the height of each bar to display the number of observations that fall in each bin.The number of bins an be set in the geom.

To compare multiple distribution visually in on graph, the frequency polygon is the right format:

ggplot(diamonds, aes(x = carat, color = cut)) + 
  geom_freqpoly(binwidth = 0.1)

Typical values

• Which values are the most common? Why?
• Which values are rare? Why? Does that match your expectations?
• Can you see any unusual patterns? What might explain them?

Let’s look at this in the context of diamonds with 3 carat or less:

smaller <- diamonds %>% 
  filter(carat < 3)
ggplot(data = smaller, mapping = aes(x = carat)) + 
  geom_histogram(binwidth = 0.01)

Unusual values

Missing values

2. Covariation