Navigate to this site:

http://rpubs.com/dzchilds/ct_tidy_intro

It contains a copy of this presentation.

A tbl object (pronounced “tibble”) is essentially a special kind of data frame. Tidyverse functions tend to produce these. They work the same as a data frame, but with a few small differences… e.g. compact printing:

storms <- read_csv("STORMS.CSV")
storms

## # A tibble: 2,747 x 11
##    name    year month   day  hour   lat  long pressure  wind type   seasday
##    <chr>  <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>    <dbl> <dbl> <chr>    <dbl>
##  1 Allis…  1995     6     3     0  17.4 -84.3     1005    30 Tropi…       3
##  2 Allis…  1995     6     3     6  18.3 -84.9     1004    30 Tropi…       3
##  3 Allis…  1995     6     3    12  19.3 -85.7     1003    35 Tropi…       3
##  4 Allis…  1995     6     3    18  20.6 -85.8     1001    40 Tropi…       3
##  5 Allis…  1995     6     4     0  22   -86        997    50 Tropi…       4
##  6 Allis…  1995     6     4     6  23.3 -86.3      995    60 Tropi…       4
##  7 Allis…  1995     6     4    12  24.7 -86.2      987    65 Hurri…       4
##  8 Allis…  1995     6     4    18  26.2 -86.2      988    65 Hurri…       4
##  9 Allis…  1995     6     5     0  27.6 -86.1      988    65 Hurri…       5
## 10 Allis…  1995     6     5     6  28.5 -85.6      990    60 Tropi…       5
## # … with 2,737 more rows

Next in your script, add these lines to convert storms and iris to tibbles…

storms <- as_tibble(storms)

iris <- as_tibble(iris)

You don’t have to do this (dplyr is fine with normal data frames) but it will ensure your output matches the presentation.

In addition to printing a tbl or data.frame, we can use the glimpse function to obtain different summary information about variables:

glimpse(storms)

## Observations: 2,747
## Variables: 11
## $ name     <chr> "Allison", "Allison", "Allison", "Allison", "Allison", …
## $ year     <dbl> 1995, 1995, 1995, 1995, 1995, 1995, 1995, 1995, 1995, 1…
## $ month    <dbl> 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6…
## $ day      <dbl> 3, 3, 3, 3, 4, 4, 4, 4, 5, 5, 5, 5, 6, 6, 6, 6, 7, 7, 7…
## $ hour     <dbl> 0, 6, 12, 18, 0, 6, 12, 18, 0, 6, 12, 18, 0, 6, 12, 18,…
## $ lat      <dbl> 17.4, 18.3, 19.3, 20.6, 22.0, 23.3, 24.7, 26.2, 27.6, 2…
## $ long     <dbl> -84.3, -84.9, -85.7, -85.8, -86.0, -86.3, -86.2, -86.2,…
## $ pressure <dbl> 1005, 1004, 1003, 1001, 997, 995, 987, 988, 988, 990, 9…
## $ wind     <dbl> 30, 30, 35, 40, 50, 60, 65, 65, 65, 60, 60, 45, 30, 35,…
## $ type     <chr> "Tropical Depression", "Tropical Depression", "Tropical…
## $ seasday  <dbl> 3, 3, 3, 3, 4, 4, 4, 4, 5, 5, 5, 5, 6, 6, 6, 6, 7, 7, 7…

This is similar to str — the glimpse function tells us what variables are in storm as well as the type of each variable. (The glimpse function actually belongs to the tibble package, but we don’t need to worry about that…)

dplyr implements a grammar of data manipulation to enable you manipulate data and summarise the information in a data set (e.g. group means).

Advantages of using dplyr

• Provides a consistent framework for data manipulation
• Designed to work well with the ggplot2 plotting system
• Fast compared to many base R functions
• Allows you to work with data stored in many ways (e.g. in a database)

It is helpful to classify the verbs according to what they work on:

• observations (rows): filter & slice & arrange
• variables (columns): select & rename & mutate
• summarise: summarise (or summarize)

(This classification only works if your data are tidy, i.e. there is one observation per row and one column per variable. Make sure you read about this idea in the course book)

We use select to extract a subset of variables for further analysis. Using select looks like this:

select(data, Variable1, Variable2, ...)

Arguments

• data: a data.frame or tbl object
• VariableX: names of variables in data

iris_fewer <- select(iris, Petal.Length:Species)
iris_fewer

## # A tibble: 150 x 3
##    Petal.Length Petal.Width Species
##           <dbl>       <dbl> <fct>  
##  1          1.4         0.2 setosa 
##  2          1.4         0.2 setosa 
##  3          1.3         0.2 setosa 
##  4          1.5         0.2 setosa 
##  5          1.4         0.2 setosa 
##  6          1.7         0.4 setosa 
##  7          1.4         0.3 setosa 
##  8          1.5         0.2 setosa 
##  9          1.4         0.2 setosa 
## 10          1.5         0.1 setosa 
## # … with 140 more rows

Scatter plots are used to show the relationship between 2 continuous variables. Using the iris dataset, let’s examine the relationship between petal length and petal width.

STEP 1:

We use the aes function inside the ggplot function to specify which variables we plan to display. We also have to specify where the data are:

plt <- ggplot(iris, aes(x = Petal.Width, y = Petal.Length))

All we did here was make a ggplot object.

We can try to print the plot to the screen:

plt

This produces an empty plot because we haven’t added a layer using a geom_ function yet.

STEP 2:

We want to make a scatter plot so we need to use the geom_point function:

plt <- plt + geom_point()

Notice that all we do is “add” the required layer. Now we have something to plot:

plt

STEP 3:

Maybe we should improve the axis labels? To do this, we need to “add” labels information using the labs function

plt <- plt + labs(x = "Petal Width (cm)", y = "Petal Length (cm)")
plt

This just adds some new information about labelling to the prexisting ggplot object. Now it prints with improved axis labels:

Doing it all in one go… We don’t have to build a plot object up in separate steps and then explicitly “print” it to the Console. If we just want to make the plot in one go we can do it like this:

ggplot(iris, aes(x = Petal.Width, y = Petal.Length)) + 
  geom_point() + 
  labs(x = "Petal Width (cm)", y = "Petal Length (cm)")

ggplot(iris, aes(x = Petal.Width, y = Petal.Length)) + 
  geom_point(colour = "blue", size = 3, alpha = 0.5) + 
  labs(x = "Petal Width (cm)", y = "Petal Length (cm)")

We want to make the following scatter plot. It shows mean wind speed against mean pressure, where the means are calculated for each combination of storm name and type. The storm type of each point is delineated by its colour.

Histograms summarise the relative frequency of different values of a variable. Look at the first 56 values of pressure variable in storms:

storms $ pressure[1:56]

##  [1] 1005 1004 1003 1001  997  995  987  988  988  990  990  993  993  994
## [15]  995  995  992  990  988  984  982  984  989  993  995  996  997 1000
## [29]  997  990  992  992  993 1019 1019 1018 1017 1016 1013 1011 1009 1007
## [43] 1004 1001  997  997  997  997  996  995  993  991  990  989 1012 1012

To get a sense of how frequent different values are we can “bin” the data. Here are the frequencies of pressure variable values, using 8 bins:

table(cut(storms $ pressure, breaks = 8))

## 
##        (905,919]        (919,934]        (934,948]        (948,962] 
##                5               23               80              164 
##        (962,976]        (976,990]      (990,1e+03] (1e+03,1.02e+03] 
##              307              558              943              667

(You don’t need to remember this R code)

We use histograms to understand the distribution of a variable. They summarise the number of observations occuring in a contiguous series of bins. We can use geom_histogram to construct a histogram. Here is an example:

ggplot(storms, aes(x = pressure)) + 
  geom_histogram(colour = "darkgrey", fill = "grey", binwidth=10) + 
  labs(x = "Pressure", y = "Count")  

ggplot(iris, aes(x = Petal.Length / Petal.Width)) + 
  geom_histogram(binwidth=0.5) + 
  labs(x = "Petal Eccentricity", y = "Count")  

We use dot plots to explore the distribution of variables when we have relatively few observations (e.g. < 100). Here is an example:

setosa <- filter(iris, Species == "setosa")
ggplot(setosa, aes(x = Sepal.Length)) + 
  geom_dotplot(binwidth = 0.1)  

N.B. — The y-scale is meaingless in this plot!

setosa <- filter(iris, Species == "setosa")
ggplot(setosa, aes(x = Sepal.Length)) + 
  geom_dotplot(binwidth = 0.1) +
  scale_y_continuous( <????> ) +
  theme_<????>()

setosa <- filter(iris, Species == "setosa")
ggplot(setosa, aes(x = Sepal.Length)) + 
  geom_dotplot(binwidth = 0.1) +
  scale_y_continuous(NULL, breaks = NULL) + # <- remove the y-axis
  theme_classic()                           # <- remove the grid lines

Box and whisker plots summarise the distributions of a variable at different levels of a categorical variable. Here is an example:

Each box-and-whisker shows the group median (line) and the interquartile range (“boxes”). The vertical lines (“whiskers”) highlight the range of the rest of the data in each group. Potential outliers are plotted individually.

You can guess which geom_ function we use to make a boxplot…

ggplot(iris, aes(x = Species, y = Petal.Length/Petal.Width)) + 
  geom_boxplot() + 
  labs(x = "Species", y = "Eccentricty") + 
  theme_minimal(base_size = 14)

ggplot(storms, aes(x = type, y = wind)) + 
  geom_boxplot(fill= "lightgrey") + 
  labs(x = "Type of storm", y = "Wind Speed (mph)") + 
  theme_classic(base_size = 14)

We can save a plot using the ggsave function with + when we’re building our plot…

# version 1. 
ggplot(setosa, aes(x = Sepal.Length)) + 
  geom_dotplot(binwidth = 0.1) + 
  ggsave("Sepal_dotplot.pdf") # <- use ggsave as part of a ggplot construct

Maybe don’t use this method.

Or we can save a plot using the ggsave function on its own after we make the figure…

# version 2. 
ggplot(setosa, aes(x = Sepal.Length)) + 
  geom_dotplot(binwidth = 0.1)
# use ggsave on its own *after* making the figure
ggsave("Sepal_dotplot.pdf")

This method is probably better.

# make the plot 
ggplot(storms, aes(x = type, y = wind)) + 
  geom_boxplot() + 
  labs(x = "Type of storm", y = "Wind Speed")
# save it
ggsave("Windspeed_boxplot.pdf", height = 4, width = 4)

We typically use a barplot to summarise differences among groups. We can use geom_bar to make barplots but it will simply count the number of observations by default:

ggplot(storms, aes(x = factor(year))) + # <- what is the 'factor' function doing?
  geom_bar() + 
  labs(x = "Year", y = "Number of Storms")  

This is not really very useful. Usually we use a bar plot to compare summary statistics (e.g. the mean).

First we need to calculate the summary statistic. We can do this with the group_by and summarise function from the dplyr package. For example:

# step 1
pl_stats <- 
  iris %>%
  group_by(Species) %>% 
  summarise(mean_pl = mean(Petal.Length))

Second, make the plot. If we want to use a bar plot to compare a summary statistic (e.g. the mean) across groups we should use the geom_col function:

# step 2 
ggplot(pl_stats, aes(x = Species, y = mean_pl)) + 
  geom_col() + 
  labs(y = "Mean Petal Length (cm)")

# step 1 - use dplyr to calculate the means
wind.means <- 
  storms %>% group_by(year) %>% 
  summarise(mean= mean(wind))
# step 2 - make the plot
ggplot(wind.means, aes(x = factor(year), y = mean)) + 
  geom_col(fill="darkgrey") + 
  labs(x = "Year", y = "Wind speed (mph)") 

The standard error is one option here:

\[ \text{Standard Error} = \frac{\text{Standard Deviation}}{\sqrt{\text{Sample Size}}} \]

The standard error is one option here:

\[ \text{Standard Error} = \frac{\text{Standard Deviation}}{\sqrt{\text{Sample Size}}} \]

We need to repeat the dplyr, but now include a calculation of the standard errors along with the means:

# step 1
pl_stats <- 
  iris %>%
  group_by(Species) %>% 
  summarise(mean_pl = mean(Petal.Length),
            se = sd(Petal.Length) / sqrt(n())) # <- New calculation

Once we have the two bits of information, we include these by adding two layers via two different geom_ functions: geom_col and geom_errorbar. We also need to define a couple of new aesthetics…

# step 2 
ggplot(pl_stats, 
       aes(x = Species, y = mean_pl, 
           ymin = mean_pl - se, ymax = mean_pl + se)) + 
  geom_col(fill = "grey", width = 0.7) + 
  geom_errorbar(width = 0.25) + 
  labs(y = "Mean Petal Length (cm)")

# step 1 - use dplyr to calculate the means
wind.means <- 
  storms %>% group_by(year) %>% 
  summarise(mean = mean(wind), 
            se = sd(wind)/sqrt(n()))

# step 2 - make the plot
ggplot(wind.means, aes(x = factor(year), y = mean, 
                       ymin = mean - se, ymax = mean + se)) + 
  geom_col(fill="darkgrey") + 
  geom_errorbar(width = 0.25) + 
  labs(x = "Year", y = "Wind speed (mph)")

## Calculate the mean and standard errors for each diet at each time point
pltdata<- group_by(ChickWeight, Time, Diet) %>% 
  summarise(mn = mean(weight), se = sd(weight)/sqrt(n()))
## Plot the means over time - remembering to colour by the diet
plta <- ggplot(pltdata, aes(x=Time, y = mn, colour = Diet)) + 
  geom_point() + 
  geom_line() + ## Unsurprisingly the function for adding lines to our plot is geom_line
  theme_classic() + 
  labs(y = "Mean weight (g)", x = "Time (days)")
plta

## Filter the summary data to only include the final weights
pltdata2 <- ungroup(pltdata) %>% 
  filter(Time==max(Time))
## Make a bar plot of the means and standard errors
pltb <- ggplot(pltdata2, aes(x=Diet, y = mn, ymin = mn-se, ymax = mn+se)) + 
  geom_col(fill = 'cornflowerblue', colour = "black") + 
  geom_errorbar(width = 0.3) + 
  labs(y = "Final weight (g)") + 
  theme_classic()
pltb

We can then use the plot_grid function to make a panel plot containing both graphs. This function is from the cowplot package so you’ll need to have that loaded (remember if you haven’t used it before then you’ll need to install it first using install.packages).

library(cowplot)
plot_grid(plta, pltb, labels = c("a)", "b)"))