Introduction to Tidyverse

1 Data wrangling

1.1 The gapminder dataset

1.1.1 Loading the gapminder and dplyr packages

Before you can work with the gapminder dataset, you’ll need to load two R packages that contain the tools for working with it, then display the gapminder dataset so that you can see what it contains.

Download the tidyverse for beginners Cheat Sheet

# Load the gapminder package

Warning messages:
1: In readChar(file, size, TRUE) : truncating string with embedded nuls
2: In readChar(file, size, TRUE) : truncating string with embedded nuls
3: In readChar(file, size, TRUE) : truncating string with embedded nuls

library(gapminder)
# Load the dplyr package
library(dplyr)
# Load the ggplot2 package
library(ggplot2)

gapminder

How many observations (rows) are in the dataset?

nrow(gapminder)

[1] 1704

1.2 The filter verb

1.2.1 Filtering for one year

The filter verb extracts particular observations based on a condition. In this exercise you’ll filter for observations from a particular year.

gapminder %>%  
  filter(year == 2007)

1.2.2 Filtering for one country and one year

You can also use the filter() verb to set two conditions, which could retrieve a single observation.

Just like in the last exercise, you can do this in two lines of code, starting with gapminder %>% and having the filter() on the second line. Keeping one verb on each line helps keep the code readable. Note that each time, you’ll put the pipe %>% at the end of the first line (like gapminder %>%); putting the pipe at the beginning of the second line will throw an error.

gapminder %>% 
  filter(year == 2007, country== "United States")

1.3 The arrange verb

1.3.1 arrange()

You use arrange() to sort observations in ascending or descending order of a particular variable.

gapminder %>% 
  arrange(desc(gdpPercap))

You’ll often need to use the pipe operator (%>%) to combine multiple dplyr verbs in a row.

gapminder %>% 
  filter(year==2007) %>% 
  arrange(desc(gdpPercap))

1.4 The mutate verb

gapminder %>% 
  mutate(pop = pop/1000000)

Adding a new variable (new column)

gapminder %>% 
  mutate(gdp = gdpPercap * pop)

# Filter, mutate, and arrange the gapminder dataset
gapminder %>%
  filter(year == 2007) %>% 
  mutate(lifeExpMonths = 12 * lifeExp) %>% 
  arrange(desc(lifeExpMonths))

1.4.1 Using mutate to change or create a column

Suppose we want life expectancy to be measured in months instead of years: you’d have to multiply the existing value by 12. You can use the mutate() verb to change this column, or to create a new column that’s calculated this way.

# Use mutate to change lifeExp to be in months
gapminder %>%
  mutate(lifeExp = 12 * lifeExp)

# Use mutate to create a new column called lifeExpMonths
gapminder %>%
  mutate(lifeExpMonths = 12 * lifeExp)

1.4.2 Combining filter, mutate, and arrange

In this exercise, you’ll combine all three of the verbs you’ve learned in this chapter, to find the countries with the highest life expectancy, in months, in the year 2007.

# Filter, mutate, and arrange the gapminder dataset
gapminder %>%
  filter(year == 2007) %>% 
  mutate(lifeExpMonths = 12 * lifeExp) %>% 
  arrange(desc(lifeExpMonths))

NA

2 Data Visualization

2.1 Visualizing with ggplot2

2.1.0.1 Variable Assignment

Throughout the exercises in this chapter, you’ll be visualizing a subset of the gapminder data from the year 1952. First, you’ll have to load the ggplot2 package, and create a gapminder_1952 dataset to visualize.

gapminder_2007 <- gapminder %>% 
  filter(year == 2007)

gapminder_2007

# Create gapminder_1952
gapminder_1952 <- gapminder %>%
  filter(year == 1952)

2.1.1 Comparing population and GDP per capita

To create a scatter plot with GDP per capita on the x-axis and life expectancy on the y-axis (the code for that graph is shown here). When you’re exploring data visually, you’ll often need to try different combinations of variables and aesthetics.

gapminder_1952 <- gapminder %>%
  filter(year == 1952)

# Change to put pop on the x-axis and gdpPercap on the y-axis
ggplot(gapminder_1952, aes(x = pop, y = gdpPercap)) +
  geom_point()

2.1.2 Comparing population and life expectancy

In this exercise, you’ll use ggplot2 to create a scatter plot from scratch, to compare each country’s population with its life expectancy in the year 1952.

gapminder_1952 <- gapminder %>%
  filter(year == 1952)

# Create a scatter plot with pop on the x-axis and lifeExp on the y-axis
ggplot(gapminder_1952, aes(x = pop, y = lifeExp)) +
  geom_point()

2.1.2.1 Other Visualizations

When you’re exploring data visually, you’ll often need to try different combinations of variables and aesthetics.

ggplot(gapminder_2007, aes(x = gdpPercap, y = lifeExp)) +
  geom_point()

2.2 Log Scales

2.2.1 Putting the x-axis on a log scale

ggplot(gapminder_2007, aes(x = gdpPercap, y = lifeExp)) +
  geom_point() +
  scale_x_log10()

You previously created a scatter plot with population on the x-axis and life expectancy on the y-axis. Since population is spread over several orders of magnitude, with some countries having a much higher population than others, it’s a good idea to put the x-axis on a log scale.

# Change this plot to put the x-axis on a log scale
ggplot(gapminder_1952, aes(x = pop, y = lifeExp)) +
  geom_point() +
  scale_x_log10()

2.2.2 Putting the x- and y- axes on a log scale

Suppose you want to create a scatter plot with population on the x-axis and GDP per capita on the y-axis. Both population and GDP per-capita are better represented with log scales, since they vary over many orders of magnitude.

ggplot(gapminder_1952, aes(x = pop, y = gdpPercap)) +
  geom_point() +
  scale_x_log10() +
  scale_y_log10()

2.3 Additional aesthetics

2.3.0.1 Color

In this lesson you learned how to use the color aesthetic, which can be used to show which continent each point in a scatter plot represents.

ggplot(gapminder_2007, aes(x = gdpPercap, y = lifeExp, color = continent)) +
    geom_point() +
    scale_x_log10()

2.3.1 Adding size and color to a plot

In the last exercise, you created a scatter plot communicating information about each country’s population, life expectancy, and continent. Now you’ll use the size of the points to communicate even more.

# Scatter plot comparing pop and lifeExp, with color representing continent
ggplot(gapminder_1952, aes(x = pop, y = lifeExp, color = continent, size = gdpPercap)) +
    geom_point() +
    scale_x_log10()

Now you’ll use the size of the points to communicate even more.

ggplot(gapminder_2007, aes(x = gdpPercap, y = lifeExp, color = continent, size = pop)) +
    geom_point() +
    scale_x_log10()

# Add the size aesthetic to represent a country's gdpPercap
ggplot(gapminder_1952, aes(x = pop, y = lifeExp, color = continent, size = gdpPercap)) +
  geom_point() +
  scale_x_log10()

2.4 Faceting

2.4.1 Creating a subgraph for each continent

Divide a graph into subplots based on one of its variables, such as the continent.

ggplot(gapminder_2007, aes(x = gdpPercap, y = lifeExp)) +
    geom_point() +
    scale_x_log10() +
    facet_wrap(~ continent)

# Scatter plot comparing pop and lifeExp, faceted by continent
ggplot(gapminder_1952, aes(x = pop, y = lifeExp)) +
    geom_point() +
    scale_x_log10() +
    facet_wrap(~ continent)

2.4.2 Faceting by year

You can create a graph showing all the country-level data from 1952 to 2007, to understand how global statistics have changed over time.

# Scatter plot comparing gdpPercap and lifeExp, with color representing continent
# and size representing population, faceted by year
ggplot(gapminder, aes(x = gdpPercap, y = lifeExp, color = continent, size = pop)) +
  geom_point() +
  scale_x_log10() +
  facet_wrap(~ year)

3 Grouping and summarizing

3.1 The summarize verb

3.1.1 Summarizing the median life expectancy

You’ve seen how to find the mean life expectancy and the total population across a set of observations, but mean() and sum() are only two of the functions R provides for summarizing a collection of numbers. Here, you’ll learn to use the median() function in combination with summarize().

gapminder %>% 
  summarize(meanLifeExp = mean(lifeExp))

gapminder %>% 
  filter(year == 2007) %>% 
  summarize(meanLifeExp = mean(lifeExp),
            totalPop = sum(as.numeric(pop)))

gapminder %>% 
  summarize(medianLifeExp = median(lifeExp))

3.1.2 Summarizing the median life expectancy in 1957

Rather than summarizing the entire dataset, you may want to find the median life expectancy for only one particular year. In this case, you’ll find the median in the year 1957.

gapminder %>% 
  filter(year == 1957) %>% 
  summarize(medianLifeExp = median(lifeExp))

3.1.3 Summarizing multiple variables in 1957

The summarize() verb allows you to summarize multiple variables at once. In this case, you’ll use the median() function to find the median life expectancy and the max() function to find the maximum GDP per capita.

# Filter for 1957 then summarize the median life expectancy and the maximum GDP per capita
gapminder %>% 
  filter(year == 1957) %>% 
  summarize(medianLifeExp = median(lifeExp),
  maxGdpPercap = max(gdpPercap))

3.2 The group_by verb

3.2.1 Summarizing by year

In a previous exercise, you found the median life expectancy and the maximum GDP per capita in the year 1957. Now, you’ll perform those two summaries within each year in the dataset, using the group_by verb.

gapminder %>% 
  group_by(year) %>% 
  summarize(meanLifeExp = mean(lifeExp),
            totalPop = sum(as.numeric(pop)))

`summarise()` ungrouping output (override with `.groups` argument)

3.2.2 Summarizing by continent

You can group by any variable in your dataset to create a summary. Rather than comparing across time, you might be interested in comparing among continents. You’ll want to do that within one year of the dataset: let’s use 1957.

gapminder %>% 
  filter(year == 2007) %>% 
  group_by(continent) %>% 
  summarize(meanLifeExp = mean(lifeExp),
            totalPop = sum(as.numeric(pop)))

`summarise()` ungrouping output (override with `.groups` argument)

3.2.3 Summarizing by continent and year

Instead of grouping just by year, or just by continent, you’ll now group by both continent and year to summarize within each.

gapminder %>% 
  group_by(year, continent) %>% 
  summarize(meanLifeExp = mean(lifeExp),
            totalPop = sum(as.numeric(pop)))

`summarise()` regrouping output by 'year' (override with `.groups` argument)

3.3 Visualized summarized data

3.3.1 Visualizing median life expectancy over time

In the last chapter, you summarized the gapminder data to calculate the median life expectancy within each year. This code is provided for you, and is saved (with <-) as the by_year dataset.

Now you can use the ggplot2 package to turn this into a visualization of changing life expectancy over time.

by_year <- gapminder %>% 
  group_by(year) %>% 
  summarize(totalPop = sum(as.numeric(pop)),
  meanLifeExp = mean(lifeExp))

`summarise()` ungrouping output (override with `.groups` argument)

#plotting the data
ggplot(by_year, aes(x = year, y = totalPop)) +
  geom_point() +
  expand_limits(y = 0) #specify y axis to star at zero

3.3.2 Visualizing median GDP per capita per continent over time

In the last exercise you were able to see how the median life expectancy of countries changed over time. Now you’ll examine the median GDP per capita instead, and see how the trend differs among continents.

by_year_continent <- gapminder %>% 
  group_by(year, continent) %>% 
  summarize(totalPop = sum(as.numeric(pop)),
  meanLifeExp = mean(lifeExp))

`summarise()` regrouping output by 'year' (override with `.groups` argument)

ggplot(by_year_continent, aes(x = year, y = totalPop, color = continent)) +
  geom_point() +
  expand_limits(y = 0)

3.3.2.1 Comparing median life expectancy and median GDP per continent in 2007

In these exercises you’ve generally created plots that show change over time. But as another way of exploring your data visually, you can also use ggplot2 to plot summarized data to compare continents within a single year.

# Summarize the median GDP and median life expectancy per continent in 2007
by_continent_2007 <- gapminder %>%
  filter(year == 2007) %>%
  group_by(continent) %>%
  summarize(medianLifeExp = median(lifeExp),
  medianGdpPercap = median(gdpPercap))

`summarise()` ungrouping output (override with `.groups` argument)

# Use a scatter plot to compare the median GDP and median life expectancy
ggplot(by_continent_2007, aes(x = medianGdpPercap, y = medianLifeExp, color = continent)) +
  geom_point() +
  expand_limits(y = 0)

4 Types of visualizations

4.1 Line Plots

4.1.1 Visualizing median GDP per capita over time

A line plot is useful for visualizing trends over time. In this exercise, you’ll examine how the median GDP per capita has changed over time.

geom_line()

ggplot(by_year_continent, aes(x = year, y = totalPop, color = continent)) +
  geom_line() +
  expand_limits(y = 0)

# Summarize the median gdpPercap by year, then save it as by_year
by_year <- gapminder %>% 
  group_by(year) %>% 
  summarize(medianGdpPercap = median(gdpPercap))

`summarise()` ungrouping output (override with `.groups` argument)

# Create a line plot showing the change in medianGdpPercap over time
ggplot(by_year, aes(x = year, y = medianGdpPercap)) +
  geom_line() +
  expand_limits(y = 0)

4.1.2 Visualizing median GDP per capita by continent over time

In the last exercise you used a line plot to visualize the increase in median GDP per capita over time. Now you’ll examine the change within each continent.

# Summarize the median gdpPercap by year & continent, save as by_year_continent
by_year_continent <- gapminder %>% 
  group_by(year, continent) %>% 
  summarize(medianGdpPercap = median(gdpPercap))

`summarise()` regrouping output by 'year' (override with `.groups` argument)

# Create a line plot showing the change in medianGdpPercap by continent over time
ggplot(by_year_continent, aes(x = year, y = medianGdpPercap, color = continent)) +
  geom_line() +
  expand_limits(y = 0)

4.2 Bar plots

4.2.1 Visualizing median GDP per capita by continent

A bar plot is useful for visualizing summary statistics, such as the median GDP in each continent.

# Summarize the median gdpPercap by continent in 1952
by_continent <- gapminder %>%
  filter(year == 1952) %>%
  group_by(continent) %>%
  summarize(medianGdpPercap = median(gdpPercap))

`summarise()` ungrouping output (override with `.groups` argument)

# Create a bar plot showing medianGdp by continent
ggplot(by_continent, aes(x = continent, y = medianGdpPercap)) +
  geom_col()

4.2.2 Visualizing GDP per capita by country in Oceania

Bar plot comparing the GDP per capita between the two countries in the Oceania continent (Australia and New Zealand).

# Filter for observations in the Oceania continent in 1952
oceania_1952 <- gapminder %>%
  filter(year == 1952, continent == "Oceania")

# Create a bar plot of gdpPercap by country
ggplot(oceania_1952, aes(x = country, y = gdpPercap)) +
  geom_col()

4.3 Histograms

4.3.1 Visualizing population

A histogram is useful for examining the distribution of a numeric variable. In this exercise, you’ll create a histogram showing the distribution of country populations (by millions) in the year 1952.

ggplot(gapminder_2007, aes(x = lifeExp)) +
  geom_histogram()

Using Binwidth

ggplot(gapminder_2007, aes(x = lifeExp)) +
  geom_histogram(binwidth = 5)

gapminder_1952 <- gapminder %>%
  filter(year == 1952) %>%
  mutate(pop_by_mil = pop / 1000000)

# Create a histogram of population (pop_by_mil)
ggplot(gapminder_1952, aes(x = pop_by_mil)) +
  geom_histogram(bins = 50)

4.3.2 Visualizing population with x-axis on a log scale

In the last exercise you created a histogram of populations across countries. You might have noticed that there were several countries with a much higher population than others, which causes the distribution to be very skewed, with most of the distribution crammed into a small part of the graph. (Consider that it’s hard to tell the median or the minimum population from that histogram).

To make the histogram more informative, you can try putting the x-axis on a log scale.

gapminder_1952 <- gapminder %>%
  filter(year == 1952)

# Create a histogram of population (pop), with x on a log scale
ggplot(gapminder_1952, aes(x = pop)) +
  geom_histogram(bins = 50) +
  scale_x_log10()

4.4 Box plots

4.4.1 Comparing GDP per capita across continents

A boxplot is useful for comparing a distribution of values across several groups. In this exercise, you’ll examine the distribution of GDP per capita by continent. Since GDP per capita varies across several orders of magnitude, you’ll need to put the y-axis on a log scale.

ggplot(gapminder_2007, aes(x = continent, y = lifeExp)) +
  geom_boxplot()

gapminder_1952 <- gapminder %>%
  filter(year == 1952)

# Create a boxplot comparing gdpPercap among continents
ggplot(gapminder_1952, aes(x = continent, y = gdpPercap)) +
  geom_boxplot() +
  scale_y_log10()

4.5 Adding a title to your graph

There are many other options for customizing a ggplot2 graph, which you can learn about in other DataCamp courses. You can also learn about them from online resources, which is an important skill to develop.

As the final exercise in this course, you’ll practice looking up ggplot2 instructions by completing a task we haven’t shown you how to do.

# Add a title to this graph: "Comparing GDP per capita across continents"
ggplot(gapminder_1952, aes(x = continent, y = gdpPercap)) +
  geom_boxplot() +
  scale_y_log10() +
  ggtitle("Comparing GDP per capita across continents")