The tidyverse is a collection of powerful R packages
for data science. They include:
readrtidyr, dplyrggplot2tibble,
purrr, stringr, forcatsYou can install them all using
(Remember, you only need to do this once!)
Pipes use the %>% operator to
take the output from a previous function call and “pipe” it through to
the next function.
This can really help with readability of code when we use multiple nested functions!
We will use the gapminder data from Jenny Bryan to
demonstrate throughout this section. Load the package
gapminder (after installing) and the corresponding
data.
Let’s preview the data.
## # A tibble: 6 × 6
## country continent year lifeExp pop gdpPercap
## <fct> <fct> <int> <dbl> <int> <dbl>
## 1 Afghanistan Asia 1952 28.8 8425333 779.
## 2 Afghanistan Asia 1957 30.3 9240934 821.
## 3 Afghanistan Asia 1962 32.0 10267083 853.
## 4 Afghanistan Asia 1967 34.0 11537966 836.
## 5 Afghanistan Asia 1972 36.1 13079460 740.
## 6 Afghanistan Asia 1977 38.4 14880372 786.
Let’s preview the data.
## # A tibble: 10 × 6
## country continent year lifeExp pop gdpPercap
## <fct> <fct> <int> <dbl> <int> <dbl>
## 1 Afghanistan Asia 1952 28.8 8425333 779.
## 2 Afghanistan Asia 1957 30.3 9240934 821.
## 3 Afghanistan Asia 1962 32.0 10267083 853.
## 4 Afghanistan Asia 1967 34.0 11537966 836.
## 5 Afghanistan Asia 1972 36.1 13079460 740.
## 6 Afghanistan Asia 1977 38.4 14880372 786.
## 7 Afghanistan Asia 1982 39.9 12881816 978.
## 8 Afghanistan Asia 1987 40.8 13867957 852.
## 9 Afghanistan Asia 1992 41.7 16317921 649.
## 10 Afghanistan Asia 1997 41.8 22227415 635.
Let’s preview the data.
## # A tibble: 15 × 6
## country continent year lifeExp pop gdpPercap
## <fct> <fct> <int> <dbl> <int> <dbl>
## 1 Afghanistan Asia 1952 28.8 8425333 779.
## 2 Afghanistan Asia 1957 30.3 9240934 821.
## 3 Afghanistan Asia 1962 32.0 10267083 853.
## 4 Afghanistan Asia 1967 34.0 11537966 836.
## 5 Afghanistan Asia 1972 36.1 13079460 740.
## 6 Afghanistan Asia 1977 38.4 14880372 786.
## 7 Afghanistan Asia 1982 39.9 12881816 978.
## 8 Afghanistan Asia 1987 40.8 13867957 852.
## 9 Afghanistan Asia 1992 41.7 16317921 649.
## 10 Afghanistan Asia 1997 41.8 22227415 635.
## 11 Afghanistan Asia 2002 42.1 25268405 727.
## 12 Afghanistan Asia 2007 43.8 31889923 975.
## 13 Albania Europe 1952 55.2 1282697 1601.
## 14 Albania Europe 1957 59.3 1476505 1942.
## 15 Albania Europe 1962 64.8 1728137 2313.
Below are the mean, median and standard deviation of the variable “pop”.
## [1] 29601212
## [1] 7023596
## [1] 106157897
What if we want the mean log population within our data set? Using base R, we might type
## [1] 15.76611
With pipes, we can use
## [1] 15.76611
Both of these are acceptable. However, when we start adding more and more functions, pipes can make your code much easier and more natural to read.
## [1] 29601212
With pipes, we can use
## [1] 29601212
Very commonly, we will want to filter our data to restrict what we plot or analyze.
For example, what if we are only interested in data from China since 1970?
## [1] FALSE FALSE FALSE FALSE FALSE FALSE
Very commonly, we will want to filter our data to restrict what we plot or analyze.
For example, what if we are only interested in data from China since 1970?
china_1980 <- gapminder %>%
select(country, year) %>%
filter(country == "China" & year >= 1970)
china_1980## # A tibble: 8 × 2
## country year
## <fct> <int>
## 1 China 1972
## 2 China 1977
## 3 China 1982
## 4 China 1987
## 5 China 1992
## 6 China 1997
## 7 China 2002
## 8 China 2007
ggplot2 is a fantastic way to make
complex publication quality images without much pre-processing of your
data.
Plots are built sequentially using layers, so it’s easy to edit and fine-tune the plots you generate.
When using ggplot2, it is essential that
your data are tidy! If they are not, the functions probably will not
look like you expect.
Let’s work through how to build a plot layer by layer.
First, let’s initialize a plot. Use the data
parameter to tell ggplot what data frame to use.
data.frame or
tibble!Add an aesthetic using the function aes() within the
initial ggplot() call.
Now ggplot knows what to plot, but it doesn’t know
how to plot it yet. Let’s add some points with
geom_point().
+
operator.ggplot(data = gapminder,
aes(x = year, y = lifeExp, group = country)) +
geom_line(color = "red", size = 0.75) ## Warning: Using `size` aesthetic for lines was deprecated in ggplot2 3.4.0.
## ℹ Please use `linewidth` instead.
We want to explore differences across continents, so let’s color by continent
aes() because we want to color by something in
our dataaes() will automatically add a
labelgeom_line(), or it
will override the aes()ggplot(data = gapminder,
aes(x = year, y = lifeExp, group = country, color = continent)) +
geom_line(size = 0.4) This is cluttered and hard to read. Let’s try separating by continents using facets!
facet_wrap~ and then the variable name you wantggplot(data = gapminder,
aes(x = year, y = lifeExp, group = country, color = continent)) +
geom_line(size = 0.4) +
geom_smooth(aes(group = continent),
se = FALSE, size = 1.5,
color = "black",
method = "loess") +
facet_wrap(~ continent) ## `geom_smooth()` using formula = 'y ~ x'
Looking better! Now let’s fine tune. First we’ll formalize the
labels on our plot using labs().
xlab(),
ylab(), ggmain(), …ggplot(data = gapminder,
aes(x = year, y = lifeExp,
group = country,
color = continent)) +
geom_line(size = 0.5) +
geom_smooth(aes(group = continent),
se = FALSE, size = 1.5,
color = "black",
method = "loess") +
facet_wrap(~ continent) +
labs(title = "Life expectancy over time by continent",
x = "Year", y = "Life Expectancy",
legend = "Continent")ggplot(data = gapminder,
aes(x = year, y = lifeExp,
group = country,
color = continent)) +
geom_line(size = 0.5) +
geom_smooth(aes(group = continent),
se = FALSE, size = 1.5,
color = "black",
method = "loess") +
facet_wrap(~ continent) +
labs(title = "Life expectancy over time by continent",
x = "Year", y = "Life Expectancy",
legend = "Continent") ggplot(data = gapminder,
aes(x = year, y = lifeExp,
group = country,
color = continent)) +
geom_line(size = 0.5) +
geom_smooth(aes(group = continent),
se = FALSE, size = 1.5, color = "black", method = "loess") +
facet_wrap(~ continent) +
labs(title = "Life expectancy over time by continent",
x = "Year", y = "Life Expectancy") +
theme(plot.title = element_text(hjust = 0.5, face = "bold", size = 14),
legend.position = "none") Great, that looks good! Now we can store it as an object.
lifeExp_plot <- ggplot(data = gapminder,
aes(x = year, y = lifeExp,
group = country,
color = continent)) +
geom_line(size = 0.5) +
geom_smooth(aes(group = continent),
se = FALSE, size = 1.5, color = "black", method = "loess") +
facet_wrap(~ continent) +
labs(title = "Life expectancy over time by continent",
x = "Year", y = "Life Expectancy") +
theme_bw(base_size = 16) +
theme(plot.title = element_text(hjust = 0.5, face = "bold"),
legend.position = "none",
axis.text.x = element_text(angle = 45, hjust = 1, vjust = 1),
panel.spacing.x = unit(0.75, "cm")) ## `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.
ggplot(data = gapminder,
aes(x = lifeExp)) +
geom_histogram(binwidth = 1,
color = "black",
fill = "lightblue") ggplot(data = gapminder,
aes(x = lifeExp)) +
geom_histogram(binwidth = 1,
color = "black",
fill = "lightblue") +
theme_bw(base_size = 20) +
labs(x = "Life Expectancy",
y = "Count")ggplot(data = gapminder,
aes(x = continent, y = lifeExp)) +
geom_boxplot(fill = "lightblue") +
labs(title = "Life expectancy by Continent",
x = "",
y = "") +
theme(plot.title =
element_text(hjust = 0.5)) +
ylim(c(20, 85)) ggplot(data = gapminder,
aes(x = continent, y = lifeExp)) +
geom_boxplot(fill = "lightblue") +
labs(title = "Life expectancy by Continent",
x = "",
y = "") +
theme(plot.title =
element_text(hjust = 0.5)) +
ylim(c(0, 85)) *Don’ts
*Do’s
xlim(), ylim()aes(), edit within
theme() or guides()geom_point(), geom_line(),
geom_histogram(), geom_bar(),
geom_boxplot(), geom_text()facet_grid(), facet_wrap() for
facetinglabs() for labelstheme_bw() to make things look nicercolor for color,
alpha for opacity, lwd/size for
thickness, shape for shape, fill for interior
color, …geom_histogram()ggplot(gapminder, aes(x = lifeExp)) +
geom_histogram() +
labs(x = "Life Expectancy",
y = "Count",
title = "Distribution of Life Expectancy") +
theme_bw(base_size = 16) +
theme(plot.title =
element_text(hjust = 0.5)) geom_boxplot()ggplot(gapminder, aes(y = lifeExp)) +
geom_boxplot() +
labs(x = "",
y = "Life Expectancy",
title = "Distribution of Life Expectancy") +
theme_bw(base_size = 16) +
theme(plot.title =
element_text(hjust = 0.5),
axis.ticks.x = element_blank(),
axis.text.x = element_blank()) +
ylim(c(0,90))geom_bar()ggplot(gapminder, aes(x = continent)) +
geom_bar() +
labs(x = "Continent",
y = "Count",
title = "Distribution of Continent") +
theme_bw(base_size = 16) +
theme(plot.title =
element_text(hjust = 0.5)) geom_boxplot()Here, we have multiple observations for each category.
ggplot(gapminder, aes(x = continent, y = lifeExp)) +
geom_boxplot() +
labs(x = "Continent",
y = "Life Expectancy",
title = "Life Expectancy by Continent") +
theme_bw(base_size = 16) +
theme(plot.title =
element_text(hjust = 0.5)) geom_bar() (with argument
stat = "identity")
Here we have only one observation per category.
mean_lifeExp <- gapminder %>%
group_by(continent) %>%
summarise(mean_lifeExp = mean(lifeExp))
ggplot(mean_lifeExp, aes(x = continent, y = mean_lifeExp)) +
geom_bar(stat = "identity") +
labs(x = "Continent",
y = "Mean Life Expectancy",
title = "Mean Life Expectancy by Continent") +
theme_bw(base_size = 16) +
theme(plot.title =
element_text(hjust = 0.5)) geom_point()ggplot(gapminder, aes(y = lifeExp, x = log(gdpPercap))) +
geom_point() +
labs(y = "Life expectancy",
x = "Log GDP per capita",
title = "Life Expectancy by Log GDP per capita") +
ylim(c(0,90)) +
theme_bw(base_size = 16) +
theme(plot.title =
element_text(hjust = 0.5)) geom_bar() setting x and fill
within aes()gapminder <- gapminder %>%
mutate(lifeExpCat = ifelse(lifeExp > 60, TRUE, FALSE))
ggplot(gapminder, aes(x = continent, fill = lifeExpCat)) +
geom_bar() +
xlab("Continent") +
ylab("Count") +
labs(fill = "Life Exp. > 60") +
ggtitle("Distribution of Continent and Life Expectancy") +
theme_bw(base_size = 16) +
theme(plot.title =
element_text(hjust = 0.5)) Let’s preview the data.
## # A tibble: 6 × 6
## country continent year lifeExp pop gdpPercap
## <fct> <fct> <int> <dbl> <int> <dbl>
## 1 Afghanistan Asia 1952 28.8 8425333 779.
## 2 Afghanistan Asia 1957 30.3 9240934 821.
## 3 Afghanistan Asia 1962 32.0 10267083 853.
## 4 Afghanistan Asia 1967 34.0 11537966 836.
## 5 Afghanistan Asia 1972 36.1 13079460 740.
## 6 Afghanistan Asia 1977 38.4 14880372 786.
select() to choose certain columns.## # A tibble: 1,704 × 2
## lifeExp gdpPercap
## <dbl> <dbl>
## 1 28.8 779.
## 2 30.3 821.
## 3 32.0 853.
## 4 34.0 836.
## 5 36.1 740.
## 6 38.4 786.
## 7 39.9 978.
## 8 40.8 852.
## 9 41.7 649.
## 10 41.8 635.
## # … with 1,694 more rows
## # ℹ Use `print(n = ...)` to see more rows
Use select() to choose certain columns. We can also use
select() to remove certain columns by putting a
- symbol in front.
## # A tibble: 1,704 × 4
## country continent year pop
## <fct> <fct> <int> <int>
## 1 Afghanistan Asia 1952 8425333
## 2 Afghanistan Asia 1957 9240934
## 3 Afghanistan Asia 1962 10267083
## 4 Afghanistan Asia 1967 11537966
## 5 Afghanistan Asia 1972 13079460
## 6 Afghanistan Asia 1977 14880372
## 7 Afghanistan Asia 1982 12881816
## 8 Afghanistan Asia 1987 13867957
## 9 Afghanistan Asia 1992 16317921
## 10 Afghanistan Asia 1997 22227415
## # … with 1,694 more rows
## # ℹ Use `print(n = ...)` to see more rows
select() has a number of helper functions to select
columns (see ?select). Notably:
starts_with() for prefixesends_with() for suffixescontains() for strings## [1] "country" "continent" "year" "lifeExp" "pop" "gdpPercap"
## # A tibble: 6 × 2
## country continent
## <fct> <fct>
## 1 Afghanistan Asia
## 2 Afghanistan Asia
## 3 Afghanistan Asia
## 4 Afghanistan Asia
## 5 Afghanistan Asia
## 6 Afghanistan Asia
select() has a number of helper functions to select
columns (see ?select). Notably:
starts_with() for prefixesends_with() for suffixescontains() for strings## [1] "country" "continent" "year" "lifeExp" "pop" "gdpPercap"
## # A tibble: 6 × 3
## lifeExp pop gdpPercap
## <dbl> <int> <dbl>
## 1 28.8 8425333 779.
## 2 30.3 9240934 821.
## 3 32.0 10267083 853.
## 4 34.0 11537966 836.
## 5 36.1 13079460 740.
## 6 38.4 14880372 786.
select() has a number of helper functions to select
columns (see ?select). Notably:
starts_with() for prefixesends_with() for suffixescontains() for strings## [1] "country" "continent" "year" "lifeExp" "pop" "gdpPercap"
## # A tibble: 6 × 1
## gdpPercap
## <dbl>
## 1 779.
## 2 821.
## 3 853.
## 4 836.
## 5 740.
## 6 786.
select() has a number of helper functions to select
columns (see ?select). Notably:
starts_with() for prefixesends_with() for suffixescontains() for stringscategory_01, category_02,
category_03, …)Create and transform variables from existing variables using
mutate().
mutate(NEW_VAR_NAME = ...).
## # A tibble: 6 × 7
## country continent year lifeExp pop gdpPercap gdp
## <fct> <fct> <int> <dbl> <int> <dbl> <dbl>
## 1 Afghanistan Asia 1952 28.8 8425333 779. 6567086330.
## 2 Afghanistan Asia 1957 30.3 9240934 821. 7585448670.
## 3 Afghanistan Asia 1962 32.0 10267083 853. 8758855797.
## 4 Afghanistan Asia 1967 34.0 11537966 836. 9648014150.
## 5 Afghanistan Asia 1972 36.1 13079460 740. 9678553274.
## 6 Afghanistan Asia 1977 38.4 14880372 786. 11697659231.
Create and transform variables from existing variables using
mutate().
mutate(NEW_VAR_NAME = ...).
Note that we can use existing columns, including those we created
within the same call to mutate().
## # A tibble: 6 × 8
## country continent year lifeExp pop gdpPercap gdp logGDP
## <fct> <fct> <int> <dbl> <int> <dbl> <dbl> <dbl>
## 1 Afghanistan Asia 1952 28.8 8425333 779. 6567086330. 22.6
## 2 Afghanistan Asia 1957 30.3 9240934 821. 7585448670. 22.7
## 3 Afghanistan Asia 1962 32.0 10267083 853. 8758855797. 22.9
## 4 Afghanistan Asia 1967 34.0 11537966 836. 9648014150. 23.0
## 5 Afghanistan Asia 1972 36.1 13079460 740. 9678553274. 23.0
## 6 Afghanistan Asia 1977 38.4 14880372 786. 11697659231. 23.2
group_by()
You can use group_by() to define a grouping of the
rows of your data based on the columns.
Notice that it doesn’t change our data, except for that we can now see that it has 5 groups.
## # A tibble: 4 × 6
## # Groups: continent [1]
## country continent year lifeExp pop gdpPercap
## <fct> <fct> <int> <dbl> <int> <dbl>
## 1 Afghanistan Asia 1952 28.8 8425333 779.
## 2 Afghanistan Asia 1957 30.3 9240934 821.
## 3 Afghanistan Asia 1962 32.0 10267083 853.
## 4 Afghanistan Asia 1967 34.0 11537966 836.
group_by()## # A tibble: 4 × 6
## # Groups: continent [1]
## country continent year lifeExp pop gdpPercap
## <fct> <fct> <int> <dbl> <int> <dbl>
## 1 Zimbabwe Africa 1992 60.4 10704340 693.
## 2 Zimbabwe Africa 1997 46.8 11404948 792.
## 3 Zimbabwe Africa 2002 40.0 11926563 672.
## 4 Zimbabwe Africa 2007 43.5 12311143 470.
group_by()
You can use group_by() to define a grouping of the
rows of your data based on the columns.
We can remove the groups with ungroup().
## # A tibble: 6 × 6
## country continent year lifeExp pop gdpPercap
## <fct> <fct> <int> <dbl> <int> <dbl>
## 1 Afghanistan Asia 1952 28.8 8425333 779.
## 2 Afghanistan Asia 1957 30.3 9240934 821.
## 3 Afghanistan Asia 1962 32.0 10267083 853.
## 4 Afghanistan Asia 1967 34.0 11537966 836.
## 5 Afghanistan Asia 1972 36.1 13079460 740.
## 6 Afghanistan Asia 1977 38.4 14880372 786.
summarize()
summarize() (or summarise(), if you
prefer) will apply functions to rows of a data frame. If a data frame is
grouped, the function will be applied by group.
gapminder %>%
group_by(continent) %>%
summarize(mean_lifeExp = mean(lifeExp, na.rm = TRUE),
max_gdp = max(gdpPercap))## # A tibble: 5 × 3
## continent mean_lifeExp max_gdp
## <fct> <dbl> <dbl>
## 1 Africa 48.9 21951.
## 2 Americas 64.7 42952.
## 3 Asia 60.1 113523.
## 4 Europe 71.9 49357.
## 5 Oceania 74.3 34435.
summarize()gapminder %>%
group_by(continent) %>%
summarize(summary_lifeExp = mean(lifeExp, na.rm = TRUE), median(lifeExp, na.rm = TRUE), sd(lifeExp, na.rm = TRUE))## # A tibble: 5 × 4
## continent summary_lifeExp `median(lifeExp, na.rm = TRUE)` sd(lifeExp, na.rm …¹
## <fct> <dbl> <dbl> <dbl>
## 1 Africa 48.9 47.8 9.15
## 2 Americas 64.7 67.0 9.35
## 3 Asia 60.1 61.8 11.9
## 4 Europe 71.9 72.2 5.43
## 5 Oceania 74.3 73.7 3.80
## # … with abbreviated variable name ¹`sd(lifeExp, na.rm = TRUE)`
arrange()
We can order the rows of the data using arrange().
Usually, you do this after some call to
summarize().
gapminder %>%
group_by(continent) %>%
summarize(mean_lifeExp = mean(lifeExp, na.rm = TRUE),
max_gdp = max(gdpPercap)) %>%
arrange(mean_lifeExp)## # A tibble: 5 × 3
## continent mean_lifeExp max_gdp
## <fct> <dbl> <dbl>
## 1 Africa 48.9 21951.
## 2 Asia 60.1 113523.
## 3 Americas 64.7 42952.
## 4 Europe 71.9 49357.
## 5 Oceania 74.3 34435.
arrange()
We can order the rows of the data using arrange().
Usually, you do this after some call to summarize(). We can
use desc() to put in decreasing order.
gapminder %>%
group_by(continent) %>%
summarize(mean_lifeExp = mean(lifeExp, na.rm = TRUE),
max_gdp = max(gdpPercap)) %>%
arrange(desc(mean_lifeExp))## # A tibble: 5 × 3
## continent mean_lifeExp max_gdp
## <fct> <dbl> <dbl>
## 1 Oceania 74.3 34435.
## 2 Europe 71.9 49357.
## 3 Americas 64.7 42952.
## 4 Asia 60.1 113523.
## 5 Africa 48.9 21951.
pull() to extract a single column as a
vector. Notice the difference between## # A tibble: 4 × 1
## lifeExp
## <dbl>
## 1 28.8
## 2 30.3
## 3 32.0
## 4 34.0
## [1] 28.801 30.332 31.997 34.020 36.088 38.438 39.854 40.822 41.674 41.763
pull() to extract a single column as a
vector. Notice the difference between## # A tibble: 4 × 1
## continent
## <fct>
## 1 Asia
## 2 Asia
## 3 Asia
## 4 Asia
## # A tibble: 20 × 1
## continent
## <fct>
## 1 Asia
## 2 Asia
## 3 Asia
## 4 Asia
## 5 Asia
## 6 Asia
## 7 Asia
## 8 Asia
## 9 Asia
## 10 Asia
## 11 Asia
## 12 Asia
## 13 Europe
## 14 Europe
## 15 Europe
## 16 Europe
## 17 Europe
## 18 Europe
## 19 Europe
## 20 Europe
rename(data, NEW_VAR_NAME = OLD_VAR_NAME)
Recall, want to change the variable lifeExp to life_exp
## # A tibble: 6 × 6
## country continent year lifeExp pop gdpPercap
## <fct> <fct> <int> <dbl> <int> <dbl>
## 1 Afghanistan Asia 1952 28.8 8425333 779.
## 2 Afghanistan Asia 1957 30.3 9240934 821.
## 3 Afghanistan Asia 1962 32.0 10267083 853.
## 4 Afghanistan Asia 1967 34.0 11537966 836.
## 5 Afghanistan Asia 1972 36.1 13079460 740.
## 6 Afghanistan Asia 1977 38.4 14880372 786.
rename(data, NEW_VAR_NAME = OLD_VAR_NAME)
Recall, want to change the variable lifeExp to life_exp
## # A tibble: 1,704 × 6
## country continent year life_exp pop gdpPercap
## <fct> <fct> <int> <dbl> <int> <dbl>
## 1 Afghanistan Asia 1952 28.8 8425333 779.
## 2 Afghanistan Asia 1957 30.3 9240934 821.
## 3 Afghanistan Asia 1962 32.0 10267083 853.
## 4 Afghanistan Asia 1967 34.0 11537966 836.
## 5 Afghanistan Asia 1972 36.1 13079460 740.
## 6 Afghanistan Asia 1977 38.4 14880372 786.
## 7 Afghanistan Asia 1982 39.9 12881816 978.
## 8 Afghanistan Asia 1987 40.8 13867957 852.
## 9 Afghanistan Asia 1992 41.7 16317921 649.
## 10 Afghanistan Asia 1997 41.8 22227415 635.
## # … with 1,694 more rows
## # ℹ Use `print(n = ...)` to see more rows