This course introduces a tidyverse environment, which is a collection of R packages for transforming and visualizing data. This is not the only set of tools in R, but it’s a powerful and popular approach for exploring data. To dive into a dataset and discover interesting things using Tidyverse environment. The two popular packages dplyr and ggplot2 are used to manipulate and visualize data. In detail, the objective of this course introduce:
gapminder datasetGapminder tracks economic and social indicators like life expectancy and GDP per capita of countries over time. To take a brief look at the data, the function glimpse(data) could be used.
Exercise 1.1. Loading the gapminder and dplyr packages
# Load the gapminder package for dataset
# Load the dplyr package for function glimpse
library(gapminder)
library(dplyr)
library(tidyverse)
library(tibble)
library(magrittr)
dplyr::glimpse(gapminder)## Rows: 1,704
## Columns: 6
## $ country <fct> "Afghanistan", "Afghanistan", "Afghanistan", "Afghanistan", ~
## $ continent <fct> Asia, Asia, Asia, Asia, Asia, Asia, Asia, Asia, Asia, Asia, ~
## $ year <int> 1952, 1957, 1962, 1967, 1972, 1977, 1982, 1987, 1992, 1997, ~
## $ lifeExp <dbl> 28.801, 30.332, 31.997, 34.020, 36.088, 38.438, 39.854, 40.8~
## $ pop <int> 8425333, 9240934, 10267083, 11537966, 13079460, 14880372, 12~
## $ gdpPercap <dbl> 779.4453, 820.8530, 853.1007, 836.1971, 739.9811, 786.1134, ~
Exercise 1.2. Understanding a data frame
After loading the gapminder dataset, we can examine for a better understanding it. To do so, type gapminder in your R terminal, and see how many observations are there in the dataset?
The correct answer is: a.
tibble data frameTibble is dataframe, which tweaks (ie. make small adjustments) some older behaviors. What do “older behavior” mean in this context? After importing a dataframe, people from older behaviors (like me, hihi) have the habit of assessing such following information from the data:
Exercise 1.3. Assess the iris data from older behaviors by means of following functions: str(), head(), nrow() and ncol().
data("iris")
str(iris)
head(iris)
nrow(iris)
ncol(iris)Tibble is a modern dataframe, which takes into account older behaviors. In other words, tibble is lazy dataframe, but convenient. In R environment, the function as_tibble() from the package tibble could be used to convert traditional dataframe to tibble.
tibble::as_tibble(iris)## # A tibble: 150 x 5
## Sepal.Length Sepal.Width Petal.Length Petal.Width Species
## <dbl> <dbl> <dbl> <dbl> <fct>
## 1 5.1 3.5 1.4 0.2 setosa
## 2 4.9 3 1.4 0.2 setosa
## 3 4.7 3.2 1.3 0.2 setosa
## 4 4.6 3.1 1.5 0.2 setosa
## 5 5 3.6 1.4 0.2 setosa
## 6 5.4 3.9 1.7 0.4 setosa
## 7 4.6 3.4 1.4 0.3 setosa
## 8 5 3.4 1.5 0.2 setosa
## 9 4.4 2.9 1.4 0.2 setosa
## 10 4.9 3.1 1.5 0.1 setosa
## # ... with 140 more rows
Beside of taking into account older behaviors, tibble possesses some characteristics:
When printed, each column will be assigned in 4 types:
logical vectors: denote <lgl>, which contain TRUE or FALSE.integer vectors: denote <int>, which contain integers.double vectors: denote <dbl>, which contain real numbers.character vectors: denote <chr>, which contain strings made with "__".All vectors can also contain the missing value, denote NA. Collectively integer and double vectors are known as numeric vectors.
filter() verbThe objective of the filter() function is to extract a subset of observations in a dataset based on a defined condition.
Exercise 1.4. Filtering observations for one year
Extracting particular observations based on a condition, eg. extracting data for Vietnam.
# Filter the gapminder dataset for Vietnam
gapminder %>%
filter(country=="Vietnam")## # A tibble: 12 x 6
## country continent year lifeExp pop gdpPercap
## <fct> <fct> <int> <dbl> <int> <dbl>
## 1 Vietnam Asia 1952 40.4 26246839 605.
## 2 Vietnam Asia 1957 42.9 28998543 676.
## 3 Vietnam Asia 1962 45.4 33796140 772.
## 4 Vietnam Asia 1967 47.8 39463910 637.
## 5 Vietnam Asia 1972 50.3 44655014 700.
## 6 Vietnam Asia 1977 55.8 50533506 714.
## 7 Vietnam Asia 1982 58.8 56142181 707.
## 8 Vietnam Asia 1987 62.8 62826491 821.
## 9 Vietnam Asia 1992 67.7 69940728 989.
## 10 Vietnam Asia 1997 70.7 76048996 1386.
## 11 Vietnam Asia 2002 73.0 80908147 1764.
## 12 Vietnam Asia 2007 74.2 85262356 2442.
Exercise 1.5. Filtering for one country and one year
You can also use the filter() verb to set two conditions. Note that we can do this in one line of code by starting with gapminder %>%, then having the filter().
However, we should Keep one verb on each line, this could help the code being readable. At each code line, we put the pipe %>% at the end of the first line (like gapminder %>%), then putting the verb at the beginning of the second line.
# Filter for Vietnam in 2002
gapminder %>%
filter(country=="Vietnam", year=="2002")## # A tibble: 1 x 6
## country continent year lifeExp pop gdpPercap
## <fct> <fct> <int> <dbl> <int> <dbl>
## 1 Vietnam Asia 2002 73.0 80908147 1764.
arrange() verbThe objective of the arrange() function is to sort observations in a dataset in ascending or descending order. The statement for:
data %>% arrange(variable), and fordata %>% arrange(desc(variable))We can flexiable combine filter() and arrange() verb to extract a subset of observations, and then sort the subset data extracted.
# statement
# data %>% filter(variable_1 == condition) %>% ## extracting
# arrange(variable_2) ## descending orderExercise 1.6. Arranging observations by life expectancy
You use arrange() to sort observations in ascending or descending order of a particular variable. In this case, you’ll sort the dataset based on the lifeExp variable.
# Sort in ascending order of lifeExp
gapminder %>%
arrange(lifeExp)## # A tibble: 1,704 x 6
## country continent year lifeExp pop gdpPercap
## <fct> <fct> <int> <dbl> <int> <dbl>
## 1 Rwanda Africa 1992 23.6 7290203 737.
## 2 Afghanistan Asia 1952 28.8 8425333 779.
## 3 Gambia Africa 1952 30 284320 485.
## 4 Angola Africa 1952 30.0 4232095 3521.
## 5 Sierra Leone Africa 1952 30.3 2143249 880.
## 6 Afghanistan Asia 1957 30.3 9240934 821.
## 7 Cambodia Asia 1977 31.2 6978607 525.
## 8 Mozambique Africa 1952 31.3 6446316 469.
## 9 Sierra Leone Africa 1957 31.6 2295678 1004.
## 10 Burkina Faso Africa 1952 32.0 4469979 543.
## # ... with 1,694 more rows
# Sort in descending order of lifeExp
gapminder %>%
arrange(desc(lifeExp))## # A tibble: 1,704 x 6
## country continent year lifeExp pop gdpPercap
## <fct> <fct> <int> <dbl> <int> <dbl>
## 1 Japan Asia 2007 82.6 127467972 31656.
## 2 Hong Kong, China Asia 2007 82.2 6980412 39725.
## 3 Japan Asia 2002 82 127065841 28605.
## 4 Iceland Europe 2007 81.8 301931 36181.
## 5 Switzerland Europe 2007 81.7 7554661 37506.
## 6 Hong Kong, China Asia 2002 81.5 6762476 30209.
## 7 Australia Oceania 2007 81.2 20434176 34435.
## 8 Spain Europe 2007 80.9 40448191 28821.
## 9 Sweden Europe 2007 80.9 9031088 33860.
## 10 Israel Asia 2007 80.7 6426679 25523.
## # ... with 1,694 more rows
Exercise 1.7. Filtering and arranging
You’ll often need to use the pipe operator (%>%) to combine multiple dplyr verbs in a row. In this case, you’ll combine a filter() with an arrange() to find the highest population countries in a particular year.
# Filter for the year 1957, then arrange in descending order of population
gapminder %>%
filter(year==1957) %>%
arrange(desc(pop))## # A tibble: 142 x 6
## country continent year lifeExp pop gdpPercap
## <fct> <fct> <int> <dbl> <int> <dbl>
## 1 China Asia 1957 50.5 637408000 576.
## 2 India Asia 1957 40.2 409000000 590.
## 3 United States Americas 1957 69.5 171984000 14847.
## 4 Japan Asia 1957 65.5 91563009 4318.
## 5 Indonesia Asia 1957 39.9 90124000 859.
## 6 Germany Europe 1957 69.1 71019069 10188.
## 7 Brazil Americas 1957 53.3 65551171 2487.
## 8 United Kingdom Europe 1957 70.4 51430000 11283.
## 9 Bangladesh Asia 1957 39.3 51365468 662.
## 10 Italy Europe 1957 67.8 49182000 6249.
## # ... with 132 more rows
mutate() verbThe verb mutate, from the English dictionary, means to change from one thing or type of thing into another. In tidyverse environment, the objective of mutate is to: (1) change a variable and/or (2) add a new variable.
Statement for changing a variable is as data %>% mutate(var_1 = function(var_1)); Let’s say: replace var_1 by a new variable, which is a function of var_1. As consequence, the var_1 in the dataset is going to be dropped.
Statement for adding a new variable is as data %>% mutate(var_2 = function(var_1)). Let’s say: create var_2, which is a function of var_1, and the var_1 is still in the dataset.
Exercise 1.8. Using mutate to change or create a column
Suppose we want life expectancy to be measured in months instead of years: you’d have to multiply the existing value by 12. You can use the mutate() verb to change this column, or to create a new column that’s calculated this way.
# Ex1. Use mutate to change lifeExp to be in months
gapminder %>%
mutate(lifeExp = lifeExp*12)## # A tibble: 1,704 x 6
## country continent year lifeExp pop gdpPercap
## <fct> <fct> <int> <dbl> <int> <dbl>
## 1 Afghanistan Asia 1952 346. 8425333 779.
## 2 Afghanistan Asia 1957 364. 9240934 821.
## 3 Afghanistan Asia 1962 384. 10267083 853.
## 4 Afghanistan Asia 1967 408. 11537966 836.
## 5 Afghanistan Asia 1972 433. 13079460 740.
## 6 Afghanistan Asia 1977 461. 14880372 786.
## 7 Afghanistan Asia 1982 478. 12881816 978.
## 8 Afghanistan Asia 1987 490. 13867957 852.
## 9 Afghanistan Asia 1992 500. 16317921 649.
## 10 Afghanistan Asia 1997 501. 22227415 635.
## # ... with 1,694 more rows
# Ex2. Use mutate to create a new column called lifeExpMonths
gapminder %>%
mutate(lifeExpMonths = lifeExp*12)## # A tibble: 1,704 x 7
## country continent year lifeExp pop gdpPercap lifeExpMonths
## <fct> <fct> <int> <dbl> <int> <dbl> <dbl>
## 1 Afghanistan Asia 1952 28.8 8425333 779. 346.
## 2 Afghanistan Asia 1957 30.3 9240934 821. 364.
## 3 Afghanistan Asia 1962 32.0 10267083 853. 384.
## 4 Afghanistan Asia 1967 34.0 11537966 836. 408.
## 5 Afghanistan Asia 1972 36.1 13079460 740. 433.
## 6 Afghanistan Asia 1977 38.4 14880372 786. 461.
## 7 Afghanistan Asia 1982 39.9 12881816 978. 478.
## 8 Afghanistan Asia 1987 40.8 13867957 852. 490.
## 9 Afghanistan Asia 1992 41.7 16317921 649. 500.
## 10 Afghanistan Asia 1997 41.8 22227415 635. 501.
## # ... with 1,694 more rows
As you can see, the value of lifeExp from example 1 is same as that of lifeExpMonths from example 2. Additionally, var lifeExp was dropped from example 1, but not example 2.
Exercise 1.9. Combining filter(), mutate(), and arrange()
In this exercise, you’ll combine all three of the verbs you’ve learned in this chapter, to find the countries with the highest life expectancy, in months, in the year 2007.
# Filter, mutate, and arrange the gapminder dataset
gapminder %>% filter(year == 2007) %>%
mutate(lifeExpMonths = 12*lifeExp) %>%
arrange(desc(lifeExpMonths))## # A tibble: 142 x 7
## country continent year lifeExp pop gdpPercap lifeExpMonths
## <fct> <fct> <int> <dbl> <int> <dbl> <dbl>
## 1 Japan Asia 2007 82.6 127467972 31656. 991.
## 2 Hong Kong, China Asia 2007 82.2 6980412 39725. 986.
## 3 Iceland Europe 2007 81.8 301931 36181. 981.
## 4 Switzerland Europe 2007 81.7 7554661 37506. 980.
## 5 Australia Oceania 2007 81.2 20434176 34435. 975.
## 6 Spain Europe 2007 80.9 40448191 28821. 971.
## 7 Sweden Europe 2007 80.9 9031088 33860. 971.
## 8 Israel Asia 2007 80.7 6426679 25523. 969.
## 9 France Europe 2007 80.7 61083916 30470. 968.
## 10 Canada Americas 2007 80.7 33390141 36319. 968.
## # ... with 132 more rows
pivot_longer() and pivot_wider()pivot_longer()Let’s take the example_eagle_nests for an illustration. Figure below shows our starting point (i.e. the wide format) and expected results (ie. the long format).
# https://dcl-wrangle.stanford.edu/pivot_basic.html
library(dcldata)
library(tidyr)
# data(package = "dcldata")
example_eagle_nests## # A tibble: 3 x 3
## region `2007` `2009`
## <chr> <dbl> <dbl>
## 1 Pacific 1039 2587
## 2 Southwest 51 176
## 3 Rocky Mountains and Plains 200 338
The command lines below illustrates how to use pivot_longer() for the purpose of converting from wide format to long format. The concept of the command lines can be explained in 4 steps.
example_eagle_nests %>%
pivot_longer(
cols = c(`2007`, `2009`),
names_to = "year",
values_to = "num_nests"
)## # A tibble: 6 x 3
## region year num_nests
## <chr> <chr> <dbl>
## 1 Pacific 2007 1039
## 2 Pacific 2009 2587
## 3 Southwest 2007 51
## 4 Southwest 2009 176
## 5 Rocky Mountains and Plains 2007 200
## 6 Rocky Mountains and Plains 2009 338
Step 1: Identify the variables from dataset. From example_eagle_nests data, there are three variables in this dataset:
region: The US region where the nests are located.year: The year the nests were found.num_nests: The number of nests found.Step 2: Identify the columns to pivot. Identify which columns are keeping the data from being tidy. In our example, those columns are 2007 and 2009.
cols argument controls which columns will be stored in the new column of the pivot.cols argument, the columns can be specified by other arguments, for instance: contains(), starts_with().2007)Step 3: Name new columns of pivot_longer(), which consists of all the columns identified in Step 2.
names_to argument controls the name of the column that will store the old column names. In our case, we want to name this new column “year”. The argument is called names_to because you’re specifying which column to move the column names to.Note that the argument to names_to has to be in quotes, while the arguments cols do not. It’s easy to get confused about which pivot arguments need to be quoted. Here’s the general rule: if you’re identifying an existing column (e.g., 2007), do not quote. If you’re talking about a column that does not currently exist (e.g., year), quote it.
Step 4: Name the column that will store the column values. That column will store the values from 2007 and 2009.
values_to controls the name of column for the values. In example_eagle_nests, the column values represent the number of nests, so we’ll name the new column “num_nests”.pivot_wider()Let’s take the example_eagle_nests for an illustration. Figure below shows our starting point (i.e. the wide format) and expected results (ie. the long format).
As you could have some idea from the previous section, here is the data in form of wide format.
example_eagle_nests_tidy## # A tibble: 6 x 3
## region year num_nests
## <chr> <chr> <dbl>
## 1 Pacific 2007 1039
## 2 Pacific 2009 2587
## 3 Southwest 2007 51
## 4 Southwest 2009 176
## 5 Rocky Mountains and Plains 2007 200
## 6 Rocky Mountains and Plains 2009 338
pivot_wider() turns the values from one column and turns them into column names, also converting from wide to long format. These command lines could be interpreted by 2 steps.
example_eagle_nests_tidy %>%
pivot_wider(
names_from = year,
values_from = num_nests
)## # A tibble: 3 x 3
## region `2007` `2009`
## <chr> <dbl> <dbl>
## 1 Pacific 1039 2587
## 2 Southwest 51 176
## 3 Rocky Mountains and Plains 200 338
Step 1: Identify the column whose values will supply the column names. In our example, we want the unique values from year to become column names.
names_from argument controls which column is pivoted into column namesStep 2: Identify the column whose values will supply the column values.
values_from argument identify num_nests as the column supplying the values of 2007 and 2009.# load iris data
library(tibble)
library(dplyr)
iris_tib <- as.tibble(iris)## Warning: `as.tibble()` was deprecated in tibble 2.0.0.
## Please use `as_tibble()` instead.
## The signature and semantics have changed, see `?as_tibble`.
## This warning is displayed once every 8 hours.
## Call `lifecycle::last_lifecycle_warnings()` to see where this warning was generated.
iris_tib %>% mutate("ID"=c(1:nrow(iris)))## # A tibble: 150 x 6
## Sepal.Length Sepal.Width Petal.Length Petal.Width Species ID
## <dbl> <dbl> <dbl> <dbl> <fct> <int>
## 1 5.1 3.5 1.4 0.2 setosa 1
## 2 4.9 3 1.4 0.2 setosa 2
## 3 4.7 3.2 1.3 0.2 setosa 3
## 4 4.6 3.1 1.5 0.2 setosa 4
## 5 5 3.6 1.4 0.2 setosa 5
## 6 5.4 3.9 1.7 0.4 setosa 6
## 7 4.6 3.4 1.4 0.3 setosa 7
## 8 5 3.4 1.5 0.2 setosa 8
## 9 4.4 2.9 1.4 0.2 setosa 9
## 10 4.9 3.1 1.5 0.1 setosa 10
## # ... with 140 more rows
# using pivot_longer
# - data: data frame to pivot
# - cols: columns to become one new column in longer format.
# for selecting multiple columns: cols = c("var1", "var2")
# - names_to: "new column name"
# - values_to: "new value column name"
iris_tib_long <- pivot_longer(iris_tib,
cols = "Sepal.Length":"Petal.Width",
names_to = "iris_part",
values_to = "measure_cm")
iris_tib_long## # A tibble: 600 x 3
## Species iris_part measure_cm
## <fct> <chr> <dbl>
## 1 setosa Sepal.Length 5.1
## 2 setosa Sepal.Width 3.5
## 3 setosa Petal.Length 1.4
## 4 setosa Petal.Width 0.2
## 5 setosa Sepal.Length 4.9
## 6 setosa Sepal.Width 3
## 7 setosa Petal.Length 1.4
## 8 setosa Petal.Width 0.2
## 9 setosa Sepal.Length 4.7
## 10 setosa Sepal.Width 3.2
## # ... with 590 more rows
# using pivot_wider
# - data: data frame to pivot
# - cols: identifier columns not to pivot.
# for selecting multiple columns: cols = c("var1", "var2")
# - names_from: "new column name"
# - values_from: "new value column name"
# iris_tib_wide <- pivot_wider(iris_tib_long,
# names_from = iris_part,
# values_from = measure_cm)This chapter will show you how to create informative graphs, for instance: scatterplots, histogram, and boxplot using the two packages dplyr and ggplot2.
ggplot2Assignment, denoted <-, this is a less and then a minus sign. This tibble has 142 rows, 6 columns and comes from the year 2007. In this example, we will:
gapminder_2007 dataset, which is a subset of the gapminder data from the year 1952, thengdpPercap) on the X-axis and life expectancy (lifeExp) on the Y-axis.# extract the subset data (year == 2007)
gapminder_2007 <- gapminder %>%
filter (year == 2007)
# create scatter plot
library(ggplot2)
ggplot2::ggplot(gapminder_2007,
aes(x=gdpPercap, y=lifeExp)) +
geom_point() +
theme_bw()This plot communicates some interesting information: the higher income countries tend to have higher life expectancy.
Exercise 2.1. Comparing population and life expectancy
In this exercise, you’ll use ggplot2 to create a scatter plot to compare each country’s population with its life expectancy in the year 1952. Run the command below, then explain this informative graph.
# extract the subset data
gapminder_1952 <- gapminder %>%
filter(year == 1952)
# create a scatter plot with pop on the x-axis and lifeExp on the y-axis
ggplot(gapminder_1952, aes(x=pop, y=lifeExp)) +
geom_point() +
theme_bw()One problem with the plot above is that a lot of countries get crammed into the leftmost part of the x-axis. The reason for that is because the distribution of GDP per capita (gdpPercap) spans several orders of magnitude. In this, some countries in the tens of thousands dollars and others in the hundreds. When one of your axes has that kind of distribution, it’s useful to work with a logarithmic scale. In logarithmic scale, each fixed distance represents a multiplication of the value. Below is what the scatter plot where x is represented on a log scale.
ggplot2::ggplot(gapminder_2007,
aes(x=gdpPercap, y=lifeExp)) +
geom_point() +
scale_x_log10() +
theme_bw()You can see the axis goes from \(300\) to \(30.000\) and the next step of an equal distance would be \(100.000\). On this scale, the relationship between GDP per capita and life expectancy looks more linear. As consequence, you can more easily distinguish the countries at the lower end of the spectrum.
To create this plot, we add a command line scale_x_log10() just after the geom_point(). This command line specifies that we want the x-axis on a log10 scale. It’s not necessary in this case, but if you want to put the y-axis on a log10 scale as well, you can add another command line as scale_y_log10().
Exercise 2.2. Putting the x-axis on a log scale
You previously created a scatter plot with population on the x-axis and life expectancy on the y-axis. Since population is spread over several orders of magnitude, with some countries having a much higher population than others, it’s a good idea to put the x-axis on a log scale.
gapminder_1952 <- gapminder %>%
filter(year == 1952)
# Change this plot to put the x-axis on a log scale
ggplot(gapminder_1952, aes(x = pop, y = lifeExp)) +
geom_point() +
scale_x_log10()Exercise 2.3. Putting both the x- and y- axes on a log scale
Suppose you want to create a scatter plot with population on the x-axis and GDP per capita on the y-axis. Both population and GDP per-capita are better represented with log scales, since they vary over many orders of magnitude.
gapminder_1952 <- gapminder %>%
dplyr::filter(year == 1952)
# Scatter plot comparing pop and gdpPercap, with both axes on a log scale
ggplot2::ggplot(gapminder_1952, aes(x=pop,y=gdpPercap)) +
geom_point() +
scale_x_log10() +
scale_y_log10()color and sizeBesides gdpPercap and lifeExp, we also have continent and population variables that are informative. We could examine relationships among all these variables in the same plot. Due to the fact that we already use the x-axis to represent GDP per capita and the y-axis to represent life expectancy, we will add three more aesthetics: color, size and facet. From technical point of view, the argument color allows to add information from a qualitative variable, whereas the argument size allows to add information from a quantitative variable. Additional aesthetics is a useful way to communicate the scatter plot with more than 2 quantitative variables.
ggplot2::ggplot(gapminder_2007,
aes(x=gdpPercap, y=lifeExp,
color = continent,
size = pop)) +
geom_point() +
scale_x_log10() +
theme_bw()Exercise 2.4. Adding color to a scatter plot
In this lesson you learned how to use the color aesthetic, which can be used to show which continent each point in a scatter plot represents.
gapminder_1952 <- gapminder %>%
dplyr::filter(year == 1952)
# Scatter plot comparing pop and lifeExp, with color representing continent
ggplot2::ggplot(gapminder_1952,
aes(x=pop, y=lifeExp, color=continent)) +
geom_point() +
scale_x_log10()Exercise 2.5. Adding size and color to a plot
In the last exercise, you created a scatter plot communicating information about each country’s population, life expectancy, and continent. Now you’ll use the size of the points to communicate even more.
gapminder_1952 <- gapminder %>%
dplyr::filter(year == 1952)
# Add the size aesthetic to represent a country's gdpPercap
ggplot2::ggplot(gapminder_1952,
aes(x = pop, y = lifeExp,
color = continent,
size = gdpPercap)) +
geom_point() +
scale_x_log10()Facet means one side of something many-sided. Faceting in ggplot2 divides your plot into subplots to get smaller graph for each level of the categorical variable (ie. one side, the so-call faceting). Faceting is an alternative way to the argument color for the purpose of adding a qualitative information to scatter plot.
To facet a plot, we add an option facet_wrap(~variable). Wrap means to cover or surround something with paper, cloth, or other material. Tilde sign (~) means by. The command line facet_wrap(~continent) tells ggplot() to divide data into subplots based on the continent variable
ggplot2::ggplot(gapminder_2007,
aes(x=gdpPercap, y=lifeExp)) +
geom_point() +
scale_x_log10() +
facet_wrap(~continent) +
theme_bw()Exercise 2.6. Creating a subgraph for each continent
You’ve learned to use faceting to divide a graph into subplots based on one of its variables, such as the continent.
gapminder_1952 <- gapminder %>%
filter(year == 1952)
# Scatter plot comparing pop and lifeExp, faceted by continent
ggplot2::ggplot(gapminder_1952, aes(x=pop, y=lifeExp)) +
geom_point() +
scale_x_log10() +
facet_wrap(~continent) +
theme_bw()Exercise 2.7 . Faceting by year
All of the graphs in this chapter have been visualizing statistics within one year. Now that you’re able to use faceting, however, you can create a graph showing all the country-level data from 1952 to 2007, to understand how global statistics have changed over time.
# Scatter plot comparing gdpPercap and lifeExp, with color representing continent
# and size representing population, faceted by year
ggplot(gapminder,
aes(x=gdpPercap, y=lifeExp, color=continent, size=pop)) +
geom_point() +
scale_x_log10() +
facet_wrap(~year) +
theme_bw()summarize() verb, not summarise()Suppose we want to know the average life expectancy across all countries and years in the dataset. From older fashion, I use the function mean() as follow:
mean(gapminder$lifeExp)## [1] 59.47444
With the modern fashion, the function summarize() could be used as command line below. First, take gapminder data, pipe it into summarize, and specify that we want to create a summary column called meanLifeExp.
gapminder %>%
summarize(meanLifeExp = mean(lifeExp))## # A tibble: 1 x 1
## meanLifeExp
## <dbl>
## 1 59.5
If we just compare the two coding fashion in calculating the mean, it does not make sense to learn and apply the verb summarize(). It makes more sense to ask questions about: (1) the average in a particular condition and (2) creating multiple summarize at once. Interestingly, all the statistic functions, for instance: mean(), sum(), median(), min(), max(), var(), or sd() could be used in combination with the fuction summarize().
For example, we want to find the average of life expectancy for the year 2007 in the one part, and also the media population in that year. To do that, we add a comma after the mean of the life expectancy, then add comment medianPop=median(pop). As you can see in the results, a new column totalPop is added into the output table.
gapminder %>%
filter(year == 2007) %>%
summarize(meanLifeExp = mean(lifeExp),
medianPop = median(pop),
varLifeExp = var(lifeExp),
sdPop=sd(pop))## # A tibble: 1 x 4
## meanLifeExp medianPop varLifeExp sdPop
## <dbl> <dbl> <dbl> <dbl>
## 1 67.0 10517531 146. 147621398.
Exercise 3.1. Summarizing the median life expectancy
# Summarize to find the median life expectancy
gapminder %>%
summarize(medianLifeExp = median(lifeExp))Exercise 3.2. Summarizing the median life expectancy in 1957
# Filter for 1957 then summarize the median life expectancy
gapminder %>%
filter(year==1957) %>%
summarize(medianLifeExp = median(lifeExp))Exercise 3.3. Summarizing multiple variables in 1957
# Filter for 1957, then summarize the median life expectancy
# and the maximum GDP per capita
gapminder %>%
filter(year == 1957) %>%
summarize(medianLifeExp = median(lifeExp),
maxGdpPercap=max(gdpPercap))group_by() verbWhat if we were not interested in the year 2007, but for each year in the dataset? We could re-run this code and change the year for each time, but that’s very tedious. Instead, we can used the group_by verb to summarize within group, not the entire dataset. Let’s do this by replacing the command filter(year == 2007) by group_by(year):
# define pop as numeric
# as.numeric is identical to as.double
gapminder$pop <- as.numeric(gapminder$pop)
# summarize meanLifeExp and totalPop by year
gapminder %>%
group_by(year) %>%
summarize(meanLifeExp = mean(lifeExp),
totalPop=sum(pop))## # A tibble: 12 x 3
## year meanLifeExp totalPop
## <int> <dbl> <dbl>
## 1 1952 49.1 2406957150
## 2 1957 51.5 2664404580
## 3 1962 53.6 2899782974
## 4 1967 55.7 3217478384
## 5 1972 57.6 3576977158
## 6 1977 59.6 3930045807
## 7 1982 61.5 4289436840
## 8 1987 63.2 4691477418
## 9 1992 64.2 5110710260
## 10 1997 65.0 5515204472
## 11 2002 65.7 5886977579
## 12 2007 67.0 6251013179
As you can see, we now get one row for each year. Rather than that, we have the average life expectancy and total population in 2007 within each continent. A little bit difficult, in case where we want to assess the information for both all the years and for all continents, we can add all the two variables in group_by().
gapminder %>%
group_by(year, continent) %>%
summarize(meanLifeExp = mean(lifeExp),
totalPop=sum(pop))## `summarise()` has grouped output by 'year'. You can override using the `.groups` argument.
## # A tibble: 60 x 4
## # Groups: year [12]
## year continent meanLifeExp totalPop
## <int> <fct> <dbl> <dbl>
## 1 1952 Africa 39.1 237640501
## 2 1952 Americas 53.3 345152446
## 3 1952 Asia 46.3 1395357351
## 4 1952 Europe 64.4 418120846
## 5 1952 Oceania 69.3 10686006
## 6 1957 Africa 41.3 264837738
## 7 1957 Americas 56.0 386953916
## 8 1957 Asia 49.3 1562780599
## 9 1957 Europe 66.7 437890351
## 10 1957 Oceania 70.3 11941976
## # ... with 50 more rows
Exercise 3.4. Running the code below and tell me what happens.
Remark that continent is put before year, what if this order will be changed?
gapminder$pop <- as.numeric(gapminder$pop)
gapminder %>%
group_by(continent, year) %>%
summarize(meanLifeExp = mean(lifeExp),
totalPop=sum(pop))## `summarise()` has grouped output by 'continent'. You can override using the `.groups` argument.
Exercise 3.5. Summarizing by year
In a previous exercise, you found the median life expectancy and the maximum GDP per capita in the year 1957. Now, you’ll perform those two summaries within each year in the dataset, using the group_by verb.
# Find median life expectancy and
# maximum GDP per capita in each year
gapminder %>%
group_by(year) %>%
summarize(medianLifeExp = median(lifeExp),
maxGdpPercap = max(gdpPercap))Exercise 3.6. Summarizing by continent
You can group by any variable in your dataset to create a summary. Rather than comparing across time, you might be interested in comparing among continents. You’ll want to do that within one year of the dataset: let’s use 1957.
# Find median life expectancy and
# maximum GDP per capita in each continent in 1957
gapminder %>%
filter(year == 1957) %>%
group_by(continent) %>%
summarize(medianLifeExp = median(lifeExp),
maxGdpPercap = max(gdpPercap))Exercise 3.7. Summarizing by continent and year
Instead of grouping just by year, or just by continent, you’ll now group by both continent and year to summarize within each.
# Find median life expectancy and
# maximum GDP per capita in each continent,
# per year combination
gapminder %>%
group_by(continent, year) %>%
summarize(medianLifeExp = median(lifeExp),
maxGdpPercap = max(gdpPercap))## `summarise()` has grouped output by 'continent'. You can override using the `.groups` argument.
After summarizing the data, we can graphically visualize this using ggplot2 package.
# extract the by_year data
by_year <- gapminder %>%
group_by(year) %>%
summarize(totalPop = sum(pop),
meanLifeExp=mean(lifeExp))
# scatter plot
ggplot2::ggplot(by_year, aes(x=year, y=totalPop)) +
geom_point() +
theme_bw()The resulting graph of population by year shows the change in the total population, which is going up over time. It starts under 3 times 10 to the 9th power (ie. 3 billions) and go up to more than 6 billions. This graph is a little misleading because it doesn’t include zero. In fact, we do not have a sense of how much the population grew relative to where it was when it started.
In order to add/start with zero point for the y-axis, we use a new graphical option as expand_limits(y=0) to the end of the script.
# summarize by year
by_year <- gapminder %>%
group_by(year) %>%
summarize(totalPop = sum(pop),
meanLifeExp=mean(lifeExp))
# plot
ggplot2::ggplot(by_year, aes(x=year, y=totalPop)) +
geom_point() +
expand_limits(y=0) +
theme_bw()By separated by continent and visualizing the data overtime, we can say that Asia is populated continent. And Africa has grown to surpass
# summarize by year and continent
by_year_continent <- gapminder %>%
group_by(year, continent) %>%
summarize(totalPop = sum(pop),
meanLifeExp=mean(lifeExp))## `summarise()` has grouped output by 'year'. You can override using the `.groups` argument.
# scatter plot with adding aesthetic as continent
ggplot(by_year_continent,
aes(x=year, y=totalPop, color = continent)) +
geom_point(alpha=0.7) +
expand_limits(y=0) +
theme_bw()You could often combine the verb supported by dplyr and ggplot2 as a part of data exploration phase. So it’s important to get into the habit of visualizing summarized or processed data.
Exercise 3.8. Visualizing median life expectancy over time
In the last chapter, you summarized the gapminder data to calculate the median life expectancy within each year. This code is provided for you, and is saved (with <-) as the by_year dataset. Now you can use the ggplot2 package to turn this into a visualization of changing life expectancy over time.
by_year <- gapminder %>%
group_by(year) %>%
summarize(medianLifeExp = median(lifeExp),
maxGdpPercap = max(gdpPercap))
# Create a scatter plot showing the change
# in medianLifeExp over time
ggplot2::ggplot(by_year, aes(x=year, y=medianLifeExp)) +
geom_point() +
expand_limits(y=0)Exercise 3.9 Visualizing median GDP per capita per continent over time
In the last exercise you were able to see how the median life expectancy of countries changed over time. Now you’ll examine the median GDP per capita instead, and see how the trend differs among continents.
# Summarize medianGdpPercap within each continent within each year:
by_year_continent <- gapminder %>%
group_by(year,continent) %>%
summarize(medianGdpPercap = median(gdpPercap))## `summarise()` has grouped output by 'year'. You can override using the `.groups` argument.
# Plot the change in medianGdpPercap in each continent over time
ggplot2::ggplot(by_year_continent,
aes(x=year, y=medianGdpPercap, color=continent)) +
geom_point() +
expand_limits(y=0)Exercise 3.10. Comparing median life expectancy and median GDP per continent in 2007
In these exercises you’ve generally created plots that show change over time. But as another way of exploring your data visually, you can also use ggplot2 to plot summarized data to compare continents within a single year.
# Summarize the median GDP and median life expectancy
# per continent in 2007
by_continent_2007 <- gapminder %>%
filter(year==2007) %>%
group_by(continent) %>%
summarize(medianLifeExp = median(lifeExp),
medianGdpPercap = median(gdpPercap))
# Use a scatter plot to compare the median GDP
# and median life expectancy
ggplot2::ggplot(by_continent_2007,
aes(x=medianGdpPercap, y=medianLifeExp, color=continent)) +
geom_point() +
expand_limits(y=0)A line plot is useful for visualizing trends over time. In this section, we’ll examine how the median GDP per capita has changed over time using the argument geom_line().
# Summarize the median gdpPercap by year, then save it as by_year
by_year <- gapminder %>%
group_by(year) %>%
summarize(medianGdpPercap = median(gdpPercap))
# Create a line plot showing the change in medianGdpPercap over time
ggplot2::ggplot(by_year,
aes(x=year, y=medianGdpPercap)) +
geom_line() +
expand_limits(y=0) +
theme_bw()Exercise 4.1. Visualizing median GDP per capita by continent over time
In the last exercise you used a line plot to visualize the increase in median GDP per capita over time. Now you’ll examine the change within each continent.
# Summarize the median gdpPercap by year & continent, save as by_year_continent
by_year_continent <- gapminder %>%
group_by(year,continent) %>%
summarize(medianGdpPercap = median(gdpPercap))## `summarise()` has grouped output by 'year'. You can override using the `.groups` argument.
# Create a line plot showing the change
# in medianGdpPercap by continent over time
ggplot2::ggplot(by_year_continent,
aes(x=year, y=medianGdpPercap, color=continent)) +
geom_line() +
expand_limits(y=0) +
theme_bw()Bar plots are useful for comparing values across categories. The by_continent data show the average lifeExp within each continent in the year 2007. Instead of just printing the table, we want to represent graphically. To do so, barplot could be used. In such graph, one bar represents a continent, whose height represent the mean of life expectancy.
To create a bar plot, we add a command geom_col where col short for column. Remarkly in aesthetic, x has to be a qualitative variable and y has to be a quantitative variable. Unlike scatter plot or line plot, bar plot always starts from 0.
# extract data
by_continent <- gapminder %>%
filter(year==2007) %>%
group_by(continent) %>%
summarize(meanLifeExp = mean(lifeExp))
# barplot
ggplot2::ggplot(by_continent,
aes(x=continent, y=meanLifeExp)) +
geom_col() +
theme_bw()Exercise 4.2. Visualizing median GDP per capita by continent
A bar plot is useful for visualizing summary statistics, such as the median GDP in each continent.
# summarize the median gdpPercap by continent in 1952
by_continent <- gapminder %>%
filter(year==1952) %>%
group_by(continent) %>%
summarize(medianGdpPercap = median(gdpPercap))
# create a bar plot showing medianGdp by continent
ggplot2::ggplot(by_continent,
aes(x=continent, y=medianGdpPercap)) +
geom_col() +
theme_bw()Exercise 4.3. Visualizing GDP per capita by country in Oceania
You’ve created a plot where each bar represents one continent, showing the median GDP per capita for each. But the x-axis of the bar plot doesn’t have to be the continent: you can instead create a bar plot where each bar represents a country. In this exercise, you’ll create a bar plot comparing the GDP per capita between the two countries in the Oceania continent (Australia and New Zealand).
# Filter for observations in the Oceania continent in 1952
oceania_1952 <- gapminder %>%
filter(continent=="Oceania", year==1952)
# Create a bar plot of gdpPercap by country
ggplot2::ggplot(oceania_1952, aes(x=country, y=gdpPercap)) +
geom_col() +
theme_bw()Each kind of graph offers a different way to investigate the data. Histogram allows to investigate one dimension (ie. one variable) of the data at a time. Histogram shows information about the distribution of that variable along with its shape. Below is an illustration of life expectancy across countries in the year 2007. Each bar represents a bin of life expectancies and the height represents frequency (how many countries) fall into that bin.
A histogram is created with argument geom_histogram(). Remarkly, histogram has only one aesthetic (ie. the x-axis) that is the variable we are assessing. As you can see in the graph below, most of countries have expectancies between 70 and 80. Another countries have expectancies between 40 and 65.
# extract data
gapminder_2007 <- gapminder %>%
filter(year==2007)
# plot histogram
ggplot2::ggplot(gapminder_2007, aes(x=lifeExp)) +
geom_histogram(binwidth = 5) +
theme_bw()The width of each bin in the histogram is chosen automatically, and it has a large effect on how the histogram communicates the distribution. You may need customize that width. You can do so with the binwidth option. Setting binwidth=5 means that each of the bars in the histogram represents a width of 5 years. In some cases, you need to put x-axis of a histogram on a log scale for a better understandable. To do so, you could use argument scale_x_log10() as mentioned before.
Exercise 4.4. Visualizing population
A histogram is useful for examining the distribution of a numeric variable. In this exercise, you’ll create a histogram showing the distribution of country populations (by millions) in the year 1952. Code for generating this dataset, gapminder_1952, is provided.
gapminder_1952 <- gapminder %>%
filter(year == 1952) %>%
mutate(pop_by_mil = pop / 1000000)
# Create a histogram of population (pop_by_mil)
ggplot2::ggplot(gapminder_1952, aes(x=pop_by_mil)) +
geom_histogram(bins=50) +
theme_bw()Exercise 4.5. Visualizing population with x-axis on a log scale
In the last exercise you created a histogram of populations across countries. You might have noticed that there were several countries with a much higher population than others, which causes the distribution to be very skewed, with most of the distribution crammed into a small part of the graph. (Consider that it’s hard to tell the median or the minimum population from that histogram). To make the histogram more informative, you can try putting the x-axis on a log scale.
gapminder_1952 <- gapminder %>%
filter(year == 1952)
# Create a histogram of population (pop), with x on a log scale
ggplot2::ggplot(gapminder_1952, aes(x=pop)) +
scale_x_log10() +
geom_histogram()## `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.
Histogram shows distribution of a (quantitative variable) across all continents. What if we want to compare among the continents in terms of lifeExp ? To do so, boxplot is a good solution. Boxplot is created by using argument geom_boxplot() with two information for aes: x-axis is a qualitative variable and y-axis is a quantitative variable.
ggplot2::ggplot(gapminder_2007, aes(x=continent, y=lifeExp)) +
geom_boxplot() +
theme_bw()Exercise 4.6. Comparing GDP per capita across continents
A boxplot is useful for comparing a distribution of values across several groups. In this exercise, you’ll examine the distribution of GDP per capita by continent. Since GDP per capita varies across several orders of magnitude, you’ll need to put the y-axis on a log scale.
gapminder_1952 <- gapminder %>%
filter(year == 1952)
# Create a boxplot comparing gdpPercap among continents
ggplot2::ggplot(gapminder_1952, aes(x=continent, y=gdpPercap)) +
geom_boxplot() +
scale_y_log10() +
theme_bw()Exercise 4.7. Adding a title to your graph
There are many other options for customizing a ggplot2 graph, which you can learn about in other DataCamp courses. You can also learn about them from online resources, which is an important skill to develop. As the final exercise in this course, you’ll practice looking up ggplot2 instructions by completing a task we haven’t shown you how to do.
# Add a title to this graph: "Comparing GDP per capita across continents"
ggplot(gapminder_1952, aes(x = continent, y = gdpPercap)) +
geom_boxplot() +
scale_y_log10() +
theme_bw() +
ggtitle("Comparing GDP per capita across continents")Next steps, enroll some courses:
ggplot2dplyrrbokeh