Introduction

The layered structure of ggplot2 encourages you to design and construct graphics in a structured manner. You’ve learned the basics in the previous chapter, and in this chapter you’ll get a more comprehensive task-based introduction. The goal here is not to exhaustively explore every option of every geom, but instead to show the most important tools for a given task. For more information about individual geoms, along with many more examples illustrating their use, see the documentation.

It is useful to think about the purpose of each layer before it is added. In general, there are three purposes for a layer:

  • To display the data. We plot the raw data for many reasons, relying on our skills at pattern detection to spot gross structure, local structure, and outliers. This layer appears on virtually every graphic. In the earliest stages of data exploration, it is often the only layer.

  • To display a statistical summary of the data. As we develop and explore models of the data, it is useful to display model predictions in the context of the data. Showing the data helps us improve the model, and showing the model helps reveal subtleties of the data that we might otherwise miss. Summaries are usually drawn on top of the data.

  • To add additional metadata: context, annotations, and references. A metadata layer displays background context, annotations that help to give meaning to the raw data, or fixed references that aid comparisons across panels. Metadata can be useful in the background and foreground.

    A map is often used as a background layer with spatial data. Background metadata should be rendered so that it doesn’t interfere with your perception of the data, so is usually displayed underneath the data and formatted so that it is minimally perceptible. That is, if you concentrate on it, you can see it with ease, but it doesn’t jump out at you when you are casually browsing the plot.

    Other metadata is used to highlight important features of the data. If you have added explanatory labels to a couple of inflection points or outliers, then you want to render them so that they pop out at the viewer. In that case, you want this to be the very last layer drawn.

This chapter is broken up into the following sections, each of which deals with a particular graphical challenge. This is not an exhaustive or exclusive categorisation, and there are many other possible ways to break up graphics into different categories. Each geom can be used for many different purposes, especially if you are creative. However, this breakdown should cover many common tasks and help you learn about some of the possibilities.

In diamonds, you’ll learn about the diamonds dataset. The final three sections use this data to discuss techniques for visualising larger datasets:

  • Displaying distributions, continuous and discrete, 1d and 2d, joint and conditional, link to section.

  • Dealing with overplotting in scatterplots, a challenge with large datasets,
    link to section.

  • Displaying statistical summaries instead of the raw data, link to section.

The chapter concludes in other packages with some pointers to other useful packages built on top of ggplot2.

Basic plot types

These geoms are the fundamental building blocks of ggplot2. They are useful in their own right, but are also used to construct more complex geoms. Most of these geoms are associated with a named plot: when that geom is used by itself in a plot, that plot has a special name.

Each of these geoms is two dimensional and requires both x and y aesthetics. All of them understand colour (or color) and size aesthetics, and the filled geoms (bar, tile and polygon) also understand fill.

  • geom_area() draws an area plot, which is a line plot filled to the y-axis (filled lines). Multiple groups will be stacked on top of each other.

  • geom_bar(stat = "identity") makes a bar chart. We need stat = "identity" because the default stat automatically counts values (so is essentially a 1d geom, see distributions. The identity stat leaves the data unchanged. Multiple bars in the same location will be stacked on top of one another.

  • geom_line() makes a line plot. The group aesthetic determines which observations are connected; see grouping for more detail. geom_line() connects points from left to right; geom_path() is similar but connects points in the order they appear in the data. Both geom_line() and geom_path() also understand the aesthetic linetype, which maps a categorical variable to solid, dotted and dashed lines.

  • geom_point() produces a scatterplot. geom_point() also understands the shape aesthetic.

  • geom_polygon() draws polygons, which are filled paths. Each vertex of the polygon requires a separate row in the data. It is often useful to merge a data frame of polygon coordinates with the data just prior to plotting. Drawing maps illustrates this concept in more detail for map data.

  • geom_rect(), geom_tile() and geom_raster() draw rectangles. geom_rect() is parameterised by the four corners of the rectangle, xmin, ymin, xmax and ymax. geom_tile() is exactly the same, but parameterised by the center of the rect and its size, x, y, width and height. geom_raster() is a fast special case of geom_tile() used when all the tiles are the same size. .

Each geom is shown in the code below. Observe the different axis ranges for the bar, area and tile plots: these geoms take up space outside the range of the data, and so push the axes out.

df <- data.frame(
  x = c(3, 1, 5), 
  y = c(2, 4, 6), 
  label = c("a","b","c")
)
p <- ggplot(df, aes(x, y, label = label)) + 
  labs(x = NULL, y = NULL) + # Hide axis label
  theme(plot.title = element_text(size = 12)) # Shrink plot title
p + geom_point() + ggtitle("point")
p + geom_text() + ggtitle("text")
p + geom_bar(stat = "identity") + ggtitle("bar")
p + geom_tile() + ggtitle("raster")

p + geom_line() + ggtitle("line")
p + geom_area() + ggtitle("area")
p + geom_path() + ggtitle("path")
p + geom_polygon() + ggtitle("polygon")

Exercises

  1. What geoms would you use to draw each of the following named plots?

    1. Scatterplot
    2. Line chart
    3. Histogram
    4. Bar chart
    5. Pie chart

  2. What’s the difference between geom_path() and geom_polygon()? What’s the difference between geom_path() and geom_line()?

  3. What low-level geoms are used to draw geom_smooth()? What about geom_boxplot() and geom_violin()?

Labels

Adding text to a plot can be quite tricky. ggplot2 doesn’t have all the answers, but does provide some tools to make your life a little easier. The main tool is geom_text(), which adds labels at the specified x and y positions.

geom_text() has the most aesthetics of any geom, because there are so many ways to control the appearance of a text:

  • family gives the name of a font. There are only three fonts that are guaranteed to work everywhere: “sans” (the default), “serif”, or “mono”:

    df <- data.frame(x = 1, y = 3:1, family = c("sans", "serif", "mono"))
ggplot(df, aes(x, y)) + 
  geom_text(aes(label = family, family = family))

    It’s trickier to include a system font on a plot because text drawing is done differently by each graphics device (GD). There are five GDs in common use (png(), pdf(), on screen devices for Windows, Mac and Linux), so to have a font work everywhere you need to configure five devices in five different ways. Two packages simplify the quandary a bit:

    Both approaches have pros and cons, so you will to need to try both of them and see which works best for your needs.

  • fontface specifies the face: “plain” (the default), “bold” or “italic”. 

    df <- data.frame(x = 1, y = 3:1, face = c("plain", "bold", "italic"))
ggplot(df, aes(x, y)) + 
  geom_text(aes(label = face, fontface = face))

  • You can adjust the alignment of the text with the hjust (“left”, “center”, “right”, “inward”, “outward”) and vjust (“bottom”, “middle”, “top”, “inward”, “outward”) aesthetics. The default alignment is centered. One of the most useful alignments is “inward”: it aligns text towards the middle of the plot: 

    df <- data.frame(
  x = c(1, 1, 2, 2, 1.5),
  y = c(1, 2, 1, 2, 1.5),
  text = c(
    "bottom-left", "bottom-right", 
    "top-left", "top-right", "center"
  )
)
ggplot(df, aes(x, y)) +
  geom_text(aes(label = text))
ggplot(df, aes(x, y)) +
  geom_text(aes(label = text), vjust = "inward", hjust = "inward")

  • size controls the font size. Unlike most tools, ggplot2 uses mm, rather than the usual points (pts). This makes it consistent with other size units in ggplot2. (There are 72.27 pts in a inch, so to convert from points to mm, just multiply by 72.27 / 25.4).

  • angle specifies the rotation of the text in degrees.

You can map data values to these aesthetics, but use restraint: it is hard to percieve the relationship between variables mapped to these aesthetics. geom_text() also has three parameters. Unlike the aesthetics, these only take single values, so they must be the same for all labels:

  • Often you want to label existing points on the plot. You don’t want the text to overlap with the points (or bars etc), so it’s useful to offset the text a little. The nudge_x and nudge_y parameters allow you to nudge the text a little horizontally or vertically:

    df <- data.frame(trt = c("a", "b", "c"), resp = c(1.2, 3.4, 2.5))
ggplot(df, aes(resp, trt)) + 
  geom_point() + 
  geom_text(aes(label = paste0("(", resp, ")")), nudge_y = -0.25) + 
  xlim(1, 3.6)

    (Note that I manually tweaked the x-axis limits to make sure all the text fit on the plot.)

  • If check_overlap = TRUE, overlapping labels will be automatically removed. The algorithm is simple: labels are plotted in the order they appear in the data frame; if a label would overlap with an existing point, it’s omitted. This is not incredibly useful, but can be handy. 

    ggplot(mpg, aes(displ, hwy)) + 
  geom_text(aes(label = model)) + 
  xlim(1, 8)
ggplot(mpg, aes(displ, hwy)) + 
  geom_text(aes(label = model), check_overlap = TRUE) + 
  xlim(1, 8)

A variation on geom_text() is geom_label(): it draws a rounded rectangle behind the text. This makes it useful for adding labels to plots with busy backgrounds: 

label <- data.frame(
  waiting = c(55, 80), 
  eruptions = c(2, 4.3), 
  label = c("peak one", "peak two")
)

ggplot(faithfuld, aes(waiting, eruptions)) +
  geom_tile(aes(fill = density)) + 
  geom_label(data = label, aes(label = label))

Labelling data well poses some challenges:

  • Text does not affect the limits of the plot. Unfortunately there’s no way to make this work since a label has an absolute size (e.g. 3 cm), regardless of the size of the plot. This means that the limits of a plot would need to be different depending on the size of the plot — there’s just no way to make that happen with ggplot2. Instead, you’ll need to tweak xlim() and ylim() based on your data and plot size.

  • If you want to label many points, it is difficult to avoid overlaps. check_overlap = TRUE is useful, but offers little control over which labels are removed. There are a number of techniques available for base graphics, like maptools::pointLabel(), but they’re not trivial to port to the grid graphics used by ggplot2. If all else fails, you may need to manually label points in a drawing tool.

Text labels can also serve as an alternative to a legend. This usually makes the plot easier to read because it puts the labels closer to the data. The directlabels package, by Toby Dylan Hocking, provides a number of tools to make this easier: 

ggplot(mpg, aes(displ, hwy, colour = class)) + 
  geom_point()

ggplot(mpg, aes(displ, hwy, colour = class)) + 
  geom_point(show.legend = FALSE) +
  directlabels::geom_dl(aes(label = class), method = "smart.grid")

Directlabels provides a number of position methods. smart.grid is a reasonable place to start for scatterplots, but there are other methods that are more useful for frequency polygons and line plots. See the directlabels website, http://directlabels.r-forge.r-project.org, for other techniques.

Annotations

Annotations add metadata to your plot. But metadata is just data, so you can use:

  • geom_text() to add text descriptions or to label points Most plots will not benefit from adding text to every single observation on the plot, but labelling outliers and other important points is very useful.

  • geom_rect() to highlight interesting rectangular regions of the plot. geom_rect() has aesthetics xmin, xmax, ymin and ymax.

  • geom_line(), geom_path() and geom_segment() to add lines. All these geoms have an arrow parameter, which allows you to place an arrowhead on the line. Create arrowheads with arrow(), which has arguments angle, length, ends and type.

  • geom_vline(), geom_hline() and geom_abline() allow you to add reference lines (sometimes called rules), that span the full range of the plot.

Typically, you can either put annotations in the foreground (using alpha if needed so you can still see the data), or in the background. With the default background, a thick white line makes a useful reference: it’s easy to see but it doesn’t jump out at you.

To show off the basic idea, we’ll draw a time series of unemployment:

ggplot(economics, aes(date, unemploy)) + 
  geom_line()

We can annotate this plot with which president was in power at the time. There is little new in this code - it’s a straightforward manipulation of existing geoms. There is one special thing to note: the use of -Inf and Inf as positions. These refer to the top and bottom (or left and right) limits of the plot. 

presidential <- subset(presidential, start > economics$date[1])

ggplot(economics) + 
  geom_rect(
    aes(xmin = start, xmax = end, fill = party), 
    ymin = -Inf, ymax = Inf, alpha = 0.2, 
    data = presidential
  ) + 
  geom_vline(
    aes(xintercept = as.numeric(start)), 
    data = presidential,
    colour = "grey50", alpha = 0.5
  ) + 
  geom_text(
    aes(x = start, y = 2500, label = name), 
    data = presidential, 
    size = 3, vjust = 0, hjust = 0, nudge_x = 50
  ) + 
  geom_line(aes(date, unemploy)) + 
  scale_fill_manual(values = c("blue", "red"))

You can use the same technique to add a single annotation to a plot, but it’s a bit fiddly because you have to create a one row data frame:

yrng <- range(economics$unemploy)
xrng <- range(economics$date)
caption <- paste(strwrap("Unemployment rates in the US have 
  varied a lot over the years", 40), collapse = "\n")

ggplot(economics, aes(date, unemploy)) + 
  geom_line() + 
  geom_text(
    aes(x, y, label = caption), 
    data = data.frame(x = xrng[1], y = yrng[2], caption = caption), 
    hjust = 0, vjust = 1, size = 4
  )

It’s easier to use the annotate() helper function which creates the data frame for you: 

ggplot(economics, aes(date, unemploy)) + 
  geom_line() + 
  annotate("text", x = xrng[1], y = yrng[2], label = caption,
    hjust = 0, vjust = 1, size = 4
  )

Annotations, particularly reference lines, are also useful when comparing groups across facets. In the following plot, it’s much easier to see the subtle differences if we add a reference line.

ggplot(diamonds, aes(log10(carat), log10(price))) + 
  geom_bin2d() + 
  facet_wrap(~cut, nrow = 1)


mod_coef <- coef(lm(log10(price) ~ log10(carat), data = diamonds))
ggplot(diamonds, aes(log10(carat), log10(price))) + 
  geom_bin2d() + 
  geom_abline(intercept = mod_coef[1], slope = mod_coef[2], 
    colour = "white", size = 1) + 
  facet_wrap(~cut, nrow = 1)

Collective geoms

Geoms can be roughly divided into individual and collective geoms. An individual geom draws a distinct graphical object for each observation (row). For example, the point geom draws one point per row. A collective geom displays multiple observations with one geometric object. This may be a result of a statistical summary, like a boxplot, or may be fundamental to the display of the geom, like a polygon. Lines and paths fall somewhere in between: each line is composed of a set of straight segments, but each segment represents two points. How do we control the assignment of observations to graphical elements? This is the job of the group aesthetic.

By default, the group aesthetic is mapped to the interaction of all discrete variables in the plot. This often partitions the data correctly, but when it does not, or when no discrete variable is used in a plot, you’ll need to explicitly define the grouping structure by mapping group to a variable that has a different value for each group.

There are three common cases where the default is not enough, and we will consider each one below. In the following examples, we will use a simple longitudinal dataset, Oxboys, from the nlme package. It records the heights (height) and centered ages (age) of 26 boys (Subject), measured on nine occasions (Occasion). Subject and Occassion are stored as ordered factors. 

data(Oxboys, package = "nlme")
head(Oxboys)
#>   Subject     age height Occasion
#> 1       1 -1.0000    140        1
#> 2       1 -0.7479    143        2
#> 3       1 -0.4630    145        3
#> 4       1 -0.1643    147        4
#> 5       1 -0.0027    148        5
#> 6       1  0.2466    150        6

Multiple groups, one aesthetic

In many situations, you want to separate your data into groups, but render them in the same way. In other words, you want to be able to distinguish individual subjects, but not identify them. This is common in longitudinal studies with many subjects, where the plots are often descriptively called spaghetti plots. For example, the following plot shows the growth trajectory for each boy (each Subject): 

ggplot(Oxboys, aes(age, height, group = Subject)) + 
  geom_point() + 
  geom_line()

If you incorrectly specify the grouping variable, you’ll get a characteristic sawtooth appearance:

ggplot(Oxboys, aes(age, height)) + 
  geom_point() + 
  geom_line()

If a group isn’t defined by a single variable, but instead by a combination of multiple variables, use interaction() to combine them, e.g. aes(group = interaction(school_id, student_id)).

Different groups on different layers

Sometimes we want to plot summaries that use different levels of aggregation: one layer might display individuals, while another displays an overall summary. Building on the previous example, suppose we want to add a single smooth line, showing the overall trend for all boys. If we use the same grouping in both layers, we get one smooth per boy: 

ggplot(Oxboys, aes(age, height, group = Subject)) + 
  geom_line() + 
  geom_smooth(method = "lm", se = FALSE)

This is not what we wanted; we have inadvertently added a smoothed line for each boy. Grouping controls both the display of the geoms, and the operation of the stats: one statistical transformation is run for each group.

Instead of setting the grouping aesthetic in ggplot(), where it will apply to all layers, we set it in geom_line() so it applies only to the lines. There are no discrete variables in the plot so the default grouping variable will be a constant and we get one smooth:

ggplot(Oxboys, aes(age, height)) + 
  geom_line(aes(group = Subject)) + 
  geom_smooth(method = "lm", size = 2, se = FALSE)

Overriding the default grouping

Some plots have a discrete x scale, but you still want to draw lines connecting across groups. This is the strategy used in interaction plots, profile plots, and parallel coordinate plots, among others. For example, imagine we’ve drawn boxplots of height at each measurement occasion: 

ggplot(Oxboys, aes(Occasion, height)) + 
  geom_boxplot()

There is one discrete variable in this plot, Occassion, so we get one boxplot for each unique x value. Now we want to overlay lines that connect each individual boy. Simply adding geom_line() does not work: the lines are drawn within each occassion, not across each subject:

ggplot(Oxboys, aes(Occasion, height)) + 
  geom_boxplot() +
  geom_line(colour = "#3366FF", alpha = 0.5)

To get the plot we want, we need to override the grouping to say we want one line per boy:

ggplot(Oxboys, aes(Occasion, height)) + 
  geom_boxplot() +
  geom_line(aes(group = Subject), colour = "#3366FF", alpha = 0.5)

Matching aesthetics to graphic objects

A final important issue with collective geoms is how the aesthetics of the individual observations are mapped to the aesthetics of the complete entity. What happens when different aesthetics are mapped to a single geometric element?

Lines and paths operate on an off-by-one principle: there is one more observation than line segment, and so the aesthetic for the first observation is used for the first segment, the second observation for the second segment and so on. This means that the aesthetic for the last observation is not used:

df <- data.frame(x = 1:3, y = 1:3, colour = c(1,3,5))

ggplot(df, aes(x, y, colour = factor(colour))) + 
  geom_line(aes(group = 1), size = 2) +
  geom_point(size = 5)

ggplot(df, aes(x, y, colour = colour)) + 
  geom_line(aes(group = 1), size = 2) +
  geom_point(size = 5)

You could imagine a more complicated system where segments smoothly blend from one aesthetic to another. This would work for continuous variables like size or colour, but not for discrete variables, and is not used in ggplot2. If this is the behaviour you want, you can perform the linear interpolation yourself:

xgrid <- with(df, seq(min(x), max(x), length = 50))
interp <- data.frame(
  x = xgrid,
  y = approx(df$x, df$y, xout = xgrid)$y,
  colour = approx(df$x, df$colour, xout = xgrid)$y  
)
ggplot(interp, aes(x, y, colour = colour)) + 
  geom_line(size = 2) +
  geom_point(data = df, size = 5)

An additional limitation for paths and lines is that line type must be constant over each individual line. In R there is no way to draw a line which has varying line type.

For all other collective geoms, like polygons, the aesthetics from the individual components are only used if they are all the same, otherwise the default value is used. It’s particularly clear why this makes sense for fill: how would you colour a polygon that had a different fill colour for each point on its border?

These issues are most relevant when mapping aesthetics to continuous variables, because, as described above, when you introduce a mapping to a discrete variable, it will by default split apart collective geoms into smaller pieces. This works particularly well for bar and area plots, because stacking the individual pieces produces the same shape as the original ungrouped data:


ggplot(mpg, aes(class)) + 
  geom_bar()
ggplot(mpg, aes(class, fill = drv)) + 
  geom_bar()

If you try to map fill to a continuous variable in the same way, it doesn’t work. The default grouping will only be based on class, so each bar will be given multiple colours. Since a bar can only display one colour, it will use the default grey. To show multiple colours, we need multiple bars for each class, which we can get by overriding the grouping:

ggplot(mpg, aes(class, fill = hwy)) + 
  geom_bar()
ggplot(mpg, aes(class, fill = hwy, group = hwy)) + 
  geom_bar()

The bars will be stacked in the order defined by the grouping variable. If you need fine control, you’ll need to create a factor with levels ordered as needed.

Exercises

  1. Draw a boxplot of hwy for each value of cyl, without turning cyl into a factor. What extra aesthetic do you need to set?

  2. Modify the following plot so that you get one boxplot per integer value value of displ.

    ggplot(mpg, aes(displ, cty)) + 
  geom_boxplot()

  3. When illustrating the difference between mapping continuous and discrete colours to a line, the discrete example needed aes(group = 1). Why? What happens if that is omitted? What’s the difference between aes(group = 1) and aes(group = 2)? Why?

  4. How many bars are in each of the following plots?

    ggplot(mpg, aes(drv)) + 
  geom_bar()

ggplot(mpg, aes(drv, fill = hwy, group = hwy)) + 
  geom_bar()

library(dplyr)  
mpg2 <- mpg %>% arrange(hwy) %>% mutate(id = seq_along(hwy)) 
ggplot(mpg2, aes(drv, fill = hwy, group = id)) + 
  geom_bar()

    (Hint: try adding an outline around each bar with colour = "white")

  5. Install the babynames package. It contains data about the popularity of babynames in the US. Run the following code and fix the resulting graph. Why does this graph make me unhappy?

    library(babynames)
hadley <- dplyr::filter(babynames, name == "Hadley")
ggplot(hadley, aes(year, n)) + 
  geom_line()

Surface plots

ggplot2 does not support true 3d surfaces. However, it does support many common tools for representing 3d surfaces in 2d: contours, coloured tiles and bubble plots. These all work similarly, differing only in the aesthetic used for the third dimension. 

ggplot(faithfuld, aes(eruptions, waiting)) + 
  geom_contour(aes(z = density, colour = ..level..))

ggplot(faithfuld, aes(eruptions, waiting)) + 
  geom_raster(aes(fill = density))

# Bubble plots work better with fewer observations
small <- faithfuld[seq(1, nrow(faithfuld), by = 10), ]
ggplot(small, aes(eruptions, waiting)) + 
  geom_point(aes(size = density), alpha = 1/3) + 
  scale_size_area()

For interactive 3d plots, including true 3d surfaces, see RGL, http://rgl.neoscientists.org/about.shtml.

Drawing maps

There are four types of map data you might want to visualise: vector boundaries, point metadata, area metadata, and raster images. Typically, assembling these datasets is the most challenging part of drawing maps. Unfortunately ggplot2 can’t help you with that part of the analysis, but I’ll provide some hints about other R packages that you might want to look at.

I’ll illustrate each of the four types of map data with some maps of Michigan.

Vector boundaries

Vector boundaries are defined by a data frame with one row for each “corner” of a geographical region like a country, state, or county. It requires four variables:

  • lat and long, giving the location of a point.
  • group, a unique identifier for each contiguous region.
  • id, the name of the region.

Separate group and id variables are necessary because sometimes a geographical unit isn’t a contiguous polygon. For example, Hawaii is composed of multiple islands that can’t be drawn using a single polygon.

The following code extracts that data from the built in maps package using ggplot2::map_data(). The maps package isn’t particularly accurate or up-to-date, but it’s built into R so it’s a reasonable place to start. 

mi_counties <- map_data("county", "michigan") %>% 
  select(lon = long, lat, group, id = subregion)
head(mi_counties)
#>     lon  lat group     id
#> 1 -83.9 44.9     1 alcona
#> 2 -83.4 44.9     1 alcona
#> 3 -83.4 44.9     1 alcona
#> 4 -83.3 44.8     1 alcona
#> 5 -83.3 44.8     1 alcona
#> 6 -83.3 44.8     1 alcona

You can visualise vector boundary data with geom_polygon(): 

ggplot(mi_counties, aes(lon, lat)) +
  geom_polygon(aes(group = group)) + 
  coord_quickmap()

ggplot(mi_counties, aes(lon, lat)) +
  geom_polygon(aes(group = group), fill = NA, colour = "grey50") + 
  coord_quickmap()

Note the use of coord_quickmap(): it’s a quick and dirty adjustment that ensures that the aspect ratio of the plot is set correctly.

Other useful sources of vector boundary data are:

  • The USAboundaries package, https://github.com/ropensci/USAboundaries which contains state, county and zip code data for the US. As well as current boundaries, it also has state and county boundaries going back to the 1600s.

  • The tigris package, https://github.com/walkerke/tigris, makes it easy to access the US Census TIGRIS shapefiles. It contains state, county, zipcode, and census tract boundaries, as well as many other useful datasets.

  • The rnaturalearth package bundles up the free, high-quality data from http://naturalearthdata.com/. It contains country borders, and borders for the top-level region within each country (e.g. states in the USA, regions in France, counties in the UK).

  • The osmar package, https://cran.r-project.org/package=osmar wraps up the OpenStreetMap API so you can access a wide range of vector data including indvidual streets and buildings

  • You may have your own shape files (.shp). You can load them into R with maptools::readShapeSpatial().

These sources all generate spatial data frames defined by the sp package. You can convert them into a data frame with fortify():

library(USAboundaries)
c18 <- us_boundaries(as.Date("1820-01-01"))
c18df <- fortify(c18)
#> Regions defined for each Polygons
head(c18df)
#>    long lat order  hole piece id group
#> 1 -87.6  35     1 FALSE     1  4   4.1
#> 2 -87.6  35     2 FALSE     1  4   4.1
#> 3 -87.6  35     3 FALSE     1  4   4.1
#> 4 -87.6  35     4 FALSE     1  4   4.1
#> 5 -87.5  35     5 FALSE     1  4   4.1
#> 6 -87.3  35     6 FALSE     1  4   4.1

ggplot(c18df, aes(long, lat)) + 
  geom_polygon(aes(group = group), colour = "grey50", fill = NA) +
  coord_quickmap()

Point metadata

Point metadata connects locations (defined by lat and lon) with other variables. For example, the code below extracts the biggest cities in MI (as of 2006):

mi_cities <- maps::us.cities %>% 
  tbl_df() %>%
  filter(country.etc == "MI") %>%
  select(-country.etc, lon = long) %>%
  arrange(desc(pop))
mi_cities
#> # A tibble: 36 × 5
#>                  name    pop   lat   lon capital
#>                 <chr>  <int> <dbl> <dbl>   <int>
#> 1          Detroit MI 871789  42.4 -83.1       0
#> 2     Grand Rapids MI 193006  43.0 -85.7       0
#> 3           Warren MI 132537  42.5 -83.0       0
#> 4 Sterling Heights MI 127027  42.6 -83.0       0
#> 5          Lansing MI 117236  42.7 -84.5       2
#> 6            Flint MI 115691  43.0 -83.7       0
#> # ... with 30 more rows

We could show this data with a scatterplot, but it’s not terribly useful without a reference. You almost always combine point metadata with another layer to make it interpretable.

ggplot(mi_cities, aes(lon, lat)) + 
  geom_point(aes(size = pop)) + 
  scale_size_area() + 
  coord_quickmap()

ggplot(mi_cities, aes(lon, lat)) + 
  geom_polygon(aes(group = group), mi_counties, fill = NA, colour = "grey50") +
  geom_point(aes(size = pop), colour = "red") + 
  scale_size_area() + 
  coord_quickmap()

Raster images

Instead of displaying context with vector boundaries, you might want to draw a traditional map underneath. This is called a raster image. The easiest way to get a raster map of a given area is to use the ggmap package, which allows you to get data from a variety of online mapping sources including OpenStreetMap and Google Maps. Downloading the raster data is often time consuming so it’s a good idea to cache it in a rds file. 

# if (file.exists("mi_raster.rds")) {
#   mi_raster <- readRDS("mi_raster.rds")
# } else {
#   bbox <- c(
#     min(mi_counties$lon), min(mi_counties$lat), 
#     max(mi_counties$lon), max(mi_counties$lat)
#   )
#   mi_raster <- ggmap::get_openstreetmap(bbox, scale = 8735660)
#   saveRDS(mi_raster, "mi_raster.rds")
# }

(Finding the appropriate scale required a lot of manual tweaking.)

You can then plot it with: 

ggmap::ggmap(mi_raster)

ggmap::ggmap(mi_raster) + 
  geom_point(aes(size = pop), mi_cities, colour = "red") + 
  scale_size_area()

If you have raster data from the raster package, you can convert it to the form needed by ggplot2 with the following code:

df <- as.data.frame(raster::rasterToPoints(x))
names(df) <- c("lon", "lat", "x")

ggplot(df, aes(lon, lat)) + 
  geom_raster(aes(fill = x))

Area metadata

Sometimes metadata is associated not with a point, but with an area. For example, we can create mi_census which provides census information about each county in MI:

mi_census <- midwest %>%
  tbl_df() %>%
  filter(state == "MI") %>% 
  mutate(county = tolower(county)) %>%
  select(county, area, poptotal, percwhite, percblack)
mi_census
#> # A tibble: 83 × 5
#>    county  area poptotal percwhite percblack
#>     <chr> <dbl>    <int>     <dbl>     <dbl>
#> 1  alcona 0.041    10145      98.8     0.266
#> 2   alger 0.051     8972      93.9     2.374
#> 3 allegan 0.049    90509      95.9     1.600
#> 4  alpena 0.034    30605      99.2     0.114
#> 5  antrim 0.031    18185      98.4     0.126
#> 6  arenac 0.021    14931      98.4     0.067
#> # ... with 77 more rows

We can’t map this data directly because it has no spatial component. Instead, we must first join it to the vector boundaries data. This is not particularly space efficient, but it makes it easy to see exactly what data is being plotted. Here I use dplyr::left_join() to combine the two datasets and create a choropleth map. 

census_counties <- left_join(mi_census, mi_counties, by = c("county" = "id"))
census_counties
#> # A tibble: 1,472 × 8
#>   county  area poptotal percwhite percblack   lon   lat group
#>    <chr> <dbl>    <int>     <dbl>     <dbl> <dbl> <dbl> <dbl>
#> 1 alcona 0.041    10145      98.8     0.266 -83.9  44.9     1
#> 2 alcona 0.041    10145      98.8     0.266 -83.4  44.9     1
#> 3 alcona 0.041    10145      98.8     0.266 -83.4  44.9     1
#> 4 alcona 0.041    10145      98.8     0.266 -83.3  44.8     1
#> 5 alcona 0.041    10145      98.8     0.266 -83.3  44.8     1
#> 6 alcona 0.041    10145      98.8     0.266 -83.3  44.8     1
#> # ... with 1,466 more rows

ggplot(census_counties, aes(lon, lat, group = county)) + 
  geom_polygon(aes(fill = poptotal)) + 
  coord_quickmap()

ggplot(census_counties, aes(lon, lat, group = county)) + 
  geom_polygon(aes(fill = percwhite)) + 
  coord_quickmap()

Revealing uncertainty

If you have information about the uncertainty present in your data, whether it be from a model or from distributional assumptions, it’s a good idea to display it. There are four basic families of geoms that can be used for this job, depending on whether the x values are discrete or continuous, and whether or not you want to display the middle of the interval, or just the extent:

  • Discrete x, range: geom_errorbar(), geom_linerange()
  • Discrete x, range & center: geom_crossbar(), geom_pointrange()
  • Continuous x, range: geom_ribbon()
  • Continuous x, range & center: geom_smooth(stat = "identity")

These geoms assume that you are interested in the distribution of y conditional on x and use the aesthetics ymin and ymax to determine the range of the y values. If you want the opposite, see coord_flip. 

y <- c(18, 11, 16)
df <- data.frame(x = 1:3, y = y, se = c(1.2, 0.5, 1.0))

base <- ggplot(df, aes(x, y, ymin = y - se, ymax = y + se))
base + geom_crossbar()
base + geom_pointrange()
base + geom_smooth(stat = "identity")

base + geom_errorbar()
base + geom_linerange()
base + geom_ribbon()

Because there are so many different ways to calculate standard errors, the calculation is up to you. For very simple cases, ggplot2 provides some tools in the form of summary functions described below, otherwise you will have to do it yourself. The modelling chapter contains more advice on extracting confidence intervals from more sophisticated models.

Weighted data

When you have aggregated data where each row in the dataset represents multiple observations, you need some way to take into account the weighting variable. We will use some data collected on Midwest states in the 2000 US census in the built-in midwest data frame. The data consists mainly of percentages (e.g., percent white, percent below poverty line, percent with college degree) and some information for each county (area, total population, population density).

There are a few different things we might want to weight by:

  • Nothing, to look at numbers of counties.
  • Total population, to work with absolute numbers.
  • Area, to investigate geographic effects. (This isn’t useful for midwest, but would be if we had variables like percentage of farmland.)

The choice of a weighting variable profoundly affects what we are looking at in the plot and the conclusions that we will draw. There are two aesthetic attributes that can be used to adjust for weights. Firstly, for simple geoms like lines and points, use the size aesthetic:

# Unweighted
ggplot(midwest, aes(percwhite, percbelowpoverty)) + 
  geom_point()

# Weight by population
ggplot(midwest, aes(percwhite, percbelowpoverty)) + 
  geom_point(aes(size = poptotal / 1e6)) + 
  scale_size_area("Population\n(millions)", breaks = c(0.5, 1, 2, 4))

For more complicated grobs which involve some statistical transformation, we specify weights with the weight aesthetic. These weights will be passed on to the statistical summary function. Weights are supported for every case where it makes sense: smoothers, quantile regressions, boxplots, histograms, and density plots. You can’t see this weighting variable directly, and it doesn’t produce a legend, but it will change the results of the statistical summary. The following code shows how weighting by population density affects the relationship between percent white and percent below the poverty line.

# Unweighted
ggplot(midwest, aes(percwhite, percbelowpoverty)) + 
  geom_point() + 
  geom_smooth(method = lm, size = 1)

# Weighted by population
ggplot(midwest, aes(percwhite, percbelowpoverty)) + 
  geom_point(aes(size = poptotal / 1e6)) + 
  geom_smooth(aes(weight = poptotal), method = lm, size = 1) +
  scale_size_area(guide = "none")

When we weight a histogram or density plot by total population, we change from looking at the distribution of the number of counties, to the distribution of the number of people. The following code shows the difference this makes for a histogram of the percentage below the poverty line: 

ggplot(midwest, aes(percbelowpoverty)) +
  geom_histogram(binwidth = 1) + 
  ylab("Counties")

ggplot(midwest, aes(percbelowpoverty)) +
  geom_histogram(aes(weight = poptotal), binwidth = 1) +
  ylab("Population (1000s)")

Diamonds data

To demonstrate tools for large datasets, we’ll use the built in diamonds dataset, which consists of price and quality information for ~54,000 diamonds:

diamonds
#> # A tibble: 53,940 × 10
#>   carat       cut color clarity depth table price     x     y     z
#>   <dbl>     <ord> <ord>   <ord> <dbl> <dbl> <int> <dbl> <dbl> <dbl>
#> 1  0.23     Ideal     E     SI2  61.5    55   326  3.95  3.98  2.43
#> 2  0.21   Premium     E     SI1  59.8    61   326  3.89  3.84  2.31
#> 3  0.23      Good     E     VS1  56.9    65   327  4.05  4.07  2.31
#> 4  0.29   Premium     I     VS2  62.4    58   334  4.20  4.23  2.63
#> 5  0.31      Good     J     SI2  63.3    58   335  4.34  4.35  2.75
#> 6  0.24 Very Good     J    VVS2  62.8    57   336  3.94  3.96  2.48
#> # ... with 5.393e+04 more rows

The data contains the four C’s of diamond quality: carat, cut, colour and clarity; and five physical measurements: depth, table, x, y and z, as described in Figure .

\begin{figure}[htbp] \centering \includegraphics[width=0.8\linewidth]{diagrams/diamond-dimensions} \caption{How the variables x, y, z, table and depth are measured.} \label{fig:diamond-dim} \end{figure}

The dataset has not been well cleaned, so as well as demonstrating interesting facts about diamonds, it also shows some data quality problems.

Displaying distributions

There are a number of geoms that can be used to display distributions, depending on the dimensionality of the distribution, whether it is continuous or discrete, and whether you are interested in the conditional or joint distribution.

For 1d continuous distributions the most important geom is the histogram, geom_histogram(): 

ggplot(diamonds, aes(depth)) + 
  geom_histogram()
#> `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.
ggplot(diamonds, aes(depth)) + 
  geom_histogram(binwidth = 0.1) + 
  xlim(55, 70)
#> Warning: Removed 45 rows containing non-finite values (stat_bin).

It is important to experiment with binning to find a revealing view. You can change the binwidth, specify the number of bins, or specify the exact location of the breaks. Never rely on the default parameters to get a revealing view of the distribution. Zooming in on the x axis, xlim(55, 70), and selecting a smaller bin width, binwidth = 0.1, reveals far more detail.

When publishing figures, don’t forget to include information about important parameters (like bin width) in the caption.

If you want to compare the distribution between groups, you have a few options:

  • Show small multiples of the histogram, facet_wrap(~ var).
  • Use colour and a frequency polygon, geom_freqpoly() .
  • Use a “conditional density plot”, geom_histogram(position = "fill").

The frequency polygon and conditional density plots are shown below. The conditional density plot uses position_fill() to stack each bin, scaling it to the same height. This plot is perceptually challenging because you need to compare bar heights, not positions, but you can see the strongest patterns. 

ggplot(diamonds, aes(depth)) + 
  geom_freqpoly(aes(colour = cut), binwidth = 0.1, na.rm = TRUE) +
  xlim(58, 68) + 
  theme(legend.position = "none")
ggplot(diamonds, aes(depth)) + 
  geom_histogram(aes(fill = cut), binwidth = 0.1, position = "fill",
    na.rm = TRUE) +
  xlim(58, 68) + 
  theme(legend.position = "none")

(I’ve suppressed the legends to focus on the display of the data.)

Both the histogram and frequency polygon geom use the same underlying statistical transformation: stat = "bin". This statistic produces two output variables: count and density. By default, count is mapped to y-position, because it’s most interpretable. The density is the count divided by the total count multiplied by the bin width, and is useful when you want to compare the shape of the distributions, not the overall size.

An alternative to a bin-based visualisation is a density estimate. geom_density() places a little normal distribution at each data point and sums up all the curves. It has desirable theoretical properties, but is more difficult to relate back to the data. Use a density plot when you know that the underlying density is smooth, continuous and unbounded. You can use the adjust parameter to make the density more or less smooth. 

ggplot(diamonds, aes(depth)) +
  geom_density(na.rm = TRUE) + 
  xlim(58, 68) + 
  theme(legend.position = "none")
ggplot(diamonds, aes(depth, fill = cut, colour = cut)) +
  geom_density(alpha = 0.2, na.rm = TRUE) + 
  xlim(58, 68) + 
  theme(legend.position = "none")

Note that the area of each density estimate is standardised to one so that you lose information about the relative size of each group.

The histogram, frequency polygon and density display a detailed view of the distribution. However, sometimes you want to compare many distributions, and it’s useful to have alternative options that sacrifice quality for quantity. Here are three options:

  • geom_boxplot(): the box-and-whisker plot shows five summary statistics along with individual “outliers”. It displays far less information than a histogram, but also takes up much less space.

    You can use boxplot with both categorical and continuous x. For continuous x, you’ll also need to set the group aesthetic to define how the x variable is broken up into bins. A useful helper function is cut_width(): 

    ggplot(diamonds, aes(clarity, depth)) + 
  geom_boxplot()
ggplot(diamonds, aes(carat, depth)) + 
  geom_boxplot(aes(group = cut_width(carat, 0.1))) + 
  xlim(NA, 2.05)
#> Warning: Removed 997 rows containing non-finite values (stat_boxplot).

  • geom_violin(): the violin plot is a compact version of the density plot. The underlying computation is the same, but the results are displayed in a similar fashion to the boxplot: 

    ggplot(diamonds, aes(clarity, depth)) + 
  geom_violin()
ggplot(diamonds, aes(carat, depth)) + 
  geom_violin(aes(group = cut_width(carat, 0.1))) + 
  xlim(NA, 2.05)
#> Warning: Removed 997 rows containing non-finite values (stat_ydensity).

  • geom_dotplot(): draws one point for each observation, carefully adjusted in space to avoid overlaps and show the distribution. It is useful for smaller datasets.

Exercises

  1. What binwidth tells you the most interesting story about the distribution of carat?

  2. Draw a histogram of price. What interesting patterns do you see?

  3. How does the distribution of price vary with clarity?

  4. Overlay a frequency polygon and density plot of depth. What computed variable do you need to map to y to make the two plots comparable? (You can either modify geom_freqpoly() or geom_density().)

Dealing with overplotting

The scatterplot is a very important tool for assessing the relationship between two continuous variables. However, when the data is large, points will be often plotted on top of each other, obscuring the true relationship. In extreme cases, you will only be able to see the extent of the data, and any conclusions drawn from the graphic will be suspect. This problem is called overplotting.

There are a number of ways to deal with it depending on the size of the data and severity of the overplotting. The first set of techniques involves tweaking aesthetic properties. These tend to be most effective for smaller datasets:

  • Very small amounts of overplotting can sometimes be alleviated by making the points smaller, or using hollow glyphs. The following code shows some options for 2000 points sampled from a bivariate normal distribution. 

    df <- data.frame(x = rnorm(2000), y = rnorm(2000))
norm <- ggplot(df, aes(x, y)) + xlab(NULL) + ylab(NULL)
norm + geom_point()
norm + geom_point(shape = 1) # Hollow circles
norm + geom_point(shape = ".") # Pixel sized

  • For larger datasets with more overplotting, you can use alpha blending (transparency) to make the points transparent. If you specify alpha as a ratio, the denominator gives the number of points that must be overplotted to give a solid colour. Values smaller than ~\(1/500\) are rounded down to zero, giving completely transparent points. 

    norm + geom_point(alpha = 1 / 3)
norm + geom_point(alpha = 1 / 5)
norm + geom_point(alpha = 1 / 10)

  • If there is some discreteness in the data, you can randomly jitter the points to alleviate some overlaps with geom_jitter(). This can be particularly useful in conjunction with transparency. By default, the amount of jitter added is 40% of the resolution of the data, which leaves a small gap between adjacent regions. You can override the default with width and height arguments.

Alternatively, we can think of overplotting as a 2d density estimation problem, which gives rise to two more approaches:

  • Bin the points and count the number in each bin, then visualise that count (the 2d generalisation of the histogram), geom_bin2d(). Breaking the plot into many small squares can produce distracting visual artefacts. (D. B. Carr et al. 1987) suggests using hexagons instead, and this is implemented in geom_hex(), using the hexbin package (D. Carr, Lewin-Koh, and Mächler 2014).

    The code below compares square and hexagonal bins, using parameters bins and binwidth to control the number and size of the bins. 

    norm + geom_bin2d()
norm + geom_bin2d(bins = 10)

    norm + geom_hex()
norm + geom_hex(bins = 10)

  • Estimate the 2d density with stat_density2d(), and then display using one of the techniques for showing 3d surfaces in surfaces.

  • If you are interested in the conditional distribution of y given x, then the techniques of displaying distributions will also be useful.

Another approach to dealing with overplotting is to add data summaries to help guide the eye to the true shape of the pattern within the data. For example, you could add a smooth line showing the centre of the data with geom_smooth() or use one of the summaries below.

Statistical summaries

geom_histogram() and geom_bin2d() use a familiar geom, geom_bar() and geom_raster(), combined with a new statistical transformation, stat_bin() and stat_bin2d(). stat_bin() and stat_bin2d() combine the data into bins and count the number of observations in each bin. But what if we want a summary other than count? So far, we’ve just used the default statistical transformation associated with each geom. Now we’re going to explore how to use stat_summary_bin() to stat_summary_2d() to compute different summaries.

Let’s start with a couple of examples with the diamonds data. The first example in each pair shows how we can count the number of diamonds in each bin; the second shows how we can compute the average price.

ggplot(diamonds, aes(color)) + 
  geom_bar()

ggplot(diamonds, aes(color, price)) + 
  geom_bar(stat = "summary_bin", fun.y = mean)

ggplot(diamonds, aes(table, depth)) + 
  geom_bin2d(binwidth = 1, na.rm = TRUE) + 
  xlim(50, 70) + 
  ylim(50, 70)

ggplot(diamonds, aes(table, depth, z = price)) + 
  geom_raster(binwidth = 1, stat = "summary_2d", fun = mean, 
    na.rm = TRUE) + 
  xlim(50, 70) + 
  ylim(50, 70)

To get more help on the arguments associated with the two transformations, look at the help for stat_summary_bin() and stat_summary_2d(). You can control the size of the bins and the summary functions. stat_summary_bin() can produce y, ymin and ymax aesthetics, also making it useful for displaying measures of spread. See the docs for more details. You’ll learn more about how geoms and stats interact in stats.

These summary functions are quite constrained but are often useful for a quick first pass at a problem. If you find them restraining, you’ll need to do the summaries yourself. See group-wise summaries for more details.

Add-on packages

If the built-in tools in ggplot2 don’t do what you need, you might want to use a special purpose tool built into one of the packages built on top of ggplot2. Some of the packages that I was familiar with when the book was published include:

A great place to track new extensions is http://www.ggplot2-exts.org, by Daniel Emaasit.

References

Carr, D. B., R. J. Littlefield, W. L. Nicholson, and J. S. Littlefield. 1987. “Scatterplot Matrix Techniques for Large N.” Journal of the American Statistical Association 82 (398): 424–36.

Carr, Dan, Nicholas Lewin-Koh, and Martin Mächler. 2014. Hexbin: Hexagonal Binning Routines.