Introduction
Writing functions with tidy eval
Writing functions is essential for the clarity and robustness of your code. Functions have several advantages:
They prevents inconsistencies because it enforces multiple computations to follow a single recipe.
They emphasise what varies (the arguments) and what is constant (every other component of the computation).
They make change easier because you only need to modify one place.
They make your code clearer if you give the function and its arguments informative names.
The process for creating a function is straightforward. First recognise duplication in your code. A good rule of thumb is to create a function when you have copy-pasted a piece of code three times. Can you spot the copy-paste mistake in this code duplication?
(df$a - min(df$a)) / (max(df$a) - min(df$a))
(df$b - min(df$b)) / (max(df$b) - min(df$b))
(df$c - min(df$c)) / (max(df$c) - min(df$c))
(df$d - min(df$d)) / (max(df$d) - min(df$c))Now identify the variying parts of the expression and give them a name. x is an easy choice but it is often a good idea to reflect the type of argument expected in the name. In our case we expect a numeric vector:
(num - min(num)) / (max(num) - min(num))
(num - min(num)) / (max(num) - min(num))
(num - min(num)) / (max(num) - min(num))
(num - min(num)) / (max(num) - min(num))We can now create a function with a relevant name:
rescale01 <- function(num) {
}Fill it with our deduplicated code:
rescale01 <- function(num) {
(num - min(num)) / (max(num) - min(num))
}And refactor a little to reduce duplication further and handle more cases:
rescale01 <- function(num) {
rng <- range(num, na.rm = TRUE, finite = TRUE)
(num - rng[[1]]) / (rng[[2]] - rng[[1]])
}Now you can reuse your function any place you need it:
rescale01(df$a)
rescale01(df$b)
rescale01(df$c)
rescale01(df$d)Reducing code duplication is as much needed with tidyverse grammars as with ordinary computations. Unfortunately, the straighforward process to create functions breaks down with grammars like dplyr. To see why, let’s try to reproduce the process with a duplicated pipeline:
df1 %>% group_by(x1) %>% summarise(mean = mean(y1))
df2 %>% group_by(x2) %>% summarise(mean = mean(y2))
df3 %>% group_by(x3) %>% summarise(mean = mean(y3))
df4 %>% group_by(x4) %>% summarise(mean = mean(y4))We first abtract out the varying parts by giving them informative names:
data %>% group_by(group_var) %>% summarise(mean = mean(summary_var))And wrap the pipeline with a function taking these argument names:
grouped_mean <- function(data, group_var, summary_var) {
data %>%
group_by(group_var) %>%
summarise(mean = mean(summary_var))
}This function looks good, but unfortunately doesn’t actually work:
grouped_mean(mtcars, cyl, mpg)
#> Error in grouped_df_impl(data, unname(vars), drop): Column `group_var` is unknownInstead of grouping the data frame by the variable cyl, dplyr complains that the variable group_var is unknown. This may seem surprising but will become clearer once you understand the difference between regular functions and quoting functions. The key to programming tidyverse grammars is to understand what is special about quoting functions and what special steps are needed to be effective at programming with them.
Evaluating functions versus quoting functions
R functions can be categorised in two broad categories: evaluating functions and quoting functions 1. These functions differ in the way they get their arguments. Evaluating functions take arguments as values. It does not matter what the expression supplied as argument is or which objects it contains. R computes the argument value following the standard rules of evaluation which the function receives passively 2.
The simplest regular function is identity(). It evaluates its single argument and returns the value. Because only the final value of the argument matters, all of these statements are completely equivalent:
identity(6)
#> [1] 6
identity(2 * 3)
#> [1] 6
a <- 2
b <- 3
identity(a * b)
#> [1] 6On the other hand, a quoting function is not passed the value of an expression, it is passed the expression itself. We say the argument has been automatically quoted. The quoted expression might be evaluated a bit later or might not be evaluated at all. The simplest quoting function is quote(). It automatically quotes its argument and returns the quoted expression without any evaluation. Because only the expression passed as argument matters, none of these statements are equivalent:
quote(6)
#> [1] 6
quote(2 * 3)
#> 2 * 3
quote(a * b)
#> a * bOther familiar quoting operators are "" and ~. The "" operator quotes a piece of text at parsing time and returns a string. This prevents the text from being interpreted as some R code to evaluate. The tilde operator is similar to the quote() function in that it prevents R code from being automatically evaluated and returns a quoted expression in the form of a formula. The expression is then used to define a statistical model in modelling functions. The three following expressions are doing something similar, they are quoting their input:
"a * b"
#> [1] "a * b"
~a * b
#> ~a * b
quote(a * b)
#> a * bThe first statement returns a quoted string and the other two return quoted code in a formula or as a bare expression.
Quoting and evaluating in mundane R code
As an R programmer, you are probably already familiar with the distinction between quoting and evaluating functions. Take the case of subsetting a data frame column by name. The [[ and $ operators are both standard for this task but they are used in very different situations. The former supports indirect references like variables or expressions that represent a column name while the latter takes a column name directly:
df <- data.frame(
y = 1,
var = 2
)
df$y
#> [1] 1
var <- "y"
df[[var]]
#> [1] 1Technically, [[ is an evaluating function while $ is a quoting function. You can indirectly refer to columns with [[ because the subsetting index is evaluated, allowing indirect references. The following expressions are completely equivalent:
df[[var]] # Indirect
#> [1] 1
df[["y"]] # Direct
#> [1] 1But these are not:
df$var # Direct
#> [1] 2
df$y # Direct
#> [1] 1The following table summarises the fundamental asymmetry between the two subsetting methods:
| Quoted | Evaluated | |
|---|---|---|
| Direct | df$y |
df[["y"]] |
| Indirect | ??? | df[[var]] |
Detecting quoting functions
Because they work so differently to standard R code, it is important to recognise auto-quoted arguments. The documentation of the quoting function should normally tell you if an argument is quoted and evaluated in a special way. You can also detect quoted arguments by yourself with some experimentation. Let’s take the following expressions involving quoting and evaluating functions:
library(MASS)
#>
#> Attaching package: 'MASS'
#> The following object is masked from 'package:dplyr':
#>
#> select
mtcars2 <- subset(mtcars, cyl == 4)
sum(mtcars2$am)
#> [1] 8
rm(mtcars2)A good indication that an argument is auto-quoted and evaluated in a special way is that the argument will not work correctly outside of its original context. Let’s try to break down each of these expressions in two steps by storing the arguments in an intermediary variable:
temp <- MASS #> Error in eval(expr, envir, enclos): object 'MASS' not found temp <- "MASS" library(temp) #> Error in library(temp): there is no package called 'temp'We get these errors because there is no
MASSobject for R to find, andpkgis interpreted bylibrary()directly as a package name rather than as an indirect reference. Let’s try to break down thesubset()expression:temp <- cyl == 4 #> Error in eval(expr, envir, enclos): object 'cyl' not foundR cannot find
cylbecause we haven’t specified where to find the column. This object exists only inside themtcarsdata frame.temp <- mtcars$am sum(temp) #> [1] 13It worked!
sum()is an evaluating function and the indirect reference was resolved as expected.mtcars2 <- mtcars temp <- "mtcars2" rm(temp) exists("mtcars2") #> [1] TRUE exists("temp") #> [1] FALSEThis time there was no error, but we have accidentally removed the variable
tempinstead of the variable it was referring to. This is becauserm()auto-quotes its arguments.
Unquoting
In practice, functions that evaluate their arguments are easier to program with because they support both direct and indirect references. For quoting functions, a piece of syntax is missing. We need the ability of unquoting arguments.
Unquoting in base R
Base R provides three different ways of allowing direct references:
An extra function that evaluates its arguments. For instance the evaluating variant of the
$operator is[[.An extra parameter that switches off auto-quoting. For instance
library()evaluates its first argument if you setcharacter.onlytoTRUE:temp <- "MASS" library(temp, character.only = TRUE)An extra parameter that evaluates its argument. If you have a list of object names to pass to
rm(), use thelistargument:temp <- "mtcars2" rm(list = temp) exists("mtcars2") #> [1] FALSE
There is no general unquoting convention in base R so you have to read the documentation to figure out how to unquote an argument. Many functions like subset() or transform() do not provide any unquoting option at all.
Unquoting in the tidyverse!!
All quoting functions in the tidyverse support a single unquotation mechanism, the !! operator (pronounced bang-bang). You can use !! to cancel the automatic quotation and supply indirect references everywhere an argument is automatically quoted.
First let’s create a couple of variables that hold references to columns from the mtcars data frame. A simple way of creating these references is to use the fundamental quoting function quote():
# Variables referring to columns `cyl` and `mpg`
x_var <- quote(cyl)
y_var <- quote(mpg)
x_var
#> cyl
y_var
#> mpgHere are a few examples of how !! is used throughout the tidyverse to unquote such references.
In dplyr most verbs quote their arguments:
library("dplyr") by_cyl <- mtcars %>% group_by(!!x_var) %>% # Refer to x_var summarise(mean = mean(!!y_var)) # Refer to y_varIn ggplot2
aes()is the main quoting function:library("ggplot2") ggplot(mtcars, aes(!!x_var, !!y_var)) + # Refer to x_var and y_var geom_point()
ggplot2 also features
vars()which is useful for facetting:ggplot(mtcars, aes(disp, drat)) + geom_point() + facet_grid(vars(!!x_var)) # Refer to x_var
Indirect references in quoting functions are rarely useful in scripts but they are invaluable for writing functions. With !! we can now easily fix our wrapper function. as we’ll see in the following section.
Understanding !! with qq_show()
At this point it is normal if the concept of unquoting still feels nebulous. A good way of practicing this operation is to see for yourself what it is really doing. To that end the qq_show() function from the rlang package performs unquoting and prints the result at the screen. Here is what !! is really doing in the dplyr example (I’ve broken up the pipeline in two steps for readability):
rlang::qq_show(mtcars %>% group_by(!!x_var))
#> mtcars %>% group_by(cyl)
rlang::qq_show(data %>% summarise(mean = mean(!!y_var)))
#> data %>% summarise(mean = mean(mpg))Similarly for the ggplot2 pipeline:
rlang::qq_show(ggplot(mtcars, aes(!!x_var, !!y_var)))
#> ggplot(mtcars, aes(cyl, mpg))
rlang::qq_show(facet_grid(vars(!!x_var)))
#> facet_grid(vars(cyl))As you can see, unquoting a variable that contains a reference to the column cyl is equivalent to directly supplying cyl to the dplyr function.