Hai Nguyen
July 20, 2020
Writing for and while loops is useful when programming but not particularly easy when working interactively on the multi-line expressions with curly braces. R has some functions which implement looping in a compact form to make your life easier.
apply(): Apply a function over the margins of an array
lapply(): Loop over a list and evaluate a function on each element
sapply(): Same as lapply but try to simplify the result
tapply(): Apply a function over subsets of a vector
mapply(): Multivariate version of lapply
An auxiliary function split is also useful, particularly in conjunction with lapply.
is used to a evaluate a function (often an anonymous one) over the margins of an array.
str(apply) ## function (X, MARGIN, FUN, ...)(x <- matrix(rnorm(50), 10, 5))## [,1] [,2] [,3] [,4] [,5]
## [1,] -1.0310711 0.90390937 0.01964287 0.5745165 -0.766270174
## [2,] 0.6758126 1.11955684 1.72053643 -0.6996398 0.437348621
## [3,] 0.7006198 1.65270426 1.69259234 1.1712015 -0.345623624
## [4,] -0.7181044 0.51083904 -0.17559944 0.6565499 1.953092325
## [5,] 1.0773979 -1.34292264 0.68188868 0.1759386 0.007072689
## [6,] 1.0293794 -1.19915960 0.65957225 -1.4806250 0.956631020
## [7,] -2.0083147 -0.83924075 0.82076425 -1.2455842 1.073592270
## [8,] 1.2701617 -0.01227269 1.96432596 -1.0569905 -0.570676168
## [9,] 0.4219844 -0.53843305 -1.18291764 -0.5567100 0.134087440
## [10,] -0.4967885 -0.43171007 1.60093310 0.5293532 -1.745164307apply(x, 2, mean)## [1] 0.09210771 -0.01767293 0.78017388 -0.19319897 0.11340901apply(x, 1, sum)## [1] -0.29927257 3.25361470 4.87149436 2.22677739 0.59937526 -0.03420197
## [7] -2.19878310 1.59454830 -1.72198888 -0.54337656For sums and means of matrix dimension, we have some shortcuts.
The shortcut functions are much faster, but you won’t notice unless you’re using a large matrix.
Quantiles of the rows of a matrix.
x## [,1] [,2] [,3] [,4] [,5]
## [1,] -1.0310711 0.90390937 0.01964287 0.5745165 -0.766270174
## [2,] 0.6758126 1.11955684 1.72053643 -0.6996398 0.437348621
## [3,] 0.7006198 1.65270426 1.69259234 1.1712015 -0.345623624
## [4,] -0.7181044 0.51083904 -0.17559944 0.6565499 1.953092325
## [5,] 1.0773979 -1.34292264 0.68188868 0.1759386 0.007072689
## [6,] 1.0293794 -1.19915960 0.65957225 -1.4806250 0.956631020
## [7,] -2.0083147 -0.83924075 0.82076425 -1.2455842 1.073592270
## [8,] 1.2701617 -0.01227269 1.96432596 -1.0569905 -0.570676168
## [9,] 0.4219844 -0.53843305 -1.18291764 -0.5567100 0.134087440
## [10,] -0.4967885 -0.43171007 1.60093310 0.5293532 -1.745164307apply(x, 1, quantile, probs = c(0.25, 0.75))## [,1] [,2] [,3] [,4] [,5] [,6] [,7]
## 25% -0.7662702 0.4373486 0.7006198 -0.1755994 0.007072689 -1.199160 -1.2455842
## 75% 0.5745165 1.1195568 1.6527043 0.6565499 0.681888676 0.956631 0.8207643
## [,8] [,9] [,10]
## 25% -0.5706762 -0.5567100 -0.4967885
## 75% 1.2701617 0.1340874 0.5293532Average matrix in an array
(a <- array(rnorm(2*2*2), c(2,2,2)))## , , 1
##
## [,1] [,2]
## [1,] 0.2591642 0.17255734
## [2,] 0.9093289 -0.07930584
##
## , , 2
##
## [,1] [,2]
## [1,] 0.1101806 -2.085360
## [2,] -0.3388593 1.072092apply(a, c(1, 2), mean)## [,1] [,2]
## [1,] 0.1846724 -0.9564013
## [2,] 0.2852348 0.4963930rowMeans(a, dims = 2)## [,1] [,2]
## [1,] 0.1846724 -0.9564013
## [2,] 0.2852348 0.4963930lappy takes three arguments: (1) a list x; (2) a function (or the name of a function) FUN; (3) other arguments via its … argument. If X is not a list, it will be coerced to a list using as.list.
lapply## function (X, FUN, ...)
## {
## FUN <- match.fun(FUN)
## if (!is.vector(X) || is.object(X))
## X <- as.list(X)
## .Internal(lapply(X, FUN))
## }
## <bytecode: 0x000000001359e438>
## <environment: namespace:base>The actual looping is done internally in C code.
(x <- list(a = 1:5, b = rnorm(10)))## $a
## [1] 1 2 3 4 5
##
## $b
## [1] -0.6084364 -0.2700525 -0.7124922 0.3789660 -1.4247590 -0.4049345
## [7] -0.5588601 -0.9837195 0.1162195 -1.1353509lapply(x, mean)## $a
## [1] 3
##
## $b
## [1] -0.560342(x <- list(a = 1:4, b = rnorm(10), c = rnorm(20, 1), d = rnorm(100, 5)))## $a
## [1] 1 2 3 4
##
## $b
## [1] 1.88522968 -0.25760691 -0.51344382 -1.55381720 -1.07710681 -0.02123826
## [7] 0.38423390 0.81227516 -0.25570059 2.48007116
##
## $c
## [1] 0.9930088 0.6751436 0.3840175 1.1282443 1.0218116 -0.1696535
## [7] 0.2361099 1.9140203 1.0741944 3.2193143 0.6730833 0.5871357
## [13] -0.8233353 2.0760040 1.0948577 0.9959635 0.7397966 1.5934328
## [19] 1.5437283 1.0068675
##
## $d
## [1] 4.812006 3.278230 4.772490 4.443637 3.601023 5.281555 4.413869 3.967899
## [9] 5.149140 4.949383 5.486253 4.146709 6.130135 6.262920 6.063631 4.940577
## [17] 5.566428 7.000207 3.732414 5.115969 3.035732 6.717552 5.905937 4.545468
## [25] 5.638412 3.538398 3.749885 3.098488 4.684065 4.959130 2.567905 4.664561
## [33] 4.186793 5.580668 5.731018 5.592876 5.357009 3.899093 4.114198 4.726685
## [41] 4.822234 5.171641 4.975139 4.237813 4.053931 4.682428 6.569071 4.630326
## [49] 5.032692 5.327807 4.288836 4.277815 5.380265 5.041099 4.217692 5.753348
## [57] 5.684015 4.396820 3.801821 5.107160 5.405357 4.099499 4.996491 5.227409
## [65] 5.701729 7.412333 5.649993 5.174684 3.684981 4.819665 6.334634 6.044096
## [73] 5.587551 6.395636 3.995892 4.856189 6.253564 3.137049 5.355125 4.140680
## [81] 2.976703 7.349088 7.317833 6.559780 5.207212 5.918800 5.314546 5.142212
## [89] 5.921429 6.169547 4.278664 4.301989 5.557067 5.808300 5.962258 2.815505
## [97] 7.085227 4.725385 5.074762 3.561242lapply(x, mean)## $a
## [1] 2.5
##
## $b
## [1] 0.1882896
##
## $c
## [1] 0.9981873
##
## $d
## [1] 5.001804x <- 1:4
lapply(x, runif)## [[1]]
## [1] 0.4404748
##
## [[2]]
## [1] 0.9978680 0.5347963
##
## [[3]]
## [1] 0.6679951 0.7153561 0.2647626
##
## [[4]]
## [1] 0.4141603 0.1615729 0.3792328 0.7778865lapply(x, runif, min = 0, max = 10)## [[1]]
## [1] 2.814084
##
## [[2]]
## [1] 1.348820 3.253221
##
## [[3]]
## [1] 1.066413 5.438469 2.493258
##
## [[4]]
## [1] 0.296358 9.155762 9.653269 8.423878(x <- list(a = matrix(1:4, 2, 2), b = matrix(1:6, 3, 2)))## $a
## [,1] [,2]
## [1,] 1 3
## [2,] 2 4
##
## $b
## [,1] [,2]
## [1,] 1 4
## [2,] 2 5
## [3,] 3 6lapply(x, function(elt) { elt[,1] })## $a
## [1] 1 2
##
## $b
## [1] 1 2 3sapply will try to simplify the result of lapply if possible.
If the result is a list where every element is length 1, then a vector is returned
If the result is a list where every element is a vector of the same length (> 1), a matrix is returned.
If it can’t figure things out, a list is returned
(x <- list(a = 1:4, b = rnorm(10), c = rnorm(20, 1), d = rnorm(100, 5)))## $a
## [1] 1 2 3 4
##
## $b
## [1] 0.2473592 -0.4727938 0.4734193 1.4971177 -0.7217773 0.4933681
## [7] -0.8989252 0.7773786 1.3065079 0.7172061
##
## $c
## [1] -1.0967026 0.4155884 1.2464292 0.6190762 2.5006363 1.2379720
## [7] 0.8273583 0.4839609 0.9699183 1.0215664 2.1990205 1.7318226
## [13] -0.1373359 1.0014293 2.0140557 1.4705253 1.5272838 1.0792480
## [19] 1.8399190 2.1481270
##
## $d
## [1] 5.006823 6.258993 4.578858 6.188938 4.890430 5.004921 4.877430 3.987656
## [9] 3.199062 6.102523 4.812663 5.439237 4.009996 5.540833 6.050725 3.576186
## [17] 5.252031 6.912252 6.775728 3.847964 4.617222 4.755816 4.945036 4.919109
## [25] 4.813571 5.953022 3.540179 5.382188 5.090180 5.892860 3.037103 4.758389
## [33] 4.255644 6.305509 6.106328 5.925032 5.519261 3.761310 6.141934 6.384631
## [41] 5.226351 3.723580 4.681138 4.657985 6.055419 4.381368 4.962422 5.357029
## [49] 5.122552 2.671165 3.951705 4.062986 6.040330 5.963598 3.701612 6.482807
## [57] 5.324660 5.198466 6.553609 6.734663 5.200484 5.033292 7.109848 4.796868
## [65] 4.489079 5.139901 3.952234 6.864073 3.829663 7.213185 4.776950 5.019568
## [73] 6.134496 4.278921 4.764974 3.924969 5.563213 7.616459 5.293957 5.246241
## [81] 5.118313 4.905841 4.649952 5.194855 5.626327 2.224251 5.349928 3.114690
## [89] 4.521199 5.044313 4.288622 4.363280 4.400662 5.560830 5.449870 4.842079
## [97] 5.856831 5.493063 5.018798 4.781371lapply(x, mean)## $a
## [1] 2.5
##
## $b
## [1] 0.3418861
##
## $c
## [1] 1.154995
##
## $d
## [1] 5.073245sapply(x, mean) ## a b c d
## 2.5000000 0.3418861 1.1549949 5.0732447takes a vector or other objects and splits it into groups determined by a factor or list of factors.
str(split)## function (x, f, drop = FALSE, ...)where
x is a vector (or list) or data frame
f is a factor (or coerced to one) or a list of factors
drop indicates whether empty factors levels should be dropped
The combination of split() and a function like lapply() or sapply() is a common paradigm in R.
x<-c(a=rnorm(10),b=0:10,c=rnorm(10,1))
f<-gl(3,10)
split(x,f)## Warning in split.default(x, f): data length is not a multiple of split variable## $`1`
## a1 a2 a3 a4 a5 a6 a7
## -0.9446156 -0.4691625 0.4206769 -0.5490512 -1.0181029 -0.9283220 1.7213578
## a8 a9 a10 c10
## 0.3487996 -1.2544942 -1.9072324 0.1763271
##
## $`2`
## b1 b2 b3 b4 b5 b6 b7 b8 b9 b10
## 0 1 2 3 4 5 6 7 8 9
##
## $`3`
## b11 c1 c2 c3 c4 c5
## 10.00000000 3.15250193 1.06363094 0.52030626 1.59394396 -0.63609454
## c6 c7 c8 c9
## 0.62120617 -0.03441994 0.07935555 1.37163819lapply(split(x, f), mean)## Warning in split.default(x, f): data length is not a multiple of split variable## $`1`
## [1] -0.4003472
##
## $`2`
## [1] 4.5
##
## $`3`
## [1] 1.773207data(airquality, package = "datasets")
head(airquality)## Ozone Solar.R Wind Temp Month Day
## 1 41 190 7.4 67 5 1
## 2 36 118 8.0 72 5 2
## 3 12 149 12.6 74 5 3
## 4 18 313 11.5 62 5 4
## 5 NA NA 14.3 56 5 5
## 6 28 NA 14.9 66 5 6s <- split(airquality, airquality$Month)
str(s)## List of 5
## $ 5:'data.frame': 31 obs. of 6 variables:
## ..$ Ozone : int [1:31] 41 36 12 18 NA 28 23 19 8 NA ...
## ..$ Solar.R: int [1:31] 190 118 149 313 NA NA 299 99 19 194 ...
## ..$ Wind : num [1:31] 7.4 8 12.6 11.5 14.3 14.9 8.6 13.8 20.1 8.6 ...
## ..$ Temp : int [1:31] 67 72 74 62 56 66 65 59 61 69 ...
## ..$ Month : int [1:31] 5 5 5 5 5 5 5 5 5 5 ...
## ..$ Day : int [1:31] 1 2 3 4 5 6 7 8 9 10 ...
## $ 6:'data.frame': 30 obs. of 6 variables:
## ..$ Ozone : int [1:30] NA NA NA NA NA NA 29 NA 71 39 ...
## ..$ Solar.R: int [1:30] 286 287 242 186 220 264 127 273 291 323 ...
## ..$ Wind : num [1:30] 8.6 9.7 16.1 9.2 8.6 14.3 9.7 6.9 13.8 11.5 ...
## ..$ Temp : int [1:30] 78 74 67 84 85 79 82 87 90 87 ...
## ..$ Month : int [1:30] 6 6 6 6 6 6 6 6 6 6 ...
## ..$ Day : int [1:30] 1 2 3 4 5 6 7 8 9 10 ...
## $ 7:'data.frame': 31 obs. of 6 variables:
## ..$ Ozone : int [1:31] 135 49 32 NA 64 40 77 97 97 85 ...
## ..$ Solar.R: int [1:31] 269 248 236 101 175 314 276 267 272 175 ...
## ..$ Wind : num [1:31] 4.1 9.2 9.2 10.9 4.6 10.9 5.1 6.3 5.7 7.4 ...
## ..$ Temp : int [1:31] 84 85 81 84 83 83 88 92 92 89 ...
## ..$ Month : int [1:31] 7 7 7 7 7 7 7 7 7 7 ...
## ..$ Day : int [1:31] 1 2 3 4 5 6 7 8 9 10 ...
## $ 8:'data.frame': 31 obs. of 6 variables:
## ..$ Ozone : int [1:31] 39 9 16 78 35 66 122 89 110 NA ...
## ..$ Solar.R: int [1:31] 83 24 77 NA NA NA 255 229 207 222 ...
## ..$ Wind : num [1:31] 6.9 13.8 7.4 6.9 7.4 4.6 4 10.3 8 8.6 ...
## ..$ Temp : int [1:31] 81 81 82 86 85 87 89 90 90 92 ...
## ..$ Month : int [1:31] 8 8 8 8 8 8 8 8 8 8 ...
## ..$ Day : int [1:31] 1 2 3 4 5 6 7 8 9 10 ...
## $ 9:'data.frame': 30 obs. of 6 variables:
## ..$ Ozone : int [1:30] 96 78 73 91 47 32 20 23 21 24 ...
## ..$ Solar.R: int [1:30] 167 197 183 189 95 92 252 220 230 259 ...
## ..$ Wind : num [1:30] 6.9 5.1 2.8 4.6 7.4 15.5 10.9 10.3 10.9 9.7 ...
## ..$ Temp : int [1:30] 91 92 93 93 87 84 80 78 75 73 ...
## ..$ Month : int [1:30] 9 9 9 9 9 9 9 9 9 9 ...
## ..$ Day : int [1:30] 1 2 3 4 5 6 7 8 9 10 ...lapply(s, function(x) {colMeans(x[, c("Ozone", "Solar.R", "Wind")])})## $`5`
## Ozone Solar.R Wind
## NA NA 11.62258
##
## $`6`
## Ozone Solar.R Wind
## NA 190.16667 10.26667
##
## $`7`
## Ozone Solar.R Wind
## NA 216.483871 8.941935
##
## $`8`
## Ozone Solar.R Wind
## NA NA 8.793548
##
## $`9`
## Ozone Solar.R Wind
## NA 167.4333 10.1800sapply(s, function(x) {colMeans(x[, c("Ozone", "Solar.R", "Wind")])})## 5 6 7 8 9
## Ozone NA NA NA NA NA
## Solar.R NA 190.16667 216.483871 NA 167.4333
## Wind 11.62258 10.26667 8.941935 8.793548 10.1800sapply(s, function(x) {colMeans(x[, c("Ozone", "Solar.R", "Wind")], na.rm = TRUE)})## 5 6 7 8 9
## Ozone 23.61538 29.44444 59.115385 59.961538 31.44828
## Solar.R 181.29630 190.16667 216.483871 171.857143 167.43333
## Wind 11.62258 10.26667 8.941935 8.793548 10.18000(x <- rnorm(10))## [1] 1.1255224 0.7878017 1.3855688 -2.5194260 -0.5611056 -0.2157305
## [7] -1.8926559 1.0024931 -0.6658225 -0.8463519(f1 <- gl(2, 5))## [1] 1 1 1 1 1 2 2 2 2 2
## Levels: 1 2(f2 <- gl(5, 2))## [1] 1 1 2 2 3 3 4 4 5 5
## Levels: 1 2 3 4 5interaction(f1,f2)## [1] 1.1 1.1 1.2 1.2 1.3 2.3 2.4 2.4 2.5 2.5
## Levels: 1.1 2.1 1.2 2.2 1.3 2.3 1.4 2.4 1.5 2.5str(split(x, list(f1, f2)))## List of 10
## $ 1.1: num [1:2] 1.126 0.788
## $ 2.1: num(0)
## $ 1.2: num [1:2] 1.39 -2.52
## $ 2.2: num(0)
## $ 1.3: num -0.561
## $ 2.3: num -0.216
## $ 1.4: num(0)
## $ 2.4: num [1:2] -1.89 1
## $ 1.5: num(0)
## $ 2.5: num [1:2] -0.666 -0.846str(split(x, list(f1, f2), drop = TRUE))## List of 6
## $ 1.1: num [1:2] 1.126 0.788
## $ 1.2: num [1:2] 1.39 -2.52
## $ 1.3: num -0.561
## $ 2.3: num -0.216
## $ 2.4: num [1:2] -1.89 1
## $ 2.5: num [1:2] -0.666 -0.846tapply() is used to apply a function over subsets of a vector. It can be thought of as a combination of split() and sapply() for vectors only. I’ve been told that the “t” in tapply() refers to “table”, but that is unconfirmed.
str(tapply)## function (X, INDEX, FUN = NULL, ..., default = NA, simplify = TRUE)The arguments to tapply() are as follows:
X is a vector
INDEX is a factor or a list of factors (or else they are coerced to factors)
FUN is a function to be applied
... contains other arguments to be passed FUN
simplify, should we simplify the result?
(x<-c(rnorm(10),runif(10),rnorm(10,1)))## [1] -0.742486902 -0.070762281 0.002881839 -0.710468701 -0.162545999
## [6] -0.650324656 0.747627718 -0.476738016 0.280498012 -1.109172238
## [11] 0.501468772 0.973928299 0.231771613 0.121027709 0.247712406
## [16] 0.082727066 0.138982071 0.026310932 0.101041305 0.494293835
## [21] 0.432121124 1.821933271 1.221449489 0.653440932 -0.313743403
## [26] 2.638999760 1.003363312 2.284005889 0.254836007 0.848715109(f<- gl(3,10))## [1] 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 3 3 3 3 3 3 3 3 3 3
## Levels: 1 2 3lapply(split(x,f), mean)## $`1`
## [1] -0.2891491
##
## $`2`
## [1] 0.2919264
##
## $`3`
## [1] 1.084512tapply(x, f, mean)## 1 2 3
## -0.2891491 0.2919264 1.0845121# take group means without simplification
tapply(x, f, mean, simplify=FALSE)## $`1`
## [1] -0.2891491
##
## $`2`
## [1] 0.2919264
##
## $`3`
## [1] 1.084512tapply(x, f, range)## $`1`
## [1] -1.1091722 0.7476277
##
## $`2`
## [1] 0.02631093 0.97392830
##
## $`3`
## [1] -0.3137434 2.6389998The mapply() function is a multivariate apply of sorts which applies a function in parallel over a set of arguments. Recall that lapply() and friends only iterate over a single R object. What if you want to iterate over multiple R objects in parallel? This is what mapply() is for.
str(mapply)## function (FUN, ..., MoreArgs = NULL, SIMPLIFY = TRUE, USE.NAMES = TRUE)The arguments to mapply() are
FUN is a function to apply
... contains R objects to apply over
MoreArgs is a list of other arguments to FUN.
SIMPLIFY indicates whether the result should be simplified
The mapply() function has a different argument order from lapply() because the function to apply comes first rather than the object to iterate over. The R objects over which we apply the function are given in the … argument because we can apply over an arbitrary number of R objects.
list(rep(1,4), rep(2,3), rep(3,2), rep(4,1))## [[1]]
## [1] 1 1 1 1
##
## [[2]]
## [1] 2 2 2
##
## [[3]]
## [1] 3 3
##
## [[4]]
## [1] 4Instead we can do
mapply(rep, 1:4, 4:1)## [[1]]
## [1] 1 1 1 1
##
## [[2]]
## [1] 2 2 2
##
## [[3]]
## [1] 3 3
##
## [[4]]
## [1] 4The mapply() function can be use to automatically “vectorize” a function. What this means is that it can be used to take a function that typically only takes single arguments and create a new function that can take vector arguments. This is often needed when you want to plot functions.
noise <- function(n, mean, sd){rnorm(n, mean, sd)}
noise(5,1,2)## [1] -2.7969593 -0.1059235 -0.2955518 2.7436791 2.3326748noise(1:5, 1:5, 2) # which we not expect## [1] -2.2182590 1.6834528 2.9972340 0.7390324 4.4569444mapply(noise, 1:5, 1:5, 2)## [[1]]
## [1] 3.938343
##
## [[2]]
## [1] 1.694360 1.820809
##
## [[3]]
## [1] 0.1820214 7.3719112 2.1322949
##
## [[4]]
## [1] 5.048006 7.787257 5.501711 3.128530
##
## [[5]]
## [1] 4.128487 2.177265 6.277762 4.487320 2.068892# same as
list(noise(1,1,2), noise(2,2,2),
noise(3,3,2), noise(4,4,2),
noise(5,5,2))## [[1]]
## [1] 4.926713
##
## [[2]]
## [1] -0.1078601 1.3106243
##
## [[3]]
## [1] 1.987970 6.125909 2.531185
##
## [[4]]
## [1] 1.652755 4.077457 3.528184 4.238129
##
## [[5]]
## [1] 5.139765 8.001115 6.384753 6.493480 2.304212(x <- rnorm(10))## [1] 1.1010948 -0.3903914 1.4121778 1.2528377 -1.5214054 0.7708671
## [7] -0.7369374 0.4660334 -2.3615079 -0.9189918sumsq <- function(mu, sigma, x) {sum(((x - mu) / sigma)^2)}
sumsq(1:10, 1:10, x) # this is not what we want## [1] 9.545619mapply(sumsq, 1:10, 1:10, MoreArgs = list(x = x))## [1] 26.87155 14.68100 12.28627 11.40181 10.97125 10.72594 10.57115 10.46623
## [9] 10.39125 10.33544vsumsq <- Vectorize(sumsq, c("mu", "sigma"))
vsumsq(1:10, 1:10, x)## [1] 26.87155 14.68100 12.28627 11.40181 10.97125 10.72594 10.57115 10.46623
## [9] 10.39125 10.33544The loop functions in R are very powerful because they allow you to conduct a series of operations on data using a compact form
The operation of a loop function involves iterating over an R object (e.g. a list or vector or matrix), applying a function to each element of the object, and the collating the results and returning the collated results.
Loop functions make heavy use of anonymous functions, which exist for the life of the loop function but are not stored anywhere
The split() function can be used to divide an R object in to subsets determined by another variable which can subsequently be looped over using loop functions.
Roger Peng, R Programming for Data Science, Chapter 16, Loop Functions 2019-09-18