From Chapter 3 of Beckerman, Childs and Petchey: Getting Started with R
Data Management, manipulation and exploration using commands from dplyr(), part of the tidyverse() package
In this exercise we will use the compensation data set, which has 40 observations of the root stock mass and mass of fruit harvested, for apple trees in both grazed and ungrazed conditions.
Find this on the Teams/Files/RStuff/data folder and save it into your own RStuff/data folder.
We will see how a few commands from the dplyr() package enable us to select portions of the data, or to manipulate it in some way. This stage of data analysis is often the most time consuming. The dplyr() commands make it straightforward to do in R.
Start a new script and save it into your RStuff/scripts folder
Begin with these commented lines, and don’t forget to add more comments to your own script as you go through it, so that you later understand what each part is doing.
# dplyr exercises
# your name
# the date
Set the working directory to be your Rstuff/data folder
See if you can do this using the Files window.
Clear R’s brain
rm(list=ls())
Load tidyverse into your session
library(tidyverse)
If this throws an error, then you need to type install.packages('tidyverse') into your console window, not in your script. This will install tidyverse(). You should then run the library(tidyverse) line again.
Import the compensation data set into a data frame called ‘compensation’
compensation<-read_csv('compensation.csv')
Inspect the data
glimpse(compensation)
Note that you can also inspect the data by clicking on the arrow against its name in the Environment pane.
Use summary() to find the mean values of the Fruit and Root columns.
summary(compensation)
summary() gives useful summary statistics of each column of a data set.
Subsetting the data
dplyr() provides several commands that let you extract subests of a larger data set.
Select a subset of columns
select() allows you to choose or exclude whichever columns you want
Use select() to pick out the Fruit column.
Save the output as an object called ‘Fruits’.
Fruits<-select(compensation,Fruit)
As is always the case with commands from dplyr(), the first argument of select() is the name of the data frame from which you want to select some columns. This is followed in this case by the names of the columns you wish to select.
Use select() to pick out all the columns except the Root column.
select() will leave out any column prefaced with a -sign.
Save the output as an object called ‘notRoot’.
notRoot<-select(compensation,-Root)
Choose particular rows
We use slice() to pick out particular rows (but see also filter())
Use slice() to grab the second row of compensation
row_2<-slice(compensation,2)
Use slice() to grab the second to the 10th rows
row_2_to_10<-slice(compensation,2:10)
Use slice() and to grab rows 2, 3 and 10.
row_2310<-slice(compensation,2,3,10)
What kind of object does slice() return?
Are the row numbers still 2, 3, and 10?
Choose rows that satisfy some condition
We use filter() to do this.
Use filter() to pick out those rows for which Fruit is greater than 50.
big_fruit<-filter(compensation,Fruit>50)
Use filter() and the logical OR symbol ‘|’ to pick rows where Fruit is greater than 80 or less than 20
extreme_fruit<-filter(compensation,Fruit>60 | Fruit<20)
Use filter() and the logical AND symbol ‘&’ to pick rows where Fruit is less than 80 AND greater than 20
medium_fruit<-filter(compensation,Fruit<80 & Fruit>20)
Note that we have saved each of these selections to a named object, so now can use them.
Sorting
Use arrange() to sort the data by the Fruit column in ascending order.
comp_fruit_ascending<-arrange(compensation,Fruit)
Use arrange() to sort the data by the Fruit column in descending order
comp_fruit_descending<-arrange(compensation,-Fruit)
TopTip 1: you can use more than one dplyr() command in one line of code.
Use select() and filter () in one line of code to pick out the rootstock column, but only for those rows where the fruit production is > 80. Put the result into an obect called largeFruit
largeRoot<-select(filter(compensation,Fruit>80),Root)
TopTip 2: Use the pipe symbol %>%
We can do the same thing using the really handy pipe symbol %>% like this:
largeFruit<-compensation %>%
filter(Fruit>80) %>%
select(Root)
You can think of the pipe symbol as meaning ‘and then’. It feeds the result of each stage into the next stage. Check that this gives the same result as the select(), filter () combination in one line of code. Which is easier to read?
Grouping and summarising
summary() gave us global means for Root and Fruit. But what if want to know if the means for each differ depending on the grazing conditions?
A combination of group() and summarise() can be used to do this:
Use group_by and summarise() to find the means of Root and Fruit in both Grazed and Ungrazed
means<-compensation %>%
group_by(Grazing) %>%
summarise(mean(Root),mean(Fruit))
LS0tCnRpdGxlOiAiZHBseXIoKSBleGVyY2lzZXMiCm91dHB1dDogaHRtbF9ub3RlYm9vawotLS0KIyMjIEZyb20gQ2hhcHRlciAzIG9mIEJlY2tlcm1hbiwgQ2hpbGRzIGFuZCBQZXRjaGV5OiBHZXR0aW5nIFN0YXJ0ZWQgd2l0aCBSCgojIyMjIERhdGEgTWFuYWdlbWVudCwgbWFuaXB1bGF0aW9uIGFuZCBleHBsb3JhdGlvbiB1c2luZyBjb21tYW5kcyBmcm9tIGBkcGx5cigpYCwgcGFydCBvZiB0aGUgYHRpZHl2ZXJzZSgpYCBwYWNrYWdlCgpJbiB0aGlzIGV4ZXJjaXNlIHdlIHdpbGwgdXNlIHRoZSBgY29tcGVuc2F0aW9uYCBkYXRhIHNldCwgd2hpY2ggaGFzIDQwIG9ic2VydmF0aW9ucyBvZiB0aGUgcm9vdCBzdG9jayBtYXNzIGFuZCBtYXNzIG9mIGZydWl0IGhhcnZlc3RlZCwgZm9yIGFwcGxlIHRyZWVzIGluIGJvdGggZ3JhemVkIGFuZCB1bmdyYXplZCBjb25kaXRpb25zLiAgCgpGaW5kIHRoaXMgb24gdGhlIFRlYW1zL0ZpbGVzL1JTdHVmZi9kYXRhIGZvbGRlciBhbmQgc2F2ZSBpdCBpbnRvIHlvdXIgb3duIFJTdHVmZi9kYXRhIGZvbGRlci4gIAoKV2Ugd2lsbCBzZWUgaG93IGEgZmV3IGNvbW1hbmRzIGZyb20gdGhlIGBkcGx5cigpYCBwYWNrYWdlIGVuYWJsZSB1cyB0byBzZWxlY3QgcG9ydGlvbnMgb2YgdGhlIGRhdGEsIG9yIHRvIG1hbmlwdWxhdGUgaXQgaW4gc29tZSB3YXkuIFRoaXMgc3RhZ2Ugb2YgZGF0YSBhbmFseXNpcyBpcyBvZnRlbiB0aGUgbW9zdCB0aW1lIGNvbnN1bWluZy4gVGhlIGBkcGx5cigpYCBjb21tYW5kcyBtYWtlIGl0IHN0cmFpZ2h0Zm9yd2FyZCB0byBkbyBpbiBSLgoKU3RhcnQgYSBuZXcgc2NyaXB0IGFuZCBzYXZlIGl0IGludG8geW91ciBSU3R1ZmYvc2NyaXB0cyBmb2xkZXIKCkJlZ2luIHdpdGggdGhlc2UgY29tbWVudGVkIGxpbmVzLCBhbmQgZG9uJ3QgZm9yZ2V0IHRvIGFkZCBtb3JlIGNvbW1lbnRzIHRvIHlvdXIgb3duIHNjcmlwdCBhcyB5b3UgZ28gdGhyb3VnaCBpdCwgc28gdGhhdCB5b3UgbGF0ZXIgdW5kZXJzdGFuZCB3aGF0IGVhY2ggcGFydCBpcyBkb2luZy4gCgpgYGB7cn0KIyBkcGx5ciBleGVyY2lzZXMKIyB5b3VyIG5hbWUKIyB0aGUgZGF0ZQpgYGAKCiMjIyBTZXQgdGhlIHdvcmtpbmcgZGlyZWN0b3J5IHRvIGJlIHlvdXIgUnN0dWZmL2RhdGEgZm9sZGVyClNlZSBpZiB5b3UgY2FuIGRvIHRoaXMgdXNpbmcgdGhlIEZpbGVzIHdpbmRvdy4KCiMjIyBDbGVhciBSJ3MgYnJhaW4KYGBge3J9YApybShsaXN0PWxzKCkpCmBgYGAKCiMjIyBMb2FkIHRpZHl2ZXJzZSBpbnRvIHlvdXIgc2Vzc2lvbgpgYGB7cn1gCmxpYnJhcnkodGlkeXZlcnNlKQpgYGAKSWYgdGhpcyB0aHJvd3MgYW4gZXJyb3IsIHRoZW4geW91IG5lZWQgdG8gdHlwZSBgaW5zdGFsbC5wYWNrYWdlcygndGlkeXZlcnNlJylgIGludG8geW91ciBjb25zb2xlIHdpbmRvdywgbm90IGluIHlvdXIgc2NyaXB0LgpUaGlzIHdpbGwgaW5zdGFsbCBgdGlkeXZlcnNlKClgLiBZb3Ugc2hvdWxkIHRoZW4gcnVuIHRoZSBgbGlicmFyeSh0aWR5dmVyc2UpYCBsaW5lIGFnYWluLgoKIyMjIEltcG9ydCB0aGUgY29tcGVuc2F0aW9uIGRhdGEgc2V0IGludG8gYSBkYXRhIGZyYW1lIGNhbGxlZCAnY29tcGVuc2F0aW9uJwpgYGB7cn1gCmNvbXBlbnNhdGlvbjwtcmVhZF9jc3YoJ2NvbXBlbnNhdGlvbi5jc3YnKQpgYGAKCgojIyMgSW5zcGVjdCB0aGUgZGF0YQpgYGB7cn0KZ2xpbXBzZShjb21wZW5zYXRpb24pCmBgYAoKTm90ZSB0aGF0IHlvdSBjYW4gYWxzbyBpbnNwZWN0IHRoZSBkYXRhIGJ5IGNsaWNraW5nIG9uIHRoZSBhcnJvdyBhZ2FpbnN0IGl0cyBuYW1lIGluIHRoZSBFbnZpcm9ubWVudCBwYW5lLgoKVXNlIGBzdW1tYXJ5KClgIHRvIGZpbmQgdGhlIG1lYW4gdmFsdWVzIG9mIHRoZSBGcnVpdCBhbmQgUm9vdCBjb2x1bW5zLgoKYGBge3J9CnN1bW1hcnkoY29tcGVuc2F0aW9uKQpgYGAKCmBzdW1tYXJ5KClgIGdpdmVzIHVzZWZ1bCBzdW1tYXJ5IHN0YXRpc3RpY3Mgb2YgZWFjaCBjb2x1bW4gb2YgYSAgZGF0YSBzZXQuCgoKIyMjIFN1YnNldHRpbmcgdGhlIGRhdGEKCmBkcGx5cigpYCBwcm92aWRlcyBzZXZlcmFsIGNvbW1hbmRzIHRoYXQgbGV0IHlvdSBleHRyYWN0IHN1YmVzdHMgb2YgYSBsYXJnZXIgZGF0YSBzZXQuCgojIyMgU2VsZWN0IGEgc3Vic2V0IG9mIGNvbHVtbnMKCmBzZWxlY3QoKWAgYWxsb3dzIHlvdSB0byBjaG9vc2Ugb3IgZXhjbHVkZSB3aGljaGV2ZXIgY29sdW1ucyB5b3Ugd2FudAoKIyMjIyBVc2UgYHNlbGVjdCgpYCB0byBwaWNrIG91dCB0aGUgRnJ1aXQgY29sdW1uLgoKU2F2ZSB0aGUgb3V0cHV0IGFzIGFuIG9iamVjdCBjYWxsZWQgJ0ZydWl0cycuCgpgYGB7cn0KRnJ1aXRzPC1zZWxlY3QoY29tcGVuc2F0aW9uLEZydWl0KQpgYGAKCkFzIGlzIGFsd2F5cyB0aGUgY2FzZSB3aXRoIGNvbW1hbmRzIGZyb20gYGRwbHlyKClgLCB0aGUgZmlyc3QgYXJndW1lbnQgb2YgYHNlbGVjdCgpYCBpcyB0aGUgbmFtZSBvZiB0aGUgZGF0YSBmcmFtZSBmcm9tIHdoaWNoIHlvdSB3YW50IHRvIHNlbGVjdCBzb21lIGNvbHVtbnMuIFRoaXMgaXMgZm9sbG93ZWQgaW4gdGhpcyBjYXNlIGJ5IHRoZSBuYW1lcyBvZiB0aGUgY29sdW1ucyB5b3Ugd2lzaCB0byBzZWxlY3QuCgojIyMjIFVzZSBzZWxlY3QoKSB0byBwaWNrIG91dCBhbGwgdGhlIGNvbHVtbnMgZXhjZXB0IHRoZSBSb290IGNvbHVtbi4gCgpgc2VsZWN0KClgIHdpbGwgbGVhdmUgb3V0IGFueSBjb2x1bW4gcHJlZmFjZWQgd2l0aCBhIC1zaWduLgoKU2F2ZSB0aGUgb3V0cHV0IGFzIGFuIG9iamVjdCBjYWxsZWQgJ25vdFJvb3QnLgoKYGBge3J9Cm5vdFJvb3Q8LXNlbGVjdChjb21wZW5zYXRpb24sLVJvb3QpCmBgYAoKIyMjIENob29zZSBwYXJ0aWN1bGFyIHJvd3MKCldlIHVzZSBgc2xpY2UoKWAgdG8gcGljayBvdXQgcGFydGljdWxhciByb3dzIChidXQgc2VlIGFsc28gYGZpbHRlcigpYCkKCiMjIyMjIFVzZSBgc2xpY2UoKWAgdG8gZ3JhYiB0aGUgc2Vjb25kIHJvdyBvZiBjb21wZW5zYXRpb24KYGBge3J9CnJvd18yPC1zbGljZShjb21wZW5zYXRpb24sMikKYGBgCgojIyMjIFVzZSBgc2xpY2UoKWAgdG8gZ3JhYiB0aGUgc2Vjb25kIHRvIHRoZSAxMHRoIHJvd3MKYGBge3J9CnJvd18yX3RvXzEwPC1zbGljZShjb21wZW5zYXRpb24sMjoxMCkKYGBgCgojIyMjIFVzZSBgc2xpY2UoKWAgYW5kIHRvIGdyYWIgcm93cyAyLCAzIGFuZCAxMC4KYGBge3J9CnJvd18yMzEwPC1zbGljZShjb21wZW5zYXRpb24sMiwzLDEwKQpgYGAKCldoYXQga2luZCBvZiBvYmplY3QgZG9lcyBgc2xpY2UoKWAgcmV0dXJuPyAgCkFyZSB0aGUgcm93IG51bWJlcnMgc3RpbGwgIDIsIDMsIGFuZCAxMD8KCiMjIyBDaG9vc2Ugcm93cyB0aGF0IHNhdGlzZnkgc29tZSBjb25kaXRpb24KCldlIHVzZSBgZmlsdGVyKClgIHRvIGRvIHRoaXMuCgojIyMjIFVzZSBgZmlsdGVyKClgIHRvIHBpY2sgb3V0IHRob3NlIHJvd3MgZm9yIHdoaWNoIEZydWl0IGlzIGdyZWF0ZXIgdGhhbiA1MC4KYGBge3J9CmJpZ19mcnVpdDwtZmlsdGVyKGNvbXBlbnNhdGlvbixGcnVpdD41MCkKYGBgCgojIyMjIFVzZSBmaWx0ZXIoKSBhbmQgdGhlIGxvZ2ljYWwgT1Igc3ltYm9sICd8JyB0byBwaWNrIHJvd3Mgd2hlcmUgRnJ1aXQgaXMgZ3JlYXRlciB0aGFuIDgwIG9yIGxlc3MgdGhhbiAyMApgYGB7cn0KZXh0cmVtZV9mcnVpdDwtZmlsdGVyKGNvbXBlbnNhdGlvbixGcnVpdD42MCB8IEZydWl0PDIwKQpgYGAKCiMjIyMgVXNlIGZpbHRlcigpIGFuZCB0aGUgbG9naWNhbCBBTkQgc3ltYm9sICcmJyB0byBwaWNrIHJvd3Mgd2hlcmUgRnJ1aXQgaXMgbGVzcyB0aGFuIDgwIEFORCBncmVhdGVyIHRoYW4gMjAKYGBge3J9Cm1lZGl1bV9mcnVpdDwtZmlsdGVyKGNvbXBlbnNhdGlvbixGcnVpdDw4MCAmIEZydWl0PjIwKQpgYGAKCk5vdGUgdGhhdCB3ZSBoYXZlIHNhdmVkIGVhY2ggb2YgdGhlc2Ugc2VsZWN0aW9ucyB0byBhIG5hbWVkIG9iamVjdCwgc28gbm93IGNhbiB1c2UgdGhlbS4KCiMjIyBUcmFuc2Zvcm1pbmcgZGF0YSB1c2luZyBgbXV0YXRlKClgCgpGaXJzdCwgdXNlIGBoZWFkKClgIHRvIGxvb2sgYXQgdGhlIGZpcnN0IDYgcm93cyBvZiBgeW91ciBkYXRhYGNvbXBlbnNhdGlvbi4gIApIb3cgbWFueSBjb2x1bW5zIGFyZSB0aGVyZT8KCmBgYHtyfQpoZWFkKGNvbXBlbnNhdGlvbikKYGBgCgpVc2UgYG11dGF0ZSgpYCB0byBjcmVhdGUgYW4gYWRkaXRpb25hbCBjb2x1bW4gY2FsbGVkIGxvZ0ZydWl0IHdoaWNoIGlzIHRoZSBuYXR1cmFsIGxvZyBvZiB0aGUgRnJ1aXQgY29sdW1uLiAgCkRvaW5nIGEgbG9nLXRyYW5zZm9ybSBvZiBkYXRhIGlzIG9mdGVuIGEgdXNlZnVsIHRyaWNrIGluIGRhdGEgcHJlcGFyYXRpb24uCgpgYGB7cn0KY29tcGVuc2F0aW9uPC1tdXRhdGUoY29tcGVuc2F0aW9uLGxvZ0ZydWl0PWxvZyhGcnVpdCkpCmBgYAoKTm93IHVzZSBgaGVhZCgpYCB0byBsb29rIGFnYWluIGF0IHRoZSBmaXJzdCA2IHJvd3Mgb2YgeW91ciBkYXRhLiAgSG93IG1hbnkgY29sdW1ucyBhcmUgdGhlcmU/CgoKSGF2ZSB5b3UgY2hhbmdlZCB0aGUgb3JpZ2luYWwgZGF0YSBmaWxlPyBUaGUgYW5zd2VyIGlzIG5vLiBXZSBoYXZlIGNoYW5nZSB0aGUgZGF0YSBmcmFtZSBgY29tcGVuc2F0aW9uKClgIGJ1dCB0aGUgZGF0YSBmaWxlcyBpdHNlbGYgaXMgdW50b3VjaGVkLgoKIyMjIFNvcnRpbmcKCiMjIyMgVXNlIGBhcnJhbmdlKClgIHRvIHNvcnQgdGhlIGRhdGEgYnkgdGhlIEZydWl0IGNvbHVtbiBpbiBhc2NlbmRpbmcgb3JkZXIuCmBgYHtyfQpjb21wX2ZydWl0X2FzY2VuZGluZzwtYXJyYW5nZShjb21wZW5zYXRpb24sRnJ1aXQpCmBgYAoKIyMjIFVzZSBgYXJyYW5nZSgpYCB0byBzb3J0IHRoZSBkYXRhIGJ5IHRoZSBGcnVpdCBjb2x1bW4gaW4gZGVzY2VuZGluZyBvcmRlcgpgYGB7cn0KY29tcF9mcnVpdF9kZXNjZW5kaW5nPC1hcnJhbmdlKGNvbXBlbnNhdGlvbiwtRnJ1aXQpCmBgYAoKIyMjIFRvcFRpcCAxOiB5b3UgY2FuIHVzZSBtb3JlIHRoYW4gb25lIGBkcGx5cigpYCBjb21tYW5kIGluIG9uZSBsaW5lIG9mIGNvZGUuCgpVc2UgYHNlbGVjdCgpYCBhbmQgYGZpbHRlciAoKWAgaW4gb25lIGxpbmUgb2YgY29kZSB0byBwaWNrIG91dCB0aGUgcm9vdHN0b2NrIGNvbHVtbiwgYnV0IG9ubHkgZm9yIHRob3NlIHJvd3Mgd2hlcmUgdGhlIGZydWl0IHByb2R1Y3Rpb24gaXMgPiA4MC4gUHV0IHRoZSByZXN1bHQgaW50byBhbiBvYmVjdCBjYWxsZWQgbGFyZ2VGcnVpdAoKYGBge3J9CmxhcmdlUm9vdDwtc2VsZWN0KGZpbHRlcihjb21wZW5zYXRpb24sRnJ1aXQ+ODApLFJvb3QpCmBgYAoKCiMjIyBUb3BUaXAgMjogVXNlIHRoZSBwaXBlIHN5bWJvbCAlPiUKCldlIGNhbiBkbyB0aGUgc2FtZSB0aGluZyB1c2luZyB0aGUgcmVhbGx5IGhhbmR5IHBpcGUgc3ltYm9sIGAlPiVgIGxpa2UgdGhpczoKCmBgYHtyfQpsYXJnZUZydWl0PC1jb21wZW5zYXRpb24gJT4lCiAgZmlsdGVyKEZydWl0PjgwKSAlPiUKICBzZWxlY3QoUm9vdCkKYGBgCgpZb3UgY2FuIHRoaW5rIG9mIHRoZSBwaXBlIHN5bWJvbCBhcyBtZWFuaW5nICdhbmQgdGhlbicuIEl0IGZlZWRzIHRoZSByZXN1bHQgb2YgZWFjaCBzdGFnZSBpbnRvIHRoZSBuZXh0IHN0YWdlLgpDaGVjayB0aGF0IHRoaXMgZ2l2ZXMgdGhlIHNhbWUgcmVzdWx0IGFzIHRoZSBgc2VsZWN0KClgLCBgZmlsdGVyICgpYCBjb21iaW5hdGlvbiBpbiBvbmUgbGluZSBvZiBjb2RlLgpXaGljaCBpcyBlYXNpZXIgdG8gcmVhZD8KCgojIyMgR3JvdXBpbmcgYW5kIHN1bW1hcmlzaW5nCgpgc3VtbWFyeSgpYCBnYXZlIHVzIGdsb2JhbCBtZWFucyBmb3IgUm9vdCBhbmQgRnJ1aXQuIEJ1dCB3aGF0IGlmIHdhbnQgdG8ga25vdyBpZiB0aGUgbWVhbnMgZm9yIGVhY2ggZGlmZmVyIGRlcGVuZGluZyBvbiB0aGUgZ3JhemluZyBjb25kaXRpb25zPwoKQSBjb21iaW5hdGlvbiBvZiBgZ3JvdXAoKWAgYW5kIGBzdW1tYXJpc2UoKWAgY2FuIGJlIHVzZWQgdG8gZG8gdGhpczoKCiMjIyMgVXNlIGdyb3VwX2J5IGFuZCBzdW1tYXJpc2UoKSB0byBmaW5kIHRoZSBtZWFucyBvZiBSb290IGFuZCBGcnVpdCBpbiBib3RoIEdyYXplZCBhbmQgVW5ncmF6ZWQKCmBgYHtyfQptZWFuczwtY29tcGVuc2F0aW9uICU+JQogIGdyb3VwX2J5KEdyYXppbmcpICU+JQogIHN1bW1hcmlzZShtZWFuKFJvb3QpLG1lYW4oRnJ1aXQpKQpgYGAKCgoKICAK