The goal of this lab is to introduce you to R and RStudio, which you’ll be using throughout the course both to learn the statistical concepts discussed in the course and to analyze real data and come to informed conclusions. To clarify which is which: R is the name of the programming language itself and RStudio is a convenient interface.
As the labs progress, you are encouraged to explore beyond what the labs dictate; a willingness to experiment will make you a much better programmer. Before we get to that stage, however, you need to build some basic fluency in R. Today we begin with the fundamental building blocks of R and RStudio: the interface, reading in data, and basic commands.
Go ahead and launch RStudio. You should see a window that looks like the image shown below.
The panel on the lower left is where the action happens. It’s called the console. Everytime you launch RStudio, it will have the same text at the top of the console telling you the version of R that you’re running. Below that informationis the prompt. As its name suggests, this prompt is really a request: a request for a command. Initially, interacting with R is all about typing commands and interpreting the output. These commands and their syntax have evolved over decades (literally) and now provide what many users feel is a fairly natural way to access data and organize, describe, and invoke statistical computations.
The panel in the upper right contains your environment as well as a history of the commands that you’ve previously entered.
Any plots that you generate will show up in the panel in the lower right corner. This is also where you can browse your files, access help, manage packages, etc.
R is an open-source programming language, meaning that users can contribute packages that make our lives easier, and we can use them for free. For this lab, and many others in the future, we will use the following R packages:
If these packages are not already available in your R environment, install them by typing the following three lines of code into the console of your RStudio session, pressing the enter/return key after each one. Note that you can check to see which packages (and which versions) are installed by inspecting the Packages tab in the lower right panel of RStudio.
#install.packages("tidyverse")
#install.packages("openintro")You may need to select a server from which to download; any of them will work. Next, you need to load these packages in your working environment. We do this with the library function. Run the following three lines in your console.
library(tidyverse)
library(openintro)You only need to install packages once, but you need to load them each time you relaunch RStudio.
The Tidyverse packages share common philosophies and are designed to work together. You can find more about the packages in the tidyverse at tidyverse.org.
We will be using R Markdown to create reproducible lab reports. See the following videos describing why and how:
Why use R Markdown for Lab Reports?
Using R Markdown for Lab Reports in RStudio
In a nuthshell, in RStudio, go to New File -> R Markdown… Then, choose From Template and then choose Lab Report for OpenIntro Statistics Lab 1 from the list of templates.
Going forward you should refrain from typing your code directly in the console, and instead type any code (final correct answer, or anything you’re just trying out) in the R Markdown file and run the chunk using either the Run button on the chunk (green sideways triangle) or by highlighting the code and clicking Run on the top right corner of the R Markdown editor. If at any point you need to start over, you can Run All Chunks above the chunk you’re working in by clicking on the down arrow in the code chunk.
To get started, let’s take a peek at the data.
source('/DATA606/cuny_data606/Lab1/more/arbuthnot.r')You can run the command by
Ctrl-Shift-Enter, orThis command instructs R to load some data: the Arbuthnot baptism counts for boys and girls. You should see that the environment area in the upper righthand corner of the RStudio window now lists a data set called arbuthnot that has 82 observations on 3 variables. As you interact with R, you will create a series of objects. Sometimes you load them as we have done here, and sometimes you create them yourself as the byproduct of a computation or some analysis you have performed.
The Arbuthnot data set refers to the work of Dr. John Arbuthnot, an 18th century physician, writer, and mathematician. He was interested in the ratio of newborn boys to newborn girls, so he gathered the baptism records for children born in London for every year from 1629 to 1710. Once again, we can view the data by typing its name into the console.
head(arbuthnot)## year boys girls
## 1 1629 5218 4683
## 2 1630 4858 4457
## 3 1631 4422 4102
## 4 1632 4994 4590
## 5 1633 5158 4839
## 6 1634 5035 4820
However, printing the whole dataset in the console is not that useful. One advantage of RStudio is that it comes with a built-in data viewer. Click on the name arbuthnot in the Environment pane (upper right window) that lists the objects in your environment. This will bring up an alternative display of the data set in the Data Viewer (upper left window). You can close the data viewer by clicking on the x in the upper lefthand corner.
What you should see are four columns of numbers, each row representing a different year: the first entry in each row is simply the row number (an index we can use to access the data from individual years if we want), the second is the year, and the third and fourth are the numbers of boys and girls baptized that year, respectively. Use the scrollbar on the right side of the console window to examine the complete data set.
Note that the row numbers in the first column are not part of Arbuthnot’s data. R adds them as part of its printout to help you make visual comparisons. You can think of them as the index that you see on the left side of a spreadsheet. In fact, the comparison to a spreadsheet will generally be helpful. R has stored Arbuthnot’s data in a kind of spreadsheet or table called a data frame.
You can see the dimensions of this data frame as well as the names of the variables and the first few observations by typing:
glimpse(arbuthnot)## Rows: 82
## Columns: 3
## $ year <int> 1629, 1630, 1631, 1632, 1633, 1634, 1635, 1636, 1637, 1638, 1...
## $ boys <int> 5218, 4858, 4422, 4994, 5158, 5035, 5106, 4917, 4703, 5359, 5...
## $ girls <int> 4683, 4457, 4102, 4590, 4839, 4820, 4928, 4605, 4457, 4952, 4...
It is better practice to type this command into your console, since it is not necessary code to include in your solution file.
This command should output the following
Rows: 82 Columns: 3 $ year
We can see that there are 82 observations and 3 variables in this dataset. The variable names are year, boys, and girls. At this point, you might notice that many of the commands in R look a lot like functions from math class; that is, invoking R commands means supplying a function with some number of arguments. The glimpse command, for example, took a single argument, the name of a data frame.
Let’s start to examine the data a little more closely. We can access the data in a single column of a data frame separately using a command like
arbuthnot$boys## [1] 5218 4858 4422 4994 5158 5035 5106 4917 4703 5359 5366 5518 5470 5460 4793
## [16] 4107 4047 3768 3796 3363 3079 2890 3231 3220 3196 3441 3655 3668 3396 3157
## [31] 3209 3724 4748 5216 5411 6041 5114 4678 5616 6073 6506 6278 6449 6443 6073
## [46] 6113 6058 6552 6423 6568 6247 6548 6822 6909 7577 7575 7484 7575 7737 7487
## [61] 7604 7909 7662 7602 7676 6985 7263 7632 8062 8426 7911 7578 8102 8031 7765
## [76] 6113 8366 7952 8379 8239 7840 7640
This command will only show the number of boys baptized each year. The dollar sign basically says “go to the data frame that comes before me, and find the variable that comes after me”.
Notice that the way R has printed these data is different. When we looked at the complete data frame, we saw 82 rows, one on each line of the display. These data are no longer structured in a table with other variables, so they are displayed one right after another. Objects that print out in this way are called vectors; they represent a set of numbers. R has added numbers in [brackets] along the left side of the printout to indicate locations within the vector. For example, 5218 follows [1], indicating that 5218 is the first entry in the vector. And if [43] starts a line, then that would mean the first number on that line would represent the 43rd entry in the vector.
# counts of girls baptized each year
(arbuthnot$girls)## [1] 4683 4457 4102 4590 4839 4820 4928 4605 4457 4952 4784 5332 5200 4910 4617
## [16] 3997 3919 3395 3536 3181 2746 2722 2840 2908 2959 3179 3349 3382 3289 3013
## [31] 2781 3247 4107 4803 4881 5681 4858 4319 5322 5560 5829 5719 6061 6120 5822
## [46] 5738 5717 5847 6203 6033 6041 6299 6533 6744 7158 7127 7246 7119 7214 7101
## [61] 7167 7302 7392 7316 7483 6647 6713 7229 7767 7626 7452 7061 7514 7656 7683
## [76] 5738 7779 7417 7687 7623 7380 7288
# statiscal summary of girls baptized
summary(arbuthnot$girls)## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 2722 4457 5718 5535 7150 7779
# total counts of girls baptized at all
sum(arbuthnot$girls)## [1] 453841
#observation with max girls births
filter(arbuthnot,(arbuthnot$girls==max(arbuthnot$girls)))## year boys girls
## 1 1705 8366 7779
#observation with max girls births
filter(arbuthnot,(arbuthnot$girls==min(arbuthnot$girls)))## year boys girls
## 1 1650 2890 2722
Ans: There appearrs to be an updaward trend in number of girls baptized/ year with min grils 2722 in year 1650 and max girls 7779 in year 1705
R has some powerful functions for making graphics. We can create a simple plot of the number of girls baptized per year with the command
plot(x = arbuthnot$year, y = arbuthnot$girls)ggplot(data = arbuthnot, aes(x = year, y = girls)) +
geom_point()We use the ggplot() function to build plots. If you run the plotting code in your console, you should see the plot appear under the Plots tab of the lower right panel of RStudio. Notice that the command above again looks like a function, this time with arguments separated by commas.
With ggplot():
aesthetic elements of the plot, e.g. the x and the y axes.+ to specify the geometric object for the plot. Since we want to scatterplot, we use geom_point().For instance, if you wanted to visualize the above plot using a line graph, you would replace geom_point() with geom_line().
ggplot(data = arbuthnot, aes(x = year, y = girls)) +
geom_line()You might wonder how you are supposed to know the syntax for the ggplot function. Thankfully, R documents all of its functions extensively. To learn what a function does and its arguments that are available to you, just type in a question mark followed by the name of the function that you’re interested in. Try the following in your console:
#?ggplotNotice that the help file replaces the plot in the lower right panel. You can toggle between plots and help files using the tabs at the top of that panel.
Now, suppose we want to plot the total number of baptisms. To compute this, we could use the fact that R is really just a big calculator. We can type in mathematical expressions like
5218 + 4683## [1] 9901
to see the total number of baptisms in 1629. We could repeat this once for each year, but there is a faster way. If we add the vector for baptisms for boys to that of girls, R will compute all sums simultaneously.
arbuthnot$boys + arbuthnot$girls## [1] 9901 9315 8524 9584 9997 9855 10034 9522 9160 10311 10150 10850
## [13] 10670 10370 9410 8104 7966 7163 7332 6544 5825 5612 6071 6128
## [25] 6155 6620 7004 7050 6685 6170 5990 6971 8855 10019 10292 11722
## [37] 9972 8997 10938 11633 12335 11997 12510 12563 11895 11851 11775 12399
## [49] 12626 12601 12288 12847 13355 13653 14735 14702 14730 14694 14951 14588
## [61] 14771 15211 15054 14918 15159 13632 13976 14861 15829 16052 15363 14639
## [73] 15616 15687 15448 11851 16145 15369 16066 15862 15220 14928
What you will see are 82 numbers (in that packed display, because we aren’t looking at a data frame here), each one representing the sum we’re after. Take a look at a few of them and verify that they are right.
We’ll be using this new vector to generate some plots, so we’ll want to save it as a permanent column in our data frame.
arbuthnot <- arbuthnot %>%
mutate(total = boys + girls)head(arbuthnot)## year boys girls total
## 1 1629 5218 4683 9901
## 2 1630 4858 4457 9315
## 3 1631 4422 4102 8524
## 4 1632 4994 4590 9584
## 5 1633 5158 4839 9997
## 6 1634 5035 4820 9855
The %>% operator is called the piping operator. It takes the output of the previous expression and pipes it into the first argument of the function in the following one. To continue our analogy with mathematical functions, x %>% f(y) is equivalent to f(x, y).
A note on piping: Note that we can read these two lines of code as the following:
“Take the arbuthnot dataset and pipe it into the mutate function. Mutate the arbuthnot data set by creating a new variable called total that is the sum of the variables called boys and girls. Then assign the resulting dataset to the object called arbuthnot, i.e. overwrite the old arbuthnot dataset with the new one containing the new variable.”
This is equivalent to going through each row and adding up the boys and girls counts for that year and recording that value in a new column called total.
Where is the new variable? When you make changes to variables in your dataset, click on the name of the dataset again to update it in the data viewer.
You’ll see that there is now a new column called total that has been tacked onto the data frame. The special symbol <- performs an assignment, taking the output of one line of code and saving it into an object in your environment. In this case, you already have an object called arbuthnot, so this command updatesthat data set with the new mutated column.
You can make a line plot of the total number of baptisms per year with the command
ggplot(data = arbuthnot, aes(x = year, y = total)) +
geom_line()Similarly to you we computed the total number of births, you can compute the ratio of the number of boys to the number of girls baptized in 1629 with
5218 / 4683## [1] 1.114243
or you can act on the complete columns with the expression
arbuthnot <- arbuthnot %>%
mutate(boy_to_girl_ratio = boys / girls)You can also compute the proportion of newborns that are boys in 1629
5218 / (5218 + 4683)## [1] 0.5270175
or you can compute this for all years simultaneously and append it to the dataset
arbuthnot <- arbuthnot %>%
mutate(boy_ratio = boys / total)arbuthnot## year boys girls total boy_to_girl_ratio boy_ratio
## 1 1629 5218 4683 9901 1.114243 0.5270175
## 2 1630 4858 4457 9315 1.089971 0.5215244
## 3 1631 4422 4102 8524 1.078011 0.5187705
## 4 1632 4994 4590 9584 1.088017 0.5210768
## 5 1633 5158 4839 9997 1.065923 0.5159548
## 6 1634 5035 4820 9855 1.044606 0.5109082
## 7 1635 5106 4928 10034 1.036120 0.5088698
## 8 1636 4917 4605 9522 1.067752 0.5163831
## 9 1637 4703 4457 9160 1.055194 0.5134279
## 10 1638 5359 4952 10311 1.082189 0.5197362
## 11 1639 5366 4784 10150 1.121656 0.5286700
## 12 1640 5518 5332 10850 1.034884 0.5085714
## 13 1641 5470 5200 10670 1.051923 0.5126523
## 14 1642 5460 4910 10370 1.112016 0.5265188
## 15 1643 4793 4617 9410 1.038120 0.5093518
## 16 1644 4107 3997 8104 1.027521 0.5067868
## 17 1645 4047 3919 7966 1.032661 0.5080341
## 18 1646 3768 3395 7163 1.109867 0.5260366
## 19 1647 3796 3536 7332 1.073529 0.5177305
## 20 1648 3363 3181 6544 1.057215 0.5139059
## 21 1649 3079 2746 5825 1.121267 0.5285837
## 22 1650 2890 2722 5612 1.061719 0.5149679
## 23 1651 3231 2840 6071 1.137676 0.5322023
## 24 1652 3220 2908 6128 1.107290 0.5254569
## 25 1653 3196 2959 6155 1.080095 0.5192526
## 26 1654 3441 3179 6620 1.082416 0.5197885
## 27 1655 3655 3349 7004 1.091371 0.5218447
## 28 1656 3668 3382 7050 1.084565 0.5202837
## 29 1657 3396 3289 6685 1.032533 0.5080030
## 30 1658 3157 3013 6170 1.047793 0.5116694
## 31 1659 3209 2781 5990 1.153901 0.5357262
## 32 1660 3724 3247 6971 1.146905 0.5342132
## 33 1661 4748 4107 8855 1.156075 0.5361942
## 34 1662 5216 4803 10019 1.085988 0.5206108
## 35 1663 5411 4881 10292 1.108584 0.5257482
## 36 1664 6041 5681 11722 1.063369 0.5153557
## 37 1665 5114 4858 9972 1.052697 0.5128359
## 38 1666 4678 4319 8997 1.083121 0.5199511
## 39 1667 5616 5322 10938 1.055242 0.5134394
## 40 1668 6073 5560 11633 1.092266 0.5220493
## 41 1669 6506 5829 12335 1.116143 0.5274422
## 42 1670 6278 5719 11997 1.097744 0.5232975
## 43 1671 6449 6061 12510 1.064016 0.5155076
## 44 1672 6443 6120 12563 1.052778 0.5128552
## 45 1673 6073 5822 11895 1.043112 0.5105507
## 46 1674 6113 5738 11851 1.065354 0.5158214
## 47 1675 6058 5717 11775 1.059647 0.5144798
## 48 1676 6552 5847 12399 1.120575 0.5284297
## 49 1677 6423 6203 12626 1.035467 0.5087122
## 50 1678 6568 6033 12601 1.088679 0.5212285
## 51 1679 6247 6041 12288 1.034100 0.5083822
## 52 1680 6548 6299 12847 1.039530 0.5096910
## 53 1681 6822 6533 13355 1.044237 0.5108199
## 54 1682 6909 6744 13653 1.024466 0.5060426
## 55 1683 7577 7158 14735 1.058536 0.5142178
## 56 1684 7575 7127 14702 1.062860 0.5152360
## 57 1685 7484 7246 14730 1.032846 0.5080788
## 58 1686 7575 7119 14694 1.064054 0.5155165
## 59 1687 7737 7214 14951 1.072498 0.5174905
## 60 1688 7487 7101 14588 1.054359 0.5132301
## 61 1689 7604 7167 14771 1.060974 0.5147925
## 62 1690 7909 7302 15211 1.083128 0.5199527
## 63 1691 7662 7392 15054 1.036526 0.5089677
## 64 1692 7602 7316 14918 1.039092 0.5095857
## 65 1693 7676 7483 15159 1.025792 0.5063659
## 66 1694 6985 6647 13632 1.050850 0.5123973
## 67 1695 7263 6713 13976 1.081931 0.5196766
## 68 1696 7632 7229 14861 1.055748 0.5135590
## 69 1697 8062 7767 15829 1.037981 0.5093183
## 70 1698 8426 7626 16052 1.104904 0.5249190
## 71 1699 7911 7452 15363 1.061594 0.5149385
## 72 1700 7578 7061 14639 1.073219 0.5176583
## 73 1701 8102 7514 15616 1.078254 0.5188268
## 74 1702 8031 7656 15687 1.048981 0.5119526
## 75 1703 7765 7683 15448 1.010673 0.5026541
## 76 1704 6113 5738 11851 1.065354 0.5158214
## 77 1705 8366 7779 16145 1.075460 0.5181790
## 78 1706 7952 7417 15369 1.072132 0.5174052
## 79 1707 8379 7687 16066 1.090022 0.5215362
## 80 1708 8239 7623 15862 1.080808 0.5194175
## 81 1709 7840 7380 15220 1.062331 0.5151117
## 82 1710 7640 7288 14928 1.048299 0.5117899
Note that we are using the new total variable we created earlier in our calculations.
Tip: If you use the up and down arrow keys, you can scroll through your previous commands, your so-called command history. You can also access it by clicking on the history tab in the upper right panel. This will save you a lot of typing in the future.
# make plot
# calculate proportion
boysProportion <- arbuthnot$boys / (arbuthnot$boys + arbuthnot$girls)
plot(arbuthnot$year, boysProportion, type = "l", ylim=c(0.5, 0.535))
abline(lm((arbuthnot$boys/(arbuthnot$boys + arbuthnot$girls)) ~ arbuthnot$year),col='red')Finally, in addition to simple mathematical operators like subtraction and division, you can ask R to make comparisons like greater than, >, less than, <, and equality, ==. For example, we can ask if the number of births of boys outnumber that of girls in each year with the expression
arbuthnot <- arbuthnot %>%
mutate(more_boys = boys > girls)This command adds a new variable to the arbuthnot dataframe containing the values of either TRUE if that year had more boys than girls, or FALSE if that year did not (the answer may surprise you). This variable contains a different kind of data than we have encountered so far. All other columns in the arbuthnot data frame have values that are numerical (the year, the number of boys and girls). Here, we’ve asked R to create logical data, data where the values are either TRUE or FALSE. In general, data analysis will involve many different kinds of data types, and one reason for using R is that it is able to represent and compute with many of them.
In the previous few pages, you recreated some of the displays and preliminary analysis of Arbuthnot’s baptism data. Your assignment involves repeating these steps, but for present day birth records in the United States. The data are stored in a data frame called present.
To get started, let’s take a peek at the data.
source('/DATA606/cuny_data606/Lab1/more/present.R')head(present)## year boys girls
## 1 1940 1211684 1148715
## 2 1941 1289734 1223693
## 3 1942 1444365 1364631
## 4 1943 1508959 1427901
## 5 1944 1435301 1359499
## 6 1945 1404587 1330869
To find the minimum and maximum values of columns, you can use the functions min and max within a summarize() call, which you will learn more about in the following lab. Here’s an example of how to find the minimum and maximum amount of boy births in a year:
present %>%
summarize(min = min(boys), max = max(boys))## min max
## 1 1211684 2186274
#years included in the data set
present$year## [1] 1940 1941 1942 1943 1944 1945 1946 1947 1948 1949 1950 1951 1952 1953 1954
## [16] 1955 1956 1957 1958 1959 1960 1961 1962 1963 1964 1965 1966 1967 1968 1969
## [31] 1970 1971 1972 1973 1974 1975 1976 1977 1978 1979 1980 1981 1982 1983 1984
## [46] 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999
## [61] 2000 2001 2002
#data frame dimensions
dim(present)## [1] 63 3
#data frame dimensions
names(present)## [1] "year" "boys" "girls"
summary(present)## year boys girls
## Min. :1940 Min. :1211684 Min. :1148715
## 1st Qu.:1956 1st Qu.:1799857 1st Qu.:1711405
## Median :1971 Median :1924868 Median :1831679
## Mean :1971 Mean :1885600 Mean :1793915
## 3rd Qu.:1986 3rd Qu.:2058524 3rd Qu.:1965538
## Max. :2002 Max. :2186274 Max. :2082052
summary(arbuthnot)## year boys girls total boy_to_girl_ratio
## Min. :1629 Min. :2890 Min. :2722 Min. : 5612 Min. :1.011
## 1st Qu.:1649 1st Qu.:4759 1st Qu.:4457 1st Qu.: 9199 1st Qu.:1.048
## Median :1670 Median :6073 Median :5718 Median :11813 Median :1.065
## Mean :1670 Mean :5907 Mean :5535 Mean :11442 Mean :1.071
## 3rd Qu.:1690 3rd Qu.:7576 3rd Qu.:7150 3rd Qu.:14723 3rd Qu.:1.088
## Max. :1710 Max. :8426 Max. :7779 Max. :16145 Max. :1.156
## boy_ratio more_boys
## Min. :0.5027 Mode:logical
## 1st Qu.:0.5118 TRUE:82
## Median :0.5157
## Mean :0.5170
## 3rd Qu.:0.5210
## Max. :0.5362
#Calculate mean and mdedian for both datasets
present_mean <- mean(present$boys + present$girls)
present_median <- median(present$boys + present$girls)
arbuthnot_mean <- mean(arbuthnot$boys + arbuthnot$girls)
arbuthnot_medeian<- median(arbuthnot$boys + arbuthnot$girls)
#putting values in a data frame
scale_compare_df <- data.frame(present_mean, present_median, arbuthnot_mean, arbuthnot_medeian)
scale_compare_df## present_mean present_median arbuthnot_mean arbuthnot_medeian
## 1 3679515 3756547 11441.74 11813
Ans: The counts of both datasets do not appear to be on same scale
plot(present$year, (present$boys/(present$boys + present$girls)),type = 'l')
abline(lm((present$boys/(present$boys + present$girls)) ~ present$year),col='blue')Ans: As it appears in above plot the proportion of boys to girls population is greater than 0.5 which holds up to Arbuthnot’s observation which is boys being born in greater proportion than girls.
arrange (for sorting). We can arrange the data in a descending order with another function: desc (for descending order). The sample code is provided below.present$total <- present$boys + present$girls
plot(present$year, present$total, type = 'l')#using subset function to find year with most number of total births
subset(present$year, present$boys + present$girls == max(present$boys + present$girls))## [1] 1961
#to view the data point (observation)
filter(present,present$girls+present$boys==max(present$girls+present$boys))## year boys girls total
## 1 1961 2186274 2082052 4268326
present %>%
arrange(desc(total))These data come from reports by the Centers for Disease Control. You can learn more about them by bringing up the help file using the command ?present.
That was a short introduction to R and RStudio, but we will provide you with more functions and a more complete sense of the language as the course progresses.
In this course we will be using the suite of R packages from the tidyverse. The book R For Data Science by Grolemund and Wickham is a fantastic resource for data analysis in R with the tidyverse. If you are googling for R code, make sureto also include these package names in your search query. For example, instead of googling “scatterplot in R”, google “scatterplot in R with the tidyverse”.
These cheatsheets may come in handy throughout the semester:
Note that some of the code on these cheatsheets may be too advanced for this course. However the majority of it will become useful throughout the semester.