R Dataframes part 1

Julius Schmid

Dataframes

On an intuitive level, a data frame is like a matrix, with a two-dimensional rows-and-columns structure. However, it differs from a matrix in that each column may have a different mode. For instance, one column may consist of numbers, and another column might have character strings. In this sense, just as lists are the heterogeneous analogs of vectors in one dimension, data frames are the heterogeneous analogs of matrices for two-dimensional data.

Creating Data Frames

To begin, let’s take another look at our simple data frame example from Section 1.4.5:

kids <- c("Colin","Cosmo")
ages <- c(14,16)
d <- data.frame(kids,ages,stringsAsFactors=FALSE)
d # matrix-like viewpoint

The first two arguments in the call to data.frame() are clear: We wish to produce a data frame from our two vectors: kids and ages. However, that third argument, stringsAsFactors=FALSE requires more comment.

If the named argument stringsAsFactors is not specified, then by default, stringsAsFactors will be TRUE. (You can also use options() to arrange the opposite default.) This means that if we create a data frame from a character vector — in this case, kids — R will convert that vector to a factor. Because our work with character data will typically be with vectors rather than factors, we’ll set stringsAsFactors to FALSE. We’ll cover factors in Chapter 6.

Accessing Data Frames

Now that we have a data frame, let’s explore a bit. Since d is a list, we can access it as such via component index values or component names:

In this example, we want to return the first column. We can do this either by using the column index (df[[index]]) or by calling the specific column name (df[[column_name]]). In this case, the column index is 1, and the corresponding column name is kids.

d[[1]] # use column index

[1] "Colin" "Cosmo"

d$kids # use column name

[1] "Colin" "Cosmo"

But we can treat it in a matrix-like fashion as well. For example, we can view column 1. We do this by using the syntax df[desired_rows, desired_columns]. If we want to print all rows or all columns, we leave the corresponding spot empty. So, with the call d[2,] we would return the second row, and with the follwing call we can return the first column:

d[,1]

[1] "Colin" "Cosmo"

This matrix-like quality is also seen when we take d apart using the structure function str():

str(d)

'data.frame':   2 obs. of  2 variables:
 $ kids: chr  "Colin" "Cosmo"
 $ ages: num  14 16

R tells us here that d consists of two observations — our two rows — that store data on two variables — our two columns.

Consider three ways to access the first column of our data frame above:d[[1]], d[,1], and d$kids. Of these, the third would generally considered to be clearer and, more importantly, safer than the first two. This better identifies the column and makes it less likely that you will reference the wrong column. But in writing general code — say writing R packages — matrix-like notation d[,1] is needed, and it is especially handy if you are extracting subdata frames.

Extended Example: Regression Analysis of Exam Grades Continued

Let us print out the working directory first:

getwd()

[1] "/cloud/project"

Since this notebook is created in RStudio Cloud, the working directory is just “/cloud/project”.

Next, we download the examsquiz.csv file from Canvas, upload it to RStudio Cloud and import it, using the read.csv() function:

examsquiz <- read.csv("ExamsQuiz.csv",sep=",",header=TRUE)

In order to see what the examsquiz data frame looks like, we return the first 10 rows, using the head() funtion with input argument 10:

head(examsquiz,10)

We observe that there are three columns: Exam1, Exam2, and Quiz.

Other Matrix-Like Operations

Various matrix operations also apply to data frames. Most notably and usefully, we can do filtering to extract various subdata frames of interest.

Extracting Subdata Frames

As mentioned, a data frame can be viewed in row-and-column terms. In particular, we can extract subdata frames by rows or columns. Here’s an example, returning the rows 4-8 of the examsquiz data frame:

examsquiz[4:8,]

Now, we are not interested in the results of students 4-8 for all tests, but only for the Quiz:

examsquiz[4:8,3]

[1] 4 2 3 4 2

R returns an array in this case, not a single-column data frame! We confirm this by returning the class of our just generated output:

class(examsquiz[4:8,3])

[1] "numeric"

In order to return our filtered output as a data frame, we set the optional input argument “drop” to FALSE, which guarantees us that the filtered data will be returned as a data frame, as well.

examsquiz[4:8,3,drop=FALSE]

Again, return the class of our recent output, when setting the drop parameter to FALSE:

class(examsquiz[4:8,3,drop=FALSE])

[1] "data.frame"

Indeed, the filtered data is still interpreted as a data frame now.

Note that in that second call, since examsquiz[4:8,3] is a vector, R created a vector instead of another data frame. By specifying drop=FALSE, as described for the matrix case in Section 3.6, we can keep it as a (onecolumn) data frame.

We can also do filtering. Here’s how to extract the subframe of all students whose first exam score was at least 3.6:

examsquiz[examsquiz$Exam1 >= 3.6,]

More on Treatment of NA Values

Suppose the second exam score for the first student had been missing. Then we would have typed the following into that line when we were preparing the data file:

#2.0 NA 3.0

In any subsequent statistical analyses, R would do its best to cope with the missing data. However, in some situations, we need to set the option na.rm=TRUE, explicitly telling R to ignore NA values. For instance, with the missing exam score, calculating the mean score on exam 2 by calling R’s mean() function would skip that first student in finding the mean. Otherwise, R would just report NA for the mean.

Here’s a little example:

Create an array x, containing a NA component:

x <- c(1,NA,9)
mean(x)

[1] NA

The mean is determined as NA, since NA + x = NA for all real numbers x. We cannot perform useful arithmetic operations, taking NA into consideration. This is why we have to ignore NA values for these calculations, setting na.rm = TRUE:

mean(x,na.rm=TRUE)

[1] 5

In Section 2.8.2, you were introduced to the subset() function, which saves you the trouble of specifying na.rm=TRUE. You can apply it in data frames for row selection. The column names are taken in the context of the given data frame. In our example, instead of typing this:

examsquiz[examsquiz$Exam1 >= 3.6,]

We could just use the subset function instead, applying the same filter, and getting the same output:

subset(examsquiz,Exam1 >= 3.6)

NA
NA