R Dataframes part 1

Activity 9:

Dataframes

On an intuitive level, a data frame is like a matrix, with a two-dimensional rows-and columns structure. However, it differs from a matrix in that each column may have a different mode. For instance, one column may consist of numbers, and another column might have character strings. In this sense, just as lists are the heterogeneous analogs of vectors in one dimension, data frames are the heterogeneous analogs of matrices for two-dimensional data.

Creating Data Frames

#To begin, let’s take another look at our simple data frame example from Section 1.4.5:
kids <- c("RProgrammer","PythonProgrammer")
ages <- c(85212,6510)
d <- data.frame(kids,ages,stringsAsFactors=FALSE)
d # matrix-like viewpoint

-> assigning two vectors and creating a dataframe

The first two arguments in the call to data.frame() are clear: We wish to produce a data frame from our two vectors: kids and ages. However, that third argument, stringsAsFactors=FALSE requires more comment.

If the named argument stringsAsFactors is not specified, then by default, stringsAsFactors will be TRUE. (You can also use options() to arrange the opposite default.) This means that if we create a data frame from a character vector—in this case, kids—R will convert that vector to a factor. Because our work with character data will typically be with vectors rather than factors, we’ll set stringsAsFactors to FALSE. We’ll cover factors in Chapter 6.

Accessing Data Frames

Now that we have a data frame, let’s explore a bit. Since d is a list, we can access it as such via component index values or component names:

d[[1]]

[1] "RProgrammer"      "PythonProgrammer"

d$kids

[1] "RProgrammer"      "PythonProgrammer"

-> we can access the first column (kids) in two ways: by column name or by index

But we can treat it in a matrix-like fashion as well. For example, we can view column 1: view column one by index in different notation:

d[,1]

[1] "RProgrammer"      "PythonProgrammer"

This matrix-like quality is also seen when we take d apart using str():

str(d)

'data.frame':   2 obs. of  2 variables:
 $ kids: chr  "RProgrammer" "PythonProgrammer"
 $ ages: num  85212 6510

-> the str() function gives us the structure: - data.frame as structure - kids column is character - ages column is numeric

R tells us here that d consists of two observations—our two rows—that store data on two variables—our two columns.

Consider three ways to access the first column of our data frame above:d[[1]], d[,1], and d$kids. Of these, the third would generally considered to be clearer and, more importantly, safer than the first two. This better identifies the column and makes it less likely that you will reference the wrong column. But in writing general code—say writing R packages—matrix-like notation d[,1] is needed, and it is especially handy if you are extracting subdata frames.

Extended Example: Regression Analysis of Exam Grades Continued

print the working directory

getwd()

[1] "C:/Users/marxm/OneDrive/Documents/ProgrammingForData_R/acti9"

import the csv file and separate them by the comma our dataframe has headers

examsquiz <- read.csv("C:/Users/marxm/OneDrive/Documents/ProgrammingForData_R/acti9/ExamsQuiz.csv",sep=",",header=TRUE)

print the first 5 rows of the dataframe

head(examsquiz)

Other Matrix-Like Operations

Various matrix operations also apply to data frames. Most notably and usefully, we can do filtering to extract various subdata frames of interest.

Extracting Subdata Frames As mentioned, a data frame can be viewed in row-and-column terms. In particular, we can extract subdata frames by rows or columns. Here’s an example:

print the second to fifth rows of all column

examsquiz[2:5,]

return second to fifth rows of second column

examsquiz[2:5,2]

[1] 3.2 2.0 4.0 2.0

return the class of the second to fifth rows of second column -> numeric

class(examsquiz[2:5,2])

[1] "numeric"

return the class of the second to fifth rows of second column with headers

examsquiz[2:5,2,drop=FALSE]

return the class of the second to fifth rows of second column with headers -> data.frame

class(examsquiz[2:5,2,drop=FALSE])

[1] "data.frame"

Note that in that second call, since examsquiz[2:5,2] is a vector, R created a vector instead of another data frame. By specifying drop=FALSE, as described for the matrix case in Section 3.6, we can keep it as a (onecolumn) data frame.

We can also do filtering. Here’s how to extract the subframe of all students whose first exam score was at least 3.8:

return all rows and column where the exam1 score is higher than 3.8:

examsquiz[examsquiz$Exam1 >= 3.8,]

More on Treatment of NA Values

Suppose the second exam score for the first student had been missing. Then we would have typed the following into that line when we were preparing the data file:

#2.0 NA 4.0

In any subsequent statistical analyses, R would do its best to cope with the missing data. However, in some situations, we need to set the option na.rm=TRUE, explicitly telling R to ignore NA values. For instance, with the missing exam score, calculating the mean score on exam 2 by calling R’s mean() function would skip that first student in finding the mean. Otherwise, R would just report NA for the mean.

Here’s a little example:

create X with three values:

x <- c(2,NA,4)
mean(x)

[1] NA

-> mean is NA since it cannot calculate mean with an NA value

remove na for mean calculation:

mean(x,na.rm=TRUE)

[1] 3

-> na is removed and mean is calculated

In Section 2.8.2, you were introduced to the subset() function, which saves you the trouble of specifying na.rm=TRUE. You can apply it in data frames for row selection. The column names are taken in the context of the given data frame. In our example, instead of typing this:

same as before:

examsquiz[examsquiz$Exam1 >= 3.8,]

same result, different method -> subset the examsquiz dataframe

subset(examsquiz,Exam1 >= 3.8)

NA
NA