Data Manipulation

climate01.csv

climate02.csv

A data frame is a very important data type in R.

It's pretty much the de facto data structure for most tabular data and what we use for statistics.

Data frames are usually read in from a file, but R comes with many practice datasets.

We will use the iris dataset, famously used by Fisher in 1936 (http://en.wikipedia.org/wiki/Iris_flower_data_set)

• head() - see first 5 rows
• tail() - see last 5 rows
• dim() - dimensions (# rows, # columns)
• nrow() - number of rows
• ncol() - number of columns
• str() - structure of any object
• class() - class of any object
• rownames() - row names
• colnames() - column names

1. How many rows does the iris data have?

2. How many columns? What are the column names?

3. Using the str() function, how many species are in the data?

4. What classes are each of the columns?

R has many powerful subset operators and mastering them will allow you to easily perform complex operation on any kind of dataset. Dataframes are akin to a series of vectors combined into a tabular structure.

head(iris)

Index individual elements of a data frame with nameofdf[row#, column#]

iris[1,1] # first row, first column

[1] 5.1

iris[3,3] # third row, third column

[1] 1.3

Get entire rows or columns

iris[2,] # the second row 
iris[,3] # the third column

Columns can also be called by their name

iris[,"Sepal.Length"]

Use the dollar sign $ to call a column

iris$Sepal.Length

Three ways to get the first column Sepal.Length.

iris[,1]

iris[,"Sepal.Length"]

iris$Sepal.Length

Get the 5th, 7th, and 9th rows and the first two columns.

iris[c(5,7,9), 1:2]

  Sepal.Length Sepal.Width
5          5.0         3.6
7          4.6         3.4
9          4.4         2.9

1. What is the 9th entry of the Sepal.Width column? Call it x.

2. How would you get all entries of iris for the 17th row?

3. Return an object with the 1st, 4th and 7th rows of the iris dataframe?

4. Use the seq() function to get all odd rows in the iris dataset?

5. What happens when you use negative numbers? Hint: Use dim() on the original and final objects.

x <- iris$Petal.Width

logi <- x > 1 # Which values are greater than 1?

a <- iris[logi,] # Records where Petal.Width >1

x <- iris$Petal.Width

logi <- x > 1 # Which values are greater than 1?

a <- iris[logi,] # Records where Petal.Width >1

This is the same as:

iris[iris$Petal.Width > 1, ]

Return a different column based on the subset

x <- iris$Petal.Width

logi <- x > 1 # Which values are greater than 1?

# Return all values in Petal.Length where Sepal.Width is greater than 1. 

iris[logi,"Petal.Length"]

1. Why is iris[iris>3,] a nonsensical command?

2. What about iris[iris$Sepal.Length>3]?

3. Create a histogram of petal lengths for the entire data.

4. Subset the data for petal lengths greater than two.

5. Create a histogram of your new data.

Use the data.frame function.

df <- data.frame(x=1:5, y=6:2)
df

  x y
1 1 6
2 2 5
3 3 4
4 4 3
5 5 2

Use the assignment operator <-

df$z <- 41:45
df

  x y  z
1 1 6 41
2 2 5 42
3 3 4 43
4 4 3 44
5 5 2 45

rm() does not work to remove a single column of a data frame.

df$z <- NULL
df

  x y
1 1 6
2 2 5
3 3 4
4 4 3
5 5 2

Assign NULL to the column. This is not reversible.

Many different types of logical statements can be used to subset data.

For all types of data, if we want a specific value/character/factor we use ==

Only4 <- iris[iris$Petal.Length == 4,]

versicolor_only <- iris[iris$Species == "versicolor",]

Use & (AND) if you want both statements to be true.
Use | (OR) if you want either statement to be true.

subsetX <- iris[iris$Petal.Length > 4 & iris$Species == "versicolor",]

Explain in words each of the following logical statements

1. iris[1:4,]

2. iris[c(1:15),c(1,3)]

3. iris[iris$Species == "setosa", "Petal.Width"]

4. What happens when you add a ! before a logical statment?

Hint: Compare
iris[iris$Species == "setosa",]
iris[!iris$Species == "setosa",]

Similar to data frames.

Two-dimensional and only consists of numbers.

Some functions require a matrix as an input.

a <- matrix(1:9, ncol=3, nrow=3)

Similar indexing as a data frame

a <- matrix(1:9, ncol=3, nrow=3)
a

     [,1] [,2] [,3]
[1,]    1    4    7
[2,]    2    5    8
[3,]    3    6    9

a[2,2]

[1] 5

a[2,]

[1] 2 5 8

R is not a spreadsheet program, so it is not great for direct data entry.

Therefore, for many analyses, you will want to import data.

.csv files are often the preferred format.

Before we do that, we will need to look at concept of your working directory.

Find out what your current working directory is

getwd()

[1] "C:/Users/Mike/Desktop/Dropbox/R_course/Mike"

This is the folder on your computer where R will look to open or write files.

Set your working directory

getwd()

setwd("C:/Users/Mike/Desktop/Dropbox/R_course/data")

# relative paths also work 
setwd("./data") 

See everything currently in your working directory

list.files()

Manually move the climate data that you downloaded to your working directory.

data01 <- read.csv("climate01.csv")
head(data01)

read.table() is an alternative function for importing data.

One of the largest sources of frustration with R can be importing data. Variable names often cause problems.

• Do not have spaces in variable names
  
• Use lower case letters
  
• Abbreviate when possible
  

Average Height # BAD
Average.Height # OK   
average.height # BETTER  
avg.height     # GOOD!  

1. Import climate01.csv & climate02.csv. Save them as separate data frames.
2. Compare these two datasets and identify any errors in climate02.csv

# write the file 
write.csv(iris,"iris.csv", row.names=FALSE)

# check and see
list.files()

Worksheet

Answers