Example: Assigning Folds

Let’s create a smaller data set to work with to illustrate the code.

Assigning rows to folds is like assigning people to tables as they enter a room. We have 20 people who will enter a room, and 5 tables for them to sit at. We want the same number of people at each table. To achieve this, we fill a box with 20 slips of paper. Each paper contains a number: 1, 2, 3, 4, or 5. Each person chooses a slip of paper, and the number on the paper tells the person which table to sit at.

Now, in R we aren’t actually creating slips of paper, but that is the idea. To do this in R, we are going to create a vector rather than a box to hold our table (fold) numbers. To do this, use the following:

papers <- rep(1:5, 4)

The command rep in R means “repeat”. So, this command means “Repeat the numbers 1 to 5 four times.”

In this code, we create an object called papers. This vector holds the numbers 1 through 5 each repeated 4 times.

Now, we have created our box full of papers. The next step is to have each person (each row) draw a slip of paper (a fold number) as they enter the room. To do this, we will use sampling without replacement.

set.seed(363663)
foldsSmall <- sample(papers, 20, replace = FALSE)

Recall that with the sample command, the command structure is sample(What to Sample From, How Many to Sample, Do we sample with or without replacement).

The vector foldsSmall that we have just created indicates which row in our 20 row data set belongs to each of the 5-folds. The first number in foldsSmall tells us the fold that the first row of the data is assigned to, and so on.

Back to the Penguin Data

Okay, now that we have seen an example of how to assign the rows in a data set to folds, let’s try it with the \(n=333\) rows in the penguins data set using 10-fold CV.

At this point, we have determined which rows in the data set belong to each fold, but we have not actually created the separate data sets defined by each fold. In other words, we have a vector indicating which row goes in each fold, but we have not created separate data sets for each fold yet. Let’s do that now.

Example: Creating the separate data sets

Let’s go back our smaller data set of 20 rows. We have already created the vector foldsSmall which indicates the folds each row is assigned to. Now, let’s actually figure out which rows are in each fold.

We are going to start with fold 1. Which rows in penguinsSmall are in fold 1?

which(foldsSmall == 1)

Now that we know which rows are in a fold, we want to actually create a data set that contains those rows. In other words, we want to actually create the different data sets.

To create fold 1, this involves taking all the rows from penguinsSmall that are in fold 1 and pulling them from the larger data set.

fold1 <- penguinsSmall[which(foldsSmall==1), ]
fold1

Okay, so now we can actually create the folds we need! Let’s go back our larger data set.

Back to the Penguin Data

Let’s use the \(n=333\) rows in the penguins data set using 10-fold CV.

Example: Inside the loop

The purpose of our for loop is to create the vector \(\hat{Y}\) of predictions for penguin sex on the penguinsSmall data set. This means that the first thing we need to do is create a data frame to store that \(\hat{Y}\) in.

storageSmall <- data.frame("YHat" = rep(NA,20))

We also need to load the library we will need for 5-nearest neighbors.

library(class)

Once this is done, we can start thinking about the loop. Whenever you are writing for loops with a lot of steps, it helps to write the steps that will go inside the loop before you try to build the loop structure. This means that we:

1. Set f = 1

f <- 1

2. Set fold 1 as test data and the rest of the folds as training data
3. Run 5-nearest neighbors to predict penguin sex for fold 1.
4. Store the predictions.

Building the inside first makes it easier to catch any errors. This in essence shows us what each part of the loop will do. Once it does what we want, we can write the loop!

Back to the Penguin Data

Let’s use the \(n=333\) rows in the penguins data set using 10-fold CV.

Turning in your assignment

When your Markdown document is complete, do a final knit and make sure everything compiles. You will then submit your document on Canvas. You must submit a PDF or html document to Canvas. No other formats will be accepted. Make sure you look through the final document to make sure everything has knit correctly.