Assigning Fold Numbers

Assigning rows to folds is like assigning people to tables as they enter a room. We have 20 people who will enter a room, and 5 tables for them to sit at. We want the same number of people at each table. To achieve this, we fill a box with 20 tickets. Each ticket contains a number: 1, 2, 3, 4, or 5. Each person chooses a ticket, and the number on the ticket tells the person which table to sit at.

Now, in R we aren’t actually creating tickets, but that is the idea. To do this in R, we are going to create a vector rather than a box to hold our tickets. To do this, use the following:

tickets <- rep(1:5, 4)

The command rep in R means “repeat”. So, this command means “Repeat the numbers 1 to 5 four times”. Take a look at what is in the vector.

tickets

tickets_Q4 <- 

Now that we have we have created our tickets, the next step is to have each person (each row) draw a ticket (a fold number) as they enter the restaurant. To do this, we will use sampling without replacement.

# Set the random seed
set.seed(363663)

# Draw tickets from the box
foldsSmall <- sample(tickets, 20, replace = FALSE)

Recall that with the sample command, the command structure is sample(What to Sample From, How Many to Sample, Do we sample with or without replacement). We are sampling from the box of tickets, 20 people are drawing out a ticket, and they do not put their ticket back once they draw it.

The vector foldsSmall that we have just created indicates which row in our 20 row data set belongs to each of the 5-folds. The first number in foldsSmall tells us the fold that the first row of the data is assigned to, and so on.

At this point, we have determined which rows in the penguinsSmall data set belong to each fold, but we have not actually created the separate data sets defined by each fold. In other words, each penguin is holding a ticket so they know which table number they need to go to, but we have not actually moved the penguins to their tables. Let’s do that now.

Moving rows into folds (Creating the separate data sets)

We have already created the vector foldsSmall which indicates the folds each row is assigned to. Now, let’s actually figure out which rows are in each fold.

We are going to start with fold 1. Which rows in penguinsSmall are in fold 1? To determine this, we use the following code:

which(foldsSmall == 1)

In R, == looks to see if two things match. So, 2 ==3 will return FALSE and 2==2 will return TRUE.

Now that we know which rows are in a fold, we want to actually create a data set that contains those rows. To create fold 1, this involves taking all the rows from penguinsSmall that are in fold 1 and storing them as a data set called fold1.

fold1 <- penguinsSmall[which(foldsSmall==1), ]

Okay, so now we can actually create the folds we need! Let’s go back our larger data set.

Back to the Penguin Data

Okay, now that we have seen an example of how to assign the rows in a data set to folds, let’s try it with all \(n=333\) rows in the penguins data set using 10-fold CV.

Nested For Loops

A Nested for-loop looks like this:

# Outer Loop
for( firstindex in 1:something){

  # Inner Loop 
  for( secondindex in 1:something){
  
 
  }

}

Let’s suppose for each number 1 - 5 I want to add the numbers 1 - 10. To do this using a nested for-loop we use the following:

# Outer Loop
for( i in 1:5){

  # Inner Loop 
  for( j in 1:10){
    
    print( i + j)
 
  }

}

Look over this code and make sure you know exactly what it is doing. If you have any questions, make sure you ask before you move to the next piece of the lab!!

Nested For Loop with KNN

We are now going to use a nested for-loop to determine what value of \(K\) we should choose for KNN. This basically means that we want to take our code from Question 14, which uses a for loop to use 10-fold with 5-NN, and now we want to use a 2nd for loop to allow the number of neighbors to change! Generally, I start with the inner loop and then think about the outer loop as wrapping around that inner loop and repeating it a lot of times.

Typically, the trickiest part of nested for-loops is deciding how to store the results that you need. Sometimes, there are things that you need to store inside the inner loop that do not actually need to be output at the end of the outer loop.

For instance, in our case we need to perform 10-fold CV in the inner loop, which means we need to create a vector to store all of our predicted values of \(Y\). This is not something we need at the end of the outer loop - we only need the validation RMSE for each choice of \(K\) in the outer loop.

This means we need to create two data frames:

# Create a storage space to hold the value of K and the
# validation RMSE for that choice of K

DATAFRAME 1 HERE

# Outer Loop
for( k in   ){

  # Create a storage space to hold the predictions from 
  # 10-fold CV
  
  DATAFRAME 2 HERE

  # Inner Loop 
  for( f in  ){
    
   
 
  }

}

Those are the hardest parts, but now, it’s time to fill in the rest.

Turning in your assignment

When your Markdown document is complete, do a final knit and make sure everything compiles. You will then submit your document on Canvas. You must submit a PDF or html document to Canvas. No other formats will be accepted. Make sure you look through the final document to make sure everything has knit correctly.