Info

Objective

The purpose of writing graded lab reports is to help students to stay on track and to provide summative feedback. Each lab report is just 1% of the total course mark. Please do not cheat - it is not worth it!

Your task

Solve the practical questions, knit your document into a PDF and submit to NTULearn before the deadline. The deadline is very tight because the task is simple. We are sure that everyone is capable to do it by themselves and we want to discourage taking someone else’s report and writing it with your own words.

Marking Scheme

The marking scheme is available on NTULearn.

Deadline

16 Aug 2026, midnight

Libraries

Here, we load libraries and set the random seed. Please change the random seed to the last four digits of your matriculation number.

library(tidyverse) # for manipulation with data
## ── Attaching core tidyverse packages ──────────────────────── tidyverse 2.0.0 ──
## ✔ dplyr     1.2.1     ✔ readr     2.2.0
## ✔ forcats   1.0.1     ✔ stringr   1.6.0
## ✔ ggplot2   4.0.3     ✔ tibble    3.3.1
## ✔ lubridate 1.9.5     ✔ tidyr     1.3.2
## ✔ purrr     1.2.2     
## ── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
## ✖ dplyr::filter() masks stats::filter()
## ✖ dplyr::lag()    masks stats::lag()
## ℹ Use the conflicted package (<http://conflicted.r-lib.org/>) to force all conflicts to become errors
library(ISLR) # for datasets from 'Introduction to Statistical Learning'
library(caret) # for machine learning, including KNN
## Loading required package: lattice
## 
## Attaching package: 'caret'
## 
## The following object is masked from 'package:purrr':
## 
##     lift
set.seed(1729)

Question 1

When we train a KNN model, an important step is choosing the optimal value of \(K\). Is it a good idea to try all values \(K=1,2,\dots,M_0\), where \(M_0\) is some parameter depending on \(N\) (the number of data points) and \(p\) (the number of predictors), and choose the value of \(K\) that minimizes the training error? Justify your answer.

Answer: This is a bad idea because, by definition, the training error of the KNN model with \(K=1\) is always 0. Indeed, the nearest neighbour to any training data point is the point itself, i.e., the prediction of the KNN model with \(K=1\) on any training data point is the observed value of the response variable. Using this method will just always result in \(K=1\) and that might not be optimal.

Question 2

Start with the dataset Hitters from the library ISLR.

Remove all observations for which Salary is missing.

Compute two new variables:

Split the data into 70% training and 30% test data.

Keep only Salary, BattingAverage, CareerBattingAverage, HmRun, Runs, RBI, Walks, and Years.

Print dimensions of the training dataset and the test dataset.

# Correct answer:

hitters_data <- Hitters %>%
  drop_na(Salary) %>%
  mutate(BattingAverage = Hits / AtBat, CareerBattingAverage = CHits / CAtBat) %>%
  select(Salary, BattingAverage, CareerBattingAverage,
    HmRun, Runs, RBI, Walks, Years)

ind <- runif(nrow(hitters_data)) < 0.7

train_data <- hitters_data %>% filter(ind)

test_data <- hitters_data %>% filter(!ind)

cat("Dim (training data) =", dim(train_data), "\n")
cat("Dim (test data) =", dim(test_data), "\n")
## Dim (training data) = 196 8 
## Dim (test data) = 67 8

Question 3

First, we will create a function that calculates the mean absolute error of a vector of predicted values vs a vector of reference values:

mae <- function(predicted_values, reference_values) {
  (predicted_values - reference_values) %>%
    abs() %>%
    mean()
}

Now train a KNN regression model with \(K=15\) on the training dataset and find its error on the test set.

Remember that KNN is based on distances, so the predictors should be normalized before fitting the model.

Modify the code below:

# Here is the correct answer:
knn_mod <- train(
  Salary ~ .,
  method = "knn",
  trControl = trainControl("none"),
  tuneGrid = expand.grid(k = 15),
  preProcess = c("range"),
  # preProcess = c("scale", "center"), # this will work too
  data = train_data
)

preds <- predict(knn_mod, test_data)

mae(preds, test_data$Salary)
## [1] 259.8752

Question 4

Train a set of KNN models with \(K=3,5,7,\dots,25\) on the training data and report the mean absolute error of every one of them on the test data.

# Make the table of errors here and print it

knn_model_test_error <- function(K) {
  # Input: K
  # Output: test error of the KNN model
  knn_mod <- train(
    Salary ~ .,
    method = "knn",
    trControl = trainControl("none"),
    tuneGrid = expand.grid(k = K),
    preProcess = c("range"),
    data = train_data
  )
  
  preds <- predict(knn_mod, test_data)
  
  mae(preds, test_data$Salary)
}

k_values <- seq(from = 3, to = 25, by = 2)

table_of_errors <- k_values %>%
  enframe(value = "k") %>%
  mutate(test_error = map_vec(k, knn_model_test_error))

# Here is the table of errors
table_of_errors

Write a single R command that prints the value of \(K\) that minimizes the mean absolute error. Your R command should work with any input data, i.e., you cannot just look at the table above, find the smallest error, and print the corresponding value of \(K\).

# Write your command here

table_of_errors %>%
  slice(which.min(test_error)) %>%
  select(k, test_error)

Declaration of Generative AI usage

Modify the following:

I used ChatGPT 5.6 to modify the 2025 version of this lab handout - specifically, to replace questions based on the Carseats dataset with similar questions based on the Hitters dataset

Fedor Duzhin