Purpose: Use labeled air-quality data to train, test, visualize, and compare models. You will first predict numeric ozone, then classify the categorical outcome goodOzone.

  • Framework: TRACE: Task, Read, Arrange, Create, Evaluate
  • Models: KSVM, SVM, linear regression, and Naive Bayes
  • Evidence: RMSE, percent accuracy, and comparable visualizations

1 Before You Begin

Lecture connection: SVM is a supervised-learning method. It learns a relationship between input variables and a known outcome from labeled training data, then applies that relationship to new cases. Unsupervised learning begins without a known outcome label and searches for structure such as clusters.

1.1 Learning objectives

By the end of this lab, you should be able to:

  • Distinguish supervised from unsupervised learning.
  • Investigate variables before choosing an outcome and predictors.
  • Prepare one reproducible training/test split.
  • Fit numeric and categorical prediction models.
  • Evaluate numeric predictions with RMSE.
  • Evaluate classifications with percent accuracy.
  • Justify a model choice using numerical and visual evidence.

1.2 Coding expectations

Some code is provided to establish the workflow and required object names. Complete every section marked Your code. Do not simply copy code without understanding what each object represents.

1.3 Required packages

# Install packages only if needed.
# install.packages(c("kernlab", "e1071", "ggplot2", "gridExtra"))

# Two packages are loaded for you.
library(kernlab)
library(ggplot2)

# Your code: load e1071 and gridExtra.
library(e1071)
library(gridExtra)

2 T — Part 1. Define the Task

Task: Predict numeric ozone and compare KSVM, SVM, and linear regression. Then create goodOzone and compare KSVM, SVM, and Naive Bayes classifiers.

2.1 1.1 Investigate the variables before answering

Do not select the outcome only from the lab title. First inspect the dataset and use the variable names, structures, and summaries as evidence.

# Load the built-in Air Quality data.
data("airquality")
AirQuality <- airquality

# Rename Ozone to ozone so the name matches this lab.
names(AirQuality)[names(AirQuality) == "Ozone"] <- "ozone"

# The first inspection command is provided.
head(AirQuality)

# Your code: list all variable names.
list(AirQuality)
# Your code: inspect the structure and data types.
str(AirQuality)

# Your code: summarize the distributions of all variables.
summary(AirQuality)

2.1.1 Variable investigation notes

1. Which variables are numeric?
Wind, Solar.R, Temp, Month, Day and ozone are numeric variables.

2. Which variables could serve as predictors?
Temp, Solar.R, Wind, Month, and Day may serve as variable predictors.

3. Which variable represents the quantity the models are asked to predict? Use evidence from your investigation.
The investigation shows that ozone is the variable the model predicts. Evidence from my investigation is that Solar.R, Wind, Temp, Month, and Day were used as predictor variables. It helps show how accurate the input variables and the model’s assumptions are.

2.2 1.2 Concept questions

What is the numeric outcome variable?
The numeric outcome variable is the quantitative measure used for analysis. In this study, ozone is that variable, as the model predicts it.

Why is this supervised learning?
This is supervised learning because it uses a known outcome variable (ozone) to see how variables relate and to build a model that can predict future values.

How does classification differ from numeric prediction?
Classification is different from numeric prediction because it puts observations into separate categories instead of estimating continuous values.

# Complete the comments in your own words.
# The regression models learn the relationship between independent variables and the dependent variable.
# The classification models learn to place observations into categories.
# The models apply what they learned to make predictions from new input data.

3 R — Part 2. Read and Prepare the Data

3.1 2.1 Inspect the data again

# Two inspection commands are provided.
head(AirQuality)
str(AirQuality)

# Your code: display the variable names.
names(AirQuality)

# Your code: display summary statistics.
summary(AirQuality)

3.2 2.2 Check missing values

# Count missing values in each column.
missing_before <- colSums(is.na(AirQuality))
missing_before

# Your response: Which variables contain missing values?
# The variables ozone = 37 and Solar.R = 7 contain missing values.

3.3 2.3 Replace missing values with column medians

The function below is partially provided. Complete the missing expression so each NA is replaced by the median of its own column.

AirQuality[] <- lapply(
  AirQuality,
  function(x) ifelse(
    is.na(x),
    # Your code: calculate the median while removing NA values.
    median(x,na.rm = TRUE),
    x
  )
)

# Your code: verify that no missing values remain.
any(is.na(AirQuality))

Why must missing values be handled before fitting the models?
Missing values should be handled before fitting the models because they can change how the model’s settings are set and cause wrong predictions.

4 A — Part 3. Arrange Training and Test Data

Why this matters: A model learns from training data and is evaluated on separate test data. Every model must use the same split for a fair comparison because they are tested on the same data..

4.1 3.1 Create a reproducible split

# A reproducible seed is provided.
set.seed(11)

# Calculate the number of training observations using 2/3.
training_size <- floor(nrow(AirQuality) * 2/3)

# Your code: randomly sample training-row indices.
train_index <- sample(nrow(AirQuality), training_size)

# Create the training data.
train <- AirQuality[train_index, ]

# Your code: create test data from rows not selected for training.
test <- AirQuality[-train_index, ]

# Verify the split.
nrow(AirQuality)
nrow(train)
nrow(test)
nrow(train) + nrow(test)

How many rows are in training?
The training dataset contains 102 rows.

How many rows are in testing?
There are 51 rows in testing dataset.

Why is set.seed(11) important?
The set.seed(11) is important because it makes the random division of data into training and testing parts repeatable, giving the same results every time the code is run..

4.2 3.2 Checkpoint

# These checks should run without an error.
stopifnot(sum(is.na(AirQuality)) == 0)
stopifnot(nrow(train) + nrow(test) == nrow(AirQuality))

# Your code: verify that ozone exists in both train and test.
stopifnot(exists("train")) 
stopifnot(exists("test"))

5 C/E — Part 4. Numeric Prediction

Evaluation rule: Use Root Mean Squared Error (RMSE). Lower RMSE means predicted ozone values are generally closer to observed values.

5.1 4.1 KSVM regression

Use the matrix method and kpar = "automatic". This avoids passing incompatible kernel parameters to rbfdot().

# Predictor names are defined for you.
regression_predictors <- setdiff(names(train), "ozone")

set.seed(11)
model_ksvm <- ksvm(
  x = as.matrix(train[, regression_predictors]),
  y = train$ozone,
  type = "eps-svr",
  kernel = "rbfdot",
  kpar = "automatic",
  cross = 10,
  scaled = TRUE
)

# Your code: predict test-set ozone values.
pred_ksvm <- as.numeric(
  predict(
    model_ksvm,
    as.matrix(train[, regression_predictors])
  )
)

# Residuals are observed minus predicted.
resid_ksvm <- test$ozone - pred_ksvm

# Your code: calculate RMSE.
rmse_ksvm <- sqrt(mean(test$ozone - pred_ksvm)^2)
rmse_ksvm

5.2 4.2 Visualize KSVM prediction error

Plot_ksvm_reg <- data.frame(
  Temp = test$Temp,
  Wind = test$Wind,
  observed = test$ozone,
  predicted = pred_ksvm
)

# Your code: calculate absolute error.
Plot_ksvm_reg$absolute_error <- abs(Plot_ksvm_reg$observed - Plot_ksvm_reg$predicted)

plot_ksvm_reg <- ggplot(Plot_ksvm_reg, aes(x = Temp, y = Wind)) +
  geom_point(
    aes(size = absolute_error, color = absolute_error),
    alpha = 0.75
  ) +
  # Your code: add a yellow-to-red color gradient.
   scale_color_gradient(
     low = "yellow",
     high = "red") +
  labs(
    title = "KSVM Regression: Absolute Prediction Error",
    size = "Absolute error",
    color = "Absolute error"
  ) +
  
  theme_minimal()
plot_ksvm_reg

5.3 4.3 e1071 SVM regression

set.seed(11)

# Complete the model formula and data argument.
model_svm <- svm(
  x = as.matrix(train[, regression_predictors]),
  y = train$ozone,
  type = "eps-regression",
  kernel = "radial",
  scale = TRUE
)

# Your code: predict test-set ozone.
pred_svm <- predict(model_ksvm,as.matrix(test[, regression_predictors]))

# Your code: calculate SVM RMSE.
rmse_svm <- (test$ozone - pred_svm)

Plot_svm_reg <- data.frame(
  Temp = test$Temp,
  Wind = test$Wind,
  observed = test$ozone,
  predicted = as.numeric(pred_svm)
)

# Your code: calculate absolute error.
Plot_svm_reg$absolute_error <- abs(Plot_svm_reg$observe - Plot_svm_reg$predict)

plot_svm_reg <- ggplot(Plot_svm_reg, aes(x = Temp, y = Wind)) +
  geom_point(aes(size = absolute_error, color = absolute_error), alpha = 0.75) +
  scale_color_gradient(low = "yellow", high = "red") +
  labs(title = "SVM Regression: Absolute Prediction Error") +
  theme_minimal()

rmse_svm
plot_svm_reg

5.4 4.4 Linear regression

# Your code: fit a linear model predicting ozone from all other variables.
model_lm <- lm(ozone ~ ., data = AirQuality)

# Your code: predict test-set ozone.
pred_lm <- predict(model_lm,AirQuality)

# Your code: calculate LM RMSE.
rmse_lm <- (airquality$ozone - model_lm$predictive)

Plot_lm_reg <- data.frame(
  Temp = test$Temp,
  Wind = test$Wind,
  observed = test$ozone,
  predicted = as.numeric(pred_lm)
)

Plot_lm_reg$absolute_error <- abs(
  Plot_lm_reg$observed - Plot_lm_reg$predicted
)

# Complete the missing ggplot layer.
plot_lm_reg <- ggplot(Plot_lm_reg, 
                      aes(x = Temp, y = Wind)) +
  geom_point(aes(size = absolute_error,
                 color = absolute_error), alpha = 0.75)+
  scale_color_gradient(low = "yellow", high = "red") +
  labs(title = "Linear Regression: Absolute Prediction Error") +
  theme_minimal()

rmse_lm
plot_lm_reg

6 E — Part 5. Compare Numeric Models

regression_results <- data.frame(
  Model = c("KSVM", "SVM", "LM"),
  RMSE = c(rmse_ksvm, rmse_svm, rmse_lm)
)

# Your code: sort from the lowest to highest RMSE.
regression_results <- regression_results [order(regression_results$RMSE),]
                                          
regression_results

# Your code: display the three regression plots in one row.
regression_plot <- gridExtra::grid.arrange(
  plot_ksvm_reg,
  plot_svm_reg,
  plot_lm_reg,
  nrow = 1
)

regression_plot

Which model has the lowest RMSE? Cite the values.
The KSVM model has an RMSE of 13.62520, the lowest among all models examined in this study. Consequently, the KSVM model’s predictions are, on average, closer to the true values than those produced by the other models.

Where do the plots show larger errors?
The evaluation results and the visualizations show that the SVM model has the largest prediction errors. This matches its RMSE of 25.65767, the highest among all the models tested.

Does the visual evidence support the RMSE ranking? Explain.
The visual evidence supports the ranking observed by the RMSE metric. We plotted each model’s predictions next to the real values, and the resulting scatter diagrams showed a clear match with the RMSE order. Therefore, KSVM had the smallest errors, LM larger, and SVM the largest; visual evidence matches the RMSE ranking.

7 A — Part 6. Create the Classification Label

Lecture connection: Classification learns from labeled examples. Here, goodOzone is the category the models will predict.

7.1 6.1 Create goodOzone

Follow the required rule exactly:

  • goodOzone = 1 when ozone is below the full-data mean.
  • goodOzone = 0 when ozone is equal to or above the mean.
# Calculate the mean ozone value.
mean_ozone <- mean(AirQuality$ozone)

# Complete the categorization rule.
AirQuality$goodOzone <- ifelse(
   AirQuality$ozone > mean_ozone,
    TRUE,  # value when condition is TRUE:
    FALSE  # value when condition is FALSE
)

# Convert the label to a factor.
AirQuality$goodOzone <- factor(
  AirQuality$goodOzone,
  levels = c(0, 1)
)

mean_ozone
table(AirQuality$goodOzone)
str(AirQuality$goodOzone)

# Reuse the original split indices.
train_class <- AirQuality[train_index, ]
test_class <- AirQuality[-train_index, ]

# Do not use ozone to predict a label created directly from ozone.
class_predictors <- c("Solar.R", "Wind", "Temp", "Month", "Day")

Why is numeric ozone excluded from the classification predictors?
The numeric ozone is excluded from the classification predictors because the variable could threaten the model’s integrity and compromise its reliability.

In your own words, what do 1 and 0 represent?
Within the KSVM model, the numeric values 1 and 0 denote the two classification categories. A value of 1 signifies good ozone, whereas a value of 0 denotes not good ozone.

8 C/E — Part 7. Classification Models

Evaluation rule: Use percent correctly classified. Higher accuracy indicates more correct predictions on the test data.

8.1 7.1 Categorical KSVM

set.seed(11)
model_ksvm_class <- ksvm(
  x = as.matrix(train_class[, class_predictors]),
  y = train_class$goodOzone,
  type = "C-svc",
  kernel = "rbfdot",
  kpar = "automatic",
  cross = 10,
  scaled = TRUE
)

# Your code: predict test labels using the same predictors.
pred_ksvm_class <- predict(
  model_ksvm_class,
  as.matrix(train_class[, class_predictors])
)

# Match the factor levels to the actual outcome.
pred_ksvm_class <- factor(
  pred_ksvm_class,
  levels = levels(test_class$goodOzone)
)

# Your code: calculate percent correctly classified.
accuracy_ksvm <- mean(pred_ksvm_class == test_class$goodOzone)
accuracy_ksvm

8.2 7.2 Prepare and plot KSVM results

# Complete the comparison labels.
correct_ksvm <- ifelse(
  pred_ksvm_class == test_class$goodOzone,
  "correct",  # value for a match
  "wrong"     # mismatch
)

Plot_ksvm <- data.frame(
  correct = factor(correct_ksvm, levels = c("correct", "wrong")),
  Temp = test_class$Temp,
  Wind = test_class$Wind,
  goodOzone = test_class$goodOzone,
  Predict = pred_ksvm_class
)

colnames(Plot_ksvm) <- c("correct", "Temp", "Wind", "goodOzone", "Predict")

plot_ksvm_class <- ggplot(Plot_ksvm, aes(x = Temp, y = Wind)) +
  geom_point(
    aes(shape = Predict, color = goodOzone, size = correct),
    alpha = 0.8
  ) +
  scale_color_manual(values = c("aquamarine", "#dc3545")) +
  # Your code: make wrong predictions larger than correct predictions.
  scale_size_manual(values = c("correct" = 1, "wrong" = 3)) +
  labs(
    title = "KSVM Classification",
    color = "Actual goodOzone",
    shape = "Predicted",
    size = "Result"
  ) +
  theme_minimal()
plot_ksvm_class

8.3 7.3 e1071 SVM classifier

set.seed(11)

model_svm_class <- svm(
  x = train_class[, class_predictors],
  y = train_class$goodOzone,
  type = "C-classification",
  kernel = "radial",
  scale = TRUE
)

# Your code: predict test labels.
pred_svm_class <-predict(model_svm_class,
                as.matrix(test_class[, 
                  class_predictors])
)
pred_svm_class <- factor(
  pred_svm_class,
  levels = levels(test_class$goodOzone)
)

# Your code: calculate percent correct.
accuracy_svm <- mean(
  pred_svm_class == test_class$goodOzone
  ) * 100

Plot_svm <- data.frame(
  correct = factor(
    ifelse(pred_svm_class == test_class$goodOzone, "correct", "wrong"),
    levels = c("correct", "wrong")
  ),
  Temp = test_class$Temp,
  Wind = test_class$Wind,
  goodOzone = test_class$goodOzone,
  Predict = pred_svm_class
)

# Your code: create a plot comparable to the KSVM plot.
plot_svm_class <- ggplot(Plot_svm,
       aes(x = Temp, y = Wind)) +
  geom_point(
    aes(shape = Predict,
        color = goodOzone, 
        size = correct),
    alpha = 0.8
  ) +
  scale_color_manual(values = c("aquamarine", "#dc3545")) +
  # Your code: make wrong predictions larger than correct predictions.
  scale_size_manual(values = c("correct" = 1, "wrong" = 3)) +
  labs(
    title = "SVM Classification",
    color = "Actual goodOzone",
    shape = "Predicted",
    size = "Result"
  ) +
  theme_minimal()

plot_svm_class

8.4 7.4 Naive Bayes classifier

# The model structure is provided.
model_nb <- naiveBayes(
  x = train_class[, class_predictors],
  y = train_class$goodOzone
)

# Your code: predict test labels.
pred_nb <- predict(model_nb,test_class[, class_predictors]
)
pred_nb <- factor(pred_nb, levels = levels(test_class$goodOzone))

# Your code: calculate percent correct.
accuracy_nb <- mean(pred_nb == test_class$goodOzone) * 100

Plot_nb <- data.frame(
  correct = factor(
    ifelse(pred_nb == test_class$goodOzone, "correct", "wrong"),
    levels = c("correct", "wrong")
  ),
  Temp = test_class$Temp,
  Wind = test_class$Wind,
  goodOzone = test_class$goodOzone,
  Predict = pred_nb
)

# Your code: create a plot comparable to the other classification plots.
plot_nb_class <- ggplot(Plot_nb,
       aes(x = Temp, y = Wind)) +
  geom_point(
    aes(shape = Predict,
        color = goodOzone, 
        size = correct),
    alpha = 0.8
  ) +
  scale_color_manual(values = c("aquamarine", "#dc3545")) +
  # Your code: make wrong predictions larger than correct predictions.
  scale_size_manual(values = c("correct" = 1, "wrong" = 3)) +
  labs(
    title = "Naive Bayes Classification",
    color = "Actual goodOzone",
    shape = "Predicted",
    size = "Result"
  ) +
  theme_minimal()

accuracy_nb
plot_nb_class

9 E — Part 8. Compare Classifiers

classification_results <- data.frame(
  Model = c("KSVM", "SVM", "Naive Bayes"),
  Accuracy = c(accuracy_ksvm, accuracy_svm, accuracy_nb)
)

# Your code: sort from highest to lowest accuracy.
classification_results <- classification_results[order(classification_results$Accuracy, decreasing = TRUE),]
classification_results

# Your code: display all three classification plots in one row.
classification_plot <- gridExtra::grid.arrange(
  plot_ksvm_class,
  plot_svm_class,
  plot_nb_class,
  nrow = 1
)

classification_plot

Which classifier has the highest accuracy? Cite the values.
The SVM model attained the greatest accuracy (84.3137%) of all models evaluated in this study, with Naive Bayes (74.5098%) and KSVM (52.9412%) following. Consequently, the SVM model yielded the most precise classifications.

Where are wrong predictions concentrated?
Incorrect predictions cluster within broad categories, making it difficult for the model to capture subtle differences among observations and leading to a higher misclassification rate.

Do the models misclassify the same observations? Cite the plots.
The models do not misclassify the same observations; plots of 150 cases show that misclassifications vary across models, indicating that each model makes errors on different instances.

10 E — Part 9. Explain and Justify

Final TRACE step: Connect the task, data investigation, arrangement, modeling, and evaluation evidence.

10.1 9.1 Best numeric model

Identify the strongest numeric model using its RMSE and error plot.

KSVM is the strongest numeric model, with the lowest RMSE of 13.62520. Its error plot shows smaller prediction errors and fewer large deviations than the LM and SVM models.

10.2 9.2 Best classifier

Identify the strongest classifier using its accuracy and classification plot.

The SVM, with an accuracy of 84.31%, is the most effective classifier, as indicated by its performance metrics and classification plot.

10.3 9.3 RMSE versus accuracy

Explain why RMSE and accuracy answer different evaluation questions.

RMSE and accuracy address distinct evaluation questions because they measure different aspects of model performance. Therefore, accuracy is employed for classification tasks, while RMSE is applied to numeric regression problems.

10.4 9.4 Supervised versus unsupervised learning

Explain why labels make this supervised learning and what would change if goodOzone were unavailable.

Because we have both the input and the correct output label, this is supervised learning, and it helps us predict the output for new or unseen data. But if goodOzone were not available, the model would not have the correct output label to learn from, making supervised learning impossible.

11 Submission Checklist