Question 8.2
Use a simulation to show tree bias with different granularities.
I will create a simulated dataset generated by a non-linear function. I will then train the tree based models with varying levels of pruning. I will then look at the MSE on the training set and test set in relation to the complexity of the tree.
set.seed(42)
n_sample <- 350
nonlinear_function <- function(x){
sin(1.25 * x) + 2 * cos(.25*x)
}
x <- runif(n_sample, 1, 25)
f_of_x <- nonlinear_function(x)
noise <- rnorm(n_sample, 0, 2)
y <- f_of_x + noise
df <- data.frame(y=y, x=x)
in_train <- createDataPartition(df$y, p = .8, list = FALSE, times = 1)
train_df <- df[in_train,]
test_df <- df[-in_train,]
results <- data.frame(Granularity = c(NA), MSE = c(NA), data = c(NA)) %>% na.omit()
get_mse <- function(model, data){
y_hat <- predict(model, data)
mse <- mean((y_hat - data$y)^2)
return(mse)
}
for(depth in seq(1:10)){
rtree_model <- rpart(y ~ x, data = train_df, control=rpart.control(maxdepth=depth))
results <- rbind(results, data.frame(Granularity = depth, MSE = get_mse(rtree_model, train_df), data = "Training"))
results <- rbind(results, data.frame(Granularity = depth, MSE = get_mse(rtree_model, test_df), data = "Test"))
}
ggplot(results, aes(Granularity, MSE, color = data, group = data)) +
geom_line() +
geom_point() +
scale_color_brewer(palette = "Set1") +
theme(legend.position = "bottom", legend.title = element_blank())
One can see that as the granularity of the tree model increases, the MSE on the training set decreases. However the MSE on the test set initially begins to decline then increases again as the model starts to overfit the training data.
