Linear regression

#Linear regession Homework [75pts] let’s analyze the correlation between displacement and Highway miles per gallon but this time we will only consider cars with 4 cylinder engines.

Setup block

#reset workspace
rm(list = ls())
#load libraries
library(dplyr)
library(ggplot2)
#load the data
mpg <- read.csv("https://tinyurl.com/didampg2022")

#0. [1pts] rename this file to include your name

#1. [5pts] p1:using dplyr functions create a new dataset called mpg_4c that only includes cars with 4 cylinder engines. p2:we will use this dataframe for the rest of the homework so double check using head or View or table that only 4 cilinder cars are in the dataset mpg_4c

#p1
mpg_4c <- mpg %>% filter(Cyl == 4)

#p2
head(mpg_4c)

#2. [10pts] p1:using geom_jitter plot the displacement on the x and the highway mpg on the y axis. p2: add appropriate axis labels. make sure you are using the mpg_4c dataset

ggplot(mpg_4c) +
  geom_jitter(aes(x=Displ, y= Hwy.MPG), width=0.5) +
  xlab("Engine Displacement (L)") +
  ylab("Highway MPG")

#3. [5pts] Describe this trend.

The plot suggests that cars with bigger engines don’t go as far on gallon of gas. the Smaller the engine the more miles you can get per gallon.

#4. [12pts]Split the dataset in a 1/5 ratio 4/5 to the training data, 1/5 to the validation data. follow the example from class, use the names mpg4.train mpg4.valid for the two resulting dataframes make sure you are using the 4 cylinder dataset everywhere

set.seed(13)


split <- 0.8

rows  <- nrow(mpg_4c)

train.entries <- sample(rows, rows*split)
mpg_4c.train <- mpg_4c[train.entries, ]

mpg_4c.valid  <- mpg_4c[-train.entries,  ]

#5. [5pts]Generate the linear model and save it in the model variable

model <- lm(Hwy.MPG~ Displ, data=mpg_4c.train)

#6. [2pts] using summary print the model

summary(model)

#7. [5pts] using the information from summary write the y hat equation:

yhat =

#8. [10pts] calculate the residuals for both training and testing data. use mutate to store yhat and the residual as columns in each dataset

#training data
mpg_4c.train <- mpg_4c.train %>%
    mutate(yhat = predict(model, newdata=mpg_4c.train)) %>%
    mutate(residual = Hwy.MPG - yhat)
  
#validation data
mpg_4c.valid <- mpg_4c.valid %>%
    mutate(yhat = predict(model, newdata=mpg_4c.valid)) %>%
    mutate(residual = Hwy.MPG - yhat)
  
  

#9. [10pts] make a plot of y_hat on the x axis and the residuals on the y axis using geom_jitter with a width of 1 for the validation data. include a geom_hline on y=0 and a limit the y axis to the range [-20,20]

ggplot(mpg_4c.valid, aes(x = yhat, y = residual)) +
  geom_jitter(width = 1) + 
  geom_hline(yintercept = 0, linetype = "dashed", color = "red") + 
  ylim(-20, 20) +
  xlab("Predicted Mileage (mpg)") +
  ylab("Residual (mpg)")

#10. [5pts] looking at this graph what can you tell me about this set of residuals: p1:does it show a trend? p2:how is the spread? Looking at this set of residuals it seems as though there is a lack of a trend because the line of best fit is smack dab in the middle of the graph. so it’s equally divided.

#11. [5pts] p1:darw a histogram of the residuals , limit the x axis to -20,20, p2:visually estimate the range where most of the observations fall.

  ggplot(mpg_4c.valid) +
  geom_histogram(aes(x=residual), bins=50, color="black") +
  xlim(-20, 20) +
  xlab("Residual (miles per gallon)")
  cat("the range is [A,B]")

#12. [5pts] are we overfitting? compare the standard deviation of the residuals from training and validation data and write wether we are overfitting or not and why.

sd(mpg_4c.train$residual)
sd(mpg_4c.valid$residual)
#overfitting or not overfitting?

based on how close the standard deviations and close they are I would say that we are not overfitting