Use the Abalone data set. Download it from Canvas and read as a data frame in R ( call it abalone_df ). Click the option “Strings as factors”.

About the Abalone data set. Use the following link to read more info about this data set:

https://archive.ics.uci.edu/ml/datasets/abalone

The outcome variable to predict is the last one, which records the number of rings in the abalone (the number of rings, in turn, is related to the abalone’s age).

We will use only one predictor: the second to last variable, which represents the shell weight.

abalone_df <- read.csv("E:/S9510/CAP3330/abalone.csv", stringsAsFactors=TRUE)
View(abalone_df)
str(abalone_df)
## 'data.frame':    4177 obs. of  9 variables:
##  $ Sex           : Factor w/ 3 levels "F","I","M": 3 3 1 3 2 2 1 1 3 1 ...
##  $ Length        : num  0.455 0.35 0.53 0.44 0.33 0.425 0.53 0.545 0.475 0.55 ...
##  $ Diameter      : num  0.365 0.265 0.42 0.365 0.255 0.3 0.415 0.425 0.37 0.44 ...
##  $ Height        : num  0.095 0.09 0.135 0.125 0.08 0.095 0.15 0.125 0.125 0.15 ...
##  $ Whole.weight  : num  0.514 0.226 0.677 0.516 0.205 ...
##  $ Shucked.weight: num  0.2245 0.0995 0.2565 0.2155 0.0895 ...
##  $ Viscera.weight: num  0.101 0.0485 0.1415 0.114 0.0395 ...
##  $ Shell.weight  : num  0.15 0.07 0.21 0.155 0.055 0.12 0.33 0.26 0.165 0.32 ...
##  $ Rings         : int  15 7 9 10 7 8 20 16 9 19 ...

Questions:

  1. Evaluate the statistical significance of the equation.
abalone_simple_out= lm(Rings ~ Shell.weight, data = abalone_df)

summary(abalone_simple_out)
## 
## Call:
## lm(formula = Rings ~ Shell.weight, data = abalone_df)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -5.9830 -1.6005 -0.5843  0.9390 15.6334 
## 
## Coefficients:
##              Estimate Std. Error t value Pr(>|t|)    
## (Intercept)   6.46212    0.07715   83.76   <2e-16 ***
## Shell.weight 14.53568    0.27908   52.08   <2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 2.51 on 4175 degrees of freedom
## Multiple R-squared:  0.3938, Adjusted R-squared:  0.3937 
## F-statistic:  2713 on 1 and 4175 DF,  p-value: < 2.2e-16

The PV for the b1 coefficient is very small (PV < 2e-16). This PV is less than alpha. Therefore, we can reject Ho and support Ha. The data give us evidence to conclude that there is a statistically significant relationship between shell weight and Rings.

  1. Evaluate the practical significance of the equation based on RSE.
#Coefficient of variation based on RSE
(summary(abalone_simple_out)$sigma/mean(abalone_df$Rings))*100
## [1] 25.27259

We can claim that the RSE is not low enough because the coefficient of variation is not below 20 %. Thus, the practical significance (prediction quality, the performance) of the equation is not as good as we wanted.

  1. Do the plot of the residuals versus the predicted values.
plot(predict(abalone_simple_out), residuals(abalone_simple_out), xlab = "Predict", ylab="Residuals")

abline(h=0, col = "red")

What assumptions can you asses using this plot?

Assess these assumptions!

The plot is showing some outliers and the residuals getting spread in a parabolic shape as the predicted value increase. Therefore, the variability is not the same for all values. Assumption 4 is not satisfied

  1. Evaluate assumption 3
shapiro.test(residuals(abalone_simple_out))
## 
##  Shapiro-Wilk normality test
## 
## data:  residuals(abalone_simple_out)
## W = 0.89287, p-value < 2.2e-16

p-value < 2.2e-16 is less than alpha (0.05); therefore, we reject Ho and support Ha and conclude that the residuals do NOT follow a Normal distribution. Thus, assumption 3 is NOT satisfied.

  1. Add the predictor “Viscera.weight” to the previous equation. Does it improve the performance (predictions quality) of the equation?

Using Adjusted R squared

# Compute the % increase

(summary (lm(Rings ~ Shell.weight + Viscera.weight, data = abalone_df))$adj.r.squared - summary (lm(Rings ~ Shell.weight, data = abalone_df))$adj.r.squared) / summary (lm(Rings ~ Shell.weight, data = abalone_df))$adj.r.squared
## [1] 0.06209265