R Notebook

I have used two information sources Understanding Diagnostic Plots for Linear Regression Analysis and Q-Q Plots Explained

library(datasets)
library(lmtest)
library(car)
library(outliers)

# Load the longley dataset
data(longley)

# Estimate a linear regression model of GNP on all other variables
model <<- lm(GNP ~ ., data = longley)
summary(model)

Call:
lm(formula = GNP ~ ., data = longley)

Residuals:
    Min      1Q  Median      3Q     Max 
-3.9488 -1.3209  0.2327  0.7264  5.3511 

Coefficients:
               Estimate Std. Error t value Pr(>|t|)   
(Intercept)  -3.021e+04  9.310e+03  -3.245  0.01007 * 
GNP.deflator  1.508e+00  6.190e-01   2.437  0.03757 * 
Unemployed   -1.860e-01  4.733e-02  -3.929  0.00347 **
Armed.Forces -5.913e-02  3.252e-02  -1.818  0.10240   
Population    4.829e+00  1.388e+00   3.480  0.00694 **
Year          1.543e+01  4.944e+00   3.121  0.01230 * 
Employed     -3.148e+00  2.944e+00  -1.070  0.31268   
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 2.858 on 9 degrees of freedom
Multiple R-squared:  0.9995,    Adjusted R-squared:  0.9992 
F-statistic:  3022 on 6 and 9 DF,  p-value: 2.409e-14

par(mfrow=c(2,2)) # Change the panel layout to 2 x 2
plot(model)
par(mfrow=c(1,1)) # Change back to 1 x 1

# you can use the influential outliers as follows
#outlierTest(model)

Brief overview of each plot

Residuals vs Fitted Plot

This plot shows the residuals on the y-axis and the fitted values on the x-axis. It is used to identify nonlinearity, heteroscedasticity, and outlier observations. A horizontal line at zero indicates that the residuals have a mean of zero, while a random scatter of points around this line indicates that the model assumptions are met. If the plot shows any patterns or trends, such as a U-shaped or inverted U-shaped curve, this suggests nonlinearity in the model. If the plot shows that the spread of the residuals is not constant across the range of fitted values, this suggests heteroscedasticity. Outlier observations may also be identified as points that fall far outside the expected range.

knitr::include_graphics("RV.png") 

In Case 1, the residuals seem to be uniformly distributed but You can see parabola in Case 2, where the non-linear relationship was not explained by the model and was left out in the residuals.

Normal Q-Q Plot

This plot shows the quantiles of the residuals on the y-axis and the theoretical quantiles of a normal distribution on the x-axis. It is used to identify deviations from normality in the distribution of the residuals. If the points fall close to a straight line, this suggests that the residuals are approximately normally distributed. If the points deviate from a straight line, this suggests that the residuals are not normally distributed and may require transformation.

knitr::include_graphics("QQ.png") 

knitr::include_graphics("QQ2.png") 

Scale-Location Plot

This plot shows the square root of the standardized residuals on the y-axis and the fitted values on the x-axis. It is used to identify heteroscedasticity in the residuals. If the points are randomly scattered around a horizontal line, this suggests that the residuals have constant variance across the range of fitted values. If the points form a funnel shape or show a trend, this suggests that the variance of the residuals changes across the range of fitted values.

knitr::include_graphics("SL.png") 

In Case 1, the residuals appear randomly spread. Whereas, in Case 2, the residuals begin to spread wider along the x-axis as it passes around 5. Because the residuals spread wider and wider, the red smooth line is not horizontal and shows a steep angle in Case 2.

Residuals vs Leverage Plot

This plot helps us to find influential cases (i.e., subjects) if any. Not all outliers are influential in linear regression analysis (whatever outliers mean). Even though data have extreme values, they might not be influential to determine a regression line. That means, the results wouldn’t be much different if we either include or exclude them from analysis. They follow the trend in the majority of cases and they don’t really matter; they are not influential. On the other hand, some cases could be very influential even if they look to be within a reasonable range of the values. They could be extreme cases against a regression line and can alter the results if we exclude them from analysis. Another way to put it is that they don’t get along with the trend in the majority of the cases.

Unlike the other plots, this time patterns are not relevant. We watch out for outlying values at the upper right corner or at the lower right corner. Those spots are the places where cases can be influential against a regression line. Look for cases outside of a dashed line, Cook’s distance. When cases are outside of the Cook’s distance (meaning they have high Cook’s distance scores), the cases are influential to the regression results. The regression results will be altered if we exclude those cases.

knitr::include_graphics("RL.png") 

Case 1 is the typical look when there is no influential case, or cases. You can barely see Cook’s distance lines (a red dashed line) because all cases are well inside of the Cook’s distance lines. In Case 2, a case is far beyond the Cook’s distance lines (the other residuals appear clustered on the left because the second plot is scaled to show larger area than the first plot). The plot identified the influential observation as #49. If I exclude the 49th case from the analysis, the slope coefficient changes from 2.14 to 2.68 and R2 from .757 to .851. Pretty big impact!