Q1:Find a Data Set of Which You Can Fit Multiple Linear Regression and Interpret Your Results

Introduction

Multiple Linear Regression (MLR) is a statistical method used to examine the relationship between one dependent variable and multiple independent variables. In this analysis, the Boston housing dataset from the MASS package is used to investigate factors affecting median house values (medv). The variables average number of rooms (rm), percentage of lower-status population (lstat), and pupil-teacher ratio (ptratio) are used as predictors. The objective is to develop a regression model, evaluate the influence of these factors on housing prices, and assess the model’s overall performance.

load the libraries

# Load package
library(MASS)

Import Dataset

# Load dataset
data(Boston)

# View first rows
head(Boston)
##      crim zn indus chas   nox    rm  age    dis rad tax ptratio  black lstat
## 1 0.00632 18  2.31    0 0.538 6.575 65.2 4.0900   1 296    15.3 396.90  4.98
## 2 0.02731  0  7.07    0 0.469 6.421 78.9 4.9671   2 242    17.8 396.90  9.14
## 3 0.02729  0  7.07    0 0.469 7.185 61.1 4.9671   2 242    17.8 392.83  4.03
## 4 0.03237  0  2.18    0 0.458 6.998 45.8 6.0622   3 222    18.7 394.63  2.94
## 5 0.06905  0  2.18    0 0.458 7.147 54.2 6.0622   3 222    18.7 396.90  5.33
## 6 0.02985  0  2.18    0 0.458 6.430 58.7 6.0622   3 222    18.7 394.12  5.21
##   medv
## 1 24.0
## 2 21.6
## 3 34.7
## 4 33.4
## 5 36.2
## 6 28.7

Fit Multiple Linear Regression Model

# Multiple Linear Regression
model <- lm(
  medv ~ rm + lstat + ptratio,
  data = Boston
)

# Model summary
summary(model)
## 
## Call:
## lm(formula = medv ~ rm + lstat + ptratio, data = Boston)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -14.4871  -3.1047  -0.7976   1.8129  29.6559 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)    
## (Intercept) 18.56711    3.91320   4.745 2.73e-06 ***
## rm           4.51542    0.42587  10.603  < 2e-16 ***
## lstat       -0.57181    0.04223 -13.540  < 2e-16 ***
## ptratio     -0.93072    0.11765  -7.911 1.64e-14 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 5.229 on 502 degrees of freedom
## Multiple R-squared:  0.6786, Adjusted R-squared:  0.6767 
## F-statistic: 353.3 on 3 and 502 DF,  p-value: < 2.2e-16

Model Diagnostics

Check whether the regression assumptions are reasonable.

# Diagnostic plots
par(mfrow = c(2, 2))
plot(model)

## Predicted Values

# Generate predicted values
Boston$Predicted_MEDV <- predict(model)

# View first predicted values
head(Boston$Predicted_MEDV)
## [1] 31.16836 25.76746 32.13917 31.08041 30.38659 27.21765

Actual vs Predicted Values

# Scatter plot of actual vs predicted values
plot(
  Boston$medv,
  Boston$Predicted_MEDV,
  main = "Actual vs Predicted House Values",
  xlab = "Actual MEDV",
  ylab = "Predicted MEDV",
  pch = 19,
  col = "blue"
)

# Reference line
abline(
  a = 0,
  b = 1,
  col = "red",
  lwd = 2
)

Interpretation of the Results

A multiple linear regression model was fitted to predict median house value (MEDV) using the average number of rooms (rm), percentage of lower-status population (lstat), and pupil-teacher ratio (ptratio). The overall model was highly significant (F = 353.3, p < 0.001), with an R-squared value of 0.679, indicating that approximately 68% of the variation in house values is explained by the three predictors. All predictors were statistically significant. House values increased as the average number of rooms increased, while higher values of lstat and ptratio were associated with lower house values.

Diagnostic plots were examined to assess the regression assumptions. The Residuals vs Fitted plot showed a slight curved pattern, suggesting that the relationship between the predictors and house values may not be perfectly linear. The Q-Q plot indicated some departure from normality, particularly in the upper tail, implying the presence of a few outliers. The Scale-Location plot suggested mild heteroscedasticity, as the spread of residuals increased slightly for larger fitted values. The Residuals vs Leverage plot identified a few observations with relatively high leverage, although none appeared to have an excessive influence on the model.

The Actual versus Predicted Values plot showed that most points were clustered around the reference line, indicating that the model provides reasonably accurate predictions. However, larger prediction errors were observed for some extreme house values, especially among the most expensive properties. Overall, the model demonstrates good predictive performance, although the diagnostic results suggest that model improvements, such as incorporating nonlinear terms or variable transformations, could further enhance accuracy.

Q2.Variable Selection in Multiple Linear Regression

Forward Selection

Start with no variables → add the most significant one step by step until no improvement.

Backward Elimination

Start with all variables → remove the least significant one step by step until all are significant.

Stepwise Selection

Combination of forward + backward → variables can be added or removed at each step.

Best Subset Selection

Tests all possible variable combinations → chooses best model using AIC, BIC, or Adjusted R².

Selection Criteria:


1.p-value (< 0.05) → significance test
2.AIC/BIC (lower is better) → model quality
3.Adjusted R² (higher is better) → explained variance


END