Assignment 1:Multiple Linear Regression Analysis Using Housing Data in R

1. Introduction

Multiple Linear Regression (MLR) is a statistical technique used to examine the relationship between a dependent variable and two or more independent variables. It helps researchers understand how several factors simultaneously influence an outcome variable.

In this assignment, the Boston Housing dataset is used to investigate factors affecting housing prices. The analysis focuses on determining how the number of rooms, crime rate, pupil-teacher ratio, and lower-status population percentage influence the median value of owner-occupied homes.

2.Objectives

The objectives of this study are:

To fit a multiple linear regression model using housing data.
To identify significant predictors of housing prices.
To evaluate the goodness of fit of the model.
To interpret regression coefficients and statistical significance.

3.Dataset Description

The Boston Housing dataset contains information collected from various suburbs of Boston.

The variables used in this analysis are:

medv:Median value of owner-occupied homes (in $1000s)
rm Average number of rooms per dwelling
lstat :Percentage of lower status population
ptratio :Pupil-teacher ratio by town
crim :Per capita crime rate

Dependent Variable

medv (Housing Price)

Independent Variables

rm
lstat
ptratio
crim

4.Loading the Dataset

# install.packages("MASS")

library(MASS)   #Load the MASS package so that the Boston dataset can be used

# Load data
data("Boston")

# View first observations
head(Boston)

##      crim zn indus chas   nox    rm  age    dis rad tax ptratio  black lstat
## 1 0.00632 18  2.31    0 0.538 6.575 65.2 4.0900   1 296    15.3 396.90  4.98
## 2 0.02731  0  7.07    0 0.469 6.421 78.9 4.9671   2 242    17.8 396.90  9.14
## 3 0.02729  0  7.07    0 0.469 7.185 61.1 4.9671   2 242    17.8 392.83  4.03
## 4 0.03237  0  2.18    0 0.458 6.998 45.8 6.0622   3 222    18.7 394.63  2.94
## 5 0.06905  0  2.18    0 0.458 7.147 54.2 6.0622   3 222    18.7 396.90  5.33
## 6 0.02985  0  2.18    0 0.458 6.430 58.7 6.0622   3 222    18.7 394.12  5.21
##   medv
## 1 24.0
## 2 21.6
## 3 34.7
## 4 33.4
## 5 36.2
## 6 28.7

The dataset contains 506 observations and 14 variables.

5.Exploratory Data Analysis

Summary Statistics

summary(Boston)

##       crim                zn             indus            chas        
##  Min.   : 0.00632   Min.   :  0.00   Min.   : 0.46   Min.   :0.00000  
##  1st Qu.: 0.08205   1st Qu.:  0.00   1st Qu.: 5.19   1st Qu.:0.00000  
##  Median : 0.25651   Median :  0.00   Median : 9.69   Median :0.00000  
##  Mean   : 3.61352   Mean   : 11.36   Mean   :11.14   Mean   :0.06917  
##  3rd Qu.: 3.67708   3rd Qu.: 12.50   3rd Qu.:18.10   3rd Qu.:0.00000  
##  Max.   :88.97620   Max.   :100.00   Max.   :27.74   Max.   :1.00000  
##       nox               rm             age              dis        
##  Min.   :0.3850   Min.   :3.561   Min.   :  2.90   Min.   : 1.130  
##  1st Qu.:0.4490   1st Qu.:5.886   1st Qu.: 45.02   1st Qu.: 2.100  
##  Median :0.5380   Median :6.208   Median : 77.50   Median : 3.207  
##  Mean   :0.5547   Mean   :6.285   Mean   : 68.57   Mean   : 3.795  
##  3rd Qu.:0.6240   3rd Qu.:6.623   3rd Qu.: 94.08   3rd Qu.: 5.188  
##  Max.   :0.8710   Max.   :8.780   Max.   :100.00   Max.   :12.127  
##       rad              tax           ptratio          black       
##  Min.   : 1.000   Min.   :187.0   Min.   :12.60   Min.   :  0.32  
##  1st Qu.: 4.000   1st Qu.:279.0   1st Qu.:17.40   1st Qu.:375.38  
##  Median : 5.000   Median :330.0   Median :19.05   Median :391.44  
##  Mean   : 9.549   Mean   :408.2   Mean   :18.46   Mean   :356.67  
##  3rd Qu.:24.000   3rd Qu.:666.0   3rd Qu.:20.20   3rd Qu.:396.23  
##  Max.   :24.000   Max.   :711.0   Max.   :22.00   Max.   :396.90  
##      lstat            medv      
##  Min.   : 1.73   Min.   : 5.00  
##  1st Qu.: 6.95   1st Qu.:17.02  
##  Median :11.36   Median :21.20  
##  Mean   :12.65   Mean   :22.53  
##  3rd Qu.:16.95   3rd Qu.:25.00  
##  Max.   :37.97   Max.   :50.00

This summary statistics provide information about the minimum, maximum, mean, and median values of the variables.

Data Quality Assessment:

Checking Missing Values

colSums(is.na(Boston))

##    crim      zn   indus    chas     nox      rm     age     dis     rad     tax 
##       0       0       0       0       0       0       0       0       0       0 
## ptratio   black   lstat    medv 
##       0       0       0       0

If all values are zero, the dataset contains no missing values.

Checking Duplicate

sum(duplicated(Boston))

## [1] 0

Duplicate observations can bias model estimates and should be removed if present and this dataset has no duplicates

Distribution of Housing Prices

hist(Boston$medv,
     main = "Distribution of Housing Prices",
     xlab = "Median House Value",
     col = "lightblue")

This histogram shows how housing prices are distributed across the dataset.

Boxplot

library(ggplot2)
ggplot(Boston, aes(y = medv)) + geom_boxplot()

The boxplot identifies potential outliers in housing prices.

Correlation Analysis

variables <- Boston[, c("medv","rm","lstat","ptratio","crim")]

cor(variables)

##               medv         rm      lstat    ptratio       crim
## medv     1.0000000  0.6953599 -0.7376627 -0.5077867 -0.3883046
## rm       0.6953599  1.0000000 -0.6138083 -0.3555015 -0.2192467
## lstat   -0.7376627 -0.6138083  1.0000000  0.3740443  0.4556215
## ptratio -0.5077867 -0.3555015  0.3740443  1.0000000  0.2899456
## crim    -0.3883046 -0.2192467  0.4556215  0.2899456  1.0000000

Correlation values indicate the strength and direction of relationships between variables.

Positive correlations indicate variables move in the same direction.
Negative correlations indicate inverse relationships.
Strong correlations suggest potential predictors of housing prices.

Multiple Linear Regression Model

The mathematical model is:

\[ medv = \beta_0 + \beta_1 rm + \beta_2 lstat + \beta_3 ptratio + \beta_4 crim + \varepsilon \]

Where:

$medv$ = Housing price
$rm$ = Number of rooms
$lstat$ = Lower-status population percentage
$ptratio$ = Pupil-teacher ratio
$crim$ = Crime rate
$\varepsilon$ = Error term

Fitting the Regression Model

model <- lm(medv ~ rm + lstat + ptratio + crim,
            data = Boston)

summary(model)

## 
## Call:
## lm(formula = medv ~ rm + lstat + ptratio + crim, data = Boston)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -14.5492  -3.1454  -0.9357   1.6894  30.1563 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)    
## (Intercept) 16.92330    3.97564   4.257 2.48e-05 ***
## rm           4.61862    0.42716  10.812  < 2e-16 ***
## lstat       -0.53431    0.04564 -11.708  < 2e-16 ***
## ptratio     -0.88969    0.11883  -7.487 3.19e-13 ***
## crim        -0.06544    0.03081  -2.124   0.0342 *  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 5.211 on 501 degrees of freedom
## Multiple R-squared:  0.6815, Adjusted R-squared:  0.6789 
## F-statistic:   268 on 4 and 501 DF,  p-value: < 2.2e-16

The regression output provides coefficient estimates, p-values, R-squared values, and overall model significance.

Intercept:

The intercept represents the expected housing price when all predictor variables are equal to zero.

Number of Rooms (rm):

A positive coefficient indicates that houses with more rooms tend to have higher prices.

Lower Status Population (lstat):

A negative coefficient suggests that areas with higher percentages of lower-status populations tend to have lower housing prices.

Pupil-Teacher Ratio (ptratio):

A negative coefficient implies that areas with larger class sizes tend to have lower housing prices.

Crime Rate (crim):

A negative coefficient indicates that housing prices decrease as crime rates increase.

P-values:

Variables with p-values less than 0.05 are considered statistically significant predictors.

6. Goodness of Fit(Model Performance)

R-Squared

summary(model)$r.squared

## [1] 0.6814923

The R-squared value indicates the proportion of variation in housing prices explained by the model.

Adjusted R-Squared

summary(model)$adj.r.squared

## [1] 0.6789494

Adjusted R-squared accounts for the number of predictors included in the model.

ANOVA Test

anova(model)

## Analysis of Variance Table
## 
## Response: medv
##            Df  Sum Sq Mean Sq  F value    Pr(>F)    
## rm          1 20654.4 20654.4 760.5665 < 2.2e-16 ***
## lstat       1  6622.6  6622.6 243.8658 < 2.2e-16 ***
## ptratio     1  1711.3  1711.3  63.0168 1.355e-14 ***
## crim        1   122.5   122.5   4.5115   0.03416 *  
## Residuals 501 13605.5    27.2                       
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

The ANOVA table evaluates whether the regression model is statistically significant overall.

A p-value less than 0.05 indicates that the overall regression model is statistically significant.

Diagnostic Analysis(Linear Assumption)

The assumptions of multiple linear regression can be assessed using diagnostic plots.

par(mfrow = c(2,2))
plot(model)

The diagnostic plots evaluate:

Linearity
Homoscedasticity
Normality of residuals
Influential observations

Prediction Example

Predict the value of a house with:

6 rooms
lstat = 10
ptratio = 18
crim = 0.5

new_house <- data.frame(
  rm = 6,
  lstat = 10,
  ptratio = 18,
  crim = 0.5
)

predict(model, new_house)

##        1 
## 23.24478

The predicted value represents the estimated median housing price for a house with the specified characteristics.

7. Discussion

The analysis demonstrates that housing prices are influenced by several socioeconomic and structural factors.

The findings include:

Houses with more rooms generally have higher values.
Higher crime rates tend to reduce property values.
Areas with poorer educational indicators often exhibit lower housing prices.
Demographic characteristics significantly influence housing markets.

8.Conclusion

This study applied multiple linear regression to investigate factors affecting housing prices using the Boston Housing dataset.

The results show that:

Number of rooms positively influences housing prices.
Crime rate negatively affects housing prices.
Higher percentages of lower-status populations reduce housing values.
Larger pupil-teacher ratios are associated with lower housing prices.

The regression model explains a substantial proportion of variation in housing prices and provides useful insights into the determinants of residential property values.

ASSignment 2:Variable Selection Methods in Multiple Linear Regression

1.Introduction

In Multiple Linear Regression, researchers often have many potential predictor variables available for analysis. However, including all variables in a model may not always be desirable because some variables may be irrelevant, redundant, or highly correlated with others.

Variable selection methods are techniques used to identify the most important predictor variables that contribute significantly to explaining the dependent variable. These methods help improve model accuracy, reduce complexity, and enhance interpretability.

This assignment discusses the major variable selection methods used in regression analysis, including their advantages, disadvantages, and practical applications.

2. What is Variable Selection?

Variable selection refers to the process of selecting a subset of relevant independent variables for inclusion in a regression model.

The general multiple regression model is:

\[ Y = \beta_0 + \beta_1X_1 + \beta_2X_2 + \cdots + \beta_pX_p + \varepsilon \]

Where:

1. = Dependent variable
(X_1, X_2, …, X_p) = Independent variables
(_0) = Intercept
(_1, _2, …, _p) = Regression coefficients
() = Error term

The goal of variable selection is to determine which predictors should remain in the final model.

3.Importance of Variable Selection

Variable selection is important because it:

Improves model interpretation.
Reduces model complexity.
Removes irrelevant variables.
Reduces overfitting.
Improves prediction accuracy.
Reduces multicollinearity problems.

4. Variable Selection Methods

The most common methods include:

Forward Selection
Backward Elimination
Stepwise Selection
Best Subset Selection

1.Forward Selection Method

Description

Forward Selection starts with an empty model and adds variables one at a time based on their contribution to the model.

Steps

Start with no predictor variables.
Add the most significant variable.
Continue adding variables until no significant improvement occurs.

Advantages

Simple to understand.
Computationally efficient.
Suitable when many variables exist.

Disadvantages

Once added, variables cannot be removed.
May miss important variable combinations.

2. Backward Elimination Method

Description

Backward Elimination begins with all predictor variables and removes the least significant variable at each step.

Steps

Fit the full model.
Identify the variable with the highest p-value.
Remove the variable if it is not significant.
Repeat until all remaining variables are significant.

Advantages

Considers all variables initially.
Often produces strong predictive models.

Disadvantages

Requires larger datasets.
Computationally more intensive.

3.Stepwise Selection Method

Description

Stepwise Selection combines Forward Selection and Backward Elimination.

Variables can be added or removed at each stage.

Steps

Start with an initial model.
Add significant variables.
Remove insignificant variables.
Continue until no further improvement is possible.

Advantages

More flexible than forward and backward methods.
Produces balanced models.

Disadvantages

Results may vary across datasets.
Can be unstable in some situations.

4.Best Subset Selection

Description

Best Subset Selection evaluates all possible combinations of predictor variables and identifies the best-performing model.

Advantages

Considers all possible models.
Often identifies the best subset of predictors.

Disadvantages

Computationally expensive.
Less practical for very large datasets.

5.Model Selection Criteria

Several statistical measures are used to compare models.

Adjusted R-Squared

Adjusted R-squared measures the proportion of variation explained while accounting for the number of predictors.

Higher values indicate better models.

AIC (Akaike Information Criterion)

AIC is used to compare competing models.

Lower AIC values indicate better models.

BIC (Bayesian Information Criterion)

BIC penalizes model complexity more strongly than AIC.

Lower BIC values indicate better models.

P-values

Variables with p-values less than 0.05 are generally considered statistically significant.

6.Practical Applications

Variable selection methods are commonly used in:

Finance
Healthcare
Agriculture
Environmental Science
Housing Market Analysis
Business Analytics

7. Advantages of Variable Selection

Improves interpretation.
Reduces unnecessary variables.
Improves prediction accuracy.
Reduces overfitting.
Simplifies analysis.

8. Limitations of Variable Selection

Different methods may produce different results.
Important variables may occasionally be excluded.
Results may depend on sample size.
Multicollinearity can affect selection decisions.

6.Conclusion

Variable selection is a critical stage in regression modeling. It helps identify the most important predictors while removing unnecessary variables. Common methods include Forward Selection, Backward Elimination, Stepwise Selection, and Best Subset Selection.

Each method has strengths and limitations, and the choice depends on the research objectives and dataset characteristics. Proper variable selection leads to simpler, more interpretable, and more accurate regression models.

9.References

James, G., Witten, D., Hastie, T., & Tibshirani, R. (2021). An Introduction to Statistical Learning. Springer.
Venables, W. N., & Ripley, B. D. (2002). Modern Applied Statistics with S. Springer.
Kutner, M. H., Nachtsheim, C. J., Neter, J., & Li, W. (2005). Applied Linear Statistical Models.
R Core Team. R: A Language and Environment for Statistical Computing.

Assignments

Alfred NDAZIGARUYE

2026-06-04

Assignment 1:Multiple Linear Regression Analysis Using Housing Data in R

1. Introduction

2.Objectives

3.Dataset Description

Dependent Variable

Independent Variables

4.Loading the Dataset

5.Exploratory Data Analysis

Summary Statistics

Data Quality Assessment:

Checking Missing Values

Checking Duplicate

Distribution of Housing Prices

Boxplot

Correlation Analysis

Multiple Linear Regression Model

Fitting the Regression Model

Intercept:

Number of Rooms (rm):

Lower Status Population (lstat):

Pupil-Teacher Ratio (ptratio):

Crime Rate (crim):

P-values:

6. Goodness of Fit(Model Performance)

R-Squared

Adjusted R-Squared

ANOVA Test

Diagnostic Analysis(Linear Assumption)

Prediction Example

7. Discussion

8.Conclusion

ASSignment 2:Variable Selection Methods in Multiple Linear Regression

1.Introduction

2. What is Variable Selection?

3.Importance of Variable Selection

4. Variable Selection Methods

1.Forward Selection Method

Description

Steps

Advantages

Disadvantages

2. Backward Elimination Method

Description

Steps

Advantages

Disadvantages

3.Stepwise Selection Method

Description

Steps

Advantages

Disadvantages

4.Best Subset Selection

Description

Advantages

Disadvantages

5.Model Selection Criteria

Adjusted R-Squared

AIC (Akaike Information Criterion)

BIC (Bayesian Information Criterion)

P-values

6.Practical Applications

7. Advantages of Variable Selection

8. Limitations of Variable Selection

6.Conclusion

9.References