Adventist University of Central Africa (AUCA)
Faculty of Information Technology
Masters of Big Data Analytics
Assignment of multiple linear regression
Jean Aime Irakuzwa
Registration No: 20251MBI030
2026-06-04
#HW 1: Find a data set of which you can fit multiple linear regression and interpret your results
# Install missing packages automatically
required_packages <- c(
"tidyverse", "lubridate", "ggplot2", "scales",
"corrplot", "GGally", "caret", "Metrics",
"gridExtra", "zoo", "ggrepel", "factoextra","car","broom","glmnet"
)
for (pkg in required_packages) {
if (!requireNamespace(pkg, quietly = TRUE)) {
install.packages(pkg, repos = "https://cloud.r-project.org")
}
}
library(tidyverse)
library(lubridate)
library(ggplot2)
library(scales)
library(corrplot)
library(GGally)
library(caret)
library(Metrics)
library(gridExtra)
library(zoo)
library(ggrepel)
library(factoextra)
library(broom)
library(car)
insurance <- read_csv("E:/Document/BIG Data Masters/Course Semester 1/Semester 1/R Programming for Data Science/insurance.csv")summary(insurance)## age sex bmi children
## Min. :18.00 Length:1338 Min. :15.96 Min. :0.000
## 1st Qu.:27.00 Class :character 1st Qu.:26.30 1st Qu.:0.000
## Median :39.00 Mode :character Median :30.40 Median :1.000
## Mean :39.21 Mean :30.66 Mean :1.095
## 3rd Qu.:51.00 3rd Qu.:34.69 3rd Qu.:2.000
## Max. :64.00 Max. :53.13 Max. :5.000
## smoker region charges
## Length:1338 Length:1338 Min. : 1122
## Class :character Class :character 1st Qu.: 4740
## Mode :character Mode :character Median : 9382
## Mean :13270
## 3rd Qu.:16640
## Max. :63770Exploratory Data Analysis
# Check for missing values
colSums(is.na(insurance))## age sex bmi children smoker region charges
## 0 0 0 0 0 0 0Distribution of charges
ggplot(insurance, aes(x = charges)) +
geom_histogram(bins = 40, fill = "#378ADD", color = "white") +
labs(title = "Distribution of Medical Charges", x = "Charges ($)", y = "Count") +
theme_minimal()
Scatter: Age vs Charges which is colored by smoker
ggplot(insurance, aes(x = age, y = charges, color = smoker)) +
geom_point(alpha = 0.5) +
scale_color_manual(values = c("no" = "#378ADD", "yes" = "#D85A30")) +
labs(title = "Age vs Charges", x = "Age", y = "Charges ($)") +
theme_minimal()
Scatter: BMI vs Charges
ggplot(insurance, aes(x = bmi, y = charges, color = smoker)) +
geom_point(alpha = 0.5) +
scale_color_manual(values = c("no" = "#378ADD", "yes" = "#D85A30")) +
labs(title = "BMI vs Charges", x = "BMI", y = "Charges ($)") +
theme_minimal()
Train / Test Split (80/20)
set.seed(42)
n <- nrow(insurance)
train_idx <- sample(seq_len(n), size = floor(0.8 * n))
train_df <- insurance[ train_idx, ]
test_df <- insurance[-train_idx, ]
cat("Training rows:", nrow(train_df), "\n")## Training rows: 1070cat("Test rows :", nrow(test_df), "\n")## Test rows : 268Fit Multiple Linear Regression
model <- lm(charges ~ age + sex + bmi + children + smoker + region,
data = train_df)
summary(model)##
## Call:
## lm(formula = charges ~ age + sex + bmi + children + smoker +
## region, data = train_df)
##
## Residuals:
## Min 1Q Median 3Q Max
## -10943.7 -2979.6 -988.6 1499.1 30235.0
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) -12386.93 1122.34 -11.037 < 2e-16 ***
## age 261.04 13.47 19.378 < 2e-16 ***
## sexmale -230.54 375.70 -0.614 0.539590
## bmi 353.48 32.41 10.906 < 2e-16 ***
## children 516.05 154.12 3.348 0.000842 ***
## smokeryes 23771.14 465.43 51.074 < 2e-16 ***
## regionnorthwest -544.63 537.05 -1.014 0.310761
## regionsoutheast -1274.10 541.39 -2.353 0.018785 *
## regionsouthwest -1325.16 537.80 -2.464 0.013895 *
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 6100 on 1061 degrees of freedom
## Multiple R-squared: 0.7489, Adjusted R-squared: 0.747
## F-statistic: 395.6 on 8 and 1061 DF, p-value: < 2.2e-16Model Evaluation on Test Set
predictions <- predict(model, newdata = test_df)
# Metrics
ss_res <- sum((test_df$charges - predictions)^2)
ss_tot <- sum((test_df$charges - mean(test_df$charges))^2)
r2 <- 1 - ss_res / ss_tot
rmse <- sqrt(mean((test_df$charges - predictions)^2))
mae <- mean(abs(test_df$charges - predictions))
cat("\n--- Test Set Performance ---\n")##
## --- Test Set Performance ---cat(sprintf("R² : %.4f\n", r2))## R² : 0.7575cat(sprintf("RMSE : $%.2f\n", rmse))## RMSE : $5926.98cat(sprintf("MAE : $%.2f\n", mae))## MAE : $4148.59Diagnostic Plots
par(mfrow = c(2, 2))
plot(model)
par(mfrow = c(1, 1))results_df <- data.frame(
actual = test_df$charges,
predicted = predictions,
residual = test_df$charges - predictions
)
ggplot(results_df, aes(x = actual, y = predicted)) +
geom_point(alpha = 0.4, color = "#378ADD") +
geom_abline(slope = 1, intercept = 0, color = "#D85A30", linetype = "dashed") +
labs(title = "Actual vs Predicted Charges",
x = "Actual ($)", y = "Predicted ($)") +
theme_minimal()
Interpretation of Results
The multiple linear regression model was used to examine the factors that influence medical insurance charges. The model achieved an R-squared value of 0.7489, indicating that approximately 74.89% of the variation in insurance charges is explained by the independent variables included in the model.
The results show that age, BMI, number of children, and smoking status are significant predictors of insurance charges. Specifically, insurance charges increase as age and BMI increase. Similarly, individuals with more children tend to have slightly higher insurance costs.
Smoking status has the strongest effect on insurance charges. On average, smokers incur substantially higher medical insurance charges than non-smokers, holding all other factors constant.
The variable sex was not statistically significant, suggesting that there is no meaningful difference in insurance charges between males and females in this dataset.
The test set R-squared value of 0.7575 indicates that the model performs well on unseen data and has good predictive capability.
Conclusion
The analysis shows that smoking status, age, BMI, and number of children significantly affect medical insurance charges. Among these factors, smoking has the greatest impact. Overall, the model provides a good fit and can be used to predict insurance charges with reasonable accuracy.
HW 2: Read about variable selection methods
Variable Selection Methods Using the Insurance Dataset
Introduction
Variable selection is an important step in multiple linear regression
because it helps identify the most relevant predictors while reducing
model complexity. This study applies different variable selection
methods for example the Insurance dataset, where the dependent variable
is charges and the independent variables are
age, sex, bmi,
children, smoker, and region.
1. Filter Methods
Filter methods evaluate variables independently of a predictive model. For continuous variables, correlation analysis can be used to identify relationships between predictors and the response variable.
library(corrplot)
insurance_num <- insurance[, c("age", "bmi", "children", "charges")]
cor_matrix <- cor(insurance_num)
corrplot(cor_matrix, method = "circle")
Interpretation
The correlation matrix shows the strength of the relationship between
variables. Variables with higher correlations with charges
are considered more important predictors. In the insurance dataset, age
and BMI usually show positive correlations with insurance charges.
2. Wrapper Methods – Stepwise Regression
Stepwise regression selects variables by adding or removing predictors based on model performance.
full_model <- lm(charges ~ ., data = insurance)
stepwise_model <- step(full_model, direction = "both")## Start: AIC=23316.43
## charges ~ age + sex + bmi + children + smoker + region
##
## Df Sum of Sq RSS AIC
## - sex 1 5.7164e+06 4.8845e+10 23315
## <none> 4.8840e+10 23316
## - region 3 2.3343e+08 4.9073e+10 23317
## - children 1 4.3755e+08 4.9277e+10 23326
## - bmi 1 5.1692e+09 5.4009e+10 23449
## - age 1 1.7124e+10 6.5964e+10 23717
## - smoker 1 1.2245e+11 1.7129e+11 24993
##
## Step: AIC=23314.58
## charges ~ age + bmi + children + smoker + region
##
## Df Sum of Sq RSS AIC
## <none> 4.8845e+10 23315
## - region 3 2.3320e+08 4.9078e+10 23315
## + sex 1 5.7164e+06 4.8840e+10 23316
## - children 1 4.3596e+08 4.9281e+10 23325
## - bmi 1 5.1645e+09 5.4010e+10 23447
## - age 1 1.7151e+10 6.5996e+10 23715
## - smoker 1 1.2301e+11 1.7186e+11 24996summary(stepwise_model)##
## Call:
## lm(formula = charges ~ age + bmi + children + smoker + region,
## data = insurance)
##
## Residuals:
## Min 1Q Median 3Q Max
## -11367.2 -2835.4 -979.7 1361.9 29935.5
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) -11990.27 978.76 -12.250 < 2e-16 ***
## age 256.97 11.89 21.610 < 2e-16 ***
## bmi 338.66 28.56 11.858 < 2e-16 ***
## children 474.57 137.74 3.445 0.000588 ***
## smokeryes 23836.30 411.86 57.875 < 2e-16 ***
## regionnorthwest -352.18 476.12 -0.740 0.459618
## regionsoutheast -1034.36 478.54 -2.162 0.030834 *
## regionsouthwest -959.37 477.78 -2.008 0.044846 *
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 6060 on 1330 degrees of freedom
## Multiple R-squared: 0.7509, Adjusted R-squared: 0.7496
## F-statistic: 572.7 on 7 and 1330 DF, p-value: < 2.2e-16Interpretation
Stepwise regression identifies the combination of predictors that provides the best balance between model complexity and predictive power. Variables retained in the final model are considered important predictors of insurance charges.
3. Regularization Methods – LASSO
LASSO performs variable selection by shrinking less important coefficients toward zero.
library(glmnet)
x <- model.matrix(charges ~ ., insurance)[, -1]
y <- insurance$charges
lasso_model <- cv.glmnet(x, y, alpha = 1)
plot(lasso_model)
coef(lasso_model, s = "lambda.min")## 9 x 1 sparse Matrix of class "dgCMatrix"
## lambda.min
## (Intercept) -11566.9505
## age 252.9685
## sexmale .
## bmi 322.5230
## children 418.4944
## smokeryes 23659.0533
## regionnorthwest .
## regionsoutheast -547.1304
## regionsouthwest -513.6591Interpretation
Variables with coefficients equal to zero are excluded from the model. Variables with non-zero coefficients are considered significant predictors of insurance charges.
4. Best Subset Selection
Best subset selection evaluates all possible combinations of predictors and identifies the best-performing model.
library(leaps)
regfit <- regsubsets(charges ~ ., data = insurance, nvmax = 10)
summary(regfit)## Subset selection object
## Call: regsubsets.formula(charges ~ ., data = insurance, nvmax = 10)
## 8 Variables (and intercept)
## Forced in Forced out
## age FALSE FALSE
## sexmale FALSE FALSE
## bmi FALSE FALSE
## children FALSE FALSE
## smokeryes FALSE FALSE
## regionnorthwest FALSE FALSE
## regionsoutheast FALSE FALSE
## regionsouthwest FALSE FALSE
## 1 subsets of each size up to 8
## Selection Algorithm: exhaustive
## age sexmale bmi children smokeryes regionnorthwest regionsoutheast
## 1 ( 1 ) " " " " " " " " "*" " " " "
## 2 ( 1 ) "*" " " " " " " "*" " " " "
## 3 ( 1 ) "*" " " "*" " " "*" " " " "
## 4 ( 1 ) "*" " " "*" "*" "*" " " " "
## 5 ( 1 ) "*" " " "*" "*" "*" " " "*"
## 6 ( 1 ) "*" " " "*" "*" "*" " " "*"
## 7 ( 1 ) "*" " " "*" "*" "*" "*" "*"
## 8 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## regionsouthwest
## 1 ( 1 ) " "
## 2 ( 1 ) " "
## 3 ( 1 ) " "
## 4 ( 1 ) " "
## 5 ( 1 ) " "
## 6 ( 1 ) "*"
## 7 ( 1 ) "*"
## 8 ( 1 ) "*"Interpretation
The best subset model is selected based on criteria such as Adjusted R², Cp, or BIC. Predictors included in the best model are considered the most important.
5. Boruta Method
Boruta uses random forests to determine feature importance.
insurance$sex <- as.factor(insurance$sex)
insurance$smoker <- as.factor(insurance$smoker)
insurance$region <- as.factor(insurance$region)
library(Boruta)
boruta_out <- Boruta(charges ~ ., data = insurance, doTrace = 2)
plot(boruta_out)
Interpretation
Boruta classifies variables into three groups:
- Confirmed: Important variables.
- Rejected: Unimportant variables.
- Tentative: Variables requiring further evaluation.
Variables marked as Confirmed are the strongest predictors of insurance charges.
Conclusion
All variable selection methods were applied to identify the most important factors affecting insurance charges. Across the methods, smoking status, age, BMI, and number of children were consistently selected as important predictors. Smoking status was found to have the greatest impact on insurance charges. These results demonstrate that variable selection techniques can improve model interpretability and help build more efficient predictive models.