HW 10
Victoria
Introduction:
In this homework, you will apply logistic regression to a real-world dataset: the Pima Indians Diabetes Database. This dataset contains medical records from 768 women of Pima Indian heritage, aged 21 or older, and is used to predict the onset of diabetes (binary outcome: 0 = no diabetes, 1 = diabetes) based on physiological measurements.
The data is publicly available from the UCI Machine Learning Repository and can be imported directly.
Dataset URL: https://raw.githubusercontent.com/jbrownlee/Datasets/master/pima-indians-diabetes.data.csv
Columns (no header in the CSV, so we need to assign them manually):
- Pregnancies: Number of times pregnant
- Glucose: Plasma glucose concentration (2-hour test)
- BloodPressure: Diastolic blood pressure (mm Hg)
- SkinThickness: Triceps skin fold thickness (mm)
- Insulin: 2-hour serum insulin (mu U/ml)
- BMI: Body mass index (weight in kg/(height in m)^2)
- DiabetesPedigreeFunction: Diabetes pedigree function (a function scoring genetic risk)
- Age: Age in years
- Outcome: Class variable (0 = no diabetes, 1 = diabetes)
Task Overview: You will load the data, build a logistic regression model to predict diabetes onset using a subset of predictors (Glucose, BMI, Age), interpret the model, evaluate it with a confusion matrix and metrics, and analyze the ROC curve and AUC.
Cleaning the dataset Don’t change the following code
library(tidyverse)## ── Attaching core tidyverse packages ──────────────────────── tidyverse 2.0.0 ──
## ✔ dplyr 1.1.4 ✔ readr 2.1.6
## ✔ forcats 1.0.1 ✔ stringr 1.5.2
## ✔ ggplot2 4.0.0 ✔ tibble 3.3.0
## ✔ lubridate 1.9.4 ✔ tidyr 1.3.1
## ✔ purrr 1.1.0
## ── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
## ✖ dplyr::filter() masks stats::filter()
## ✖ dplyr::lag() masks stats::lag()
## ℹ Use the conflicted package (<http://conflicted.r-lib.org/>) to force all conflicts to become errorsurl <- "https://raw.githubusercontent.com/jbrownlee/Datasets/master/pima-indians-diabetes.data.csv"
data <- read.csv(url, header = FALSE)
colnames(data) <- c("Pregnancies", "Glucose", "BloodPressure", "SkinThickness", "Insulin", "BMI", "DiabetesPedigreeFunction", "Age", "Outcome")
data$Outcome <- as.factor(data$Outcome)
# Handle missing values (replace 0s with NA because 0 makes no sense here)
data$Glucose[data$Glucose == 0] <- NA
data$BloodPressure[data$BloodPressure == 0] <- NA
data$BMI[data$BMI == 0] <- NA
colSums(is.na(data))## Pregnancies Glucose BloodPressure
## 0 5 35
## SkinThickness Insulin BMI
## 0 0 11
## DiabetesPedigreeFunction Age Outcome
## 0 0 0Question 1: Create and Interpret a Logistic Regression Model - Fit a logistic regression model to predict Outcome using Glucose, BMI, and Age.
Provide the model summary.
Calculate and interpret R²: 1 - (model\(deviance / model\)null.deviance). What does it indicate about the model’s explanatory power?
## Enter your code here
model <- glm(Outcome ~ Glucose +BMI + Age, data = data, family = binomial)
summary(model)##
## Call:
## glm(formula = Outcome ~ Glucose + BMI + Age, family = binomial,
## data = data)
##
## Coefficients:
## Estimate Std. Error z value Pr(>|z|)
## (Intercept) -9.032377 0.711037 -12.703 < 2e-16 ***
## Glucose 0.035548 0.003481 10.212 < 2e-16 ***
## BMI 0.089753 0.014377 6.243 4.3e-10 ***
## Age 0.028699 0.007809 3.675 0.000238 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## (Dispersion parameter for binomial family taken to be 1)
##
## Null deviance: 974.75 on 751 degrees of freedom
## Residual deviance: 724.96 on 748 degrees of freedom
## (16 observations deleted due to missingness)
## AIC: 732.96
##
## Number of Fisher Scoring iterations: 4# r square
r_square <- 1 - (model$deviance/model$null.deviance)
r_square## [1] 0.25626Interpretation r-square:
The r-square value of ~0.26 means that the model explains ~ 26% of the variation which indicates moderate explanotory power and according to the coeficients provided glucose is typically the strongest predictor, followed by BMI and age.
What does the intercept represent (log-odds of diabetes when predictors are zero)? The intercept represents the log-odds of having diabetes when all predictors are 0. The amount resulted for the intercept in this case is very negative because the predictors being zero do not reflect a real scenario.
For each predictor (Glucose, BMI, Age), does a one-unit increase raise or lower the odds of diabetes? Are they significant (p-value < 0.05)?
Glucose’s coeficient is positive so an increase would raise the odds of diabetes and since the p-value < 2e-16 it is highly significant.
BMI’s coeficient is also positive so an increase would also raise the odds of diabetes. The p-value = 4.3e-10 which is also highly significant.
Age’s coeficient is positive as well so again an increase would also raise the odds of diabetes and since the p-value = 0.000238 it is moderately significant.
Question 2: Confusion Matrix and Important Metric
Predict probabilities using the fitted model.
Create predicted classes with a 0.5 threshold (1 if probability > 0.5, else 0).
Build a confusion matrix (Predicted vs. Actual Outcome).
Calculate and report the metrics:
Accuracy: (TP + TN) / Total Sensitivity (Recall): TP / (TP + FN) Specificity: TN / (TN + FP) Precision: TP / (TP + FP)
Use the following starter code
# Keep only rows with no missing values in Glucose, BMI, or Age
data_subset <- data[complete.cases(data[, c("Glucose", "BMI", "Age")]), ]
#Create a numeric version of the outcome (0 = no diabetes, 1 = diabetes).This is required for calculating confusion matrices.
data_subset$Outcome_num <- ifelse(data_subset$Outcome == "1", 1, 0)
# Predicted probabilities
predicted_probs <- predict(model, newdata = data_subset, type = "response")
# Predicted classes
predicted_classes <- ifelse(predicted_probs > 0.5, 1, 0)
# Confusion matrix
confusion <- table(
Predicted = factor(predicted_classes, levels = c(0, 1)),
Actual = factor(data_subset$Outcome_num, levels = c(0, 1))
)
confusion## Actual
## Predicted 0 1
## 0 429 114
## 1 59 150#Extract Values:
TN <- 429
FP <- 59
FN <- 114
TP <- 150
#Metrics
accuracy <- (TP + TN) / (TP + TN + FP + FN)
sensitivity <- TP / (TP + FN)
specificity <- TN / (TN + FP)
precision <- TP / (TP + FP)
cat("Accuracy:", round(accuracy, 3), "\nSensitivity:", round(sensitivity, 3), "\nSpecificity:", round(specificity, 3), "\nPrecision:", round(precision, 3))## Accuracy: 0.77
## Sensitivity: 0.568
## Specificity: 0.879
## Precision: 0.718The model has moderate accuracy and only detects 57% of diabetes cases, but it misses 43%. The model correctly identifies 88% of non-diabetic individuals meaning false positives are relatively low. Lastly since precision is about 72% the results are reliable, but could be improved.
Interpret: How well does the model perform? Is it better at detecting diabetes (sensitivity) or non-diabetes (specificity)? Why might this matter for medical diagnosis?
The model has moderate accuracy and only detects 57% of diabetes cases, but it misses 43%. The model correctly identifies 88% of non-diabetic individuals meaning false positives are relatively low. Lastly since precision is about 72% the results are reliable, but could be improved
We can conclude the model is better at specificity than sensitivity because it performed better detecting non-diabetes cases. Underperforming sensitivity in this case means diabetic patients go undetected so they might not receive treatment and increase risk of complications.
Question 3: ROC Curve, AUC, and Interpretation
Plot the ROC curve, use the “data_subset” from Q2.
Calculate AUC.
#Enter your code here
library(pROC)## Warning: package 'pROC' was built under R version 4.5.2## Type 'citation("pROC")' for a citation.##
## Attaching package: 'pROC'## The following objects are masked from 'package:stats':
##
## cov, smooth, varroc_obj <- roc(response = data_subset$Outcome_num,
predictor = predicted_probs)## Setting levels: control = 0, case = 1## Setting direction: controls < casesplot(roc_obj,
col ="green",
lwd = 3,
main = "ROC Curve for Diabetes")
auc_value <- auc(roc_obj)
auc_value## Area under the curve: 0.828What does AUC indicate (0.5 = random, 1.0 = perfect)?
The AUC = 0.82 which means the model is very good at distinguishing between diabetic and non-diabetic individuals.
For diabetes diagnosis, prioritize sensitivity (catching cases) or specificity (avoiding false positives)? Suggest a threshold and explain.
Sensitivity should be prioritized meaning catching true diabetics cases because a false negative/ missed case can be dangerous if left untreated. A lower threshhold than the one currently used(0.5) could be benificial to capture the missing cases.