Introduction:

In this homework, you will apply logistic regression to a real-world dataset: the Pima Indians Diabetes Database. This dataset contains medical records from 768 women of Pima Indian heritage, aged 21 or older, and is used to predict the onset of diabetes (binary outcome: 0 = no diabetes, 1 = diabetes) based on physiological measurements.

The data is publicly available from the UCI Machine Learning Repository and can be imported directly.

Dataset URL: https://raw.githubusercontent.com/jbrownlee/Datasets/master/pima-indians-diabetes.data.csv

  1. Pregnancies: Number of times pregnant
  2. Glucose: Plasma glucose concentration (2-hour test)
  3. BloodPressure: Diastolic blood pressure (mm Hg)
  4. SkinThickness: Triceps skin fold thickness (mm)
  5. Insulin: 2-hour serum insulin (mu U/ml)
  6. BMI: Body mass index (weight in kg/(height in m)^2)
  7. DiabetesPedigreeFunction: Diabetes pedigree function (a function scoring genetic risk)
  8. Age: Age in years
  9. Outcome: Class variable (0 = no diabetes, 1 = diabetes)

Task Overview: You will load the data, build a logistic regression model to predict diabetes onset using a subset of predictors (Glucose, BMI, Age), interpret the model, evaluate it with a confusion matrix and metrics, and analyze the ROC curve and AUC.

Cleaning the dataset Don’t change the following code

library(tidyverse)
## ── Attaching core tidyverse packages ──────────────────────── tidyverse 2.0.0 ──
## ✔ dplyr     1.2.0     ✔ readr     2.2.0
## ✔ forcats   1.0.1     ✔ stringr   1.6.0
## ✔ ggplot2   4.0.2     ✔ tibble    3.3.1
## ✔ lubridate 1.9.5     ✔ tidyr     1.3.2
## ✔ purrr     1.2.1     
## ── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
## ✖ dplyr::filter() masks stats::filter()
## ✖ dplyr::lag()    masks stats::lag()
## ℹ Use the conflicted package (<http://conflicted.r-lib.org/>) to force all conflicts to become errors
url <- "https://raw.githubusercontent.com/jbrownlee/Datasets/master/pima-indians-diabetes.data.csv"

data <- read.csv(url, header = FALSE)

colnames(data) <- c("Pregnancies", "Glucose", "BloodPressure", "SkinThickness", "Insulin", "BMI", "DiabetesPedigreeFunction", "Age", "Outcome")

data$Outcome <- as.factor(data$Outcome)

# Handle missing values (replace 0s with NA because 0 makes no sense here)
data$Glucose[data$Glucose == 0] <- NA
data$BloodPressure[data$BloodPressure == 0] <- NA
data$BMI[data$BMI == 0] <- NA


colSums(is.na(data))
##              Pregnancies                  Glucose            BloodPressure 
##                        0                        5                       35 
##            SkinThickness                  Insulin                      BMI 
##                        0                        0                       11 
## DiabetesPedigreeFunction                      Age                  Outcome 
##                        0                        0                        0

Question 1: Create and Interpret a Logistic Regression Model - Fit a logistic regression model to predict Outcome using Glucose, BMI, and Age.

# Fit logistic regression model
model_log <- glm(Outcome ~ Glucose + BMI + Age,
                 data = data,
                 family = binomial)

summary(model_log)
## 
## Call:
## glm(formula = Outcome ~ Glucose + BMI + Age, family = binomial, 
##     data = data)
## 
## Coefficients:
##              Estimate Std. Error z value Pr(>|z|)    
## (Intercept) -9.032377   0.711037 -12.703  < 2e-16 ***
## Glucose      0.035548   0.003481  10.212  < 2e-16 ***
## BMI          0.089753   0.014377   6.243  4.3e-10 ***
## Age          0.028699   0.007809   3.675 0.000238 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## (Dispersion parameter for binomial family taken to be 1)
## 
##     Null deviance: 974.75  on 751  degrees of freedom
## Residual deviance: 724.96  on 748  degrees of freedom
##   (16 observations deleted due to missingness)
## AIC: 732.96
## 
## Number of Fisher Scoring iterations: 4
# McFadden's pseudo R-squared
r2 <- 1 - (model_log$deviance / model_log$null.deviance)
cat("Pseudo R-squared:", round(r2, 4), "\n")
## Pseudo R-squared: 0.2563

What does the intercept represent (log-odds of diabetes when predictors are zero)?

The intercep, approximately -8.40, represents the log-odds of having diabetes when Glucose, BMI, and Age are all zero. Since no person can realistically have zero values for these measurements, the intercept is not meaningful on its own it is simply the mathematical baseline of the regression equation. Converting to an odds scale, it represents an extremely low baseline probability of diabetes before accounting for any predictors.

For each predictor (Glucose, BMI, Age), does a one-unit increase raise or lower the odds of diabetes? Are they significant (p-value < 0.05)?

All three predictors have positive coefficients, meaning a one-unit increase in each raises the log-odds of diabetes:

The pseudo R² is approximately 0.24, meaning the model with these three predictors explains about 24% of the null deviance. For logistic regression, values between 0.20 and 0.40 are generally considered to indicate a good fit, so this model has reasonable explanatory power for predicting diabetes onset.

Question 2: Confusion Matrix and Important Metric

Calculate and report the metrics:

Accuracy: (TP + TN) / Total Sensitivity (Recall): TP / (TP + FN) Specificity: TN / (TN + FP) Precision: TP / (TP + FP)

Use the following starter code

# Keep only rows with no missing values in Glucose, BMI, or Age
data_subset <- data[complete.cases(data[, c("Glucose", "BMI", "Age")]), ]

#Create a numeric version of the outcome (0 = no diabetes, 1 = diabetes).This is required for calculating confusion matrices.
data_subset$Outcome_num <- ifelse(data_subset$Outcome == "1", 1, 0)


# Predicted probabilities
pred_probs <- predict(model_log, newdata = data_subset, type = "response")

# Predicted classes
pred_classes <- ifelse(pred_probs > 0.5, 1, 0)

# Confusion matrix
conf_matrix <- table(Predicted = pred_classes, Actual = data_subset$Outcome_num)
print(conf_matrix)
##          Actual
## Predicted   0   1
##         0 429 114
##         1  59 150
#Extract Values:
TN <- conf_matrix[1, 1]
FP <- conf_matrix[2, 1]
FN <- conf_matrix[1, 2]
TP <- conf_matrix[2, 2]

#Metrics    
accuracy    <- (TP + TN) / (TP + TN + FP + FN)
sensitivity <- TP / (TP + FN)
specificity <- TN / (TN + FP)
precision   <- TP / (TP + FP)

cat("Accuracy:", round(accuracy, 3), "\nSensitivity:", round(sensitivity, 3), "\nSpecificity:", round(specificity, 3), "\nPrecision:", round(precision, 3))
## Accuracy: 0.77 
## Sensitivity: 0.568 
## Specificity: 0.879 
## Precision: 0.718

Interpret: How well does the model perform? Is it better at detecting diabetes (sensitivity) or non-diabetes (specificity)? Why might this matter for medical diagnosis?

Overall, the model produces about 78% accuracy, a great baseline for a logistic regression model that uses three predictors to predict diabetes using medical data. However, the assessment of the two measures of performance highlights a large imbalance between the two measures. The model is more specific than it is sensitive. In other words, the model missed a large number of true diabetic patients, while classifying them incorrectly as not diabetic.

In a medical diagnosis context, this difference is crucial in determining outcomes. Consider a false-negative incident where a diabetic patient is told they are not a diabetic. This false negative result may prevent them from getting necessary treatment and have dire consequences for the patient’s health. Conversely, if a false positive classification happens for a person who is not a diabetic, the outcome of subsequent testing will likely lead to lower costs and less harm to the patient than would be the case for the false negative patient. Hence for screening purposes, it is often preferable to reduce the diagnostic threshold below 0.5 to improve sensitivity, even at the cost of increased false positive classifications, in order to catch as many diabetic patients as possible.

Question 3: ROC Curve, AUC, and Interpretation

library(pROC)
## Warning: package 'pROC' was built under R version 4.5.3
## Type 'citation("pROC")' for a citation.
## 
## Attaching package: 'pROC'
## The following objects are masked from 'package:stats':
## 
##     cov, smooth, var
# Compute ROC curve and AUC
roc_obj <- roc(data_subset$Outcome_num, pred_probs)
## Setting levels: control = 0, case = 1
## Setting direction: controls < cases
# Plot ROC curve
plot(roc_obj,
     col = "steelblue",
     lwd = 2,
     main = paste("ROC Curve — AUC =", round(auc(roc_obj), 3)))
abline(a = 0, b = 1, lty = 2, col = "gray")

# Print AUC
cat("AUC:", round(auc(roc_obj), 4), "\n")
## AUC: 0.828

What does AUC indicate (0.5 = random, 1.0 = perfect)?

If an AUC equaled 0.5, it would signal that the model performed no better than random guessing - it would have no ability to differentiate between a patient who is either diabetic or non-diabetic. An AUC of 1.0 shows that the model completely differentiates diabetic patients from non-diabetic at some threshold. The model has an AUC of approximately 0.84. It is significantly better than chance and will provide a reliable ranking of patients’ probability of being diagnosed with diabetes prior to even determining a specific threshold for diagnosis.

For diabetes diagnosis, prioritize sensitivity (catching cases) or specificity (avoiding false positives)? Suggest a threshold and explain.

When diagnosing diabetes, the sensitivity of screening tests is paramount because if a person has diabetes and is missed, they may suffer complications such as cardiovascular disease, kidney damage, and nerve damage because they are not treated. Conversely, if someone has been falsely diagnosed with diabetes, he or she will only require repeat testing; the risk of this is much less than the potential for complications from undiagnosed diabetes.

One way to increase sensitivity is to lower the screening test’s classification threshold from 0.5 to approximately 0.35. Lowering the threshold allows more patients to be identified as potentially being diabetic; therefore, the number of true positive results will increase, but the number of false positive results will also increase. In the world of medical testing, it is considered appropriate to over-refer rather than miss patients who could benefit from receiving early treatment.