Data 101 - HW 10
Prasanna Sadashivaiah
Introduction:
In this homework, you will apply logistic regression to a real-world dataset: the Pima Indians Diabetes Database. This dataset contains medical records from 768 women of Pima Indian heritage, aged 21 or older, and is used to predict the onset of diabetes (binary outcome: 0 = no diabetes, 1 = diabetes) based on physiological measurements.
The data is publicly available from the UCI Machine Learning Repository and can be imported directly.
Dataset URL: https://raw.githubusercontent.com/jbrownlee/Datasets/master/pima-indians-diabetes.data.csv
Columns (no header in the CSV, so we need to assign them manually): 1. Pregnancies: Number of times pregnant 2. Glucose: Plasma glucose concentration (2-hour test) 3. BloodPressure: Diastolic blood pressure (mm Hg) 4. SkinThickness: Triceps skin fold thickness (mm) 5. Insulin: 2-hour serum insulin (mu U/ml) 6. BMI: Body mass index (weight in kg/(height in m)^2) 7. DiabetesPedigreeFunction: Diabetes pedigree function (a function scoring genetic risk) 8. Age: Age in years 9. Outcome: Class variable (0 = no diabetes, 1 = diabetes)
Task Overview: You will load the data, build a logistic regression model to predict diabetes onset using a subset of predictors (Glucose, BMI, Age), interpret the model, evaluate it with a confusion matrix and metrics, and analyze the ROC curve and AUC.
Cleaning the dataset Don’t change the following code
library(tidyverse)## ── Attaching core tidyverse packages ──────────────────────── tidyverse 2.0.0 ──
## ✔ dplyr 1.2.1 ✔ readr 2.2.0
## ✔ forcats 1.0.1 ✔ stringr 1.6.0
## ✔ ggplot2 4.0.3 ✔ tibble 3.3.1
## ✔ lubridate 1.9.5 ✔ tidyr 1.3.2
## ✔ purrr 1.2.2
## ── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
## ✖ dplyr::filter() masks stats::filter()
## ✖ dplyr::lag() masks stats::lag()
## ℹ Use the conflicted package (<http://conflicted.r-lib.org/>) to force all conflicts to become errorsurl <- "https://raw.githubusercontent.com/jbrownlee/Datasets/master/pima-indians-diabetes.data.csv"
data <- read.csv(url, header = FALSE)
colnames(data) <- c("Pregnancies", "Glucose", "BloodPressure", "SkinThickness", "Insulin", "BMI", "DiabetesPedigreeFunction", "Age", "Outcome")
data$Outcome <- as.factor(data$Outcome)
# Handle missing values (replace 0s with NA because 0 makes no sense here)
data$Glucose[data$Glucose == 0] <- NA
data$BloodPressure[data$BloodPressure == 0] <- NA
data$BMI[data$BMI == 0] <- NA
# Handle missing values (replace 0s with NA because 0 makes no sense here)
# Count missing values
colSums(is.na(data))## Pregnancies Glucose BloodPressure
## 0 5 35
## SkinThickness Insulin BMI
## 0 0 11
## DiabetesPedigreeFunction Age Outcome
## 0 0 0Question 1: Create and Interpret a Logistic Regression Model - Fit a logistic regression model to predict Outcome using Glucose, BMI, and Age.
Provide the model summary.
Calculate and interpret R²: 1 - (model\(deviance / model\)null.deviance). What does it indicate about the model’s explanatory power?
# Fit logistic regression model
model <- glm(
Outcome ~ Glucose + BMI + Age,
data = data,
family = binomial(link = "logit")
)
# Display model summary
summary(model)##
## Call:
## glm(formula = Outcome ~ Glucose + BMI + Age, family = binomial(link = "logit"),
## data = data)
##
## Coefficients:
## Estimate Std. Error z value Pr(>|z|)
## (Intercept) -9.032377 0.711037 -12.703 < 2e-16 ***
## Glucose 0.035548 0.003481 10.212 < 2e-16 ***
## BMI 0.089753 0.014377 6.243 4.3e-10 ***
## Age 0.028699 0.007809 3.675 0.000238 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## (Dispersion parameter for binomial family taken to be 1)
##
## Null deviance: 974.75 on 751 degrees of freedom
## Residual deviance: 724.96 on 748 degrees of freedom
## (16 observations deleted due to missingness)
## AIC: 732.96
##
## Number of Fisher Scoring iterations: 4# Calculate and interpret R2
model <- glm(
Outcome ~ Glucose + BMI + Age,
data = data,
family = binomial(link = "logit")
)
# Display model summary
summary(model)##
## Call:
## glm(formula = Outcome ~ Glucose + BMI + Age, family = binomial(link = "logit"),
## data = data)
##
## Coefficients:
## Estimate Std. Error z value Pr(>|z|)
## (Intercept) -9.032377 0.711037 -12.703 < 2e-16 ***
## Glucose 0.035548 0.003481 10.212 < 2e-16 ***
## BMI 0.089753 0.014377 6.243 4.3e-10 ***
## Age 0.028699 0.007809 3.675 0.000238 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## (Dispersion parameter for binomial family taken to be 1)
##
## Null deviance: 974.75 on 751 degrees of freedom
## Residual deviance: 724.96 on 748 degrees of freedom
## (16 observations deleted due to missingness)
## AIC: 732.96
##
## Number of Fisher Scoring iterations: 4# Report the R2 value
R2 <- 1 - (model$deviance / model$null.deviance)
R2## [1] 0.25626cat("Pseudo R-squared =", round(R2, 4))## Pseudo R-squared = 0.2563Interpretation: The pseudo R² value of 0.23 indicates that the logistic regression model reduces the unexplained variation in diabetes outcome by approximately 23% compared with a model containing only the intercept. This suggests that Glucose, BMI, and Age provide a moderate level of explanatory power for predicting whether a patient has diabetes. Although the model explains some of the variation in the outcome, additional predictors may be needed to improve prediction accuracy.
# Fit logistic regression model
model <- glm(
Outcome ~ Glucose + BMI + Age,
data = data,
family = binomial
)
# Model summary
summary(model)##
## Call:
## glm(formula = Outcome ~ Glucose + BMI + Age, family = binomial,
## data = data)
##
## Coefficients:
## Estimate Std. Error z value Pr(>|z|)
## (Intercept) -9.032377 0.711037 -12.703 < 2e-16 ***
## Glucose 0.035548 0.003481 10.212 < 2e-16 ***
## BMI 0.089753 0.014377 6.243 4.3e-10 ***
## Age 0.028699 0.007809 3.675 0.000238 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## (Dispersion parameter for binomial family taken to be 1)
##
## Null deviance: 974.75 on 751 degrees of freedom
## Residual deviance: 724.96 on 748 degrees of freedom
## (16 observations deleted due to missingness)
## AIC: 732.96
##
## Number of Fisher Scoring iterations: 4# Calculate pseudo R²
R2 <- 1 - (model$deviance / model$null.deviance)
cat("Pseudo R-squared =", round(R2, 4), "\n")## Pseudo R-squared = 0.2563# Odds ratios (optional but useful)
exp(coef(model))## (Intercept) Glucose BMI Age
## 0.0001194781 1.0361873320 1.0939041273 1.0291149212What does the intercept represent (log-odds of diabetes when predictors are zero)? For each predictor (Glucose, BMI, Age), does a one-unit increase raise or lower the odds of diabetes? Are they significant (p-value < 0.05)?
Conclusion
The logistic regression model shows that Glucose, BMI, and Age are all statistically significant positive predictors of diabetes. Among these variables, Glucose has the strongest effect, followed by BMI and Age. The model achieves a pseudo R² of 0.2563, indicating a moderate ability to explain diabetes outcomes and suggesting that additional predictors could improve predictive performance.
Question 2: Confusion Matrix and Important Metric
Predict probabilities using the fitted model.
Create predicted classes with a 0.5 threshold (1 if probability > 0.5, else 0).
Build a confusion matrix (Predicted vs. Actual Outcome).
Calculate and report the metrics:
Accuracy: (TP + TN) / Total Sensitivity (Recall): TP / (TP + FN) Specificity: TN / (TN + FP) Precision: TP / (TP + FP)
Use the following starter code
# Keep only rows with no missing values in Glucose, BMI, or Age
data_subset <- data[complete.cases(data[, c("Glucose", "BMI", "Age")]), ]
#Create a numeric version of the outcome (0 = no diabetes, 1 = diabetes).This is required for calculating confusion matrices.
data_subset$Outcome_num <- ifelse(data_subset$Outcome == "1", 1, 0)
# Predicted probabilities
pred_prob <- predict(model, type = "response")
# Predicted classes using 0.5 cutoff
pred_class <- ifelse(pred_prob > 0.5, 1, 0)
# Actual outcomes as numeric
actual <- as.numeric(as.character(data$Outcome))
# Remove rows excluded from model fitting due to missing values
valid_rows <- complete.cases(data[, c("Glucose", "BMI", "Age", "Outcome")])
actual <- actual[valid_rows]
# Confusion Matrix
conf_matrix <- table(
Predicted = pred_class,
Actual = actual
)
conf_matrix## Actual
## Predicted 0 1
## 0 429 114
## 1 59 150#Extract Values:
TN <- conf_matrix["0","0"]
FP <- conf_matrix["1","0"]
FN <- conf_matrix["0","1"]
TP <- conf_matrix["1","1"]
#Metrics
Accuracy <- (TP + TN) / (TP + TN + FP + FN)
Sensitivity <- TP / (TP + FN)
Specificity <- TN / (TN + FP)
Precision <- TP / (TP + FP)
cat("Accuracy:", round(Accuracy, 3), "\nSensitivity:", round(Sensitivity, 3), "\nSpecificity:", round(Specificity, 3), "\nPrecision:", round(Precision, 3))## Accuracy: 0.77
## Sensitivity: 0.568
## Specificity: 0.879
## Precision: 0.718Interpret: How well does the model perform? Is it better at detecting diabetes (sensitivity) or non-diabetes (specificity)? Why might this matter for medical diagnosis?
Model Performance Interpretation, The logistic regression model achieved the following performance metrics:
#Accuracy: 0.770 (77.0%) #Sensitivity (Recall): 0.568 (56.8%) #Specificity: 0.879 (87.9%) #Precision: 0.718 (71.8%) #Overall Performance #The model correctly classified approximately 77% of patients, indicating reasonably good overall performance. However, accuracy alone does not tell the full story, especially in medical diagnosis where the costs of different types of errors are not equal. #Sensitivity vs. Specificity #The model’s specificity (87.9%) is much higher than its sensitivity (56.8%). #Sensitivity = 56.8% #The model correctly identifies about 57% of patients who actually have diabetes. #Approximately 43% of diabetic patients are missed (false negatives). #Specificity = 87.9% #The model correctly identifies about 88% of patients who do not have diabetes. #Only about 12% of non-diabetic patients are incorrectly classified as diabetic (false positives). #Is the Model Better at Detecting Diabetes or Non-Diabetes? #The model is clearly better at detecting non-diabetes because its specificity (87.9%) is substantially higher than its sensitivity (56.8%). #This means the model is more effective at ruling out diabetes in healthy individuals than it is at identifying people who actually have diabetes. #Why Does This Matter for Medical Diagnosis? #In medical screening, sensitivity is often particularly important because failing to identify a patient who has a disease can have serious consequences. #A false negative (diabetic patient predicted as non-diabetic) may delay treatment and lifestyle interventions, potentially leading to complications such as heart disease, kidney disease, nerve damage, and vision problems. #A false positive (healthy patient predicted as diabetic) may cause additional testing and temporary anxiety, but the consequences are generally less severe. #Because this model has relatively low sensitivity, it misses a substantial proportion of diabetic patients. For a diabetes screening tool, a higher sensitivity may be preferable, even if it slightly reduces specificity. One way to improve sensitivity is to lower the classification threshold below 0.5 or include additional predictors in the model. #Precision Interpretation #The precision of 71.8% indicates that when the model predicts a patient has diabetes, it is correct about 72% of the time. This suggests that positive predictions are fairly reliable, although nearly 28% of predicted diabetic cases are actually non-diabetic.
#Conclusion #The logistic regression model demonstrates good overall accuracy and excellent specificity, but moderate sensitivity. It is better at identifying patients without diabetes than detecting those with diabetes. In a medical setting, this imbalance is important because missing true diabetes cases could delay diagnosis and treatment. Therefore, improving sensitivity would likely be a priority if the model were used for screening purposes.
Question 3: ROC Curve, AUC, and Interpretation
Plot the ROC curve, use the “data_subset” from Q2.
Calculate AUC.
#Enter your code here
library(pROC)## Type 'citation("pROC")' for a citation.##
## Attaching package: 'pROC'## The following objects are masked from 'package:stats':
##
## cov, smooth, var# Create subset used by the model (same rows retained in Q2)
data_subset <- data[complete.cases(data[, c("Glucose", "BMI", "Age", "Outcome")]), ]
# Predicted probabilities
pred_prob <- predict(model, type = "response")
# ROC Curve
roc_obj <- roc(data_subset$Outcome, pred_prob)## Setting levels: control = 0, case = 1## Setting direction: controls < cases# Plot ROC Curve
plot(
roc_obj,
col = "blue",
lwd = 3,
main = "ROC Curve for Diabetes Logistic Regression Model"
)
# Add diagonal reference line
abline(a = 0, b = 1, lty = 2, col = "red")
# Calculate AUC
auc_value <- auc(roc_obj)
auc_value## Area under the curve: 0.828What does AUC indicate (0.5 = random, 1.0 = perfect)? #The AUC can be interpreted as the probability that the model will assign a higher predicted probability to a randomly selected diabetic patient than to a randomly selected non-diabetic patient. #For example: #AUC = 0.50: The model performs no better than flipping a coin. #AUC = 0.80: There is an 80% chance that the model will rank a diabetic patient higher than a non-diabetic patient. #AUC = 1.00: The model perfectly separates diabetic and non-diabetic patients.
For diabetes diagnosis, prioritize sensitivity (catching cases) or specificity (avoiding false positives)? Suggest a threshold and explain.
#Recommendation #For diabetes screening, I would prioritize sensitivity because identifying as many true diabetes cases as possible is more important than avoiding false positives. A lower classification threshold, such as 0.35 or 0.40, would likely improve sensitivity and reduce the number of undiagnosed diabetic patients. Although this may produce more false positives, those patients can undergo additional testing for confirmation, making this trade-off acceptable in a healthcare setting.
#Conclusion #The current model favors specificity (87.9%) over sensitivity (56.8%). For diabetes screening, a lower threshold than 0.5 would be preferable because the cost of missing a true diabetes case is typically greater than the cost of conducting additional tests on someone who does not have diabetes.