DATA 621 - Homework 3 —- 11 Apr 2026

The stepwise model has the lowest AIC value, indicating it provides the best fit with the least prediction error among the models considered. As expected, the first model performed the worst, as it failed to capture much of the data’s variability. The second model, which included all variables, showed a noticeable improvement in AIC but was still not optimal. The third model, developed using stepwise selection, achieved the best performance; both forward selection and backward elimination led to the same final model, which was somewhat surprising. Based on these results, the stepwise model is the most appropriate choice, and it will be used to predict the target variable in the evaluation dataset.

step_mod <- glm(target ~ zn + rad + age + ptratio + indus + rm, family = binomial,
    data = train_df)

summary(step_mod)

Call:
glm(formula = target ~ zn + rad + age + ptratio + indus + rm, 
    family = binomial, data = train_df)

Coefficients:
             Estimate Std. Error z value Pr(>|z|)    
(Intercept) -6.837830   2.547577  -2.684  0.00727 ** 
zn          -0.054166   0.018564  -2.918  0.00353 ** 
rad          0.486803   0.111965   4.348 1.38e-05 ***
age          0.039941   0.007694   5.191 2.09e-07 ***
ptratio     -0.105136   0.069678  -1.509  0.13133    
indus        0.052943   0.027031   1.959  0.05016 .  
rm           0.439592   0.257184   1.709  0.08740 .  
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

(Dispersion parameter for binomial family taken to be 1)

    Null deviance: 645.88  on 465  degrees of freedom
Residual deviance: 277.73  on 459  degrees of freedom
AIC: 291.73

Number of Fisher Scoring iterations: 8

The final logistic regression model includes the predictors zn, rad, age, ptratio, indus, and rm. The coefficients represent changes in the log-odds of a neighborhood being classified as high crime.

Variables such as rm and zn have negative coefficients, indicating that higher values are associated with lower crime risk. In contrast, indus, rad, age, and ptratio have positive coefficients, suggesting that higher values increase the likelihood of high crime.

Overall, the coefficient signs are intuitive and align with expected relationships.

predictions <- predict(step_mod, eval_df, type = "response")

threshold <- 0.5

binary_predictions <- ifelse(predictions >= threshold, 1, 0)

eval_df$target_pred <- binary_predictions
model_names <- c("simple", "everything", "step")
aic_values <- c(simple_model$aic, everything_model$aic, step_mod$aic)

kable(cbind(model_names, aic_values), col.names = c("Model Name", "AIC Value")) |>
    kable_styling(full_width = T)

library(caret)
library(pROC)

# Convert to factor for evaluation
actual <- as.factor(eval_df$target)
predicted <- as.factor(binary_predictions)

# Confusion Matrix
conf_mat <- confusionMatrix(predicted, actual)
conf_mat

Confusion Matrix and Statistics

          Reference
Prediction  0  1
         0 23  0
         1  0 17
                                     
               Accuracy : 1          
                 95% CI : (0.9119, 1)
    No Information Rate : 0.575      
    P-Value [Acc > NIR] : 2.436e-10  
                                     
                  Kappa : 1          
                                     
 Mcnemar's Test P-Value : NA         
                                     
            Sensitivity : 1.000      
            Specificity : 1.000      
         Pos Pred Value : 1.000      
         Neg Pred Value : 1.000      
             Prevalence : 0.575      
         Detection Rate : 0.575      
   Detection Prevalence : 0.575      
      Balanced Accuracy : 1.000      
                                     
       'Positive' Class : 0          
                                     

# Extract key metrics
accuracy  <- conf_mat$overall['Accuracy']
precision <- conf_mat$byClass['Precision']
recall    <- conf_mat$byClass['Sensitivity']
specificity <- conf_mat$byClass['Specificity']
f1_score  <- conf_mat$byClass['F1']

# ROC and AUC
roc_obj <- roc(actual, predictions)
auc_value <- auc(roc_obj)

# Print results
metrics <- data.frame(
  Metric = c("Accuracy", "Precision", "Recall (Sensitivity)", 
             "Specificity", "F1 Score", "AUC"),
  Value = c(accuracy, precision, recall, specificity, f1_score, auc_value)
)

kable(metrics) %>%
  kable_styling(full_width = TRUE)

# Plot ROC Curve
plot(roc_obj, main = "ROC Curve")

To evaluate the performance of the final logistic regression model, we used several classification metrics. The confusion matrix summarizes the model’s prediction accuracy by comparing predicted and actual values. The model achieved strong performance in terms of accuracy, while precision and recall indicate a good balance between correctly identifying high-crime areas and minimizing false positives.

The F1 score further confirms this balance between precision and recall. Additionally, the ROC curve and AUC value show the model’s ability to distinguish between high- and low-crime neighborhoods. Overall, the evaluation metrics suggest that the selected model performs well and is suitable for prediction.

Prediction Results

kable(eval_df) |>
    kable_styling(full_width = T)