R Notebook

This is an R Markdown Notebook. When you execute code within the notebook, the results appear beneath the code.

Try executing this chunk by clicking the Run button within the chunk or by placing your cursor inside it and pressing Cmd+Shift+Enter.

library(dplyr)

Attaching package: ‘dplyr’

The following objects are masked from ‘package:stats’:

    filter, lag

The following objects are masked from ‘package:base’:

    intersect, setdiff, setequal, union

head(compas_scores_two_years)

# Data Cleaning and Preparation
data <- compas_scores_two_years %>%
  select(id, sex, age, race, priors_count...9, is_recid) %>%
  filter(!is.na(is_recid))

# Split into training and testing sets (75% training data) #learned about logistic training from various sources including https://stackoverflow.com/questions/58617114/logistic-regression-training-and-test-data, https://stats.stackexchange.com/questions/94770/when-to-divide-data-into-training-test-set-in-logistic-regression, https://www.datacamp.com/tutorial/logistic-regression-R
set.seed(8675309)
training_rows <- sample(nrow(data), 0.75 * nrow(data))
train_data <- data[training_rows, ]
test_data <- data[-training_rows, ]

# Fit Logistic Regression Model on Training Data
model <- glm(is_recid ~ age + race + priors_count...9, data = train_data, family = "binomial")

# Predict on Testing Data
test_data$predicted_recid <- predict(model, newdata = test_data, type = "response")

# Convert probabilities to binary outcomes
test_data$predicted_class <- ifelse(test_data$predicted_recid > 0.5, 1, 0)

# Assess Model Performance(Here we calculate the overall accuracy)
correct_predictions <- test_data$predicted_class == test_data$is_recid
accuracy <- sum(correct_predictions) / length(correct_predictions)
print(paste("Accuracy: ", accuracy))

[1] "Accuracy:  0.667960088691796"

# Assess Model Performance Across Races
accuracy_by_race <- test_data %>%
  group_by(race) %>%
  summarise(
    count = n(),
    correct = sum(predicted_class == is_recid),
    accuracy = correct / count
  )

print(accuracy_by_race)

# To visualize the results:
library(ggplot2)

Warning: package ‘ggplot2’ was built under R version 4.3.2

cbPalette <- c("#000000", "#E69F00", "#56B4E9", "#009E73", "#F0E442", "#0072B2", "#D55E00", "#CC79A7")

ggplot(accuracy_by_race, aes(x = race, y = accuracy, fill = race)) +
  geom_bar(stat = "identity") +
  scale_colour_manual(values=cbPalette) +
  theme_minimal() +
  labs(title = "Accuracy of Recidivism Prediction by Race")

#to try and understand the graphh above we look at predicted rate vs actual recidivism
average_scores <- test_data %>%
  group_by(race) %>%
  summarise(avg_predicted_class = mean(as.numeric(predicted_class), na.rm = TRUE),
            actual_recidivism_rate = mean((is_recid), na.rm = TRUE))

#Compare these averages
average_scores

#anova test
anova_result <- aov(is_recid ~ race, data = compas_scores_two_years)
summary(anova_result)

              Df Sum Sq Mean Sq F value Pr(>F)    
race           5   38.6   7.720   31.58 <2e-16 ***
Residuals   7208 1762.3   0.244                   
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

#p values for model
summary(model)

Call:
glm(formula = is_recid ~ age + race + priors_count...9, family = "binomial", 
    data = train_data)

Coefficients:
                     Estimate Std. Error z value Pr(>|z|)    
(Intercept)          1.173914   0.095492  12.293  < 2e-16 ***
age                 -0.048970   0.002800 -17.490  < 2e-16 ***
raceAsian           -0.121598   0.453322  -0.268  0.78852    
raceCaucasian       -0.124821   0.066850  -1.867  0.06188 .  
raceHispanic        -0.331490   0.108192  -3.064  0.00218 ** 
raceNative American  0.164299   0.691657   0.238  0.81223    
raceOther           -0.265813   0.133867  -1.986  0.04707 *  
priors_count...9     0.163044   0.008224  19.824  < 2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

(Dispersion parameter for binomial family taken to be 1)

    Null deviance: 7493.5  on 5409  degrees of freedom
Residual deviance: 6643.1  on 5402  degrees of freedom
AIC: 6659.1

Number of Fisher Scoring iterations: 4

Add a new chunk by clicking the Insert Chunk button on the toolbar or by pressing Cmd+Option+I.

When you save the notebook, an HTML file containing the code and output will be saved alongside it (click the Preview button or press Cmd+Shift+K to preview the HTML file).

The preview shows you a rendered HTML copy of the contents of the editor. Consequently, unlike Knit, Preview does not run any R code chunks. Instead, the output of the chunk when it was last run in the editor is displayed.