Overview

In this markdown I summarize the results from our first arrest cnn prediction test. The goal is to see how different the models predictions for release are for real images and their respective projection through the GAN. As a reminder, projected images are those for which the GAN finds the closest approximate image in its own latent space w.r.t some real mugshot.

Here I randomly sampled 2000 images from our mugshots dataset and projected each individually to obtain 2000 pairs of real-projected images. I then produce two summary plots together with out baseline table 01:

  1. Plot 1 Comparing their respective p_hat_cnn distributions. Ideally these would align as closely as possible

  2. Plot 2 Checking the correlation between p_hat_cnn_projection, the prediction for projected images, and p_hat_cnn_target, the cnn prediction for the real image counterpart. Here we want to see as close of a relationship as possible

  3. Table 01 Configuration 1 I repeat the regressions for our baseline table 01 comparing between the p_hat_cnn values for projections and targets. Note The definitions for the terms used in table 01 are above the table.

Definitions

There are two different types of images here:

  1. Targets: These are the real mugshots that I randomly sampled.

  2. Projections: These are the GAN generated images which try to approximate the real targets as close as possible.

An ideal projector will produce generated images which are indistinguishable from their target counterpart.

Table 01 - Configuration 01 - Comparing CNN output for Projections and Targets

I repeat our baseline table 01 and compare the predictive power of the p_hat_cnn values from our projections and targets. Below is an outline of the definitions:

Definitions for Table 01

As a reminder (these are the same definitions as in previous itterations), here is a list of the regression terms. I split them into definitions for our models and our features:

Model Definitions

  1. Demographic LM: This includes sex and age_arrest to predict our arrest-outcome

  2. Charge Feature LM : This includes felony_flag, gun_crime_flag, drug_crime_flag, violent_crime_flag, property_crime_flag, arrest_year

  3. XgBoost risk: As the name would suggest, this is our XgBoost risk predictor using time-varying arrest history features

Feature Definitions

  1. MTurk Features: These are our high-detail (minimum of 6 workers per image) MTurk features together with their median

  2. Kitchen Sink: The final row of the table includes all previous rows, it is our fully stacked model with all covariates.

NOTE The column titled Mugshot Targets uses the p_hat_cnn of the real mugshot targets. The column titled GAN Projections, as the name would suggest, uses the p_hat_cnn from the corresponding GAN projections.

Table 01 - Version 01 - Model Comparison
Fit measured in adjusted R squared and AUC
Model Configuration Mugshot Targets GAN Projections
Adjusted R Squared ROC AUC Adjusted R Squared ROC AUC
Single Variable Model
Demographic LM 0.0056 0.5588 0.0056 0.5588
Lower 95% C.I. −0.0013 0.5007 −0.0012 0.5007
Upper 95% C.I. 0.0206 0.6168 0.0202 0.6168
Charge Feature LM 0.1144 0.7030 0.1144 0.7030
0.0680 0.6462 0.0682 0.6462
0.1674 0.7598 0.1679 0.7598
XgBoost Risk 0.0449 0.6243 0.0449 0.6243
0.0172 0.5675 0.0157 0.5675
0.0847 0.6811 0.0907 0.6811
MTurk Features (Mean + Median) 0.0360 0.6773 0.0360 0.6773
0.0311 0.6225 0.0287 0.6225
0.1221 0.7321 0.1233 0.7321
P_hat_cnn 0.0463 0.6402 0.0429 0.6191
0.0191 0.5852 0.0189 0.5633
0.0820 0.6952 0.0783 0.6749
Combined Variable Model
Demographics + Charge Feature 0.1119 0.7039 0.1119 0.7039
0.0671 0.6476 0.0677 0.6476
0.1638 0.7601 0.1650 0.7601
Demographics + Charge Feature + Risk 0.1300 0.7185 0.1300 0.7185
0.0847 0.6618 0.0836 0.6618
0.1885 0.7752 0.1843 0.7752
Demographics + Charge Feature + Risk + MTurk (Mean + Median) 0.1537 0.7739 0.1537 0.7739
0.1301 0.7238 0.1300 0.7238
0.2532 0.8239 0.2476 0.8239
Demographics + Charge Feature + Risk + CNN 0.1419 0.7376 0.1428 0.7379
0.0942 0.6832 0.0989 0.6847
0.1996 0.7919 0.2041 0.7910
Kitchen Sink (all RHS variables included) 0.1600 0.7806 0.1635 0.7822
0.1390 0.7313 0.1397 0.7334
0.2597 0.8300 0.2594 0.8310