Quick Overview:

Here I’m comparing the predictions from the JUST FACE and the FACE + STRUCTURED DATA and JUST STRUCTURED data models. These are called ML Face CNN and ML Face Fusion and ML Structured respectively. The fusion model is a first pass that only includes data on:

The main point here is that the adjusted r-squared for the fusion model is significantly higher than just the face (as seen in table 01). This goes from 0.0332 to 0.1030. It’s also very promising that in table 2, where I’m including the charge data, the gun crime and property crime flags are insignificant with ML Face Fusion. I’m thinking this indicates the model is using that data to make its prediction !

Regression Table 01 - Baseline Models

Table No.1 - Comparing ML Face & ML Fusion
Dependent variable:
Release-Final-Outcome
Both S F
(1) (2) (3) (4) (5)
ML-Face Fusion 1.1126*** 0.5826***
(1.0517, 1.1735) (0.4557, 0.7095)
ML Structured 1.1553*** 1.0637*** 0.5329***
(1.0871, 1.2235) (0.9950, 1.1324) (0.3985, 0.6672)
ML-Face CNN 0.6960*** 0.5079*** 0.3777***
(0.6263, 0.7656) (0.4399, 0.5759) (0.3043, 0.4512)
Constant -0.1120*** -0.1532*** 0.2437*** -0.4579*** -0.3983***
(-0.1604, -0.0636) (-0.2077, -0.0987) (0.1914, 0.2959) (-0.5255, -0.3902) (-0.4670, -0.3296)
Observations 7,851 7,851 7,851 7,851 7,851
Adjusted R2 0.1030 0.0898 0.0332 0.1069 0.1132
F Statistic 902.3631*** (df = 1; 7849) 775.7189*** (df = 1; 7849) 270.3141*** (df = 1; 7849) 470.6992*** (df = 2; 7848) 335.0501*** (df = 3; 7847)
Note: p<0.1; p<0.05; p<0.01

Regression Table 02 - Including covariates used in ML-Fusion training:

To evaluate the ML-Fusion model, in this table we have to compare the adjusted R-sqrt and AUC in col (3) to col (7) … there is no more 2% gap !

Table No.2 - How much R2 can we expect from the ML-Structured ?
Dependent variable:
Release-Final-Outcome P-hat Structured
(1) (2) (3) (4) (5) (6) (7) (8)
p_hat_structured 1.155*** 0.866*** 0.818***
(0.041) (0.293) (0.290)
p_hat_cnn 0.696*** 0.605***
(0.042) (0.045)
p_hat_cnn_fusion 1.113*** 0.890***
(0.037) (0.076)
age_arrest -0.0005 0.0001 0.0005 0.001** -0.001***
(0.0004) (0.0005) (0.0005) (0.0004) (0.00002)
felony -0.212*** -0.059 -0.068 -0.057*** -0.176***
(0.010) (0.053) (0.052) (0.017) (0.0004)
gun_crime 0.044** 0.033* 0.028 0.017 0.013***
(0.018) (0.019) (0.018) (0.018) (0.001)
drug_crime 0.032** 0.029** 0.037*** 0.042*** 0.003***
(0.013) (0.013) (0.012) (0.012) (0.0005)
violent_crime -0.175*** -0.075** -0.071* -0.053*** -0.116***
(0.017) (0.038) (0.037) (0.020) (0.001)
property_crime -0.044*** -0.001 0.005 0.006 -0.049***
(0.011) (0.018) (0.018) (0.012) (0.0004)
sexM -0.088*** -0.031 0.039* -0.047*** -0.067***
(0.011) (0.023) (0.023) (0.012) (0.0004)
raceB -0.012 0.077 0.087 -0.018 -0.103***
(0.052) (0.060) (0.060) (0.052) (0.002)
Constant -0.153*** 0.244*** -0.112*** 0.977*** 0.048 -0.438 0.114 1.073***
(0.033) (0.032) (0.029) (0.055) (0.319) (0.318) (0.092) (0.002)
ROC-AUC 0.696 0.696 0.71 0.698 0.696 0.721 0.712
Observations 7,851 7,851 7,851 7,851 7,851 7,851 7,851 7,851
Adjusted R2 0.090 0.033 0.103 0.092 0.093 0.113 0.108 0.980
F Statistic 775.719*** (df = 1; 7849) 270.314*** (df = 1; 7849) 902.363*** (df = 1; 7849) 73.642*** (df = 11; 7839) 68.298*** (df = 12; 7838) 78.060*** (df = 13; 7837) 80.013*** (df = 12; 7838) 35,043.150*** (df = 11; 7839)
Note: p<0.1; p<0.05; p<0.01

Regression Table 03 - Including Current Charge Dummies and Risk

Table No.3 - Comparing ML Face & ML Fusion
Dependent variable:
Release-Final-Outcome
(1) (2) (3) (4) (5) (6)
SexM -0.0279** -0.0203 0.0025 -0.0306*** -0.0208 -0.0001
(-0.0477, -0.0080) (-0.0556, 0.0151) (-0.0185, 0.0235) (-0.0500, -0.0112) (-0.0559, 0.0143) (-0.0206, 0.0204)
RaceB 0.0360 0.1106* 0.0504 0.0098 0.0851 0.0284
(-0.0501, 0.1221) (0.0125, 0.2088) (-0.0355, 0.1363) (-0.0750, 0.0946) (-0.0120, 0.1822) (-0.0562, 0.1130)
Age 0.0010 0.0001 0.00004 0.0008 -0.0003 -0.0001
(-0.00002, 0.0020) (-0.0010, 0.0012) (-0.0010, 0.0010) (-0.0002, 0.0018) (-0.0014, 0.0008) (-0.0011, 0.0009)
† skin-tone 0.0542*** 0.0614*** 0.0469**
(0.0215, 0.0870) (0.0285, 0.0943) (0.0142, 0.0796)
Attractiveness -0.0159 -0.0139 -0.0139
(-0.0488, 0.0170) (-0.0470, 0.0192) (-0.0467, 0.0189)
Competence 0.0106 0.0141 0.0101
(-0.0223, 0.0436) (-0.0191, 0.0473) (-0.0228, 0.0430)
Dominance -0.0026 -0.0057 -0.0002
(-0.0295, 0.0243) (-0.0327, 0.0214) (-0.0271, 0.0266)
Trustworthiness 0.0312 0.0313 0.0280
(-0.0013, 0.0637) (-0.0014, 0.0640) (-0.0044, 0.0604)
Felony-Dummie -0.0653*** -0.0890* -0.1991*** -0.0606*** -0.0813 -0.1986***
(-0.0924, -0.0382) (-0.1703, -0.0077) (-0.2158, -0.1824) (-0.0876, -0.0336) (-0.1626, 0.0001) (-0.2153, -0.1819)
Gun-Crime-Dummie 0.0186 0.0343* 0.0382** 0.0170 0.0330* 0.0372**
(-0.0112, 0.0484) (0.0039, 0.0647) (0.0087, 0.0677) (-0.0128, 0.0468) (0.0027, 0.0634) (0.0077, 0.0667)
Drug-Crime-Dummie 0.0437*** 0.0340*** 0.0415*** 0.0427*** 0.0319** 0.0410***
(0.0234, 0.0640) (0.0135, 0.0544) (0.0213, 0.0617) (0.0224, 0.0630) (0.0115, 0.0523) (0.0208, 0.0612)
Violent-Crime-Dummie -0.0685*** -0.1020*** -0.1649*** -0.0662*** -0.0984*** -0.1654***
(-0.1005, -0.0365) (-0.1609, -0.0432) (-0.1920, -0.1377) (-0.0981, -0.0342) (-0.1573, -0.0395) (-0.1925, -0.1382)
Property-Crime-Dummie 0.0099 -0.0018 -0.0248** 0.0101 -0.0010 -0.0255**
(-0.0090, 0.0288) (-0.0299, 0.0264) (-0.0425, -0.0071) (-0.0088, 0.0290) (-0.0292, 0.0272) (-0.0432, -0.0078)
Risk Prediction -0.7867*** -0.8276*** -0.7759*** -0.7995*** -0.8479*** -0.7836***
(-0.8970, -0.6764) (-0.9384, -0.7169) (-0.8859, -0.6658) (-0.9095, -0.6895) (-0.9584, -0.7375) (-0.8933, -0.6738)
Well-groomed 0.0089* 0.0101** 0.0081
(0.0006, 0.0171) (0.0018, 0.0184) (-0.0001, 0.0163)
Messy 0.0032 0.0011 0.0051
(-0.0042, 0.0105) (-0.0063, 0.0085) (-0.0022, 0.0125)
ML-Face Fusion 0.7738*** 0.7971***
(0.6494, 0.8982) (0.6736, 0.9207)
ML Structured 0.6225** 0.6618**
(0.1692, 1.0759) (0.2084, 1.1152)
ML-Face CNN 0.5470*** 0.5607***
(0.4710, 0.6229) (0.4861, 0.6354)
Constant 0.2921*** 0.3846 0.5612*** 0.4007*** 0.4864 0.6734***
(0.1180, 0.4662) (-0.1181, 0.8872) (0.4200, 0.7024) (0.2445, 0.5570) (-0.0120, 0.9849) (0.5553, 0.7914)
Observations 7,851 7,851 7,851 7,851 7,851 7,851
Adjusted R2 0.1243 0.1131 0.1282 0.1233 0.1113 0.1276
F Statistic 56.6999*** (df = 20; 7830) 51.0738*** (df = 20; 7830) 58.7155*** (df = 20; 7830) 85.8911*** (df = 13; 7837) 76.6324*** (df = 13; 7837) 89.3554*** (df = 13; 7837)
Note: p<0.1; p<0.05; p<0.01
† We encode skin-tone as a continuous variable increasing in brightness