Arrest - Table No. 01 - Configurations 1 and 2

Summary

In this markdown I repeat our baseline arrest regression tables with the inclusion of:

Additional MTurk label row which now also includes additional moments (mean + median)
I split our Table 01 Configuration 01 by gender and notice a signficant MTurk effect for females.
For Table 01 Config 02 I also include the additional MTurk moments
I also repeat Table 01 Configuration 01 with the inclusion of an interaction term between sex and MTurk labels

Definitions

Demographics LM: model predicting final-arrest-outcome using sex and age_arrest
Charge Feature LM: model predicting final-arrest-outcome using felony_flag, gun_crime_flag, drug_crime_flag, violent_crime_flag, property_crime_flag, arrest_year
XgBoost risk: this is a boosted-tree using our historical and time-varying arrest-history data to predict re-arrest. We use this as a proxy for predicted risk. Note that we have always been using an XgBoost model, I am now only being more explicit with my naming scheme.
MTurk features: here I am including both the mean and median value for attractiveness, competence, dominance, trustworthiness. This model also includes skin_tone
CNN Predicted Probability: These are the predicted-probabilities from our baseline CNN.

Baselin Table: Table 01 - Config 01

This is our baseline arrest table and this includes both single and combined-variable models:

Single-Variable models:

Demographics LM: model predicting final-arrest-outcome using sex and age_arrest
Charge Feature LM: model predicting final-arrest-outcome using felony_flag, gun_crime_flag, drug_crime_flag, violent_crime_flag, property_crime_flag, arrest_year
XgBoost risk: this is a boosted-tree using our historical and time-varying arrest-history data
MTurk features: here I am including both the mean and median value for attractiveness, competence, dominance, trustworthiness. This model also includes skin_tone
CNN Predicted Probability

Combined-Variable models:

Demographic LM + Charge Feature LM (in a combined model)
Demographic LM + Charge Feature LM + XgBoost Risk
Demographic LM + Charge Feature LM + XgBoost Risk + MTurk (mean + median)
Demographic LM + Charge Feature LM + XgBoost Risk + CNN
Demographic LM + Charge Feature LM + XgBoost Risk + MTurk + CNN

NOTE For Demographic LM + Charge Feature LM we include all charge feature and demographic variables in the same model (this is more parsimonious).

Table 01 - Version 01 - Arrest Regressions
Fit measured in adjusted R squared and AUC
Model Configuration	Male-Female Combined
Model Configuration	Adjusted R Squared	ROC AUC
Single Variable Model
Demographic LM	0.0101	0.5552
Lower 95% C.I.	0.0071	0.5412
Upper 95% C.I.	0.0137	0.5692
Charge Feature LM	0.0906	0.6910
	0.0803	0.6777
	0.1018	0.7042
XgBoost Risk	0.0332	0.6098
	0.0271	0.5961
	0.0410	0.6234
MTurk Features (Mean + Median)	0.0002	0.5346
	0.0007	0.5202
	0.0052	0.5489
P_hat_cnn	0.0325	0.6222
	0.0268	0.6087
	0.0388	0.6358
Combined Variable Model
Demographics + Charge Feature	0.0973	0.7011
	0.0862	0.6880
	0.1080	0.7141
Demographics + Charge Feature + Risk	0.1123	0.7177
	0.1010	0.7050
	0.1250	0.7303
Demographics + Charge Feature + Risk + MTurk (Mean + Median)	0.1123	0.7194
	0.1028	0.7067
	0.1267	0.7320
Demographics + Charge Feature + Risk + CNN	0.1216	0.7286
	0.1104	0.7162
	0.1343	0.7409
Combined Model	0.1219	0.7307
	0.1126	0.7184
	0.1365	0.7430

Splitting by Gender: Table 01 - Config 01

This repeats the above baseline table, splitting by gender. The balance is: 1833 females and 6646 males in our validation set. The model specifications are exactly as above.

Table 01 - Version 01 - Arrest Regressions - Split By Gender
Fit measured in adjusted R squared and AUC
Model Configuration	Male-Female Combined		Male Subsample		Female Subsample
Model Configuration	Adjusted R Squared	ROC AUC	Adjusted R Squared	ROC AUC	Adjusted R Squared	ROC AUC
Single Variable Model
Demographic LM	0.0101	0.5552	0.0004	0.5120	0.0001	0.5308
Lower 95% C.I.	0.0072	0.5412	−0.0001	0.4962	−0.0005	0.4949
Upper 95% C.I.	0.0136	0.5692	0.0017	0.5278	0.0036	0.5667
Charge Feature LM	0.0906	0.6910	0.0910	0.6891	0.0692	0.6824
	0.0804	0.6777	0.0803	0.6746	0.0479	0.6477
	0.1015	0.7042	0.1034	0.7036	0.0926	0.7170
XgBoost Risk	0.0332	0.6098	0.0300	0.6045	0.0185	0.5802
	0.0266	0.5961	0.0230	0.5894	0.0081	0.5475
	0.0397	0.6234	0.0382	0.6196	0.0324	0.6129
MTurk Features (Mean + Median)	0.0002	0.5346	−0.0005	0.5338	0.0085	0.6108
	0.0006	0.5202	0.0003	0.5181	0.0090	0.5767
	0.0052	0.5489	0.0055	0.5496	0.0323	0.6449
P_hat_cnn	0.0325	0.6222	0.0220	0.5976	0.0294	0.6570
	0.0266	0.6087	0.0162	0.5823	0.0178	0.6254
	0.0387	0.6358	0.0280	0.6129	0.0433	0.6886
Combined Variable Model
Demographics + Charge Feature	0.0973	0.7011	0.0913	0.6902	0.0706	0.6861
	0.0873	0.6880	0.0797	0.6757	0.0497	0.6518
	0.1079	0.7141	0.1039	0.7047	0.0951	0.7205
Demographics + Charge Feature + Risk	0.1123	0.7177	0.1077	0.7093	0.0806	0.6983
	0.1014	0.7050	0.0952	0.6953	0.0572	0.6642
	0.1244	0.7303	0.1200	0.7233	0.1074	0.7325
Demographics + Charge Feature + Risk + MTurk (Mean + Median)	0.1123	0.7194	0.1068	0.7102	0.0864	0.7234
	0.1033	0.7067	0.0974	0.6961	0.0731	0.6916
	0.1266	0.7320	0.1237	0.7242	0.1252	0.7552
Demographics + Charge Feature + Risk + CNN	0.1216	0.7286	0.1185	0.7211	0.1034	0.7343
	0.1113	0.7162	0.1059	0.7075	0.0807	0.7035
	0.1336	0.7409	0.1310	0.7347	0.1317	0.7650
Combined Model	0.1219	0.7307	0.1180	0.7228	0.1028	0.7449
	0.1129	0.7184	0.1077	0.7092	0.0888	0.7149
	0.1372	0.7430	0.1333	0.7365	0.1417	0.7748

Baseline: Table 01 - Config 02

This is our configuration 2 baseline table, with the inclusion of the mean and median of our MTurk features. NOTE that since we have a separate row for skin-tone in this table, this is now not included in our MTurk feature row ! The columns include:

Single & Combined Variable Models without the inclusion of XgBoost risk and/or the Charge Feature LM predictions as these now form separate columns (same as in config 01)
All models with the inclusion of our risk-predictor in + XgBoost Risk
All models with the inclusion of our charge-feature lm in + Charge Feature
Fully combined models, with the inclusion of both the risk and charge models

Table 01 - Version 02 - Arrest Regressions
Fit measured in adjusted R squared and AUC
Model Configuration	No Added Variables		+ XgBoost Risk		+ Charge Feature		+ Risk and Charge
Model Configuration	Adjusted R Squared	ROC AUC	Adjusted R Squared	ROC AUC	Adjusted R Squared	ROC AUC	Adjusted R Squared	ROC AUC
p_hat_cnn	0.0325	0.6222	0.0566	0.6594	0.1116	0.7184	0.1251	0.7314
Lower 95% C.I.	0.0270	0.6087	0.0489	0.6461	0.1015	0.7059	0.1135	0.7191
Upper 95% C.I.	0.0391	0.6358	0.0649	0.6727	0.1234	0.7308	0.1376	0.7437
Demographics	0.0101	0.5552	0.0381	0.6302	0.0959	0.6981	0.1117	0.7165
	0.0070	0.5412	0.0313	0.6162	0.0867	0.6849	0.1005	0.7037
	0.0136	0.5692	0.0458	0.6442	0.1078	0.7112	0.1237	0.7292
Demographics + p_hat_cnn	0.0330	0.6218	0.0565	0.6591	0.1117	0.7186	0.1250	0.7314
	0.0270	0.6083	0.0489	0.6458	0.1004	0.7061	0.1135	0.7191
	0.0395	0.6353	0.0657	0.6724	0.1234	0.7310	0.1379	0.7436
MTurk Features (Mean + Median)	−0.0001	0.5128	0.0328	0.6102	0.0907	0.6922	0.1088	0.7134
	−0.0003	0.4985	0.0267	0.5957	0.0806	0.6789	0.0983	0.7006
	0.0014	0.5272	0.0405	0.6248	0.1025	0.7054	0.1215	0.7262
MTurk Features (Mean + Median) + Demographics + p_hat_cnn	0.0326	0.6220	0.0563	0.6595	0.1114	0.7187	0.1247	0.7315
	0.0275	0.6084	0.0488	0.6462	0.1010	0.7063	0.1143	0.7192
	0.0403	0.6355	0.0654	0.6728	0.1234	0.7312	0.1368	0.7437
Skin-Tone	0.0005	0.5331	0.0333	0.6170	0.0907	0.6931	0.1094	0.7147
	0.0005	0.5188	0.0285	0.6026	0.0821	0.6798	0.0997	0.7018
	0.0046	0.5474	0.0420	0.6314	0.1040	0.7064	0.1231	0.7275
MTurk Features + Skin-Tone + Demographics + p_hat_cnn	0.0323	0.6251	0.0563	0.6616	0.1114	0.7202	0.1252	0.7333
	0.0288	0.6116	0.0505	0.6483	0.1021	0.7077	0.1154	0.7211
	0.0416	0.6386	0.0676	0.6749	0.1248	0.7326	0.1401	0.7456

Gender Interaction

Taking a cue from the work I did last week (including interaction terms in the election tables) here I try out a few new configurations for table 1 configuration 1 (since this is a little easier to see)

Including an interaction term for sex * MTurk features on top of the Demographics model to see how much additional signal we can gain from this interaction, on top of what is explained by demographics alone.