Figures

ROC AUC for ML-Fusion

Joint Density Plots for ML-Fusion and XgBoost Risk-prediction

Joint Density Box Plot

ROC AUC for ML-Face

ROC AUC for XgBoost

ROC AUC for ML-Face, Ml-Fusion, and ML-Structured

Distribution & Calibration Plots for ML-Face

Bar Charts for Demographics + Charge Types + ML Fusion Predictions

MTurk Plots

Alpha Summary Table
Label Median Alpha
attractive 0.601
competent 0.915
dominant 0.912
messy 0.344
trustworthy 0.927
well_groomed 0.850
Summary Table
Alpha and Kappa Measures
Label Alpha Kappa
Median Alpha ICC (1,k) F DF1 DF2 p Lower Bound Upper Bound
attractive 0.601 0.956 22.746 32880 5852818 0.000 0.955 0.957
competent 0.915 0.921 12.676 32880 5852818 0.000 0.920 0.922
dominant 0.912 0.914 11.606 32880 5852818 0.000 0.913 0.915
messy 0.344 0.957 23.261 32880 5852818 0.000 0.956 0.958
trustworthy 0.927 0.922 12.836 32880 5852818 0.000 0.921 0.923
well_groomed 0.850 0.964 27.663 32880 5852818 0.000 0.963 0.964
Kappa Summary Table
Label Type ICC F DF1 DF2 p Lower Bound Upper Bound
attractive ICC2k 0.957 41.845 32880 5852640 0.000 0.953 0.960
competent ICC2k 0.924 22.719 32880 5852640 0.000 0.918 0.929
dominant ICC2k 0.916 17.653 32880 5852640 0.000 0.911 0.921
messy ICC2k 0.958 32.337 32880 5852640 0.000 0.955 0.960
trustworthy ICC2k 0.925 27.020 32880 5852640 0.000 0.918 0.931
well_groomed ICC2k 0.964 45.683 32880 5852640 0.000 0.962 0.967
Figure 8 - MTurk Plots V2
## [1] "ACCURACY TABLE:"
## # A tibble: 2 x 4
##   source performance performance_se_lower performance_se_upper
##   <chr>        <dbl>                <dbl>                <dbl>
## 1 human        0.544                0.533                0.554
## 2 model        0.606                0.595                0.616

## [1] "ACCURACY TABLE:"
## # A tibble: 2 x 4
##   treatment        performance performance_se_lower performance_se_upper
##   <fct>                  <dbl>                <dbl>                <dbl>
## 1 actual only            0.544                0.533                0.554
## 2 gan interspersed       0.592                0.570                0.614

Figure 8 - MTurk Quality PLots