Outline

This markdown contains table 01s for elections with updated MTurk labels. These now come from a side-by-side survey with 10 voters per image which is much closer to the survey conducted in Todorov et.al. (2005) (https://science.sciencemag.org/content/308/5728/1623) which make use of about 50 voters. Our validation set is now reduced to 350 observations. We also updated the CNN to take care of the trainval-test split errors we found last week. All other variable definitions are the same as before. There are two tables here:

  1. Table 01 Configuration 4 (using own-party-vote-share-jacknife in the election_lm) with updated Mturk features in the same model

  2. Table 01 Configuration 4 but with each MTurk feature entered as a univariate model to check for each individually

NOTE Since this was a trial run, these results are based on 350 observations and if we decide it’s worth pursuing, we will need to run a bigger survey to gather high detail labels for our entire validation set.

Definitions

Before jumping into regression output, I will put all definitions of important terms here:

  1. Vote-Share : In all tables below, when vote-share is a LHS variable this means the vote-share of a specific candidate election. I.e. how much of the total vote did a specific candidate get in their election.

  2. Own-Party-Vote-Share-Jacknife : This is a feature and computed as the “average vote share of my party in this state over all years excluding the current”

Table 1 - Configuration 4 - Comparison between High/Low detail

Replacing own_party_win_rate with own_party_vote_share_jacknife in the election lm. This now includes:

  1. Party (limited to Democrat vs. Republican)
  2. Vote-share

Own Party Vote-share-Jacknife is computed as: “average vote share of my party in this state over all years excluding the current”

NOTE I now include a split for high detail vs. low detail MTurk labels.

  1. Lower Detail are using regression output from the previous survey in which we have between 3-4 voters per image, but over 1,300 observations

  2. Higher Detail uses the new 10 voter per image Mturk survey, which is reduced to only 350 observations

Table 01 - Election Regressions - Comparing High/Low Detail MTurk
Fit measured in adjusted R squared and AUC
Model Configuration Election Outcome Vote Share
R-Sqrt High Detail AUC High Detail R-Sqrt Low Detail AUC Low Detail R-Sqrt High Detail R-Sqrt Low Detail
Single Variable Model
Election LM 0.0245 0.6421 0.0667 0.6421 0.0259 0.0742
Lower 95% C.I. 0.0027 0.5183 0.0462 0.6115 0.0036 0.0527
Upper 95% C.I. 0.0635 0.5412 0.0904 0.6726 0.0636 0.0974
Sex −0.0012 0.5581 0.0037 0.5183 −0.0017 0.0048
−0.0031 0.6858 0.0006 0.4962 −0.0031 0.0008
0.0174 0.6904 0.0094 0.5404 0.0136 0.0124
Skine-Tone 0.0110 0.7323 −0.0077 0.5412 0.0135 −0.0022
0.0145 0.6443 −0.0035 0.5098 0.0146 0.0005
0.1065 0.7342 0.0172 0.5726 0.1219 0.0271
MTurk Features 0.0781 0.6577 0.0127 0.5581 0.0689 −0.0003
0.0420 0.7394 0.0057 0.5264 0.0383 −0.0020
0.1377 0.6699 0.0288 0.5898 0.1260 0.0095
P_hat_cnn 0.1487 0.7432 0.1011 0.6858 0.1423 0.0804
0.0950 0.6421 0.0766 0.6566 0.0852 0.0578
0.2106 0.5183 0.1298 0.7149 0.2139 0.1079
Combined Variable Model
P_hat_cnn + MTurk Features 0.2045 0.5412 0.1053 0.6904 0.1906 0.0778
0.1558 0.5581 0.0842 0.6614 0.1386 0.0570
0.2798 0.6858 0.1366 0.7193 0.2728 0.1058
Election LM + P_hat_cnn 0.1604 0.6904 0.1579 0.7323 0.1593 0.1469
0.1069 0.7323 0.1298 0.7047 0.1031 0.1177
0.2267 0.6443 0.1908 0.7598 0.2302 0.1777
Election LM + Sex 0.0217 0.7342 0.0683 0.6443 0.0232 0.0780
0.0028 0.6577 0.0488 0.6139 0.0027 0.0573
0.0632 0.7394 0.0926 0.6748 0.0666 0.1039
Election LM + Sex + P_hat_cnn 0.1609 0.6699 0.1594 0.7342 0.1595 0.1504
0.1125 0.7432 0.1311 0.7068 0.1101 0.1238
0.2340 0.6421 0.1918 0.7616 0.2351 0.1841
Election LM + Sex + Skin-Tone 0.0300 0.5183 0.0613 0.6577 0.0300 0.0758
0.0311 0.5412 0.0536 0.6276 0.0307 0.0650
0.1353 0.5581 0.0992 0.6878 0.1571 0.1192
Election LM + Sex + Skin-Tone + P_hat_cnn 0.1789 0.6858 0.1540 0.7394 0.1667 0.1492
0.1556 0.6904 0.1358 0.7123 0.1468 0.1304
0.2915 0.7323 0.1978 0.7666 0.3000 0.1970
Election LM + Sex + Skin-Tone + MTurk 0.0972 0.6443 0.0731 0.6699 0.0773 0.0742
0.0927 0.7342 0.0662 0.6403 0.0779 0.0664
0.2201 0.6577 0.1155 0.6995 0.2168 0.1188
Election LM + Sex + Skin-Tone + MTurk + P_hat_cnn 0.2287 0.7394 0.1575 0.7432 0.2014 0.1465
0.2014 0.6699 0.1434 0.7161 0.1859 0.1317
0.3479 0.7432 0.2074 0.7702 0.3353 0.1994

Table 01 - Configuration 04 - Including Individual MTurk Features

In this table I decide to include the different MTurk features individually in the single variable model section. We can see that on the high-detail features every feature other than Attractiveness is significant.

Table 01 - Version 04 - Election Regressions - Individual High Detail MTurk Features
Fit measured in adjusted R squared and AUC
Model Configuration Election Outcome Vote Share
Adjusted R Squared ROC AUC Adjusted R squared
Single Variable Model
Election LM 0.0245 0.5708 0.0259
Lower 95% C.I. 0.0031 0.5073 0.0032
Upper 95% C.I. 0.0621 0.6343 0.0621
Sex −0.0012 0.5189 −0.0017
−0.0031 0.4723 −0.0031
0.0180 0.5654 0.0132
Skine-Tone 0.0110 0.6285 0.0135
0.0139 0.5685 0.0165
0.1047 0.6885 0.1307
Attractiveness 0.0024 0.5442 0.0225
−0.0031 0.4809 0.0023
0.0249 0.6075 0.0569
Competence 0.0307 0.6097 0.0520
0.0064 0.5479 0.0199
0.0706 0.6714 0.1025
Dominance 0.0194 0.5878 0.0271
0.0008 0.5253 0.0041
0.0585 0.6502 0.0624
Trustworthiness 0.0466 0.6191 0.0462
0.0143 0.5580 0.0148
0.0877 0.6802 0.0930
P_hat_cnn 0.1487 0.7263 0.1423
0.0929 0.6711 0.0899
0.2138 0.7816 0.2143
Combined Variable Model
Election LM + P_hat_cnn 0.1604 0.7338 0.1593
0.1064 0.6792 0.1041
0.2298 0.7884 0.2293
Election LM + Sex 0.0217 0.5798 0.0232
0.0026 0.5167 0.0022
0.0611 0.6429 0.0650
Election LM + Sex + P_hat_cnn 0.1609 0.7377 0.1595
0.1072 0.6836 0.1022
0.2352 0.7918 0.2308
Election LM + Sex + Skin-Tone 0.0300 0.6635 0.0300
0.0305 0.6042 0.0336
0.1376 0.7228 0.1578
Election LM + Sex + Skin-Tone + P_hat_cnn 0.1789 0.7792 0.1667
0.1558 0.7292 0.1480
0.2932 0.8292 0.3069
Election LM + Sex + Skin-Tone + MTurk 0.0972 0.7302 0.0773
0.0915 0.6755 0.0794
0.2222 0.7849 0.2194
Election LM + Sex + Skin-Tone + MTurk + P_hat_cnn 0.2287 0.8141 0.2014
0.2043 0.7675 0.1866
0.3480 0.8607 0.3418