Bivariate MTurk Label Regressions
We now consider a set of bivariate regressions of the MTurk labels on release outcome:
- Combined male-female regression
- subsample female
- Subsample male
- Subsample black
- Subsample not-black
I then also repeat the above regressions on the well-labeled subset of observations with more than 3 MTurkers.
Combined Male-Female
Multihead(ResNet50)
|
|
|
|
Dependent variable:
|
|
|
|
|
|
Release Outcome
|
|
|
(1)
|
(2)
|
(3)
|
(4)
|
(5)
|
|
|
|
attractiveness
|
0.005
|
|
|
|
0.002
|
|
|
(-0.002, 0.012)
|
|
|
|
(-0.008, 0.013)
|
|
|
|
|
|
|
|
|
competence
|
|
0.006
|
|
|
0.003
|
|
|
|
(-0.002, 0.014)
|
|
|
(-0.010, 0.015)
|
|
|
|
|
|
|
|
|
dominance
|
|
|
-0.003
|
|
-0.008
|
|
|
|
|
(-0.011, 0.004)
|
|
(-0.017, 0.001)
|
|
|
|
|
|
|
|
|
trustworthiness
|
|
|
|
0.007*
|
0.006
|
|
|
|
|
|
(0.0001, 0.015)
|
(-0.005, 0.017)
|
|
|
|
|
|
|
|
|
Constant
|
0.740***
|
0.736***
|
0.782***
|
0.730***
|
0.751***
|
|
|
(0.706, 0.774)
|
(0.695, 0.777)
|
(0.741, 0.823)
|
(0.695, 0.765)
|
(0.702, 0.799)
|
|
|
|
|
|
|
|
|
|
|
Observations
|
7,318
|
7,318
|
7,318
|
7,318
|
7,318
|
|
Adjusted R2
|
0.0001
|
0.00005
|
-0.0001
|
0.0002
|
0.0001
|
|
F Statistic
|
1.477 (df = 1; 7316)
|
1.330 (df = 1; 7316)
|
0.523 (df = 1; 7316)
|
2.743* (df = 1; 7316)
|
1.193 (df = 4; 7313)
|
|
|
|
Note:
|
p<0.1; p<0.05; p<0.01
|
Subsample Female
Multihead(ResNet50)
|
|
|
|
Dependent variable:
|
|
|
|
|
|
Release Outcome
|
|
|
(1)
|
(2)
|
(3)
|
(4)
|
(5)
|
(6)
|
|
|
|
attractiveness
|
0.016**
|
|
|
|
0.011
|
0.010
|
|
|
(0.003, 0.028)
|
|
|
|
(-0.007, 0.029)
|
(-0.003, 0.024)
|
|
|
|
|
|
|
|
|
|
competence
|
|
0.012
|
|
|
-0.010
|
|
|
|
|
(-0.002, 0.026)
|
|
|
(-0.033, 0.013)
|
|
|
|
|
|
|
|
|
|
|
dominance
|
|
|
0.019**
|
|
0.016
|
0.014
|
|
|
|
|
(0.005, 0.033)
|
|
(-0.0001, 0.032)
|
(-0.001, 0.030)
|
|
|
|
|
|
|
|
|
|
trustworthiness
|
|
|
|
0.014*
|
0.006
|
|
|
|
|
|
|
(0.001, 0.027)
|
(-0.014, 0.027)
|
|
|
|
|
|
|
|
|
|
|
Constant
|
0.772***
|
0.785***
|
0.752***
|
0.779***
|
0.733***
|
0.728***
|
|
|
(0.713, 0.832)
|
(0.713, 0.858)
|
(0.683, 0.821)
|
(0.715, 0.844)
|
(0.651, 0.816)
|
(0.652, 0.804)
|
|
|
|
|
|
|
|
|
|
|
|
Observations
|
1,593
|
1,593
|
1,593
|
1,593
|
1,593
|
1,593
|
|
Adjusted R2
|
0.002
|
0.001
|
0.003
|
0.001
|
0.002
|
0.003
|
|
F Statistic
|
4.473** (df = 1; 1591)
|
2.044 (df = 1; 1591)
|
5.291** (df = 1; 1591)
|
3.155* (df = 1; 1591)
|
1.853 (df = 4; 1588)
|
3.419** (df = 2; 1590)
|
|
|
|
Note:
|
p<0.1; p<0.05; p<0.01
|
Subsample Male
Multihead(ResNet50)
|
|
|
|
Dependent variable:
|
|
|
|
|
|
Release Outcome
|
|
|
(1)
|
(2)
|
(3)
|
(4)
|
(5)
|
|
|
|
attractiveness
|
-0.0002
|
|
|
|
-0.004
|
|
|
(-0.008, 0.008)
|
|
|
|
(-0.016, 0.008)
|
|
|
|
|
|
|
|
|
competence
|
|
0.003
|
|
|
0.005
|
|
|
|
(-0.006, 0.013)
|
|
|
(-0.009, 0.020)
|
|
|
|
|
|
|
|
|
dominance
|
|
|
-0.002
|
|
-0.004
|
|
|
|
|
(-0.012, 0.007)
|
|
(-0.014, 0.007)
|
|
|
|
|
|
|
|
|
trustworthiness
|
|
|
|
0.003
|
0.004
|
|
|
|
|
|
(-0.006, 0.011)
|
(-0.010, 0.017)
|
|
|
|
|
|
|
|
|
Constant
|
0.742***
|
0.725***
|
0.754***
|
0.727***
|
0.738***
|
|
|
(0.703, 0.782)
|
(0.677, 0.773)
|
(0.704, 0.803)
|
(0.686, 0.768)
|
(0.680, 0.796)
|
|
|
|
|
|
|
|
|
|
|
Observations
|
5,725
|
5,725
|
5,725
|
5,725
|
5,725
|
|
Adjusted R2
|
-0.0002
|
-0.0001
|
-0.0001
|
-0.0001
|
-0.0005
|
|
F Statistic
|
0.002 (df = 1; 5723)
|
0.315 (df = 1; 5723)
|
0.172 (df = 1; 5723)
|
0.331 (df = 1; 5723)
|
0.311 (df = 4; 5720)
|
|
|
|
Note:
|
p<0.1; p<0.05; p<0.01
|
Subsample Black
Multihead(ResNet50)
|
|
|
|
Dependent variable:
|
|
|
|
|
|
Release Outcome
|
|
|
(1)
|
(2)
|
(3)
|
(4)
|
(5)
|
|
|
|
attractiveness
|
0.004
|
|
|
|
0.004
|
|
|
(-0.006, 0.014)
|
|
|
|
(-0.010, 0.017)
|
|
|
|
|
|
|
|
|
competence
|
|
0.007
|
|
|
0.012
|
|
|
|
(-0.004, 0.018)
|
|
|
(-0.005, 0.029)
|
|
|
|
|
|
|
|
|
dominance
|
|
|
-0.005
|
|
-0.010
|
|
|
|
|
(-0.015, 0.005)
|
|
(-0.022, 0.001)
|
|
|
|
|
|
|
|
|
trustworthiness
|
|
|
|
0.003
|
-0.004
|
|
|
|
|
|
(-0.006, 0.013)
|
(-0.019, 0.011)
|
|
|
|
|
|
|
|
|
Constant
|
0.744***
|
0.729***
|
0.788***
|
0.746***
|
0.759***
|
|
|
(0.699, 0.789)
|
(0.673, 0.784)
|
(0.733, 0.844)
|
(0.700, 0.793)
|
(0.692, 0.825)
|
|
|
|
|
|
|
|
|
|
|
Observations
|
4,107
|
4,107
|
4,107
|
4,107
|
4,107
|
|
Adjusted R2
|
-0.0001
|
0.00001
|
-0.0001
|
-0.0002
|
-0.0002
|
|
F Statistic
|
0.495 (df = 1; 4105)
|
1.039 (df = 1; 4105)
|
0.611 (df = 1; 4105)
|
0.329 (df = 1; 4105)
|
0.794 (df = 4; 4102)
|
|
|
|
Note:
|
p<0.1; p<0.05; p<0.01
|
Subsample Not-Black
Multihead(ResNet50)
|
|
|
|
Dependent variable:
|
|
|
|
|
|
Release Outcome
|
|
|
(1)
|
(2)
|
(3)
|
(4)
|
(5)
|
|
|
|
attractiveness
|
0.006
|
|
|
|
0.001
|
|
|
(-0.004, 0.016)
|
|
|
|
(-0.015, 0.016)
|
|
|
|
|
|
|
|
|
competence
|
|
0.004
|
|
|
-0.009
|
|
|
|
(-0.008, 0.016)
|
|
|
(-0.028, 0.010)
|
|
|
|
|
|
|
|
|
dominance
|
|
|
-0.001
|
|
-0.005
|
|
|
|
|
(-0.013, 0.010)
|
|
(-0.018, 0.009)
|
|
|
|
|
|
|
|
|
trustworthiness
|
|
|
|
0.012*
|
0.019*
|
|
|
|
|
|
(0.001, 0.022)
|
(0.002, 0.036)
|
|
|
|
|
|
|
|
|
Constant
|
0.737***
|
0.747***
|
0.774***
|
0.710***
|
0.743***
|
|
|
(0.685, 0.790)
|
(0.686, 0.808)
|
(0.712, 0.835)
|
(0.656, 0.764)
|
(0.671, 0.814)
|
|
|
|
|
|
|
|
|
|
|
Observations
|
3,211
|
3,211
|
3,211
|
3,211
|
3,211
|
|
Adjusted R2
|
-0.00003
|
-0.0002
|
-0.0003
|
0.001
|
0.0002
|
|
F Statistic
|
0.902 (df = 1; 3209)
|
0.299 (df = 1; 3209)
|
0.035 (df = 1; 3209)
|
3.150* (df = 1; 3209)
|
1.158 (df = 4; 3206)
|
|
|
|
Note:
|
p<0.1; p<0.05; p<0.01
|
Well labeled subset:
Repeating the above regressions on the well-labeled subset:
Combined Male-Female
Multihead(ResNet50)
|
|
|
|
Dependent variable:
|
|
|
|
|
|
Release Outcome
|
|
|
(1)
|
(2)
|
(3)
|
(4)
|
(5)
|
|
|
|
attractiveness
|
0.024
|
|
|
|
-0.022
|
|
|
(-0.009, 0.057)
|
|
|
|
(-0.074, 0.029)
|
|
|
|
|
|
|
|
|
competence
|
|
0.052**
|
|
|
0.034
|
|
|
|
(0.011, 0.094)
|
|
|
(-0.037, 0.104)
|
|
|
|
|
|
|
|
|
dominance
|
|
|
0.012
|
|
0.002
|
|
|
|
|
(-0.029, 0.053)
|
|
(-0.043, 0.047)
|
|
|
|
|
|
|
|
|
trustworthiness
|
|
|
|
0.049**
|
0.043
|
|
|
|
|
|
(0.012, 0.085)
|
(-0.024, 0.110)
|
|
|
|
|
|
|
|
|
Constant
|
0.647***
|
0.499***
|
0.696***
|
0.535***
|
0.489***
|
|
|
(0.491, 0.803)
|
(0.292, 0.705)
|
(0.479, 0.914)
|
(0.363, 0.707)
|
(0.225, 0.752)
|
|
|
|
|
|
|
|
|
|
|
Observations
|
449
|
449
|
449
|
449
|
449
|
|
Adjusted R2
|
0.001
|
0.007
|
-0.002
|
0.008
|
0.004
|
|
F Statistic
|
1.426 (df = 1; 447)
|
4.378** (df = 1; 447)
|
0.217 (df = 1; 447)
|
4.678** (df = 1; 447)
|
1.406 (df = 4; 444)
|
|
|
|
Note:
|
p<0.1; p<0.05; p<0.01
|
Subsample Female
Multihead(ResNet50)
|
|
|
|
Dependent variable:
|
|
|
|
|
|
Release Outcome
|
|
|
(1)
|
(2)
|
(3)
|
(4)
|
(5)
|
(6)
|
|
|
|
attractiveness
|
0.049
|
|
|
|
0.048
|
0.043
|
|
|
(-0.003, 0.100)
|
|
|
|
(-0.031, 0.127)
|
(-0.015, 0.102)
|
|
|
|
|
|
|
|
|
|
competence
|
|
0.074
|
|
|
0.100
|
|
|
|
|
(-0.001, 0.149)
|
|
|
(-0.039, 0.238)
|
|
|
|
|
|
|
|
|
|
|
dominance
|
|
|
0.039
|
|
-0.002
|
0.014
|
|
|
|
|
(-0.024, 0.103)
|
|
(-0.079, 0.075)
|
(-0.058, 0.086)
|
|
|
|
|
|
|
|
|
|
trustworthiness
|
|
|
|
0.035
|
-0.075
|
|
|
|
|
|
|
(-0.028, 0.099)
|
(-0.196, 0.045)
|
|
|
|
|
|
|
|
|
|
|
Constant
|
0.596***
|
0.457*
|
0.636***
|
0.661***
|
0.467*
|
0.552***
|
|
|
(0.341, 0.851)
|
(0.073, 0.842)
|
(0.315, 0.957)
|
(0.346, 0.975)
|
(0.056, 0.877)
|
(0.213, 0.892)
|
|
|
|
|
|
|
|
|
|
|
|
Observations
|
101
|
101
|
101
|
101
|
101
|
101
|
|
Adjusted R2
|
0.014
|
0.016
|
0.0004
|
-0.002
|
0.001
|
0.005
|
|
F Statistic
|
2.448 (df = 1; 99)
|
2.632 (df = 1; 99)
|
1.043 (df = 1; 99)
|
0.833 (df = 1; 99)
|
1.013 (df = 4; 96)
|
1.265 (df = 2; 98)
|
|
|
|
Note:
|
p<0.1; p<0.05; p<0.01
|
Subsample Male
Multihead(ResNet50)
|
|
|
|
Dependent variable:
|
|
|
|
|
|
Release Outcome
|
|
|
(1)
|
(2)
|
(3)
|
(4)
|
(5)
|
|
|
|
attractiveness
|
0.005
|
|
|
|
-0.064
|
|
|
(-0.037, 0.047)
|
|
|
|
(-0.129, 0.0001)
|
|
|
|
|
|
|
|
|
competence
|
|
0.041
|
|
|
0.021
|
|
|
|
(-0.008, 0.089)
|
|
|
(-0.061, 0.103)
|
|
|
|
|
|
|
|
|
dominance
|
|
|
0.013
|
|
0.021
|
|
|
|
|
(-0.039, 0.066)
|
|
(-0.036, 0.079)
|
|
|
|
|
|
|
|
|
trustworthiness
|
|
|
|
0.045*
|
0.078
|
|
|
|
|
|
(0.001, 0.090)
|
(-0.001, 0.157)
|
|
|
|
|
|
|
|
|
Constant
|
0.714***
|
0.537***
|
0.666***
|
0.531***
|
0.459**
|
|
|
(0.521, 0.908)
|
(0.295, 0.779)
|
(0.383, 0.949)
|
(0.325, 0.737)
|
(0.122, 0.795)
|
|
|
|
|
|
|
|
|
|
|
Observations
|
348
|
348
|
348
|
348
|
348
|
|
Adjusted R2
|
-0.003
|
0.003
|
-0.002
|
0.005
|
0.004
|
|
F Statistic
|
0.034 (df = 1; 346)
|
1.872 (df = 1; 346)
|
0.166 (df = 1; 346)
|
2.771* (df = 1; 346)
|
1.385 (df = 4; 343)
|
|
|
|
Note:
|
p<0.1; p<0.05; p<0.01
|
Subsample Black
Multihead(ResNet50)
|
|
|
|
Dependent variable:
|
|
|
|
|
|
Release Outcome
|
|
|
(1)
|
(2)
|
(3)
|
(4)
|
(5)
|
|
|
|
attractiveness
|
0.031
|
|
|
|
-0.041
|
|
|
(-0.015, 0.076)
|
|
|
|
(-0.114, 0.033)
|
|
|
|
|
|
|
|
|
competence
|
|
0.074**
|
|
|
0.080
|
|
|
|
(0.020, 0.128)
|
|
|
(-0.017, 0.177)
|
|
|
|
|
|
|
|
|
dominance
|
|
|
0.018
|
|
0.009
|
|
|
|
|
(-0.036, 0.072)
|
|
(-0.052, 0.069)
|
|
|
|
|
|
|
|
|
trustworthiness
|
|
|
|
0.055*
|
0.031
|
|
|
|
|
|
(0.007, 0.103)
|
(-0.058, 0.120)
|
|
|
|
|
|
|
|
|
Constant
|
0.631***
|
0.404**
|
0.670***
|
0.520***
|
0.373*
|
|
|
(0.424, 0.838)
|
(0.136, 0.671)
|
(0.379, 0.962)
|
(0.300, 0.739)
|
(0.012, 0.734)
|
|
|
|
|
|
|
|
|
|
|
Observations
|
267
|
267
|
267
|
267
|
267
|
|
Adjusted R2
|
0.001
|
0.015
|
-0.003
|
0.010
|
0.007
|
|
F Statistic
|
1.239 (df = 1; 265)
|
5.144** (df = 1; 265)
|
0.309 (df = 1; 265)
|
3.592* (df = 1; 265)
|
1.495 (df = 4; 262)
|
|
|
|
Note:
|
p<0.1; p<0.05; p<0.01
|
Subsample Not-Black
Multihead(ResNet50)
|
|
|
|
Dependent variable:
|
|
|
|
|
|
Release Outcome
|
|
|
(1)
|
(2)
|
(3)
|
(4)
|
(5)
|
|
|
|
attractiveness
|
0.021
|
|
|
|
-0.005
|
|
|
(-0.029, 0.071)
|
|
|
|
(-0.079, 0.069)
|
|
|
|
|
|
|
|
|
competence
|
|
0.027
|
|
|
-0.016
|
|
|
|
(-0.038, 0.091)
|
|
|
(-0.122, 0.089)
|
|
|
|
|
|
|
|
|
dominance
|
|
|
-0.002
|
|
-0.016
|
|
|
|
|
(-0.066, 0.063)
|
|
(-0.089, 0.057)
|
|
|
|
|
|
|
|
|
trustworthiness
|
|
|
|
0.044
|
0.065
|
|
|
|
|
|
(-0.015, 0.103)
|
(-0.039, 0.169)
|
|
|
|
|
|
|
|
|
Constant
|
0.643***
|
0.609***
|
0.750***
|
0.535***
|
0.626***
|
|
|
(0.399, 0.886)
|
(0.284, 0.935)
|
(0.415, 1.085)
|
(0.254, 0.817)
|
(0.233, 1.019)
|
|
|
|
|
|
|
|
|
|
|
Observations
|
182
|
182
|
182
|
182
|
182
|
|
Adjusted R2
|
-0.003
|
-0.003
|
-0.006
|
0.003
|
-0.012
|
|
F Statistic
|
0.471 (df = 1; 180)
|
0.460 (df = 1; 180)
|
0.002 (df = 1; 180)
|
1.513 (df = 1; 180)
|
0.454 (df = 4; 177)
|
|
|
|
Note:
|
p<0.1; p<0.05; p<0.01
|
Appendix
Subsample Female with race controll
Multihead(ResNet50)
|
|
|
|
Dependent variable:
|
|
|
|
|
|
Release Outcome
|
|
|
(1)
|
(2)
|
(3)
|
(4)
|
(5)
|
(6)
|
|
|
|
race_mturk_2_not_black
|
-0.057***
|
-0.050***
|
-0.048***
|
-0.051***
|
-0.055***
|
-0.055***
|
|
|
(-0.087, -0.027)
|
(-0.080, -0.021)
|
(-0.078, -0.018)
|
(-0.081, -0.021)
|
(-0.086, -0.025)
|
(-0.085, -0.025)
|
|
|
|
|
|
|
|
|
|
attractiveness
|
0.020***
|
|
|
|
0.017
|
0.015*
|
|
|
(0.008, 0.032)
|
|
|
|
(-0.001, 0.036)
|
(0.002, 0.029)
|
|
|
|
|
|
|
|
|
|
competence
|
|
0.014
|
|
|
-0.011
|
|
|
|
|
(-0.00005, 0.028)
|
|
|
(-0.034, 0.012)
|
|
|
|
|
|
|
|
|
|
|
dominance
|
|
|
0.019**
|
|
0.014
|
0.012
|
|
|
|
|
(0.005, 0.033)
|
|
(-0.002, 0.030)
|
(-0.004, 0.027)
|
|
|
|
|
|
|
|
|
|
trustworthiness
|
|
|
|
0.016**
|
0.005
|
|
|
|
|
|
|
(0.003, 0.029)
|
(-0.015, 0.026)
|
|
|
|
|
|
|
|
|
|
|
Constant
|
0.783***
|
0.804***
|
0.778***
|
0.798***
|
0.754***
|
0.746***
|
|
|
(0.723, 0.843)
|
(0.731, 0.877)
|
(0.707, 0.850)
|
(0.733, 0.863)
|
(0.671, 0.837)
|
(0.669, 0.823)
|
|
|
|
|
|
|
|
|
|
|
|
Observations
|
1,593
|
1,593
|
1,593
|
1,593
|
1,593
|
1,593
|
|
Adjusted R2
|
0.008
|
0.005
|
0.006
|
0.006
|
0.007
|
0.008
|
|
F Statistic
|
7.094*** (df = 2; 1590)
|
4.899*** (df = 2; 1590)
|
6.207*** (df = 2; 1590)
|
5.571*** (df = 2; 1590)
|
3.280*** (df = 5; 1587)
|
5.260*** (df = 3; 1589)
|
|
|
|
Note:
|
p<0.1; p<0.05; p<0.01
|
Skin-tone and race relation (sanity check)
