Introduction

This document replicates the main results from the study `Political endorsement by Nature and trust in scientific expertise during COVID-19’ by Floyd Jiuyun Zhang published in the journal Nature. The research paper asks whether political endorsements by scientists lower trust in scientists, and whether the effects differ depending on whether individuals are Trump or Biden supporters. This is an important question to answer given current partisan polarization in the US and the politicization of science and scientists.

Hypotheses

Drawing on literature on affective political polarization in the US, the authors say that it would be reasonable to expect a right-wing backlash against scientists making political statements. This is because associating individuals or entities, like scientists, with one political party is likely to increase out-party animosity towards them.

Alternatively, the authors also say that a right-wing backlash is also consistent with Bayesian models of information economics and decision theory because if the scientists’ message is inconsistent with their prior beliefs, they may judge it negatively.

However, there are also reasons to believe that scientists making political endorsements may not affect people’s perceptions.

In the research paper, the authors hypothesize the following:

Hypothesis 1: Political endorsements by scientists will decrease Trump supporters’ trust in scientific organizations.

Hypothesis 2: Political endorsements by scientists will decrease Trump supporters’ trust in scientists.

Here, the dependent variable is trust in science and scientists, and the independent variable is political endorsements by scientists.

Experimental Design

To test their hypothesis, the researchers design an experiment. An experiment allows the researchers to make causal claims because without randomizing the treatment, we may worry that those that are exposed to political endorsements by scientists are more likely to trust in science and scientists, so if we simply compare individuals that have been exposed to the scientific political endorsements to those that have not, then the differences in perceptions may be a result of their pre-existing trust in science rather than an effect of the political endorsement

In the experiment, the author create an online survey and recruit 4,260 respondents and ask whether they support Biden or Trump. The researcher then randomly assigns half to the treatment group (\(n=1,934\)) and the other half to the control group (\(n=1,946\)). The treatment group is then exposed to Nature’s endorsement of Biden. The control group receives is shown an image of nature’s website without any political endorsements. After each group receives the treatment or placebo, the respondents are asked to answer the following questions (among others, but I focus on four) to measure the dependent variable: (1) Nature is informed when providing advice, (2) Nature is unbiased on contentious issues, (3) US scientists are informed when providing advice, and (4) US scientists are unbiased on contentious issues.

Descriptive Statistics

First, the researchers collect data on the respondents. If the treatment and control groups were successfully randomized, the two groups should be, on average, the same across confounding variables. In the table below, I show the mean values of key confounds for the treatment and control groups to see whether the two groups are similar.

Descriptive statistics by treatment group
level Control Treatment
n 1939 1946
sex_cat (%) Female 1009 (52.3) 1046 (53.9)
Male 919 (47.7) 894 (46.1)
age (mean (SD)) 4.74 (1.62) 4.70 (1.65)
race_cat (%) American Indian 24 (1.2) 30 (1.5)
Asian 99 (5.1) 97 (5.0)
Black 207 (10.7) 228 (11.7)
Native Hawaiian/Pacific Islander 9 (0.5) 8 (0.4)
Other 70 (3.6) 77 (4.0)
White 1530 (78.9) 1506 (77.4)
edu_cat (%) Associate’s/technical degree 261 (13.5) 282 (14.5)
Bachelor’s degree 517 (26.7) 541 (27.8)
Began college, no degree 387 (20.0) 390 (20.0)
High school graduate/GED 454 (23.4) 438 (22.5)
No high school 45 (2.3) 39 (2.0)
Postgraduate/professional degree 275 (14.2) 256 (13.2)
hispanic (mean (SD)) 0.15 (0.36) 0.16 (0.37)

Nature is informed when providing advice

Here I plot the results for the first of the four outcomes.

## Warning: package 'ggplot2' was built under R version 4.4.2

Regression Results - Nature is informed and unbiased when providing advice

Ok, let’s run a regression and show the results it in our document. These regression results are of the first two rows in Table 2 of the main article.

Experimental results, Nature informed and unbiased (Table 2 first two rows)
Nature is informed Nature is unbiased
(Intercept) \num{0.290} \num{0.371}
(\num{0.023}) (\num{0.022})
trump\_supporter\_treated \num{-0.854} \num{-0.633}
(\num{0.052}) (\num{0.050})
biden\_supporter\_treated \num{0.108} \num{0.045}
(\num{0.031}) (\num{0.031})
trump\_supporterTRUE \num{-0.387} \num{-0.655}
(\num{0.039}) (\num{0.040})
Num.Obs. \num{3885} \num{3885}
R2 \num{0.255} \num{0.277}
R2 Adj. \num{0.254} \num{0.276}
AIC \num{9893.0} \num{9776.6}
BIC \num{9924.3} \num{9808.0}
Log.Lik. \num{-4941.495} \num{-4883.320}
RMSE \num{0.86} \num{0.85}
Std.Errors HC3 HC3

(Here I would interpret the coefficients and the main results. Say what the coefficients are and whether they are statistically significant.)

Conclusion

In this replication, I have provided evidence that treating people with X causes them to be less likely to Y. These results are statistically significant.

More broadly, these results demonstrate that scientists should reconsider how they communicate research to the public because of (this reason) and (this other reason).