This is a preregistration of an extension project for the following study:
Eady, G., Paskhalis, T., Zilinsky, J., Bonneau, R., Nagler, J., & Tucker, J. A. (2023). Exposure to the Russian Internet Research Agency foreign influence campaign on Twitter in the 2016 US election and its relationship to attitudes and voting behavior. Nature Communications, 14(1), 62.https://doi.org/10.1038/s41467-022-35576-9
The paper I will replicate and build upon considers the impact of a the Russian foreign influence campaign on voter choice in the 2016 US election. Specifically, it examines whether exposure to the Russian foreign influence campaign through Twitter increased the chance of voters turning towards Donald Trump, either in the actual vote or at different points in the run up to the election. The paper finds no evidence of a significant relationship between exposure to Russian tweets and the probability of changing voter preferences, with the estimated coefficients being small and insignificant.
In the original study, the authors assume that any effect of exposure on voter choice is linear and does not depend on the characteristics of the individuals exposed to Russian posts. It is plausible that the effects of exposure to Russian tweets are heterogeneous, however, leading some demographics to switch their voting behaviour while not affecting others. Various different demographic characteristics may be relevant when considering heterogeneity, including age, education levels, ethnicity, and gender (e.g., Pew Research Center, 2020).
In this study, I will focus on the political stance of voters, examining whether the effects of Russian tweets on voting patterns differ between individuals with moderate and strong political views. The Pew Research Center study (2020) shows that voters rarely switch parties, an outcome consistent with the data used in Eady et al. (2023). They do find, however, that switching is more likely among less politically engaged voters. Such outcomes further suggest that foreign influence campaigns are unlikely to have an effect on the voting choices of more politically engaged voters, but have a better chance of influencing those less politically engaged. Using information in the dataset of Eady et al. (2023) on the strength of a voter’s affiliation to a particular party, the study tests the hypothesis that foreign influence campaigns have a stronger impact on those with moderate political views. Specifically, the study uses an indicator of where voters rank themselves on a scale (0 being strong Democrat and 1 strong Republican) to address this hypothesis.
A straightforward way of testing for this kind of heterogeneity or non-linearity is to use interaction terms, such as an interaction between the strength of political views and exposure to foreign influence campaigns. In this case, however, this is not so straightforward since the two extremes (0 and 1) represent strong political views and the middle values (e.g., 0.5) moderate views. This means that the variable is non-linear in the strength of political views. To deal with this, the study will redefine the variable such that higher numbers imply more moderate political views. Interacting this new variable with the exposure to foreign influence tweets will then allow for a test of the hypothesis, with a positive (and significant) coefficient on the interaction term signifying that individuals with more moderate political views are more likely to change their voter preferences.
The hypothesis that I am testing is that voters with more political moderate views are more likely to be influenced by foreign influence campaigns to change their voter preferences.
The original study found no significant relationship between exposure to foreign influence campaigns and changes to voter preferences. One potential reason for this is that they assume a linear relationship, not allowing for the possibility that the effect varies by characteristics of the voters. This analysis moves beyond that linear assumption and so has the potential to provide different results.
At the same time, another characteristic of the dataset is that few voters actually change their voter preferences, resulting in a lack of variation in the data (and the dependent variable specifically). Given this, there is also a strong possibility that the results obtained from my non-linear approach will also result in insignificant effects.
The data sets used will be the same as in the original study. This involves two main databases: 1. A YouGov survey 2. Twitter 27 election integrity data sets There is little information on the original source of these data sets or how to access them. The data set that I will be used has been combined by the original authors into a constructed data set which is available here:
Tpaskhalis (2023) Tpaskhalis/NCOMMS_RUSSIA_US_2016: Replication materials for ‘exposure to the Russian Internet Research Agency Foreign Influence Campaign on Twitter in the 2016 US election and its relationship to attitudes and voting behavior’, GitHub. Available at: https://github.com/tpaskhalis/ncomms_russia_us_2016 (Accessed: 8 May 2024).
I will test the hypothesis using a regression model as in the original study, further using OLS regression methods as in the original study. I will, however, be introducing the possibility of non-linear relationships between exposure to foreign influence campaigns and changes in voter preferences. To do this, I will be using interaction terms, which allow for me to test whether there is an additional effect of foreign influence campaigns on changes in voter preferences for voters that have more moderate political views. Specifically, this involves interacting the exposure variable with the redefined pid7 variable, the variable capturing political views.
Dependent variable(s): The dependent variables will remain the same as in the original data set these are:
Vote choice (Clinton toward Trump) - a shift in voting preference from Clinton to Trump between the period prior to the election campaign and the actual vote.
Rank (Trump > Clinton) - a shift in respondent preferences between Clinton and Trump in the period prior to the campaign and the last wave of the survey.
Vote choice (Clinton toward Trump, 3rd party, or not voting) - a shift in respondent preferences from their preferred candidate prior to the campaign to how they actually voted, including shifts from their preferred candidate to 3rd party candidates or abstaining from the vote.
The dependent variables were each measured in a similar way. At each point in time (i.e., first survey wave, third survey wave, actual vote) a preference for Trump was coded as 1 and a preference for Clinton as zero. Considering changes in these preferences between the first wave and either the third wave or the actual vote, this implies that a positive value (+1) involved a shift towards Trump, a negative value (-1) a shift towards Clinton, and a zero value implying no change in preferences.
Independent variables (IVs):
The independent variables will be the same as in the original study those being:
log(Exposure + 1) - total_exposure_russia_log
Age - age (in years)
Woman - genderFemale (dummy variable, 1 = female)
College-educated - educationCollege+
Income - income (categorical variable, 4 categories, $0-30000, $30000-70000,$70000 -120 000, $120 000+)
Person of Color - raceNon-White (binary variable, 1 = non-white)
Other/Hispanic - race_other_hispanic (binary variable, 1 = Hispanic)
Black - race_black (binary variable, 1 = Black)
Region: Northeast - regionNortheast (binary variable, 1 = Northeast)
Region: Midwest - regionMidwest (binary variable, 1 = Midwest)
Region : West - regionWest (binary variable, 1 = West)
Freq. of social media use - internet_social_media_w1 (7 point scale)
log(Total Tweets) - total_tweets_log
Party ID - pid7 - this reports information on political views on a scale from 0 to 1. This variable will be rescaled to ensure that higher numbers refer to more moderate political views
In addition to these variables, I will further include an interaction term between the exposure variable and the rescaled pid7 variable.
I will use the standard p<0.05 criteria for determining if the statistical tests suggest that the results are significantly different from those expected in the null hypothesis. I will further report information on the estimated size of the effects, further distinguishing these by the strength of voter’s political views.
No checks will be performed to determine eligibility for inclusion besides verification that each respondent answered each of the survey questions. Outliers will be included in the analysis. Cleaning was undertaken by the authors of the original paper who also excluded from their final dataset all voters without a Twitter account (and, therefore, no means of being exposed to foreign influence accounts via Twitter).
There are a few rows where the respondents have chosen to not declare how they would classify themselves in terms of their own political identity (pid7, on the scale from strong democrat to strong republican), these are missing values and have been labelled as NA. There are no respondents who answered ‘not sure’, ‘skipped’, or were not asked. Following the original paper, these observations will be excluded from the analysis. While these individuals might provide useful information and while their decision not to answer a question could also be informative (e.g., certain types of individual chose not to answer, which means that the sample is biased against individuals with these characteristics), it is not possible to include them in the analysis.
Given the focus on considering whether the strength of political views is a relevant characteristic when considering the relationship between exposure to foreign influence campaigns and changes in voter preferences, the analysis will introduce descriptive analysis of the sample in terms of strength of political views.
This preregistration form was completed in the following R environment:
R version 4.3.2 (2023-10-31)
Platform: x86_64-apple-darwin20 (64-bit)
Running under: macOS Sonoma 14.3.1
Bartlett J. (2021). OSF preregistration template.Rmd. GitHub (accessed 03/05/2023)
Bowman, S. D., DeHaven, A. C., Errington, T. M., Hardwicke, T. E., Mellor, D. T., Nosek, B. A., & Soderberg, C. K. (2020). OSF Prereg Template. https://doi.org/10.31222/osf.io/epgjd. OSF (accessed 03/05/2023)
Eady, G., Paskhalis, T., Zilinsky, J., Bonneau, R., Nagler, J., & Tucker, J. A. (2023). Exposure to the Russian Internet Research Agency foreign influence campaign on Twitter in the 2016 US election and its relationship to attitudes and voting behavior. Nature Communications, 14(1), 62.https://doi.org/10.1038/s41467-022-35576-9
Pew Research Center (2020). Voters Rarely Switch Parties, but Recent Shifts Further Educational, Racial Divergence. [online] Pew Research Center - U.S. Politics & Policy. Available at: https://www.pewresearch.org/politics/2020/08/04/voters-rarely-switch-parties-but-recent-shifts-further-educational-racial-divergence/ [Accessed 13 May 2024].