Executive Summary

Among voters in the Philadelphia 2020 primary, the average causal effects of the Neighbors and Self postcards on voter turnout were small. However, the causal effects also vary between voters. This memo outlines key insights into the impact of the postcards on voter behavior in this election. In particular, we look at the implications of voter history on the treatment effect of the postcards. Finally, we use 2020 primary data to forecast the potential impact on voter turnout of mailing Neighbors postcards in the 2024 primary.

Methods

In order to estimate the average causal effect of each of the two postcard treatments, I fit a model that considers voting history, while holding constant voters’ age group and sex. The algebraic formula for the model and table of results are shown below:

\(y_i = \beta_0 +\beta_1x_{i,1} + \beta_2x_{i,2}...+\beta_nx_{i,n} +\varepsilon_i\)

\(y_i = \beta_0 +\beta_1 Neighborhood_i + \beta_2 Self_i + \beta_3 [30,40)_i +\)
\(\beta_4 [40,50)_i + \beta_5 [50,65) _i + \beta_6 [65,121]_i +\)
\(\beta_7 M_i + \beta_8 U_i + \beta_9 General Voter_i +\)
\(\beta_{10} Low Voter_i + \beta_{11} Municipal Primary Voter_i +\)
\(\beta_{12} Municipal Voter_i + \beta_{13} Neighborhood_i General Voter_i +\)
\(\beta_{14} Self_i General Voter_i +\beta_{15} Neighborhood_i Low Voter_i +\)
\(\beta_{16} Self_i Low Voter_i +\beta_{17} Neighborhood_i MunicipalPrimaryVoter_i +\)
\(\beta_{18} Self_i Municipal Primary Voter_i +\beta_{19} Neighborhood_i Municipal Voter_i +\)
\(\beta_{20} Self_i Municipal Voter_i +\varepsilon_i\)

Figure 1: Regression Table

Likelihood of Voting in 2020 Primary
How Treatment Assignment and Voting History Predict Likelihood of Voting
Characteristic Parameter 95% CI1
(Intercept) 0.340 0.337, 0.344
treat
Control — —
Neighborhood -0.009 -0.025, 0.007
Self -0.012 -0.029, 0.006
age_bin
[18,30) — —
[30,40) -0.004 -0.007, -0.002
[40,50) 0.007 0.004, 0.010
[50,65) 0.028 0.026, 0.031
[65,121] 0.011 0.008, 0.014
sex
F — —
M -0.055 -0.057, -0.054
U -0.047 -0.049, -0.044
vote_history
General Primary Voter — —
General Voter -0.119 -0.122, -0.116
Low Voter -0.276 -0.280, -0.273
Municipal Primary Voter 0.236 0.233, 0.239
Municipal Voter 0.087 0.083, 0.091
treat * vote_history
Neighborhood * General Voter 0.013 -0.006, 0.033
Self * General Voter 0.030 0.010, 0.050
Neighborhood * Low Voter 0.010 -0.010, 0.030
Self * Low Voter 0.008 -0.012, 0.028
Neighborhood * Municipal Primary Voter 0.004 -0.015, 0.023
Self * Municipal Primary Voter 0.019 -0.001, 0.039
Neighborhood * Municipal Voter 0.025 0.000, 0.052
Self * Municipal Voter -0.003 -0.028, 0.024
Hopkins et al (2021)

1 CI = Confidence Interval

Interpretation of Model

The intercept of this model tells us the expected probability of voting in the 2020 primary for a woman in the age range of 18-30 years who is in the control group of the study and voted in the General Primary election. The estimate is 34.0% with a 95% confidence interval of 33.7% to 34.4%. This is our baseline. According to this model, neither of the two treatments have a significant effect on changing a person’s probability of voting. As you can see, the regression coefficient of -.009 for Neighborhood and -.012 for Self are very small and slightly negative, which indicate that they may actually decrease a person’s likelihood of voting. However, the confidence interval for each of these treatments includes 0, so we cannot say for sure that they have an effect one way or another for females ages 18-30, compared to the control group. However, this model also predicts that there is a larger causal treatment effect for certain groups. For example, for individuals who are Municipal Voters, the average treatment effect of Neighborhood is estimated to be 1.6%.

The average treatment effect does vary from person to person. The plot below is a useful tool for understanding the treatment effects for each type of post card on different types of voters.

Figure 2: Graph of Average Treatment Effects

This graph shows the probability distributions for how much a particular postcard is likely to increase the probability of voting, for ten different types of voters: each of the voter history categories crossed with both of the types of postcards. Each curve represents a range of possible outcomes. This visualization makes it clear that that the causal effects of the postcard treatments do in fact vary depending on a person’s voting history.

Our model makes estimations about the causal effects based on a sample of voters from a 2020 field experiment that encouraged voting by mail (Hopkins et al, 2021). The experiment involved randomly sending 46,960 registered voters postcards that encouraged them to vote by mail. Since the recipients were randomly selected from the larger population of registered voters, it is likely to be representative of the population. We also have high external validity since the voters in the sample are similar to the people we hope to generalize the findings to - all Philadelphia 2020 primary voters. Furthermore, we do not know what the actual causal effect would be on the probability of voting for each primary voter in Philadelphia in 2020 because for any given voter who received the post card, it is impossible to know whether they would have voted had they not received the treatment. This is the fundamental problem of causal inference. Still, the model here aims to estimate probable ranges for the causal effects based on a set of assumptions.

Implications for Future Elections

Not only can this data be used for campaign strategy in the current election cycle, but it can also be used to create predictive models to inform future campaigns. For example, if in the 2024 primary cycle we were considering sending out 100 neighbors postcards to increase voter turnout, we would expect the treatment effects to vary between types of voters once again.

The graph below depicts how many additional voters might be generated by sending out 100 Neighbor postcards, for each different type of voter.

Figure 3: Posterior Probability Distributions for Future Postcard Treatment

Figure 3 shows how many additional voters may be generated for each 100 Neighbor postcards sent out. As you can see, for each type of voter, there is a range of potential outcomes. For example, if you look at General election voters, in red, on the bottom, it seems very likely according to our model that the Neighbor postcards will add up to 10 additional voters; however, it is equally likely that it causes a decrease of 10 voters.

It is important to note that these models were created by looking at women in the age range of 18 to 30, so other demographic groups might have different trends. That being said, this data from the 2020 primary election should be pretty good at predicting outcomes for the 2024 primary, since they are both primary elections set in the same place over a relatively short span of time. Perhaps the most important takeaway from this model is that across types of voters, sending Neighbors postcards is unlikely to yield more than 10 additional voters per 100 postcards mailed, and could potentially have a negative impact on voter turnout.