Summary

• A House candidate’s “personal vote” is that portion of their vote that is due to the candidate’s personal attributes (popularity, name recognition, reputation, history of service and performance as an incumbent or in non-political endeavours) as opposed to their party affiliation.

• A convenient method for estimating a House candidate’s personal vote is to compute the difference between their 1st preference vote and their party’s share of 1st preferences cast for Senate in the candidate’s division.

• In this report I examine variation in this quantity for Coalition, Labor and Green House candidates in the 2016 Australian Federal election.

• Senate votes are much more dispersed across a greater number of minor party tickets/candidates than are House votes. Therefore, personal votes calculated as the difference between House and Senate vote shares are almost always positive, in that House vote shares almost always exceed Senate vote shares in a given division, for the Coalition, Labor and typically the Greens as well. Accordingly, politically meaningful insight follows from observing not the absolute magnitudes of these apparent personal votes, but the relative magnitudes of the House/Senate differneital and how this quantity varies across candidates.

• Incumbents enjoy higher personal votes than challengers, by as much as 4 to 5 percentage points. This incumbency advantage in personal votes does not appear to vary by party.

• Coalition candidates appear to have a slightly higher personal vote than Labor candidates, about 2/3s of a percentage points. Green candidates have a smaller personal vote than Labor candidates, by as much two percentage points.

• Candidate gender is not associated with the magnitude of the personal vote, save for some weak evidence suggesting male Coalition candidates have a higher personal vote than female Coalition candidates.

• Ballot position and ballot length shape the magnitude of personal votes. Being listed at the top of the House ballot papers lifts a candidate’s appartent personal vote share by 1.25 percentage points. Similarly, as the number of House candidates grows, the magnitude of the personal vote drops, by about half a percentage point for every extra House candidate. Since ballot position and ballot length effects are not personal attributes of candidates — and indeed, in the case of ballot position, assigned randomly — care should be taken when intepreting the difference between House vote and Senate vote shares as an estimate of the House candidate’s “personal vote”.

• A caveat. Consider the following possibility, where a House candidate is so well known or so popular in their electoral division that they generate coattails for their party’s Senate ticket. In this case the difference between the House vote and the Senate vote could well understate the candidate’s personal vote, in that the latter is inflated by the candidate’s popularity.

Estimating personal votes for House candidates

Adopting a technique favored by Peter “Mumble” Brent, we compute a House candidate’s “personal vote” as the difference between their 1st preference vote and their party’s share of 1st preferences cast for Senate in the candidate’s division.

This is easily implemented given the results files provided by the AEC, as the following code demonstrates.

Reading the AEC data

We begin by reading the Senate results data, containing votes at the divisional level.

baseURL <- "https://results.aec.gov.au/20499/Website/Downloads/SenateFirstPrefsByDivisionByVoteTypeDownload-20499.csv"
theFile <- "~/Downloads/SenateFirstPrefsByDivisionByVoteTypeDownload-20499.csv"

## to speed up debugging, only download once
## use saved file thereafter
if(!file.exists(theFile)){
  download.file(baseURL,theFile)
}

## read file, skipping 1st line with AEC info
out <- read_csv(theFile,
                skip=1)

We next compute division-level totals and percentages for each party.

out <- out %>%
  dplyr::rename(division=DivisionNm) %>%
  group_by(division,PartyAb) %>%
  summarise(total=sum(TotalVotes),
            State=StateAb[1]) %>%
  group_by(division) %>%
  mutate(p=total/sum(total)*100)

We create an auxiliary data set for mapping candidates to parties, recognizing that Senate tickets and Senate candidates have different labels and abbreviations in different states. This provides a series of mappings from party abbreivations in different states to some common tags, that we can merge against the House data for analysis (across states) of personal votes by party.
This gets tricky in the case of three-cornered contests, where Nationals and Liberals run against one another and in some cases we will exclude data from these seats.

labs <- list()
labs[["NSW"]] <- data.frame(LP="Coalition",
                            NP="Coalition",
                            LPNP="Coalition",
                            ALP="ALP",
                            GRN="GRN")
labs[["VIC"]] <- labs[["NSW"]]
labs[["QLD"]] <- data.frame(LNP="Coalition",
                            ALP="ALP",
                            GRN="GRN")
labs[["TAS"]] <- data.frame(LP="Coalition",ALP="ALP",GRN="GRN")
labs[["SA"]] <- labs[["TAS"]]
labs[["WA"]] <- data.frame(LP="LP",NP="NP",ALP="ALP",GRN="GRN")
labs[["NT"]] <- data.frame(CLP="Coalition",
                           ALP="ALP",
                           GRN="GRN")
labs[["ACT"]] <- labs[["TAS"]]

labs <- bind_rows(labs,.id="State") %>%
  group_by(State) %>%
  gather(PartyAb,Party,-State) %>%
  ungroup() %>%
  filter(!is.na(Party))

We now implement the selections, writing Party back onto the file of results:

out2 <- left_join(out,labs,by=c("State","PartyAb")) %>%
  group_by(State,division,Party) %>%
  summarise(p=sum(p)) %>%
  filter(Party %in% c("Coalition","LP","NP","ALP","GRN","CLP"))

House results, by CED

We now download and parse 1st preference House of Representatives results, by division.

The last line, below, computes the difference between House vote (per) and Senate vote (p), as pvote, our estimate of the House candidate’s “personal vote.”

hdata <- read_csv("https://results.aec.gov.au/20499/Website/Downloads/HouseFirstPrefsByCandidateByVoteTypeDownload-20499.csv",skip=1)

out3 <- hdata %>%
  rename(division=DivisionNm,State=StateAb) %>%
  filter(Surname!="Informal") %>%
  group_by(division) %>%
  mutate(Party=PartyAb,
         per=TotalVotes/sum(TotalVotes)*100) %>%
  select(State,division,Party,PartyNm,
         Surname,GivenNm,CandidateID,
         Incumbent=HistoricElected,
         BallotPosition,per) %>%
  mutate(Party=if_else(State!="WA",
                       recode(Party,
                              `LP`="Coalition",
                              `NP`="Coalition",
                              `LNP`="Coalition",
                              `CLP`="Coalition"),
                       Party)) %>%
  left_join(out2,by=c("State","division","Party")) %>% 
  filter(Party %in% c("ALP","LP","NP","Coalition","GRN")) %>%
  mutate(pvote = per - p)

We now drop Coalition House candidates in three-cornered contests but where the Coalition ran a joint Senate ticket, which occured in Ballarat, Bendigo, Indi, McEwan, Murray and Whitlam. The labels “LP” and “NP” apply to Western Australia, where the Coalition does not run a joint Senate ticket and the Nationals ran candidates in five seats (all held by Liberal incumbents, incidentally).

tcc <- out3 %>% 
  group_by(division,Party) %>% 
  summarise(n=n()) %>% 
  ungroup() %>% 
  filter(Party=="Coalition",n>1)

out3 <- anti_join(out3,
                  tcc,
                  by=c("division","Party"))

Results

We display estimated personal votes for House candidates in table that can filtered by state, party and incumbency, etc. Both major parties no shortage of appartent “stars”, where the House vote outpaces Senate vote shares by more than 10 percentage points. There are 15 such Labor incumbents, 11 Liberals, 8 Nationals and 7 LNP incumbents with these high levels of apparent personal votes. Incumbents – perhaps by definition – have higher personal votes than challengers, and this is true across the parties.

library(DT)
datatable(out3 %>% 
            rename(House=per,Senate=p) %>%
            arrange(desc(pvote)) %>%
            mutate(Party=recode(PartyNm,
                                `The Nationals`="NAT",
                                `Liberal National Party of Queensland`="LNP",
                                `Australian Labor Party`="ALP",
                                `Liberal`="LIB",
                                `Labor`="ALP",
                                `The Greens`="GRN",
                                `The Greens (WA)`="GRN",
                                `Country Liberals (NT)`="CLP",
                                `Australian Labor Party (Northern Territory Branch)`="ALP")) %>%
            ##filter(abb=="lib" & HistoricElected=="Y") %>%
            select(division,State,Party,Incumbent,Surname,GivenNm,
                   House,Senate,pvote),
          options=list(pageLength=20),
          elementId = "results_table",
          filter="top",rownames=FALSE) %>%
  #formatStyle(c("division","State","PartyNm",
  #            "Incumbent","Surname","GivenNm",
  #            "House","Senate","pvote"),
  #            fontFamily="AtlasGrotesk",
  #            fontSize="12px") %>%
  formatRound(~House+Senate+pvote,digits=1)

NSW Coalition incumbents

In the graph below, data points on the left show Senate votes for the Coalition in House seats with Coalition incumbents; on the right hand side of the graph are the corresponding share of 1st preferences for Coalition incumbents. The lines connecting each pair of data points tends to slope up, indicating that on average, Coalition House candidates outperformed the Senate ticket in their respective seats. For Coalition incumbents in NSW in 2016, the average difference between House and Senate vote was 8.2 percentage points; in Wentworth this difference was 9.3 percentage points.

Graphical inspection also indicates the extent to which Wentworth lies “off-trend”, again consistent with a larger than average personal vote for Malcolm Turnbull in that seat. But the graph also suggests that Wentworth is not especially unusual in this regard, relative to other NSW Coalition-held seats in 2016.

Wentworth is more “unusual” or “distinct” with respect to its large House and Senate votes for the Coalition. If in 2016, Turnbull’s apparent personal vote was hypothetically set to zero, such that the Liberal’s House vote share fell to the Coalition’s Senate vote share in the seat (53.0%), Wentworth would remain a safe Liberal seat, safer than Warringah, North Sydney or Bennelong.

d3d <- out3 %>% 
  ungroup() %>%
  filter(Party=="Coalition" & 
           Incumbent=="Y" & 
           State=="NSW") %>%
  dplyr::rename(Senate=p,House=per) %>%
  dplyr::mutate(name=stringr::str_to_title(Surname),
                indx=1:n()) %>%
  dplyr::select(division,indx,Senate,House,name,pvote) %>%
  arrange(division)

d3d <- jsonlite::toJSON(d3d)
                       
cat(
  paste(
  '<script>
    var data = ', d3d ,';
  </script>'
  , sep="")
)

Analysis

What predicts the size of the personal vote?

# Vector of smoothing methods for each plot panel
meths <- c("loess","loess","loess","NULL","NULL")

# Smoothing function with different behaviour in the different plot panels
mysmooth <- function(formula,data,...){
   meth <- eval(parse(text=meths[unique(data$PANEL)]))
   x <- match.call()
   x[[1]] <- meth
   eval.parent(x)
}

library(ggplot2)
p <- ggplot(data=out3,
       aes(x=p,y=per,
           group=Incumbent,
           color=Incumbent)) +
  geom_abline(intercept = 0, slope = 1) +
  geom_point(pch=1) +
  geom_smooth(method="mysmooth",se=FALSE) +
  facet_wrap(~Party) +
  scale_x_continuous("Senate Vote, by Division") +
  scale_y_continuous("House Vote, by Division") +
  mytheme +
  theme(legend.position = "top")
print(p)

m <- lm(pvote ~ Party + Incumbent,
        data=out3)
pander(summary(m),
       digits=2,
       caption="Regression analysis of apparent personal votes: differences by party and incumbency.")

m2 <- lm(pvote ~ Party*Incumbent,
        data=out3)
pander(anova(m,m2,test="F"),digits=2,add.significance.stars = FALSE,
       caption = "F Test, invariance of incumbency effects across parties.")

library(jsonlite)
gender <- read_json(path="../../../AEC/results/LightProgress/data/candidates.json",
                    flatten=TRUE,
                    simplifyVector = TRUE)
gender <- bind_rows(lapply(gender,
                           function(x){
                             data.frame(CandidateID=x$candidate_id,
                                        gender=x$gender)
                           }))
out3 <- left_join(out3,gender)

m3 <- update(m, ~ . + gender)
m4 <- update(m, ~ . + gender*Incumbent)
m5 <- update(m, ~ . + gender*Party)

library(memisc)
mt <- mtable("Base Model"=m,
             "+Gender"=m3,
             "+Gender*Incumbency"=m4,
             "+Gender*Party"=m5,
             digits=2,
             sdigits=2,
             signif.symbols = FALSE,
             summary.stats=c("sigma","R-squared","AIC"))
mt <- relabel(mt,"(Intercept)" = "Constant")
show_html(mt)

out3 <- left_join(out3,
                   hdata %>% 
                     dplyr::rename(division=DivisionNm) %>% 
                     group_by(division) %>% 
                     filter(BallotPosition!=999) %>% 
                     summarise(ballotLength=max(BallotPosition)),
                   by="division")

library(ggplot2)
ggplot(data=out3,
       aes(x=ballotLength,y=pvote,group=Incumbent,color=Incumbent)) + 
    geom_smooth(method="mysmooth",se=FALSE) +
  geom_point(pch=1) + 
  facet_wrap(~Party) + 
  mytheme +
  theme(legend.position = "top")

library(ggplot2)
ggplot(data=out3,
       aes(x=BallotPosition,y=pvote,group=Incumbent,color=Incumbent)) + 
    geom_smooth(method="mysmooth",se=FALSE) +
  geom_point(pch=1) + 
  facet_wrap(~Party) + 
  mytheme +
  theme(legend.position = "top")

m6 <-  update(m, ~ . + ballotLength)
m7 <-  update(m, ~ . + BallotPosition)
m8 <-  update(m, ~ . + ballotLength + BallotPosition)
m9 <-  update(m, ~ . + ballotLength + I(BallotPosition==1))
m10 <- update(m, ~ . + ballotLength * I(BallotPosition==1))

mt <- mtable("Base Model"=m,
             "+Ballot Length"=m6,
             "+Ballot Position"=m7,
             "+Length * Position"=m8,
             "+Length + 1st place"=m9,
             "+Length * 1st place"=m10,
             digits=2,
             sdigits=2,
             signif.symbols = FALSE,
             summary.stats=c("sigma","R-squared","AIC"))
mt <- relabel(mt,"(Intercept)" = "Constant")
show_html(mt)

library(GGally)
ggscatmat(out3 %>%
            ungroup() %>%
            dplyr::select(pvote,Party,division) %>%
            tidyr::spread(Party,pvote) %>%
            dplyr::select(-division))

library(modelr)
ggscatmat(out3 %>%
            add_residuals(m9) %>%
            ungroup() %>%
            dplyr::select(resid,Party,division) %>%
            tidyr::spread(Party,resid) %>%
            dplyr::select(-division))

library(rstanarm)
options(mc.cores = parallel::detectCores()-2)
m9a <- stan_lmer(pvote ~ Party + Incumbent + ballotLength + I(BallotPosition==1) + (1|division),
            data=out3)

sjPlot::tab_model(m9a,show.p = FALSE,show.aic = TRUE,digits=2,ci.hyphen = ", ",
                  bpe="mean")