At 10am AEDT tomorrow, Wednesday 15 November 2017, the Australian Bureau of Statistics will release the results of the Australian Marriage Law Postal Survey (AMLPS).

Ahead of tomorrow’s release of results, I report the results of using the ABC’s 2016 Vote Compass survey to estimate support for same-sex marriage in each of Australia’s 150 Commonwealth electoral divisions (CEDs). These estimates draw on joint work with Dr Shaun Ratcliff and Luke Mansillo, also based at the USSC at the University of Sydney.

Over one million respondents took the ABC/Vote Compass survey in 2016, administered on-line via the ABC’s web site. This is a truly massive sample relative to typical public opinion surveys, yet comes at the cost of pronounced unrepresentativeness and bias. Respondents were not randomly sampled, but self-selected to take the survey either by seeing links to the survey on the ABC’s web site, hearing or seeing references to the survey in ABC programming, or word of mouth. Australians who do not listen to ABC radio or watch ABC TV or do not visit the ABC website (including those who do not use the Internet) can be considered to have a very small probability of taking the survey. Self-selection among the ABC’s audience is also surely an issue. In short, despite the massive size of the VC data, it cannot be considered a random sample of the Australian electorate, not without weighting or other adjustments. We are attracted to the methodological challenge these data provide.

Caveats

As a predictive exercise, the analysis of the VC data provided here has some limitations:

Methods of adjustment

We consider several different approaches to adjusting the Vote Compass SSM data. Most of these use model-based approaches, similar to the “MRP” or “multivariate regression and post-stratification” approach of Andy Gelman and co-workers. Indeed, all of the results reported below rely on post-stratification to some extent. The differences among the different results turn on how the post-stratification is done, and the type of model used in the “MR” stage of “MRP”. In every instance we rely on the presence of Census data tabulated by CED.

Post-stratifying variables

The variables we rely on include:

With the exception of 1st preference vote share, these variables are available in both the VC survey and the Census. On the Census side, we use the ABS tablebuilder product to produce cell counts for the cross-classification of the variables listed above by CED; we refer to this data set of cells and cell counts as the Census post-stratifying frame.

First preference vote share is not available on the Census. We impute vote preference proportions onto each cell in the Census post-stratifying frame by fitting a model using the randomForests package in R for each CED. Predictions of vote proportions from this modeling exercise are adjusted such that aggregated over the cells for each CED, the predicted vote proportions match the actual vote proportions observed in each CED at the 2016 election.

Post-stratifying methods

The methods utilized here are:

The Bayesian additive regression tree models have the best fit in each CED, and so this approach tends to be our preferred method.

Final round of shrinkage for BART estimates

Since each CED is treated separately in the approaches listed above (except the mixed-effects GLMER method), we introduce some Bayesian shrinkage with a simple hierarchical model.

Let \(\theta_i\) be the unknown level of support for SSM in electoral division \(i = 1, \ldots, n\). The BART procedure provides \(y_i\), the BART estimate of support for SSM in CED \(i\) and \(\sigma^2_i\), the posterior variance of the corresponding BART estimate. We posit that \(y_i \sim N(\theta_i, \sigma^2_i)\) and absent any additional information we would estimate \(\theta_i\) with the BART estimate \(y_i\).

Divisional-level covariates not used in the post-stratification exercise can be used to specify a prior for \(\theta_i\), via the model \(\theta_i \sim N(z_i'\gamma, \omega^2)\). Priors on \(\gamma\) and \(\omega^2\) complete the model. Posterior inference for \(\theta_i\) thus combines both the information from VC and the Census post-stratifying frame reflected in \(y_i\) and \(\sigma^2_i\), along with additional divisional-level covariates \(z_i\). Here \(z_i\) comprises

Smoothing splines are fit over the 1st two predictors, using the jagam function in mgcv package generates model matrices for the smooth terms, that we then utilize in the Bayesian model fitting package, JAGS.

Results

National and state results

We can produce a national level estimate by summing over all Census cells in the post-stratification frame, or equivalently, over the CEDs. These results are displayed, below.

Percent in support of SSM, by jurisdiction.
Estimated Low Upper
QLD 52.7 52.2 53.2
NSW 55.0 54.6 55.4
SA 55.3 54.5 56.0
NT 56.1 53.3 58.9
TAS 56.2 54.9 57.4
WA 56.5 55.8 57.2
VIC 60.3 59.8 60.7
ACT 66.7 65.9 67.6
National 56.3 56.0 56.5

We again note that these estimates don’t allocate “don’t know” responses, and should be considered lower bounds on actual support levels for SSM.

Commonwealth electoral divisions

The following table presents results for all 150 CEDs, ranked from highest levels of estimated support for to lowest levels of the “BART + Bayes” estimates, which correspond to the Bayes estimates of \(\theta_i\) in the model described in the preceeding paragraphs, expressed as percentages. Lower and upper bounds of a 95% credible interval for the Bayes estimates are also reported.

Note that in 0 divisions we estimate that support for SSM is below 50%. Note once again that these estimates don’t allocate “don’t know” responses, and should be considered lower bounds on actual support levels for SSM.

Estimated support for SSM, by CED, from statistical adjustment of Vote Compass 2016
Rank Division State BART + Bayes lo up BART GLMER NNET GLM Raked Naive
1 Melbourne VIC 77.0 75.0 79.1 77.1 75.2 75.0 78.6 79.4 88.1
2 Sydney NSW 76.9 74.7 79.2 76.7 75.6 78.1 77.8 80.2 88.0
3 Melbourne Ports VIC 76.1 74.2 78.0 76.2 74.2 75.0 76.6 78.0 86.8
4 Grayndler NSW 73.1 70.9 75.4 73.1 73.1 75.8 73.1 75.4 87.8
5 Wentworth NSW 72.7 70.5 74.9 73.2 71.6 72.2 73.5 72.2 85.4
6 Batman VIC 72.1 68.9 75.2 72.2 71.3 70.2 70.2 69.9 88.9
7 Higgins VIC 69.5 67.8 71.4 69.5 68.6 70.1 69.7 71.3 83.2
8 Wills VIC 69.4 66.3 72.4 69.6 68.2 68.7 68.5 71.3 88.5
9 Brisbane QLD 68.5 66.6 70.5 68.7 67.4 66.5 69.7 69.8 81.1
10 Fenner ACT 67.6 66.3 68.9 67.6 67.4 66.7 67.4 68.8 79.5
11 Gellibrand VIC 67.3 64.4 70.1 66.4 68.6 64.4 66.5 65.4 86.1
12 Goldstein VIC 66.9 65.0 68.8 67.5 65.3 67.9 68.1 67.9 81.2
13 Griffith QLD 66.1 63.9 68.1 66.6 65.1 68.2 67.2 68.9 80.9
14 Canberra ACT 65.9 64.8 67.0 65.9 65.9 67.6 65.7 66.3 77.7
15 Perth WA 65.9 63.7 68.0 65.6 65.3 64.9 64.2 69.0 80.4
16 Newcastle NSW 65.3 63.0 67.4 65.8 64.4 66.0 65.6 66.6 78.4
17 Kooyong VIC 64.3 62.5 66.2 64.5 62.1 64.2 63.6 65.1 78.7
18 Isaacs VIC 64.3 61.8 66.9 65.0 62.3 66.3 65.6 62.7 77.0
19 Ballarat VIC 63.8 61.6 65.9 64.1 62.0 62.7 63.7 65.7 76.9
20 Kingsford Smith NSW 63.4 60.8 65.9 63.8 63.8 62.2 63.0 63.9 80.0
21 Hotham VIC 63.4 60.7 66.0 63.9 61.5 65.7 63.7 60.6 78.0
22 North Sydney NSW 63.3 61.5 65.4 63.7 62.3 62.0 63.2 64.2 78.2
23 Jagajaga VIC 63.1 61.1 65.2 63.2 62.8 67.2 62.4 63.8 79.2
24 Denison TAS 62.7 60.2 65.4 62.2 63.1 61.2 61.4 64.8 79.8
25 Lalor VIC 62.4 59.8 64.9 62.1 60.2 60.3 60.4 61.7 70.7
26 Dunkley VIC 62.3 60.2 64.4 62.4 60.2 62.5 62.5 62.1 76.0
27 Ryan QLD 62.3 60.6 63.9 62.7 61.4 63.2 62.8 64.1 74.9
28 Gorton VIC 62.1 59.3 64.9 63.1 60.9 62.0 62.4 62.1 70.9
29 Adelaide SA 61.9 59.9 63.8 61.8 63.3 62.8 61.9 63.7 79.0
30 Curtin WA 61.9 59.7 63.8 62.1 61.3 65.3 61.3 62.5 76.3
31 Brand WA 61.8 59.3 64.0 62.3 60.3 58.0 62.3 64.0 70.2
32 Warringah NSW 61.8 59.9 63.9 62.3 61.7 63.8 61.5 65.4 79.3
33 Lilley QLD 61.7 59.4 64.0 62.1 60.4 61.3 60.3 63.9 73.9
34 Solomon NT 61.6 58.6 64.1 61.8 61.0 57.7 61.3 61.4 71.4
35 Corio VIC 61.5 58.9 64.1 62.0 60.2 60.6 61.8 62.9 75.4
36 Fremantle WA 61.2 58.8 63.5 61.3 63.3 58.1 60.1 62.1 78.5
37 Port Adelaide SA 61.2 58.5 64.0 61.2 62.4 61.5 61.7 60.7 76.7
38 Cunningham NSW 60.9 58.3 63.4 61.6 59.0 59.2 59.8 62.6 75.1
39 Franklin TAS 60.5 58.3 62.6 61.0 60.2 60.0 60.3 62.1 72.2
40 McEwen VIC 60.0 57.9 62.1 60.4 60.4 59.1 59.9 62.1 72.9
41 Kingston SA 59.8 57.6 61.9 60.2 61.8 60.9 59.7 61.3 71.7
42 Bendigo VIC 59.7 57.5 61.8 59.8 60.2 59.4 58.4 63.0 76.5
43 Maribyrnong VIC 59.6 57.0 62.5 59.3 61.6 61.2 58.7 60.2 80.7
44 Swan WA 59.4 56.7 62.0 58.5 58.8 59.0 58.3 62.2 75.3
45 Stirling WA 58.9 56.8 61.1 58.7 58.6 56.9 58.1 59.0 75.8
46 Oxley QLD 58.7 56.3 61.1 59.4 57.2 59.3 59.2 59.2 68.6
47 Moore WA 58.7 56.5 60.8 59.4 56.8 58.2 58.7 60.0 69.7
48 Mackellar NSW 58.4 56.2 60.7 58.9 55.5 58.9 58.0 59.3 72.8
49 Hunter NSW 58.2 55.8 60.7 59.3 57.6 56.8 60.0 58.2 69.7
50 Makin SA 58.2 56.1 60.0 58.4 56.7 57.2 58.0 57.2 68.6
51 Casey VIC 58.1 55.8 60.2 58.3 57.7 58.1 57.6 61.1 73.3
52 Shortland NSW 58.1 55.7 60.5 58.7 57.8 54.7 58.0 58.8 70.6
53 La Trobe VIC 58.0 56.1 60.2 57.7 58.8 56.3 58.2 59.1 74.3
54 Scullin VIC 57.9 54.8 61.0 58.5 59.5 56.1 58.0 53.6 73.9
55 Holt VIC 57.7 54.9 60.2 57.7 56.7 62.0 57.4 53.9 67.5
56 Hindmarsh SA 57.6 55.2 59.8 57.4 57.7 56.4 57.2 57.8 75.2
57 Chisholm VIC 57.3 55.0 59.8 56.8 57.0 56.5 55.8 58.7 75.2
58 Pearce WA 57.2 54.8 59.5 57.7 56.5 54.3 59.3 58.1 67.2
59 Richmond NSW 56.9 54.4 59.6 57.0 57.6 60.4 57.0 58.2 74.7
60 Wakefield SA 56.9 54.3 59.3 57.7 57.7 55.7 57.7 58.8 68.9
61 Eden-Monaro NSW 56.7 54.7 59.0 57.3 56.5 57.3 58.6 56.1 70.8
62 Rankin QLD 56.6 54.0 59.2 57.3 53.9 58.3 56.6 59.8 63.1
63 Flinders VIC 56.6 54.1 59.0 56.2 57.2 56.3 55.7 57.8 74.6
64 McMillan VIC 56.4 54.0 58.8 56.3 55.3 55.6 56.0 57.0 71.2
65 Moncrieff QLD 56.3 53.5 59.5 56.0 54.3 58.6 55.2 58.1 69.0
66 Deakin VIC 56.3 54.3 58.1 56.1 54.9 55.9 55.2 57.2 72.3
67 Boothby SA 56.3 54.4 58.0 56.1 58.0 55.5 56.9 58.0 74.4
68 Macarthur NSW 56.3 53.6 58.9 56.6 54.2 55.4 57.1 57.3 67.1
69 Burt WA 56.2 53.7 58.7 56.0 55.1 56.7 55.7 56.1 64.3
70 Corangamite VIC 56.2 54.2 58.5 56.0 57.3 52.0 55.0 59.6 75.7
71 Paterson NSW 56.2 53.6 58.7 56.7 54.8 58.0 57.9 57.2 67.1
72 Whitlam NSW 56.2 53.5 58.8 57.0 56.0 62.0 56.6 57.0 67.2
73 Reid NSW 56.1 53.8 58.4 55.6 55.9 55.6 55.5 54.6 72.5
74 Dobell NSW 56.0 53.6 58.3 55.8 55.8 53.5 55.9 56.8 71.5
75 Herbert QLD 55.7 53.1 58.4 55.8 55.4 55.3 56.3 59.5 69.0
76 Robertson NSW 55.7 53.2 57.9 55.5 55.1 57.6 54.6 57.4 71.2
77 McPherson QLD 55.6 52.9 58.1 55.5 54.7 56.5 54.6 58.7 70.4
78 Dickson QLD 55.5 53.6 57.2 55.6 55.1 52.8 55.6 56.9 66.2
79 Hasluck WA 55.4 53.2 57.9 54.9 54.9 56.0 55.2 56.3 68.2
80 Cowan WA 55.4 53.0 57.9 55.2 56.1 53.7 54.3 54.1 68.8
81 Sturt SA 55.4 53.4 57.6 55.2 55.1 54.2 54.4 55.1 72.5
82 Aston VIC 55.4 53.1 57.5 55.3 54.5 50.8 55.0 56.0 69.4
83 Barton NSW 55.2 52.2 58.4 53.2 55.6 54.0 56.3 53.4 76.0
84 Bowman QLD 54.5 52.4 56.8 54.8 53.5 55.5 54.9 56.4 64.9
85 Forrest WA 54.5 51.8 56.8 54.8 52.6 52.8 54.4 54.6 66.5
86 Bradfield NSW 54.3 52.6 56.2 54.4 52.8 52.9 53.7 53.7 69.2
87 Petrie QLD 54.3 51.9 56.7 54.2 52.4 53.8 54.5 54.4 64.9
88 Leichhardt QLD 54.2 51.4 57.0 54.6 54.3 56.3 52.4 57.8 67.2
89 Bass TAS 54.0 51.5 56.3 53.8 55.1 55.0 52.9 56.1 69.3
90 Moreton QLD 53.7 51.4 55.9 53.0 55.8 51.0 53.8 55.3 73.2
91 Menzies VIC 53.7 51.4 55.9 53.4 52.2 52.5 53.6 52.8 69.5
92 Indi VIC 53.6 51.3 55.9 53.5 54.9 51.8 52.4 54.4 71.3
93 Watson NSW 53.6 49.9 57.2 52.7 50.4 51.1 51.3 44.1 66.7
94 Bonner QLD 53.4 51.3 55.6 53.1 53.7 48.1 53.2 54.9 68.3
95 Lyons TAS 53.3 50.4 56.3 53.1 53.7 52.6 53.0 54.4 67.7
96 Hughes NSW 53.3 51.2 55.3 53.1 50.7 53.0 52.1 53.7 68.1
97 Cook NSW 53.3 50.7 55.9 52.9 50.5 56.4 51.3 52.0 68.9
98 Chifley NSW 53.3 49.8 56.6 53.2 48.4 52.8 55.7 50.6 59.7
99 Bennelong NSW 53.2 50.9 55.2 52.9 49.5 55.2 54.6 52.0 68.5
100 Greenway NSW 53.2 50.7 55.6 52.9 49.7 55.4 53.5 53.4 62.0
101 Forde QLD 53.0 50.5 55.3 52.7 52.3 55.7 51.9 54.2 62.4
102 Blair QLD 53.0 50.6 55.6 53.3 53.6 49.9 50.5 56.4 66.0
103 Wannon VIC 53.0 50.1 55.6 53.3 51.9 56.0 54.7 52.4 70.8
104 Mayo SA 52.9 51.1 54.6 52.7 54.2 52.0 52.5 54.1 70.0
105 Fadden QLD 52.8 50.5 55.3 52.5 53.1 52.2 52.3 54.9 66.7
106 Macquarie NSW 52.8 50.6 55.1 51.9 54.6 51.5 51.3 54.4 71.4
107 Lindsay NSW 52.7 50.0 55.4 51.7 52.3 50.1 50.5 51.6 67.4
108 Calare NSW 52.5 49.9 54.9 52.8 51.2 53.4 53.9 53.1 68.3
109 Gilmore NSW 52.2 49.8 54.6 52.2 52.0 49.4 50.1 55.1 68.1
110 Page NSW 52.0 49.2 54.8 51.6 51.8 51.1 52.3 53.8 69.6
111 Parramatta NSW 51.8 49.2 54.7 51.1 48.7 49.4 49.6 50.4 64.5
112 Mitchell NSW 51.7 49.4 53.9 51.6 49.0 51.5 51.5 53.1 63.9
113 Fisher QLD 51.6 49.3 54.0 51.5 51.8 52.6 51.4 53.4 64.4
114 Gippsland VIC 51.5 49.0 54.0 51.1 49.4 49.8 52.1 51.9 67.6
115 Lingiari NT 51.4 46.9 55.4 50.8 53.4 52.0 47.9 51.5 65.9
116 Banks NSW 51.3 48.4 54.1 50.3 48.2 49.3 50.1 49.5 66.5
117 Canning WA 51.2 48.5 53.7 50.8 49.1 53.3 51.9 51.9 62.9
118 Calwell VIC 51.2 47.8 54.5 49.6 54.1 52.5 48.5 47.4 67.2
119 Tangney WA 50.9 48.9 53.0 50.7 52.0 49.3 50.7 52.0 68.0
120 Bruce VIC 50.8 48.0 53.4 49.8 49.8 49.3 49.1 48.5 65.5
121 Berowra NSW 50.8 48.9 52.7 50.5 49.5 49.1 50.8 50.5 66.6
122 Dawson QLD 50.8 47.5 53.7 52.0 49.0 59.4 49.5 54.6 61.9
123 Fairfax QLD 50.7 48.4 52.9 50.3 51.2 46.3 49.9 52.1 64.7
124 Longman QLD 50.7 48.1 53.5 50.2 49.5 46.9 51.1 52.3 61.3
125 Werriwa NSW 50.7 46.6 54.5 51.3 48.5 50.1 NA 45.5 64.1
126 Braddon TAS 50.4 47.6 53.3 50.6 51.1 53.9 49.5 52.0 63.1
127 Durack WA 50.2 46.5 53.7 50.7 51.7 49.3 50.9 57.0 66.8
128 McMahon NSW 50.1 46.5 53.7 50.6 49.1 55.5 50.1 46.9 62.5
129 Lyne NSW 50.1 47.5 52.7 50.4 47.7 48.3 52.0 51.8 65.1
130 Mallee VIC 50.0 47.0 52.9 49.7 47.3 51.7 50.4 51.5 67.3
131 Fowler NSW 50.0 45.4 54.6 52.6 50.0 51.8 51.4 48.8 65.5
132 Hume NSW 49.9 47.2 52.4 49.5 51.1 51.6 48.5 51.0 66.8
133 O’Connor WA 49.5 47.0 51.9 49.6 49.9 46.4 48.3 49.8 63.5
134 Murray VIC 49.5 46.6 52.2 48.9 47.2 46.9 49.0 49.3 67.7
135 Farrer NSW 49.0 46.3 51.7 49.9 48.9 50.6 50.5 49.3 67.9
136 Blaxland NSW 49.0 44.8 53.1 48.9 46.4 49.6 50.4 45.9 59.2
137 Wide Bay QLD 47.3 44.9 49.8 46.8 47.6 46.1 47.3 49.2 59.8
138 Riverina NSW 46.8 43.8 49.7 46.7 45.8 46.7 46.6 48.7 66.1
139 Wright QLD 46.8 44.1 49.4 46.3 47.3 42.1 44.4 48.9 61.0
140 Cowper NSW 46.5 43.7 49.1 44.6 46.9 49.5 44.1 48.5 68.4
141 Capricornia QLD 46.1 43.5 48.8 46.1 49.4 44.5 45.6 48.3 59.8
142 Parkes NSW 45.2 42.1 48.5 45.3 45.1 47.9 46.2 44.2 60.7
143 Groom QLD 44.9 42.4 47.3 44.3 42.8 41.2 44.2 46.4 57.5
144 Grey SA 44.6 41.5 47.5 44.9 47.7 43.7 46.3 47.7 61.1
145 Kennedy QLD 44.5 41.3 47.5 44.9 43.5 46.3 43.9 47.0 56.9
146 Hinkler QLD 44.1 40.6 47.4 42.5 41.5 36.8 41.4 42.4 53.1
147 New England NSW 44.0 40.9 46.6 42.8 43.8 41.8 41.7 43.8 62.3
148 Barker SA 43.7 41.4 46.2 43.0 43.8 45.9 42.8 45.2 60.2
149 Flynn QLD 42.7 39.8 45.7 42.7 44.5 49.0 40.2 46.1 56.3
150 Maranoa QLD 37.2 34.4 40.1 36.5 39.3 36.7 35.1 36.0 53.6

We compare the Bayes estimates and naive, raw estimates in the following dotplot:

Comparison of Bayes estimates and naive estimates.

Comparison of Bayes estimates and naive estimates.