Tweet-level: treatment vs control

Outcomes are scaled (z-score). Toxicity-family outcomes are transformed to deciles (ntile(..., 10)) before scaling. username_changes_count is transformed via log(x + 1) then scaled. crypto, politics, lean1, abs_lean1, and pc1 are scaled without additional transform. Deduplication on tweet_id. For lean1, abs_lean1, and pc1, deduplication is on (tweet_id, domain).

## 
## 
## ### All tweets 
## 
## n = 1463896  | users = 2267 
## 
## 
## 
## Table: Treatment vs Control (scaled coefficient, p-value) — All tweets
## 
## |outcome                |       n| scaled_coefficient| p_value|
## |:----------------------|-------:|------------------:|-------:|
## |toxicity               | 1463896|              0.003|   0.931|
## |severe_toxicity        | 1463896|              0.127|  <0.001|
## |obscene                | 1463896|              0.121|  <0.001|
## |identity_attack        | 1463896|              0.126|  <0.001|
## |insult                 | 1463896|              0.085|   0.004|
## |threat                 | 1463896|              0.091|   0.002|
## |sexual_explicit        | 1463896|              0.094|   0.002|
## |crypto                 | 1463889|              0.095|   0.001|
## |politics               | 1463889|              0.066|   0.020|
## |lean1                  |   35506|             -0.422|   0.095|
## |abs_lean1              |   35506|              0.424|   0.064|
## |pc1                    |   35506|             -0.220|   0.415|
## |scam_eq1               |  418292|              0.080|  <0.001|
## |scam_eq1_or2           |  418292|              0.107|  <0.001|
## |health_is              |  834717|              0.001|   0.963|
## |economy_is             |  834717|              0.046|   0.116|
## |science_is             |  834717|             -0.100|  <0.001|
## |politics_is            |  834717|              0.115|   0.004|
## |username_changes_count | 1463896|              0.376|  <0.001|
## 
## 
## ### Reply tweets 
## 
## n = 1007764  | users = 2064 
## 
## 
## 
## Table: Treatment vs Control (scaled coefficient, p-value) — Reply tweets
## 
## |outcome                |       n| scaled_coefficient| p_value|
## |:----------------------|-------:|------------------:|-------:|
## |toxicity               | 1007764|              0.029|   0.434|
## |severe_toxicity        | 1007764|              0.113|  <0.001|
## |obscene                | 1007764|              0.123|  <0.001|
## |identity_attack        | 1007764|              0.116|  <0.001|
## |insult                 | 1007764|              0.103|   0.002|
## |threat                 | 1007764|              0.090|   0.003|
## |sexual_explicit        | 1007764|              0.093|   0.004|
## |crypto                 | 1007763|              0.063|   0.074|
## |politics               | 1007763|              0.018|   0.561|
## |lean1                  |    4854|             -0.194|   0.098|
## |abs_lean1              |    4854|             -0.184|   0.076|
## |pc1                    |    4854|              0.270|   0.043|
## |health_is              |  584142|             -0.013|   0.431|
## |economy_is             |  584142|              0.017|   0.627|
## |science_is             |  584142|             -0.089|   0.001|
## |politics_is            |  584142|              0.059|   0.142|
## |username_changes_count | 1007764|              0.358|  <0.001|
## 
## 
## ### Primary tweets 
## 
## n = 456132  | users = 2190 
## 
## 
## 
## Table: Treatment vs Control (scaled coefficient, p-value) — Primary tweets
## 
## |outcome                |      n| scaled_coefficient| p_value|
## |:----------------------|------:|------------------:|-------:|
## |toxicity               | 456132|             -0.053|   0.307|
## |severe_toxicity        | 456132|              0.076|   0.080|
## |obscene                | 456132|              0.056|   0.305|
## |identity_attack        | 456132|              0.088|   0.021|
## |insult                 | 456132|              0.061|   0.183|
## |threat                 | 456132|              0.008|   0.869|
## |sexual_explicit        | 456132|              0.026|   0.621|
## |crypto                 | 456126|              0.123|   0.002|
## |politics               | 456126|              0.091|   0.049|
## |lean1                  |  30652|             -0.457|   0.099|
## |abs_lean1              |  30652|              0.492|   0.047|
## |pc1                    |  30652|             -0.287|   0.352|
## |scam_eq1               | 418292|              0.080|  <0.001|
## |scam_eq1_or2           | 418292|              0.107|  <0.001|
## |health_is              | 250575|              0.014|   0.428|
## |economy_is             | 250575|              0.070|   0.055|
## |science_is             | 250575|             -0.013|   0.689|
## |politics_is            | 250575|              0.120|   0.050|
## |username_changes_count | 456132|              0.412|  <0.001|

Combined table (coef + p-value)

Scaled coefficient and p-value for treatment effect (group = treatment, baseline control).

Treatment vs Control — Combined (coef + p-value)
Outcome All p-value Primary p-value Replies p-value
toxicity 0.003 0.931 -0.053 0.307 0.029 0.434
severe_toxicity 0.127 <0.001 0.076 0.080 0.113 <0.001
obscene 0.121 <0.001 0.056 0.305 0.123 <0.001
identity_attack 0.126 <0.001 0.088 0.021 0.116 <0.001
insult 0.085 0.004 0.061 0.183 0.103 0.002
threat 0.091 0.002 0.008 0.869 0.090 0.003
sexual_explicit 0.094 0.002 0.026 0.621 0.093 0.004
crypto 0.095 0.001 0.123 0.002 0.063 0.074
politics 0.066 0.020 0.091 0.049 0.018 0.561
lean1 -0.422 0.095 -0.457 0.099 -0.194 0.098
abs_lean1 0.424 0.064 0.492 0.047 -0.184 0.076
pc1 -0.220 0.415 -0.287 0.352 0.270 0.043
scam_eq1 0.080 <0.001 0.080 <0.001 NA NA
scam_eq1_or2 0.107 <0.001 0.107 <0.001 NA NA
health_is 0.001 0.963 0.014 0.428 -0.013 0.431
economy_is 0.046 0.116 0.070 0.055 0.017 0.627
science_is -0.100 <0.001 -0.013 0.689 -0.089 0.001
politics_is 0.115 0.004 0.120 0.050 0.059 0.142
username_changes_count 0.376 <0.001 0.412 <0.001 0.358 <0.001

User-average combined table

For each subset (all/reply/primary), outcomes are first averaged within user, then the same treatment-vs-control regression is run on user-level means.

User-average Treatment vs Control — Combined (coef + p-value)
Outcome All p-value Primary p-value Replies p-value
toxicity -0.340 <0.001 -0.336 <0.001 -0.289 <0.001
severe_toxicity -0.253 <0.001 -0.257 <0.001 -0.195 <0.001
obscene -0.314 <0.001 -0.286 <0.001 -0.272 <0.001
identity_attack -0.173 <0.001 -0.177 <0.001 -0.176 <0.001
insult -0.376 <0.001 -0.353 <0.001 -0.309 <0.001
threat -0.259 <0.001 -0.243 <0.001 -0.225 <0.001
sexual_explicit -0.326 <0.001 -0.274 <0.001 -0.278 <0.001
crypto 0.320 <0.001 0.325 <0.001 0.251 <0.001
politics 0.035 0.410 0.030 0.495 -0.164 <0.001
lean1 -0.044 0.479 -0.072 0.281 0.003 0.977
abs_lean1 -0.117 0.063 -0.133 0.049 -0.194 0.031
pc1 -0.022 0.721 -0.086 0.199 0.051 0.573
scam_eq1 0.353 <0.001 0.353 <0.001 NA NA
scam_eq1_or2 0.413 <0.001 0.413 <0.001 NA NA
health_is 0.003 0.960 0.081 0.170 -0.073 0.222
economy_is 0.278 <0.001 0.311 <0.001 0.135 0.029
science_is -0.030 0.599 0.167 0.005 -0.045 0.460
politics_is 0.090 0.112 0.038 0.512 -0.116 0.049
username_changes_count 0.434 <0.001 0.436 <0.001 0.408 <0.001

Weighted user-average combined table

User-level means are regressed on treatment vs control using weights equal to each user’s tweet count.

Weighted user-average Treatment vs Control — Combined (coef + p-value)
Outcome All p-value Primary p-value Replies p-value
toxicity -0.092 0.163 -0.184 0.034 -0.007 0.925
severe_toxicity -0.055 0.386 -0.092 0.297 0.007 0.915
obscene -0.098 0.109 -0.097 0.235 -0.061 0.343
identity_attack 0.079 0.215 0.014 0.852 0.111 0.105
insult -0.113 0.075 -0.182 0.037 -0.034 0.608
threat -0.019 0.756 -0.086 0.275 0.014 0.827
sexual_explicit -0.098 0.084 -0.123 0.086 -0.039 0.510
crypto 0.110 0.001 0.154 0.002 0.088 0.074
politics 0.135 0.020 0.176 0.049 0.040 0.561
lean1 -0.126 0.166 -0.231 0.052 -0.025 0.853
abs_lean1 -0.062 0.464 -0.044 0.703 -0.055 0.650
pc1 0.104 0.306 -0.045 0.768 0.218 0.097
scam_eq1 0.089 0.008 0.166 <0.001 NA NA
scam_eq1_or2 0.113 <0.001 0.185 <0.001 NA NA
health_is 0.001 0.963 0.024 0.429 -0.023 0.431
economy_is 0.085 0.116 0.125 0.055 0.039 0.627
science_is -0.230 <0.001 -0.019 0.689 -0.211 0.001
politics_is 0.242 0.004 0.243 0.050 0.127 0.142
username_changes_count 0.389 <0.001 0.430 <0.001 0.369 <0.001

Outcomes vs log(username changes + 1): Combined table

Same outcomes, but replace treatment indicator with log(username_changes_count + 1) as the key regressor.

Outcomes vs log(username changes + 1) — Combined (coef + p-value)
Outcome All p-value Primary p-value Replies p-value
toxicity 0.015 0.486 0.038 0.209 0.005 0.842
severe_toxicity 0.024 0.247 0.012 0.697 0.026 0.236
obscene 0.020 0.399 0.018 0.615 0.017 0.510
identity_attack 0.029 0.119 0.025 0.329 0.028 0.194
insult 0.015 0.468 0.020 0.510 0.014 0.550
threat 0.018 0.349 0.016 0.626 0.015 0.503
sexual_explicit 0.018 0.396 0.019 0.574 0.015 0.538
crypto 0.028 0.135 0.049 0.095 0.012 0.508
politics 0.022 0.378 0.039 0.250 0.008 0.817
lean1 0.232 0.214 0.339 0.123 -0.106 0.178
abs_lean1 0.131 0.508 0.195 0.402 -0.042 0.546
pc1 -0.371 0.124 -0.497 0.078 0.057 0.519
scam_eq1 0.049 <0.001 0.049 <0.001 NA NA
scam_eq1_or2 0.055 0.002 0.055 0.002 NA NA
health_is -0.024 0.050 -0.024 0.117 -0.025 0.066
economy_is 0.028 0.107 0.009 0.735 0.035 0.048
science_is 0.002 0.933 -0.014 0.586 0.018 0.445
politics_is 0.041 0.162 0.069 0.127 0.011 0.706

User-average outcomes vs log(username changes + 1)

User-average outcomes vs log(username changes + 1) — Combined (coef + p-value)
Outcome All p-value Primary p-value Replies p-value
toxicity -0.065 0.052 -0.053 0.120 -0.055 0.111
severe_toxicity 0.003 0.926 -0.009 0.812 0.027 0.452
obscene -0.017 0.618 -0.024 0.500 0.007 0.852
identity_attack -0.031 0.371 -0.012 0.740 -0.017 0.644
insult -0.090 0.007 -0.086 0.012 -0.075 0.031
threat 0.001 0.975 0.027 0.436 0.022 0.544
sexual_explicit 0.007 0.853 -0.002 0.959 0.013 0.715
crypto 0.148 <0.001 0.151 <0.001 0.098 0.004
politics -0.067 0.052 -0.065 0.063 -0.090 0.014
lean1 0.038 0.450 0.061 0.266 0.014 0.831
abs_lean1 -0.076 0.136 -0.074 0.187 -0.055 0.427
pc1 -0.006 0.896 -0.063 0.249 0.017 0.796
scam_eq1 0.230 <0.001 0.230 <0.001 NA NA
scam_eq1_or2 0.203 <0.001 0.203 <0.001 NA NA
health_is -0.088 0.031 -0.109 0.003 -0.043 0.370
economy_is 0.015 0.736 0.018 0.676 0.094 0.061
science_is -0.009 0.838 -0.039 0.362 0.029 0.533
politics_is -0.028 0.558 -0.058 0.220 -0.095 0.063

Weighted user-average outcomes vs log(username changes + 1)

Weighted user-average outcomes vs log(username changes + 1) — Combined (coef + p-value)
Outcome All p-value Primary p-value Replies p-value
toxicity -0.019 0.666 0.010 0.851 -0.032 0.510
severe_toxicity 0.045 0.362 0.069 0.223 0.030 0.568
obscene 0.023 0.634 0.084 0.105 0.002 0.964
identity_attack 0.048 0.289 0.072 0.128 0.022 0.645
insult -0.045 0.298 -0.021 0.700 -0.042 0.353
threat 0.045 0.258 0.051 0.235 0.040 0.358
sexual_explicit 0.031 0.484 0.044 0.357 0.002 0.972
crypto 0.033 0.135 0.061 0.095 0.016 0.508
politics 0.046 0.378 0.076 0.250 0.017 0.817
lean1 0.103 0.122 0.156 0.052 0.042 0.726
abs_lean1 -0.036 0.552 -0.081 0.284 -0.014 0.882
pc1 0.010 0.889 -0.122 0.301 0.016 0.862
scam_eq1 0.045 0.097 0.102 <0.001 NA NA
scam_eq1_or2 0.055 0.044 0.096 0.002 NA NA
health_is -0.037 0.050 -0.040 0.117 -0.044 0.066
economy_is 0.053 0.107 0.016 0.736 0.080 0.048
science_is 0.004 0.933 -0.020 0.586 0.042 0.445
politics_is 0.087 0.162 0.139 0.127 0.023 0.706

Topic prevalence: control vs treatment

Bar charts compare prevalence (mean share) of health_is, economy_is, science_is, politics_is between control and treatment for all tweets, reply tweets, and primary tweets.