Goal

The goal of this document is to execute the HTE tutorial for the inoculation against misinformation project.

Summary

In this script, I take as input the intermediate data set and output RATE, causal forests, and causal trees for all outcomes and covariate sets, as well as summary of the subgroup results, where the Romano-Wolf corrected subgroups are in separate scripts by outcome in this issue. Outcomes and covariates are listed in the Data section below. Covariate sets are:

covariates_1: pre sharing rates (misinfo and non-misinfo), pre sharing discernment, pre attn check passing
covariates_2: pre accuracy scores (misinfo and non-misinfo), pre accuracy discernment, pre attn heck passing
covariates_3: union of covariates_1 & covariates_2
covariates_4: demographics
covariates_5: union of covariates_3 & covariates_4 (i.e., all covariates)

The RATE results report:

TOC curve
Estimate & SE

The causal forest results report:

ATE by quintile, estimated using OLS and AIPW
Covariate heatmap by quintile
Best linear prediction
CATE estimation for users at each value of pre-misinfo sharing outcome (commented out for now)

The causal tree results report:

Causal tree degree diagram
For trees with more than 1 leaf, but fewer than 6 leaves (error with too many leaves):
- Tests for differences in treatment effects across leaves
- Heatmap of average covariate values within each leaf

Key Takeaways

With the notable exception of the RATE, I consistently find heterogeneity on post misinfo and non-misinfo sharing rates, as well as the post - pre difference in misinfo and non-misinfo sharing rates. This heterogeneity is consistently with respect to pre-misinfo sharing, often with respect to pre-non-misinfo sharing, and sometimes with respect to pre-misinfo accuracy scores.

RATE

I report a subset of the RATE results in the main script, showing only the 3 outcomes and 3 covariates sets on which I find results. All outcomes and covariates are in the appendix.

I find heterogeneity using RATE as follows:
- sharing discernment using pre accuracy ratings (covariates_2) (est. = 201, se = 83)
- post non-misinfo sharing rate using pre sharing rates and accuracy ratings (covariates_3) (est. = -171, se = 82)
- post misinfo accuracy score using demographics (covariates_4) (est. = 279, se = 112)
- Is this likely an error, given that this is the only method that does not find results consistent with the others?

Causal Forest

I report a subset of the causal forest results in the main script, showing all outcomes but focusing on only one covariate set: all covariates (covariates_5). All outcomes and covariates are in the appendix.

When we observe that CATE quintiles have differentiable treatment effects, the quintiles are consistently ordered by several pre-experiment outcome variables, where more sharing and higher accuracy ratings (on both misinfo and non-misinfo) yields higher treatment effects. These higher sharing rates and accuracy ratings are also among users who report spending more time and sharing a higher proportion of content on social media. Specifically, the variables on which we show a monotone relationship between quintile averages and ranking:
- pre misinfo sharing rate
- pre non-misinfo sharing rate
- pre misinfo accuracy score
- pre non-misinfo accuracy score
- hours spent on social media
- proportion of content shared on social media
We observe CATE quintiles with differentiable treatment effects on the post non-misinfo and misinfo sharing rate, and the post - pre differences in non-misinfo and misinfo sharing rates. There is also possible heterogeneity on accuracy discernment.
The highest CATE quintile consistently has (close to, fairly precise) 0 treatment effect, while the other quintiles have treatment effects in the expected direction.
Some of these outcomes only need sharing rates for a “clean” CATE quintile figure, but all outcomes have fairly clean CATE quintile figures when including ALL covariates.
The linear projections show heterogeneity with respect to pre-misinfo sharing for:
- post non-misinfo
- post misinfo sharing
- post - pre non-misinfo
- post - pre misinfo sharing

Subgroup Analysis

I find heterogeneity for post misinfo sharing and post - pre misinfo sharing with respect to:
- pre-misinfo sharing
- pre-non-misinfo sharing
- total pre-misinfo accuracy score.
Treatment effects on post non-misinfo sharing and post - pre non-misinfo sharing are only heterogeneous with respect to the pre sharing outcomes (and not misinfo accuracy score).

Causal Tree

I report a subset of the causal tree results in the main script, on which I base on my key takeaways below. All outcomes and covariates are in the appendix. The main outcomes of interest are post non-misinfo sharing, post misinfo sharing, and post - pre difference in misinfo sharing, and the main covariate sets of interest are pre-sharing only (covariates_1) and all covariates (covariates_5).

We have trees that split on pre-misinfo sharing for post non-misinfo sharing (pre-misinfo>=0.75, diff=0.10, se=0.044), post misinfo sharing (pre-misinfo>=0.58, diff=0.06, se=0.036), and post-pre misinfo sharing (pre-misinfo>=0.75, diff=0.10, se=0.039).
The post non-misinfo outcome tree splits with just the pre sharing outcomes as covariates, but the post misinfo outcome only splits on pre-misinfo when all covariates are included (the tree still has only 2 branches). The post - pre misinfo outcome tree splits on both covariate sets. Is this inconsistency indicative of an error? I’ll note that these outcomes also split under other covariate sets, but those trees seem to overly-complicate things.
The high pre-misinfo group is really a “high sharing” overall group, sharing an average of 90% of both misinfo and non-misinfo in the pre, with essentially zero discernment when split is at pre-misinfo>=0.75 (and similar for 0.58). The low pre-misinfo group shares an average of 59% of non-misinfo and 37% of misinfo, so they are more discerning. In addition, users in the high misinfo group report higher accuracy scores in the pre-survey on both misinfo and non-misinfo compared to the low misinfo group, and have lower accuracy discernment. There are only small differences on demographics, except with respect to how much content users report sharing on social media: the low misinfo group reports sharing much less. There is only a small difference in pre attention check passing.

Experimental Design

The goal of this experiment is to study the effect of different text message courses on misinformation sharing and discernment. The figure below diagrams the experimental design. We recruited participants through Facebook ads to complete a five-day text message course, plus a pre- and post-survey, for a mobile airtime payment. Participants who clicked an ad were directed to the pre-survey on Qualtrics, where they were first randomized on whether they would see an accuracy nudge in the pre- and post-survey. Conditional on completing the pre-survey, participants were randomized into one of the text message course interventions and enrolled in the course. Participants in all course interventions, except the No-course baseline, received one text message a day for five days starting on the day they completed the pre-survey. Participants in the No-course baseline received the Combo course after the post-survey to ensure we fulfilled our recruitment promise of a text message course. On the last day of the course, participants received a link to complete the post-survey on Qualtrics. Participants who completed the post-survey were paid KSH 500 (about $4 in U.S. dollars) in mobile airtime. Seven to eleven weeks later, participants who completed the post-survey were randomized on whether they would see a prime in the text message recruiting them to a follow-up survey. Participants who completed the follow-up survey were paid KSH 350 (about $3 in U.S. dollars).

knitr::include_graphics("./figures/Experimental Design.png")

Experimental Design

The pre- and post-surveys each contained 10 posts:

3 non-misinfo posts
6 misinfo posts (2 emotions, 2 reasoning, 2 combo)
1 attention check post

These posts were created by taking 15 facts/domains, then creating a non-misinfo version and 3 misinfo versions (emotions, reasoning, combo), for a total of 60 posts (plus 2 attention check posts). Participants saw each of the 15 facts at least once.

Whichever 3 facts/domains participants saw in the form of non-misinfo in the pre-survey, they saw in the form of misinfo in the post-survey, so these 3 facts/domains were repeated. Which facts/domains were repeated vary by participant.

For each posts, participants were asked 2 questions:

Would you share this post? (yes/no)
To the best of your knowledge, how accurate is the claim in the above post? (4-point Likert scale)

Data

In this script, we use two datasets.

intermediate_data_wide.csv: all data at the user level. Includes treatment conditions, covariates, and individual-aggregated outcomes. Sample of 8,684 participants who completed the pre- and post-survey.

intermediate_data_long_treatments.csv: all data at the user-post level. Includes treatment conditions, covariates, and individual-aggregated outcomes. Sample of 8,684 participants who completed the pre- and post-survey.

For more information about how these datasets are generated and a data dictionary, see memo.

Samples

This script primarily uses the sample of 3,626 participants who completed the post-survey and were assigned to either the Emotions course or Facts baseline.

Outcomes and Covariates

All Outcomes

Research Question: Do the treatment interventions…	Outcome Measures	Estimands	Notes
Sharing Outcomes
change misinfo sharing?	primary misinfo sharing rate	See equation below.	Defined only in post.
	count of misinfo posts shared $Y^M$	$\frac{E[Y^{M, post}(1)]}{6} - \frac{E[Y^{M, post}(0)]}{6}$, $(\frac{E[Y^{M, post}(1)]}{6} - \frac{E[Y^{M, pre}(1)]}{6}) - (\frac{E[Y^{M, post}(0)]}{6} - \frac{E[Y^{M, pre}(0)]}{6})$	post and post-pre misinfo sharing rate
change non-misinfo sharing?	count of non-misinfo posts shared $Y^N$	$\frac{E[Y^{N, post}(1)]}{3} - \frac{E[Y^{N, post}(0)]}{3}$, $(\frac{E[Y^{N, post}(1)]}{6} - \frac{E[Y^{N, pre}(1)]}{6}) - (\frac{E[Y^{N, post}(0)]}{6} - \frac{E[Y^{N, pre}(0)]}{6})$	post and post-pre non-misinfo sharing rate
change sharing discernment?	count of misinfo posts shared $Y^M$, count of non-misinfo posts shared $Y^N$	$(\frac{E[Y^{N, post}(1)]}{3} - \frac{E[Y^{M, post}(1)]}{6} ) - (\frac{E[Y^{N, post}(0)]}{3} - \frac{E[Y^{M, post}(0)]}{6})$	post sharing discernment
Accuracy Outcomes
change accuracy scoring of misinfo posts?	total score on misinfo posts $A^{M_1}$	$\frac{E[A^{M_1, post}(1)]}{6} - \frac{E[A^{M_1, post}(0)]}{6}$, $(\frac{E[A^{M_1, post}(1)]}{6} - \frac{E[A^{M_1, pre}(1)]}{6}) - (\frac{E[A^{M_1, post}(0)]}{6} - \frac{E[A^{M_1, pre}(0)]}{6})$	post and post-pre misinfo accuracy score
	count of misinfo posts with accuracy score > 0 $A^{M_2}$	$\frac{E[A^{M_2, post}(1)]}{6} - \frac{E[A^{M_2, post}(0)]}{6}$, $(\frac{E[A^{M_2, post}(1)]}{6} - \frac{E[A^{M_2, pre}(1)]}{6}) - (\frac{E[A^{M_2, post}(0)]}{6} - \frac{E[A^{M_2, pre}(0)]}{6})$	post and post-pre misinfo binarized accuracy score
change accuracy scoring of non-misinfo posts?	total score on non-misinfo posts $A^{N_1}$	$\frac{E[A^{N_1, post}(1)]}{3} - \frac{E[A^{N_1, post}(0)]}{3}$, $(\frac{E[A^{N_1, post}(1)]}{6} - \frac{E[A^{N_1, pre}(1)]}{6}) - (\frac{E[A^{N_1, post}(0)]}{6} - \frac{E[A^{N_1, pre}(0)]}{6})$	post and post-pre non-misinfo accuracy score
	count of non-misinfo posts with accuracy score > 0 $A^{N_2}$	$\frac{E[A^{N_2, post}(1)]}{3} - \frac{E[A^{N_2, post}(0)]}{3}$, $(\frac{E[A^{N_2, post}(1)]}{6} - \frac{E[A^{N_2, pre}(1)]}{6}) - (\frac{E[A^{N_2, post}(0)]}{6} - \frac{E[A^{N_2, pre}(0)]}{6})$	post and post-pre non-misinfo accuracy score
change accuracy discernment?	total score on misinfo posts $A^M$, total score on non-misinfo posts $A^N$	$(\frac{E[A^{N, post}(1)]}{3} - \frac{E[A^{M, post}(1)]}{6} ) - (\frac{E[A^{N, post}(0)]}{3} - \frac{E[A^{M, post}(0)]}{6})$	post accuracy discernment

knitr::include_graphics("./figures/primary_sharing_equation.png")

All Covariates

Demographics

knitr::include_graphics("./figures/demographics_table.png")

Demographics Definitions

Pre-treatment variables

Description	Variable Construction
Pre misinfo sharing rate	$\frac{E[Y^{M, pre}]}{6}$
Pre non-misinfo sharing rate	$\frac{E[Y^{N, pre}]}{3}$
Pre sharing discernment	$\frac{E[Y^{N, pre}(1)]}{3} - \frac{E[Y^{M, pre}(1)]}{6}$
Pre misinfo accuracy score	$\frac{E[A^{M, pre}]}{6}$
Pre non-misinfo accuracy score	$\frac{E[A^{N, pre}]}{3}$
Pre accuracy discernment	$\frac{E[A^{N, pre}(1)]}{3} - \frac{E[A^{M, pre}(1)]}{6}$
Pre attention check passing	$1[\text{passed attention check}]$

Setup

Load packages

# Set CRAN mirror 
options(repos = c(CRAN = "https://cran.rstudio.com"))

packages = c(
  "tidyverse", "data.table", "dtplyr", "rlang", "kableExtra", "haven", "ggcorrplot", "visdat", "VIM", "corrplot", "kableExtra", "fastDummies", "causalTree", "grf", "rpart", "glmnet", "splines", "MASS", "lmtest", "sandwich", "ggplot2", "stingr", "estimatr", "gridExtra", "repr"
)

# Load the packages, install if necessary
new_packages = packages[!(packages %in% installed.packages()[,"Package"])]
if(length(new_packages)) install.packages(new_packages, dependencies = TRUE)
lapply(packages, require, character.only = TRUE) |> invisible()

Load data

df_wide <- read.csv("../intermediate_data_wide.csv")
df_long <- read.csv("../intermediate_data_long.csv") %>% filter(pre_post != "followup")

# compute the number of rows in each dataset
nrow_wide <- nrow(df_wide)
nrow_long <- nrow(df_long)

# print the number of rows in each dataset in a statement that says "There are x rows in the wide dataset and y rows in the long dataset."
paste("There are", nrow_wide, "rows in the wide dataset. There are", nrow_long, "rows in the long dataset, which is equal to the", nrow_wide*18, "expected for 18 user-post observations for each", nrow_wide, "user in the wide dataset.")

## [1] "There are 8684 rows in the wide dataset. There are 156312 rows in the long dataset, which is equal to the 156312 expected for 18 user-post observations for each 8684 user in the wide dataset."

Helper Functions

romano_wolf_correction <- function(t.orig, t.boot) {
  abs.t.orig <- abs(t.orig)
  abs.t.boot <- abs(t.boot)
  abs.t.sorted <- sort(abs.t.orig, decreasing = TRUE)

  max.order <- order(abs.t.orig, decreasing = TRUE)
  rev.order <- order(max.order)

  M <- nrow(t.boot)
  S <- ncol(t.boot)

  p.adj <- rep(0, S)
  p.adj[1] <- mean(apply(abs.t.boot, 1, max) > abs.t.sorted[1])
  for (s in seq(2, S)) {
    cur.index <- max.order[s:S]
    p.init <- mean(apply(abs.t.boot[, cur.index, drop=FALSE], 1, max) > abs.t.sorted[s])
    p.adj[s] <- max(p.init, p.adj[s-1])
  }
  p.adj[rev.order]
}

summary_rw_lm <- function(model, indices=NULL, cov.type="HC2", num.boot=10000) {

  if (is.null(indices)) {
    indices <- 1:nrow(coef(summary(model)))
  }
  # Grab the original t values.
  summary <- coef(summary(model))[indices,,drop=FALSE]
  t.orig <- summary[, "t value"]

  # Null resampling.
  # This is a trick to speed up bootstrapping linear models.
  # Here, we don't really need to re-fit linear regressions, which would be a bit slow.
  # We know that betahat ~ N(beta, Sigma), and we have an estimate Sigmahat.
  # So we can approximate "null t-values" by
  #  - Draw beta.boot ~ N(0, Sigma-hat) --- note the 0 here, this is what makes it a *null* t-value.
  #  - Compute t.boot = beta.boot / sqrt(diag(Sigma.hat))
  Sigma.hat <- vcovHC(model, type=cov.type)[indices, indices]
  se.orig <- sqrt(diag(Sigma.hat))
  num.coef <- length(se.orig)
  beta.boot <- mvrnorm(n=num.boot, mu=rep(0, num.coef), Sigma=Sigma.hat)
  t.boot <- sweep(beta.boot, 2, se.orig, "/")
  p.adj <- romano_wolf_correction(t.orig, t.boot)

  result <- cbind(summary[,c(1,2,4),drop=F], p.adj)
  colnames(result) <- c('Estimate', 'Std. Error', 'Orig. p-value', 'Adj. p-value')
  result
}

fig <- function(width, heigth){
 options(repr.plot.width = width, repr.plot.height = heigth)
}

se_cont = function(x, na.rm=FALSE) {
  if (na.rm) x = na.omit(x)
  sqrt(var(x)/length(x))}

Define input for script

This is the main section on which any changes would be made, unless there are bugs or additional analyses are required. It:

restricts the data to emotions or facts baseline
specifies variables that are used consistently through the script (e.g., n, treatment)
defines different sets of outcomes and covariates, including main outcomes/covariate sets of interest for specific analyses

# Define data, filtering to emotions and baseline only, then redefining the treatment variable equal to 1 if treatment is Emotions, 0 if treatment is Facts Baseline
data <- df_wide %>% 
  filter(treatment == "Emotions" | treatment == "Facts Baseline") %>% 
  mutate(treatment = ifelse(treatment == "Emotions", 1, 0))

# create data that filters missing value in main outcome
data_no_na <- data %>% 
    filter(!is.na(pre_spec_outcome_rates)) 

# compute the number of rows
n <- nrow(data)

# print the number of rows
paste("There are", n, "rows in the dataset restricted to emotions and facts baseline.")

## [1] "There are 3626 rows in the dataset restricted to emotions and facts baseline."

# compute the number of rows
n_no_na <- nrow(data_no_na)

# print the number of rows
paste("There are", n_no_na, "rows in the dataset further restricted to have no missing values for the pre-specified misinformation sharing rate outcome (i.e., shared at least one non-misinfo post in the pre-survey).")

## [1] "There are 3201 rows in the dataset further restricted to have no missing values for the pre-specified misinformation sharing rate outcome (i.e., shared at least one non-misinfo post in the pre-survey)."

# Define treatment as a binary variable = 1 if treatment is Emotions, 0 if treatment is Facts Baseline
treatment <- "treatment"

# Define all potential outcomes
outcomes <- c("pre_spec_outcome_rates", "base_rate_post", "misinfo_post", "base_rate_diff", "misinfo_diff", "new_disc_post", "new_disc_diff", "misinfo_avg_acc_score_post", "misinfo_avg_acc_score_diff", "base_avg_acc_score_post", "base_avg_acc_score_diff", "new_acc_post", "new_acc_diff")

# Define interesting outcomes for RATE
outcomes_rate <- c("base_rate_post", "misinfo_post", "base_rate_diff", "misinfo_diff", "new_disc_post", "new_disc_diff", "misinfo_avg_acc_score_post", "misinfo_avg_acc_score_diff", "base_avg_acc_score_post", "base_avg_acc_score_diff", "new_acc_post", "new_acc_diff")

# Define interesting outcomes for causal tree
outcomes_int <- c("base_rate_post", "misinfo_post", "misinfo_diff")

# Define relevant outcomes for subgroup analysis
outcomes_subgroup <- c("pre_spec_outcome_rates", "base_rate_post", "misinfo_post", "base_rate_diff", "misinfo_diff", "new_disc_post")

# Define covariate sets
covariates_5 <- c("att_check_pre", "base_rate_pre", "misinfo_pre", "new_disc_pre", "misinfo_avg_acc_score_pre", "base_avg_acc_score_pre", "new_acc_pre", "age", "gender_Man", "education_High_school_or_less", "education_Some_college", "education_Bachelor_degree", "education_Graduate_degree", "marital_Married_or_in_a_domestic_partnership", "employment_Employed", "employment_Unemployed", "employment_Student", "location_Mostly_urban", "location_Suburban", "location_Mostly_rural", "religion_Christian", "religiosity_Attends", "social_media_bin_Yes", "social_media_hours", "social_media_share_80_100", "social_media_share_60_80", "social_media_share_40_60", "social_media_share_20_40", "social_media_share_0_20")

covariates_3 <- c("att_check_pre", "base_rate_pre", "misinfo_pre", "new_disc_pre", "misinfo_avg_acc_score_pre", "base_avg_acc_score_pre", "new_acc_pre")

covariates_1 <- c("att_check_pre", "base_rate_pre", "misinfo_pre", "new_disc_pre")

covariates_2 <- c("att_check_pre", "misinfo_avg_acc_score_pre", "base_avg_acc_score_pre", "new_acc_pre")

covariates_4 <- c("age", "gender_Man", "education_High_school_or_less", "education_Some_college", "education_Bachelor_degree", "education_Graduate_degree", "marital_Married_or_in_a_domestic_partnership", "employment_Employed", "employment_Unemployed", "employment_Student", "location_Mostly_urban", "location_Suburban", "location_Mostly_rural", "religion_Christian", "religiosity_Attends", "social_media_bin_Yes", "social_media_hours", "social_media_share_80_100", "social_media_share_60_80", "social_media_share_40_60", "social_media_share_20_40", "social_media_share_0_20")

# Define main covariate set for causal forest & RATE

covariates_main <- covariates_5

covariate_sets <- list(
  covariates_1 = covariates_1,
  covariates_2 = covariates_2,
  covariates_3 = covariates_3,
  covariates_4 = covariates_4,
  covariates_5 = covariates_5
)

covariate_sets_234 <- list(
  covariates_2 = covariates_2,
  covariates_3 = covariates_3,
  covariates_4 = covariates_4
)

RATE

# Loop over all covariate sets
for (covariate_set_name in names(covariate_sets)) {
  covariates <- covariate_sets[[covariate_set_name]]

    # Loop over all outcomes Y
  for (outcome in outcomes) {
    # Print Outcome and Covariates at the top of each loop
      print(paste("Outcome:", outcome))
      print(paste("Covariates:", covariate_set_name))
      
    # Define parameters - change sample for pre_spec_outcome_rates
    if (outcome == "pre_spec_outcome_rates") {
      
      # Define Y to be the outcome
      Y <- data_no_na[,outcome]
      
      # Define X to be the matrix 
      fmla <- formula(paste0("~ 0 + ", paste0(covariates, collapse="+")))
      X <- model.matrix(fmla, data_no_na)
      
      # Define W to be the treatment
      W <- data_no_na[,treatment]
      
    } else {
      
      # Define Y to be the outcome
      Y <- data[,outcome]
      
      # Define X to be the matrix 
      fmla <- formula(paste0("~ 0 + ", paste0(covariates, collapse="+")))
      X <- model.matrix(fmla, data)
      
      # Define W to be the treatment
      W <- data[,treatment]
    }

      # Train a causal forest to estimate a CATE based priority ranking
    train <- sample(1:nrow(X), nrow(X) / 2)
    cf.priority <- causal_forest(X[train, ], Y[train], W[train])

    # Compute a prioritization based on estimated treatment effects.
    priority.cate <- predict(cf.priority, X[-train, ])$predictions

    # Estimate AUTOC on held out data.
    cf.eval <- causal_forest(X[-train, ], Y[-train], W[-train])
    rate <- rank_average_treatment_effect(cf.eval, priority.cate)
    print(rate)

    # Plot the Targeting Operator Characteristic curve
    rate_plot <- plot(rate, las=1)
    print(rate_plot)

    # Add title to plot "TOC Curve for outcome"
    title(paste("TOC Curve for", outcome, covariate_set_name), line = -2, cex.main = 0.8)
  }
}

## [1] "Outcome: pre_spec_outcome_rates"
## [1] "Covariates: covariates_1"
##    estimate    std.err             target
##  0.01249719 0.01796171 priorities | AUTOC

## NULL
## [1] "Outcome: base_rate_post"
## [1] "Covariates: covariates_1"
##    estimate   std.err             target
##  0.01988524 0.0177453 priorities | AUTOC

## NULL
## [1] "Outcome: misinfo_post"
## [1] "Covariates: covariates_1"
##    estimate    std.err             target
##  0.04398594 0.01454008 priorities | AUTOC

## NULL
## [1] "Outcome: base_rate_diff"
## [1] "Covariates: covariates_1"
##    estimate    std.err             target
##  0.03593646 0.01809113 priorities | AUTOC

## NULL
## [1] "Outcome: misinfo_diff"
## [1] "Covariates: covariates_1"
##    estimate    std.err             target
##  0.04784621 0.01389008 priorities | AUTOC

## NULL
## [1] "Outcome: new_disc_post"
## [1] "Covariates: covariates_1"
##     estimate    std.err             target
##  0.004816168 0.01600716 priorities | AUTOC

## NULL
## [1] "Outcome: new_disc_diff"
## [1] "Covariates: covariates_1"
##     estimate    std.err             target
##  0.009428112 0.01487488 priorities | AUTOC

## NULL
## [1] "Outcome: misinfo_avg_acc_score_post"
## [1] "Covariates: covariates_1"
##   estimate    std.err             target
##  0.0812849 0.05939596 priorities | AUTOC

## NULL
## [1] "Outcome: misinfo_avg_acc_score_diff"
## [1] "Covariates: covariates_1"
##    estimate    std.err             target
##  0.05630572 0.06255142 priorities | AUTOC

## NULL
## [1] "Outcome: base_avg_acc_score_post"
## [1] "Covariates: covariates_1"
##    estimate   std.err             target
##  0.08658879 0.0779894 priorities | AUTOC

## NULL
## [1] "Outcome: base_avg_acc_score_diff"
## [1] "Covariates: covariates_1"
##    estimate    std.err             target
##  0.04035429 0.07074557 priorities | AUTOC

## NULL
## [1] "Outcome: new_acc_post"
## [1] "Covariates: covariates_1"
##   estimate    std.err             target
##  0.1120059 0.05490989 priorities | AUTOC

## NULL
## [1] "Outcome: new_acc_diff"
## [1] "Covariates: covariates_1"
##    estimate    std.err             target
##  0.08043332 0.08455198 priorities | AUTOC

## NULL
## [1] "Outcome: pre_spec_outcome_rates"
## [1] "Covariates: covariates_2"
##     estimate    std.err             target
##  0.003849854 0.02407433 priorities | AUTOC

## NULL
## [1] "Outcome: base_rate_post"
## [1] "Covariates: covariates_2"
##   estimate    std.err             target
##  0.0144863 0.01828793 priorities | AUTOC

## NULL
## [1] "Outcome: misinfo_post"
## [1] "Covariates: covariates_2"
##    estimate    std.err             target
##  0.02737177 0.01498756 priorities | AUTOC

## NULL
## [1] "Outcome: base_rate_diff"
## [1] "Covariates: covariates_2"
##    estimate    std.err             target
##  0.03945616 0.01921253 priorities | AUTOC

## NULL
## [1] "Outcome: misinfo_diff"
## [1] "Covariates: covariates_2"
##    estimate    std.err             target
##  0.01434535 0.01513298 priorities | AUTOC

## NULL
## [1] "Outcome: new_disc_post"
## [1] "Covariates: covariates_2"
##    estimate    std.err             target
##  0.01381564 0.01644151 priorities | AUTOC

## NULL
## [1] "Outcome: new_disc_diff"
## [1] "Covariates: covariates_2"
##    estimate    std.err             target
##  0.03662679 0.01738672 priorities | AUTOC

## NULL
## [1] "Outcome: misinfo_avg_acc_score_post"
## [1] "Covariates: covariates_2"
##    estimate    std.err             target
##  0.09283515 0.06393727 priorities | AUTOC

## NULL
## [1] "Outcome: misinfo_avg_acc_score_diff"
## [1] "Covariates: covariates_2"
##    estimate    std.err             target
##  0.02124833 0.06549956 priorities | AUTOC

## NULL
## [1] "Outcome: base_avg_acc_score_post"
## [1] "Covariates: covariates_2"
##    estimate    std.err             target
##  0.08108108 0.06500458 priorities | AUTOC

## NULL
## [1] "Outcome: base_avg_acc_score_diff"
## [1] "Covariates: covariates_2"
##    estimate    std.err             target
##  0.07281863 0.07611701 priorities | AUTOC

## NULL
## [1] "Outcome: new_acc_post"
## [1] "Covariates: covariates_2"
##  estimate   std.err             target
##  0.150773 0.0714688 priorities | AUTOC

## NULL
## [1] "Outcome: new_acc_diff"
## [1] "Covariates: covariates_2"
##   estimate    std.err             target
##  0.1880323 0.06075642 priorities | AUTOC

## NULL
## [1] "Outcome: pre_spec_outcome_rates"
## [1] "Covariates: covariates_3"
##    estimate    std.err             target
##  0.02138681 0.02068647 priorities | AUTOC

## NULL
## [1] "Outcome: base_rate_post"
## [1] "Covariates: covariates_3"
##   estimate    std.err             target
##  0.0435729 0.01518902 priorities | AUTOC

## NULL
## [1] "Outcome: misinfo_post"
## [1] "Covariates: covariates_3"
##    estimate    std.err             target
##  0.04445331 0.01285786 priorities | AUTOC

## NULL
## [1] "Outcome: base_rate_diff"
## [1] "Covariates: covariates_3"
##    estimate    std.err             target
##  0.03594299 0.01505721 priorities | AUTOC

## NULL
## [1] "Outcome: misinfo_diff"
## [1] "Covariates: covariates_3"
##    estimate    std.err             target
##  0.05914946 0.01483643 priorities | AUTOC

## NULL
## [1] "Outcome: new_disc_post"
## [1] "Covariates: covariates_3"
##    estimate   std.err             target
##  0.01013311 0.0148035 priorities | AUTOC

## NULL
## [1] "Outcome: new_disc_diff"
## [1] "Covariates: covariates_3"
##    estimate    std.err             target
##  0.01796233 0.01430654 priorities | AUTOC

## NULL
## [1] "Outcome: misinfo_avg_acc_score_post"
## [1] "Covariates: covariates_3"
##     estimate  std.err             target
##  -0.07616341 0.055424 priorities | AUTOC

## NULL
## [1] "Outcome: misinfo_avg_acc_score_diff"
## [1] "Covariates: covariates_3"
##    estimate    std.err             target
##  0.05259185 0.06011151 priorities | AUTOC

## NULL
## [1] "Outcome: base_avg_acc_score_post"
## [1] "Covariates: covariates_3"
##    estimate    std.err             target
##  0.00127287 0.07213498 priorities | AUTOC

## NULL
## [1] "Outcome: base_avg_acc_score_diff"
## [1] "Covariates: covariates_3"
##    estimate   std.err             target
##  0.07474461 0.0692469 priorities | AUTOC

## NULL
## [1] "Outcome: new_acc_post"
## [1] "Covariates: covariates_3"
##    estimate    std.err             target
##  0.02993595 0.06313902 priorities | AUTOC

## NULL
## [1] "Outcome: new_acc_diff"
## [1] "Covariates: covariates_3"
##   estimate    std.err             target
##  0.1248075 0.06112532 priorities | AUTOC

## NULL
## [1] "Outcome: pre_spec_outcome_rates"
## [1] "Covariates: covariates_4"
##    estimate    std.err             target
##  -0.0229175 0.02017134 priorities | AUTOC

## NULL
## [1] "Outcome: base_rate_post"
## [1] "Covariates: covariates_4"
##      estimate    std.err             target
##  -0.005361383 0.01873925 priorities | AUTOC

## NULL
## [1] "Outcome: misinfo_post"
## [1] "Covariates: covariates_4"
##     estimate    std.err             target
##  -0.01220184 0.01599703 priorities | AUTOC

## NULL
## [1] "Outcome: base_rate_diff"
## [1] "Covariates: covariates_4"
##      estimate    std.err             target
##  0.0007083126 0.01808525 priorities | AUTOC

## NULL
## [1] "Outcome: misinfo_diff"
## [1] "Covariates: covariates_4"
##      estimate    std.err             target
##  -0.006619351 0.01513627 priorities | AUTOC

## NULL
## [1] "Outcome: new_disc_post"
## [1] "Covariates: covariates_4"
##     estimate    std.err             target
##  0.005732953 0.01369167 priorities | AUTOC

## NULL
## [1] "Outcome: new_disc_diff"
## [1] "Covariates: covariates_4"
##     estimate    std.err             target
##  0.003185753 0.01954647 priorities | AUTOC

## NULL
## [1] "Outcome: misinfo_avg_acc_score_post"
## [1] "Covariates: covariates_4"
##    estimate   std.err             target
##  0.03478347 0.0540063 priorities | AUTOC

## NULL
## [1] "Outcome: misinfo_avg_acc_score_diff"
## [1] "Covariates: covariates_4"
##     estimate    std.err             target
##  -0.04713114 0.06677248 priorities | AUTOC

## NULL
## [1] "Outcome: base_avg_acc_score_post"
## [1] "Covariates: covariates_4"
##      estimate    std.err             target
##  -0.003726039 0.06907064 priorities | AUTOC

## NULL
## [1] "Outcome: base_avg_acc_score_diff"
## [1] "Covariates: covariates_4"
##    estimate    std.err             target
##  0.04941542 0.07791993 priorities | AUTOC

## NULL
## [1] "Outcome: new_acc_post"
## [1] "Covariates: covariates_4"
##     estimate    std.err             target
##  -0.04090337 0.06506082 priorities | AUTOC

## NULL
## [1] "Outcome: new_acc_diff"
## [1] "Covariates: covariates_4"
##   estimate    std.err             target
##  0.1174783 0.09492583 priorities | AUTOC

## NULL
## [1] "Outcome: pre_spec_outcome_rates"
## [1] "Covariates: covariates_5"
##   estimate    std.err             target
##  0.0359481 0.01813229 priorities | AUTOC

## NULL
## [1] "Outcome: base_rate_post"
## [1] "Covariates: covariates_5"
##    estimate    std.err             target
##  0.01235161 0.01433374 priorities | AUTOC

## NULL
## [1] "Outcome: misinfo_post"
## [1] "Covariates: covariates_5"
##    estimate    std.err             target
##  0.05637324 0.01173318 priorities | AUTOC

## NULL
## [1] "Outcome: base_rate_diff"
## [1] "Covariates: covariates_5"
##   estimate   std.err             target
##  0.0214914 0.0179313 priorities | AUTOC

## NULL
## [1] "Outcome: misinfo_diff"
## [1] "Covariates: covariates_5"
##    estimate   std.err             target
##  0.04717753 0.0119635 priorities | AUTOC

## NULL
## [1] "Outcome: new_disc_post"
## [1] "Covariates: covariates_5"
##    estimate    std.err             target
##  0.02687851 0.01462512 priorities | AUTOC

## NULL
## [1] "Outcome: new_disc_diff"
## [1] "Covariates: covariates_5"
##    estimate    std.err             target
##  0.01779371 0.01280788 priorities | AUTOC

## NULL
## [1] "Outcome: misinfo_avg_acc_score_post"
## [1] "Covariates: covariates_5"
##    estimate    std.err             target
##  0.07476877 0.05249391 priorities | AUTOC

## NULL
## [1] "Outcome: misinfo_avg_acc_score_diff"
## [1] "Covariates: covariates_5"
##   estimate    std.err             target
##  0.1130892 0.04707927 priorities | AUTOC

## NULL
## [1] "Outcome: base_avg_acc_score_post"
## [1] "Covariates: covariates_5"
##    estimate    std.err             target
##  0.07392699 0.07228186 priorities | AUTOC

## NULL
## [1] "Outcome: base_avg_acc_score_diff"
## [1] "Covariates: covariates_5"
##   estimate    std.err             target
##  0.1194722 0.05867015 priorities | AUTOC

## NULL
## [1] "Outcome: new_acc_post"
## [1] "Covariates: covariates_5"
##    estimate    std.err             target
##  0.09675413 0.05181803 priorities | AUTOC

## NULL
## [1] "Outcome: new_acc_diff"
## [1] "Covariates: covariates_5"
##    estimate    std.err             target
##  0.02960189 0.05801295 priorities | AUTOC

## NULL

Causal Forest

Define function

causal_forest_analysis <- function(XX, Y, W, covariates, file_name, W.hat=0.5) {
  
fmla <- formula(paste0("~ 0 +", paste0(covariates, collapse="+")))
XX <- model.matrix(fmla, data)

  # Define the outcome
  Y <- data[,outcome]
  
  # Number of rankings that the predictions will be ranking on
# (e.g., 2 for above/below median estimated CATE, 5 for estimated CATE quintiles, etc.)
num.rankings <- 5  

# Prepare for data.splitting
# Assign a fold number to each observation.
# The argument 'clusters' in the next step will mimick K-fold cross-fitting.
num.folds <- 10
folds <- sort(seq(n) %% num.folds) + 1

# Comment or uncomment depending on your setting.
# Observational setting with unconfoundedness+overlap (unknown assignment probs):
# forest <- causal_forest(X, Y, W, clusters = folds)
# Randomized settings with fixed and known probabilities (here: 0.5).
cf <- causal_forest(XX, Y, W, W.hat = 0.5, clusters = folds)

# Retrieve out-of-bag predictions.
# Predictions for observation in fold k will be computed using
# trees that were not trained using observations for that fold.
tau.hat <- predict(cf)$predictions

# Rank observations *within each fold* into quintiles according to their CATE predictions.
ranking <- rep(NA, n)
for (fold in seq(num.folds)) {
  tau.hat.quantiles <- quantile(tau.hat[folds == fold], probs = seq(0, 1, by=1/num.rankings))
  ranking[folds == fold] <- cut(tau.hat[folds == fold], tau.hat.quantiles, include.lowest=TRUE,labels=seq(num.rankings))
}

# Formula y ~ 0 + ranking + ranking:w
fmla <- paste0(outcome, " ~ 0 + ranking + ranking:", treatment)
ols.ate <- lm(fmla, data=transform(data, ranking=factor(ranking)))
ols.ate <- coeftest(ols.ate, vcov=vcovHC(ols.ate, type='HC2'))
interact <- which(grepl(":", rownames(ols.ate)))
ols.ate <- data.frame("ols", paste0("Q", seq(num.rankings)), ols.ate[interact, 1:2])
rownames(ols.ate) <- NULL # just for display
colnames(ols.ate) <- c("method", "ranking", "estimate", "std.err")
print(ols.ate)
  
  # Define the causal forest
 # cf <- causal_forest(XX, Y, W, W.hat=0.5)
  
  # Get forest predictions
  # tau.hat <- predict(cf)$predictions
  m.hat <- cf$Y.hat  # E[Y|X] estimates
  e.hat <- cf$W.hat  # e(X) := E[W|X] estimates (or known quantity)
  tau.hat <- cf$predictions  # tau(X) estimates
  # data_with_ntile <- data.frame(XX, tau.hat = tau.hat)
  
  # Estimating mu.hat(X, 1) and mu.hat(X, 0) for obs in held-out sample
# Note: to understand this, read equations 6-8 in this vignette:
# https://grf-labs.github.io/grf/articles/muhats.html
mu.hat.0 <- m.hat - e.hat * tau.hat        # E[Y|X,W=0] = E[Y|X] - e(X)*tau(X)
mu.hat.1 <- m.hat + (1 - e.hat) * tau.hat  # E[Y|X,W=1] = E[Y|X] + (1 - e(X))*tau(X)

# AIPW scores
aipw.scores <- tau.hat + W / e.hat * (Y -  mu.hat.1) - (1 - W) / (1 - e.hat) * (Y -  mu.hat.0)
ols <- lm(aipw.scores ~ 0 + factor(ranking))
forest.ate <- data.frame("aipw", paste0("Q", seq(num.rankings)), coeftest(ols, vcov=vcovHC(ols, "HC2"))[,1:2])
colnames(forest.ate) <- c("method", "ranking", "estimate", "std.err")
rownames(forest.ate) <- NULL # just for display
print(forest.ate)

# Concatenate the two results.
res <- rbind(forest.ate, ols.ate)

# Plotting the point estimate of average treatment effect 
# and 95% confidence intervals around it.
cate_plot <- ggplot(res) +
  aes(x = ranking, y = estimate, group=method, color=method) + 
  geom_point(position=position_dodge(0.2)) +
  geom_errorbar(aes(ymin=estimate-2*std.err, ymax=estimate+2*std.err), width=.2, position=position_dodge(0.2)) +
  ylab("") + xlab("") +
  theme_minimal() +
  theme(legend.position="bottom", legend.title = element_blank())
 
print(cate_plot)

# y ~ ranking + w + ranking:w
fmla <- paste0(outcome, "~ ranking + ", treatment, " + ranking:", treatment)
ols <- lm(fmla, data=transform(data, ranking=factor(ranking)))
interact <- which(sapply(names(coef(ols)), function(x) grepl(":", x)))
res <- summary_rw_lm(ols, indices=interact)
rownames(res) <- paste("Rank", 2:num.rankings, "- Rank 1") # just for display
print(res)

# Using AIPW scores computed above
ols <- lm(aipw.scores ~ 1 + factor(ranking))
res <- summary_rw_lm(ols, indices=2:num.rankings)
rownames(res) <- paste("Rank", 2:num.rankings, "- Rank 1") # just for display
print(res)
  
  # Get the best linear projection of tau(X) onto X
  print(best_linear_projection(cf, XX))
  
  # Test calibration
  print(test_calibration(cf))

# MAKE COVARIATE BY QUINTILE PLOTS
  
  df <- mapply(function(covariate) {
      # Looping over covariate names
      # Compute average covariate value per ranking (with correct standard errors)
      fmla <- formula(paste0(covariate, "~ 0 + ranking"))
      ols <- lm(fmla, data=transform(data, ranking=factor(ranking)))
      ols.res <- coeftest(ols, vcov=vcovHC(ols, "HC2"))

      # Retrieve results
      avg <- ols.res[,1]
      stderr <- ols.res[,2]

      # Tally up results
      data.frame(covariate, avg, stderr, ranking=paste0("Q", seq(num.rankings)),
                 # Used for coloring
                 scaling=pnorm((avg - mean(avg))/sd(avg)),
                 # We will order based on how much variation is 'explain' by the averages
                 # relative to the total variation of the covariate in the data
                 variation=sd(avg) / sd(data[,covariate]),
                 # String to print in each cell in heatmap below
                 labels=paste0(signif(avg, 3), "\n", "(", signif(stderr, 3), ")"))
}, covariates, SIMPLIFY = FALSE)
df <- do.call(rbind, df)

# a small optional trick to ensure heatmap will be in decreasing order of 'variation'
df$covariate <- reorder(df$covariate, order(df$variation))

# Calculate the height based on the number of covariates
heatmap_height <- max(8, length(covariates) * 2)  # Adjust the multiplier as needed

# Plot heatmap
heatmap <- ggplot(df) +
  aes(ranking, covariate) +
  geom_tile(aes(fill = scaling)) + 
  geom_text(aes(label = labels)) +
  scale_fill_gradient(low = "#E1BE6A", high = "#40B0A6") +
  theme_minimal() + 
  ylab("") + xlab("CATE estimate ranking") +
  theme(plot.title = element_text(size = 11, face = "bold"),
        axis.text = element_text(size = 11))

# Print the heatmap with the specified size
#print(heatmap, vp = grid::viewport(height = grid::unit(heatmap_height, "in")))

print(heatmap)

ggsave(file_name, plot = heatmap, width = 8, height = 12)

# PARTIAL DEPENDENCE on misinfo_pre
#selected.covariate <- "misinfo_pre"
#other.covariates <- covariates[which(covariates != selected.covariate)]

# Compute a grid of values appropriate for the selected covariate
#grid.size <- 6 # there are 6 misinfo posts
#covariate.grid <- seq(min(data[,selected.covariate]), max(data[,selected.covariate]), length.out=grid.size)

# Take median of other covariates
#medians <- apply(data[, other.covariates, F], 2, median)

# Construct a dataset
#data.grid <- data.frame(sapply(medians, function(x) rep(x, grid.size)), covariate.grid)
#colnames(data.grid) <- c(other.covariates, selected.covariate)

# Redefining fmla
#fmla <- formula(paste0("~ 0 + ", paste0(covariates, collapse="+")))

# Expand the data
#X.grid <- model.matrix(fmla, data.grid)

# Point predictions of the CATE and standard errors
#forest.pred <- predict(cf, newdata = X.grid, estimate.variance=TRUE)
#tau.hat <- forest.pred$predictions
#tau.hat.se <- sqrt(forest.pred$variance.estimates)

# Plot predictions for each group and 95% confidence intervals around them.
#data.pred <- transform(data.grid, tau.hat=tau.hat, ci.low = tau.hat - 2*tau.hat.se, ci.high = tau.hat + 2*tau.hat.se)

#te_misinfo_plot <- ggplot(data.pred) +
  #geom_line(aes_string(x=selected.covariate, y="tau.hat", group = 1), color="black") +
  #geom_errorbar(aes_string(x=selected.covariate, ymin="ci.low", ymax="ci.high", width=.2), color="blue") +
  # ylab("") +
  #ggtitle(paste0("Predicted treatment effect on '", outcome, "' varying '", selected.covariate, "' (other variables fixed at median)")) +
  #scale_x_discrete("misinfo_pre") +
  #theme_minimal() +
  #theme(plot.title = element_text(size = 11, face = "bold")) 

# Print the plot
#print(te_misinfo_plot)

}

All outcomes with all covariates (covariates_5)

for (outcome in outcomes) {
  if (outcome == "pre_spec_outcome_rates") {
    print(paste("Outcome:", outcome))
    print("Outcome has missing values.")
  } else {
      # Print Outcome at the top of each loop
      print(paste("Outcome:", outcome))
    
    # Create file name
    file_name <- sprintf("~/GitHub/First_Draft/main_analysis/new_analyses_for_submission/generated_figures/%s_%s_heatmap_plot.png", outcome, "covariates_main")
    
    # Execute analysis
      causal_forest_analysis(XX, outcome, W, covariates_main, file_name, W.hat = 0.5)
  }
}

## [1] "Outcome: pre_spec_outcome_rates"
## [1] "Outcome has missing values."
## [1] "Outcome: base_rate_post"
##   method ranking    estimate    std.err
## 1    ols      Q1 -0.16762441 0.02435951
## 2    ols      Q2 -0.11367754 0.02583901
## 3    ols      Q3 -0.07209943 0.02646444
## 4    ols      Q4 -0.06202208 0.02759869
## 5    ols      Q5 -0.03818131 0.02657526
##   method ranking    estimate    std.err
## 1   aipw      Q1 -0.16194564 0.02369534
## 2   aipw      Q2 -0.12659378 0.02506220
## 3   aipw      Q3 -0.05872461 0.02532440
## 4   aipw      Q4 -0.07119024 0.02593263
## 5   aipw      Q5 -0.04025377 0.02471885

##                   Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 0.05394688 0.03709882  0.1459943229       0.1462
## Rank 3 - Rank 1 0.09552499 0.03701875  0.0099062786       0.0217
## Rank 4 - Rank 1 0.10560233 0.03709428  0.0044401285       0.0150
## Rank 5 - Rank 1 0.12944310 0.03696620  0.0004679592       0.0024
##                   Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 0.03535186 0.03529215  0.3165593878       0.3134
## Rank 3 - Rank 1 0.10322104 0.03521866  0.0034013721       0.0091
## Rank 4 - Rank 1 0.09075540 0.03529215  0.0101641020       0.0183
## Rank 5 - Rank 1 0.12169187 0.03517024  0.0005462518       0.0012
## 
## Best linear projection of the conditional average treatment effect.
## Confidence intervals are cluster- and heteroskedasticity-robust (HC3):
## 
##                                                 Estimate  Std. Error t value
## (Intercept)                                   0.09029819  0.23049963  0.3917
## att_check_pre                                 0.01096915  0.01256765  0.8728
## base_rate_pre                                -0.05693170  0.04575122 -1.2444
## misinfo_pre                                  -0.15183111  0.05748710 -2.6411
## misinfo_avg_acc_score_pre                     0.01056746  0.01053768  1.0028
## base_avg_acc_score_pre                        0.00252840  0.00748132  0.3380
## age                                          -0.00099943  0.00187003 -0.5344
## gender_Man                                    0.01330638  0.01836806  0.7244
## education_High_school_or_less                -0.13175476  0.11826746 -1.1140
## education_Some_college                       -0.13610252  0.12087302 -1.1260
## education_Bachelor_degree                    -0.14436862  0.11973863 -1.2057
## marital_Married_or_in_a_domestic_partnership -0.03529450  0.03815243 -0.9251
## employment_Employed                           0.03107150  0.03575647  0.8690
## employment_Unemployed                        -0.00254512  0.02848549 -0.0893
## location_Mostly_urban                         0.02231239  0.02200843  1.0138
## location_Suburban                            -0.01827990  0.02224811 -0.8216
## religion_Christian                           -0.01022204  0.05659726 -0.1806
## religiosity_Attends                          -0.08925971  0.05090375 -1.7535
## social_media_bin_Yes                          0.20449855  0.22700902  0.9008
## social_media_hours                           -0.00204198  0.00323032 -0.6321
## social_media_share_80_100                    -0.05213129  0.02935529 -1.7759
## social_media_share_60_80                      0.01689772  0.03795543  0.4452
## social_media_share_40_60                     -0.02867746  0.03937979 -0.7282
## social_media_share_20_40                     -0.00286237  0.04358818 -0.0657
##                                              Pr(>|t|)   
## (Intercept)                                  0.695266   
## att_check_pre                                0.382826   
## base_rate_pre                                0.213442   
## misinfo_pre                                  0.008299 **
## misinfo_avg_acc_score_pre                    0.316012   
## base_avg_acc_score_pre                       0.735412   
## age                                          0.593067   
## gender_Man                                   0.468849   
## education_High_school_or_less                0.265336   
## education_Some_college                       0.260242   
## education_Bachelor_degree                    0.228013   
## marital_Married_or_in_a_domestic_partnership 0.354980   
## employment_Employed                          0.384918   
## employment_Unemployed                        0.928810   
## location_Mostly_urban                        0.310741   
## location_Suburban                            0.411337   
## religion_Christian                           0.856684   
## religiosity_Attends                          0.079601 . 
## social_media_bin_Yes                         0.367734   
## social_media_hours                           0.527342   
## social_media_share_80_100                    0.075838 . 
## social_media_share_60_80                     0.656203   
## social_media_share_40_60                     0.466521   
## social_media_share_20_40                     0.947645   
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## Best linear fit using forest predictions (on held-out data)
## as well as the mean forest prediction as regressors, along
## with one-sided heteroskedasticity-robust (HC3) SEs:
## 
##                                Estimate Std. Error t value         Pr(>t)    
## mean.forest.prediction          0.98984    0.15367  6.4413 0.000000000067 ***
## differential.forest.prediction  0.82885    0.26059  3.1806       0.000741 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

## [1] "Outcome: misinfo_post"
##   method ranking    estimate    std.err
## 1    ols      Q1 -0.21648295 0.02390808
## 2    ols      Q2 -0.11679449 0.02367537
## 3    ols      Q3 -0.12823666 0.02134351
## 4    ols      Q4 -0.08494745 0.02011306
## 5    ols      Q5 -0.03004045 0.01817905
##   method ranking    estimate    std.err
## 1   aipw      Q1 -0.21737075 0.02343788
## 2   aipw      Q2 -0.12669952 0.02213676
## 3   aipw      Q3 -0.13041261 0.02092392
## 4   aipw      Q4 -0.08837977 0.01918440
## 5   aipw      Q5 -0.02744132 0.01718439

##                   Estimate Std. Error      Orig. p-value Adj. p-value
## Rank 2 - Rank 1 0.09968846 0.03050535 0.0010936138423974       0.0014
## Rank 3 - Rank 1 0.08824629 0.03042877 0.0037529981731727       0.0039
## Rank 4 - Rank 1 0.13153550 0.03050019 0.0000165644289466       0.0000
## Rank 5 - Rank 1 0.18644250 0.03038766 0.0000000009406658       0.0000
##                   Estimate Std. Error       Orig. p-value Adj. p-value
## Rank 2 - Rank 1 0.09067123 0.02926527 0.00196157691057733       0.0042
## Rank 3 - Rank 1 0.08695814 0.02920433 0.00292462412762226       0.0042
## Rank 4 - Rank 1 0.12899098 0.02926527 0.00001075310378011       0.0000
## Rank 5 - Rank 1 0.18992943 0.02916418 0.00000000008413083       0.0000
## 
## Best linear projection of the conditional average treatment effect.
## Confidence intervals are cluster- and heteroskedasticity-robust (HC3):
## 
##                                                 Estimate  Std. Error t value
## (Intercept)                                  -0.02701346  0.13983640 -0.1932
## att_check_pre                                -0.01334497  0.01440041 -0.9267
## base_rate_pre                                -0.05721900  0.04587035 -1.2474
## misinfo_pre                                  -0.13053344  0.05647659 -2.3113
## misinfo_avg_acc_score_pre                    -0.00503739  0.01105401 -0.4557
## base_avg_acc_score_pre                       -0.01311743  0.01010741 -1.2978
## age                                          -0.00062516  0.00140389 -0.4453
## gender_Man                                    0.03250056  0.02158536  1.5057
## education_High_school_or_less                -0.12778412  0.08552119 -1.4942
## education_Some_college                       -0.12417835  0.07747943 -1.6027
## education_Bachelor_degree                    -0.13309133  0.08733643 -1.5239
## marital_Married_or_in_a_domestic_partnership -0.00572081  0.03861978 -0.1481
## employment_Employed                          -0.01216298  0.02872173 -0.4235
## employment_Unemployed                        -0.00416915  0.02887672 -0.1444
## location_Mostly_urban                         0.03864473  0.02620665  1.4746
## location_Suburban                            -0.00621951  0.01905957 -0.3263
## religion_Christian                           -0.02068455  0.03302555 -0.6263
## religiosity_Attends                          -0.04652254  0.03980561 -1.1687
## social_media_bin_Yes                          0.20566880  0.10487667  1.9611
## social_media_hours                           -0.00011790  0.00277738 -0.0425
## social_media_share_80_100                     0.05720671  0.03770679  1.5171
## social_media_share_60_80                      0.02658732  0.03034501  0.8762
## social_media_share_40_60                      0.00389190  0.03311960  0.1175
## social_media_share_20_40                      0.03213788  0.02908500  1.1050
##                                              Pr(>|t|)  
## (Intercept)                                   0.84683  
## att_check_pre                                 0.35414  
## base_rate_pre                                 0.21233  
## misinfo_pre                                   0.02087 *
## misinfo_avg_acc_score_pre                     0.64863  
## base_avg_acc_score_pre                        0.19444  
## age                                           0.65612  
## gender_Man                                    0.13224  
## education_High_school_or_less                 0.13522  
## education_Some_college                        0.10908  
## education_Bachelor_degree                     0.12762  
## marital_Married_or_in_a_domestic_partnership  0.88225  
## employment_Employed                           0.67197  
## employment_Unemployed                         0.88521  
## location_Mostly_urban                         0.14040  
## location_Suburban                             0.74420  
## religion_Christian                            0.53115  
## religiosity_Attends                           0.24258  
## social_media_bin_Yes                          0.04995 *
## social_media_hours                            0.96614  
## social_media_share_80_100                     0.12932  
## social_media_share_60_80                      0.38100  
## social_media_share_40_60                      0.90646  
## social_media_share_20_40                      0.26925  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## Best linear fit using forest predictions (on held-out data)
## as well as the mean forest prediction as regressors, along
## with one-sided heteroskedasticity-robust (HC3) SEs:
## 
##                                Estimate Std. Error t value
## mean.forest.prediction         0.987627   0.110357  8.9494
## differential.forest.prediction 0.985143   0.095267 10.3408
##                                               Pr(>t)    
## mean.forest.prediction         < 0.00000000000000022 ***
## differential.forest.prediction < 0.00000000000000022 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

## [1] "Outcome: base_rate_diff"
##   method ranking    estimate    std.err
## 1    ols      Q1 -0.15935271 0.02542061
## 2    ols      Q2 -0.09966865 0.02848838
## 3    ols      Q3 -0.07439803 0.03143843
## 4    ols      Q4 -0.10116379 0.02956830
## 5    ols      Q5 -0.03304432 0.02831204
##   method ranking    estimate    std.err
## 1   aipw      Q1 -0.15699949 0.02411626
## 2   aipw      Q2 -0.10808216 0.02425860
## 3   aipw      Q3 -0.07123233 0.02626445
## 4   aipw      Q4 -0.09008536 0.02581690
## 5   aipw      Q5 -0.02796094 0.02440778

##                   Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 0.05968406 0.04070062   0.142621753       0.2551
## Rank 3 - Rank 1 0.08495468 0.04060808   0.036502392       0.1028
## Rank 4 - Rank 1 0.05818893 0.04069398   0.152827793       0.2551
## Rank 5 - Rank 1 0.12630839 0.04055707   0.001858015       0.0068
##                   Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 0.04891733 0.03534053   0.166391404       0.1692
## Rank 3 - Rank 1 0.08576717 0.03526693   0.015066232       0.0401
## Rank 4 - Rank 1 0.06691413 0.03534053   0.058382937       0.1081
## Rank 5 - Rank 1 0.12903855 0.03521845   0.000251915       0.0011
## 
## Best linear projection of the conditional average treatment effect.
## Confidence intervals are cluster- and heteroskedasticity-robust (HC3):
## 
##                                                Estimate Std. Error t value
## (Intercept)                                   0.1100666  0.2310091  0.4765
## att_check_pre                                 0.0082473  0.0117181  0.7038
## base_rate_pre                                -0.0772535  0.0453255 -1.7044
## misinfo_pre                                  -0.1387045  0.0560067 -2.4766
## misinfo_avg_acc_score_pre                     0.0112136  0.0110285  1.0168
## base_avg_acc_score_pre                        0.0037165  0.0077263  0.4810
## age                                          -0.0011092  0.0019135 -0.5797
## gender_Man                                    0.0165237  0.0205339  0.8047
## education_High_school_or_less                -0.1305431  0.1237488 -1.0549
## education_Some_college                       -0.1342607  0.1243125 -1.0800
## education_Bachelor_degree                    -0.1440394  0.1233240 -1.1680
## marital_Married_or_in_a_domestic_partnership -0.0358807  0.0405998 -0.8838
## employment_Employed                           0.0296629  0.0364618  0.8135
## employment_Unemployed                        -0.0029827  0.0303447 -0.0983
## location_Mostly_urban                         0.0237736  0.0220023  1.0805
## location_Suburban                            -0.0183944  0.0216548 -0.8494
## religion_Christian                           -0.0146844  0.0539733 -0.2721
## religiosity_Attends                          -0.0845797  0.0480659 -1.7597
## social_media_bin_Yes                          0.1878764  0.2186330  0.8593
## social_media_hours                           -0.0015376  0.0034480 -0.4459
## social_media_share_80_100                    -0.0585288  0.0304304 -1.9234
## social_media_share_60_80                      0.0168087  0.0374275  0.4491
## social_media_share_40_60                     -0.0239269  0.0392455 -0.6097
## social_media_share_20_40                      0.0012101  0.0431174  0.0281
##                                              Pr(>|t|)  
## (Intercept)                                   0.63378  
## att_check_pre                                 0.48160  
## base_rate_pre                                 0.08839 .
## misinfo_pre                                   0.01331 *
## misinfo_avg_acc_score_pre                     0.30932  
## base_avg_acc_score_pre                        0.63054  
## age                                           0.56216  
## gender_Man                                    0.42104  
## education_High_school_or_less                 0.29154  
## education_Some_college                        0.28020  
## education_Bachelor_degree                     0.24289  
## marital_Married_or_in_a_domestic_partnership  0.37688  
## employment_Employed                           0.41597  
## employment_Unemployed                         0.92170  
## location_Mostly_urban                         0.27999  
## location_Suburban                             0.39569  
## religion_Christian                            0.78559  
## religiosity_Attends                           0.07855 .
## social_media_bin_Yes                          0.39022  
## social_media_hours                            0.65567  
## social_media_share_80_100                     0.05451 .
## social_media_share_60_80                      0.65339  
## social_media_share_40_60                      0.54212  
## social_media_share_20_40                      0.97761  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## Best linear fit using forest predictions (on held-out data)
## as well as the mean forest prediction as regressors, along
## with one-sided heteroskedasticity-robust (HC3) SEs:
## 
##                                Estimate Std. Error t value           Pr(>t)    
## mean.forest.prediction          0.99653    0.15517  6.4222 0.00000000007587 ***
## differential.forest.prediction  0.75072    0.22579  3.3249        0.0004466 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

## [1] "Outcome: misinfo_diff"
##   method ranking    estimate    std.err
## 1    ols      Q1 -0.23557262 0.02381805
## 2    ols      Q2 -0.11241593 0.02524032
## 3    ols      Q3 -0.11260875 0.02431281
## 4    ols      Q4 -0.07459381 0.02262413
## 5    ols      Q5 -0.02252129 0.01935879
##   method ranking   estimate    std.err
## 1   aipw      Q1 -0.2303789 0.02321410
## 2   aipw      Q2 -0.1172193 0.02244118
## 3   aipw      Q3 -0.1235869 0.02062905
## 4   aipw      Q4 -0.0896141 0.01908440
## 5   aipw      Q5 -0.0239520 0.01757731

##                  Estimate Std. Error       Orig. p-value Adj. p-value
## Rank 2 - Rank 1 0.1231567 0.03279345 0.00017570034994591       0.0005
## Rank 3 - Rank 1 0.1229639 0.03272664 0.00017446380625230       0.0005
## Rank 4 - Rank 1 0.1609788 0.03280821 0.00000096725692130       0.0000
## Rank 5 - Rank 1 0.2130513 0.03267339 0.00000000007971338       0.0000
##                  Estimate Std. Error        Orig. p-value Adj. p-value
## Rank 2 - Rank 1 0.1131596 0.02926928 0.000112504469264967       0.0004
## Rank 3 - Rank 1 0.1067921 0.02920832 0.000259598314902611       0.0006
## Rank 4 - Rank 1 0.1407648 0.02926928 0.000001576241506207       0.0000
## Rank 5 - Rank 1 0.2064269 0.02916817 0.000000000001759187       0.0000
## 
## Best linear projection of the conditional average treatment effect.
## Confidence intervals are cluster- and heteroskedasticity-robust (HC3):
## 
##                                                 Estimate  Std. Error t value
## (Intercept)                                  -0.02466034  0.13735918 -0.1795
## att_check_pre                                -0.01562568  0.01511128 -1.0340
## base_rate_pre                                -0.06013584  0.04665400 -1.2890
## misinfo_pre                                  -0.13118409  0.05916039 -2.2174
## misinfo_avg_acc_score_pre                    -0.00577701  0.01068626 -0.5406
## base_avg_acc_score_pre                       -0.01201710  0.00918216 -1.3087
## age                                          -0.00054504  0.00138203 -0.3944
## gender_Man                                    0.03090367  0.02054947  1.5039
## education_High_school_or_less                -0.13678117  0.09228228 -1.4822
## education_Some_college                       -0.13438268  0.08609645 -1.5608
## education_Bachelor_degree                    -0.14526993  0.09519275 -1.5261
## marital_Married_or_in_a_domestic_partnership -0.00396764  0.03695058 -0.1074
## employment_Employed                          -0.00979552  0.02600430 -0.3767
## employment_Unemployed                        -0.00605804  0.02806124 -0.2159
## location_Mostly_urban                         0.03645848  0.02559819  1.4243
## location_Suburban                            -0.00766540  0.01987901 -0.3856
## religion_Christian                           -0.01413253  0.03583298 -0.3944
## religiosity_Attends                          -0.03380125  0.03807765 -0.8877
## social_media_bin_Yes                          0.19920426  0.10652099  1.8701
## social_media_hours                            0.00006435  0.00277759  0.0232
## social_media_share_80_100                     0.05078680  0.03503629  1.4495
## social_media_share_60_80                      0.02535038  0.03097827  0.8183
## social_media_share_40_60                      0.00199959  0.03381613  0.0591
## social_media_share_20_40                      0.03102360  0.02773094  1.1187
##                                              Pr(>|t|)  
## (Intercept)                                   0.85753  
## att_check_pre                                 0.30119  
## base_rate_pre                                 0.19749  
## misinfo_pre                                   0.02666 *
## misinfo_avg_acc_score_pre                     0.58882  
## base_avg_acc_score_pre                        0.19070  
## age                                           0.69333  
## gender_Man                                    0.13270  
## education_High_school_or_less                 0.13837  
## education_Some_college                        0.11865  
## education_Bachelor_degree                     0.12708  
## marital_Married_or_in_a_domestic_partnership  0.91450  
## employment_Employed                           0.70643  
## employment_Unemployed                         0.82909  
## location_Mostly_urban                         0.15446  
## location_Suburban                             0.69981  
## religion_Christian                            0.69331  
## religiosity_Attends                           0.37477  
## social_media_bin_Yes                          0.06155 .
## social_media_hours                            0.98152  
## social_media_share_80_100                     0.14727  
## social_media_share_60_80                      0.41322  
## social_media_share_40_60                      0.95285  
## social_media_share_20_40                      0.26333  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## Best linear fit using forest predictions (on held-out data)
## as well as the mean forest prediction as regressors, along
## with one-sided heteroskedasticity-robust (HC3) SEs:
## 
##                                Estimate Std. Error t value
## mean.forest.prediction          0.99279    0.11251  8.8237
## differential.forest.prediction  1.01373    0.08643 11.7290
##                                               Pr(>t)    
## mean.forest.prediction         < 0.00000000000000022 ***
## differential.forest.prediction < 0.00000000000000022 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

## [1] "Outcome: new_disc_post"
##   method ranking    estimate    std.err
## 1    ols      Q1 0.014360927 0.02223173
## 2    ols      Q2 0.006984372 0.02312896
## 3    ols      Q3 0.031069323 0.02208895
## 4    ols      Q4 0.014946103 0.02215821
## 5    ols      Q5 0.050989447 0.02178959
##   method ranking    estimate    std.err
## 1   aipw      Q1 0.015805668 0.02201834
## 2   aipw      Q2 0.008543599 0.02302381
## 3   aipw      Q3 0.031397889 0.02188964
## 4   aipw      Q4 0.016445813 0.02204276
## 5   aipw      Q5 0.052738178 0.02188484

##                      Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 -0.0073765553 0.03156805     0.8152532       0.9593
## Rank 3 - Rank 1  0.0167083959 0.03150944     0.5959599       0.9139
## Rank 4 - Rank 1  0.0005851753 0.03156657     0.9852108       0.9836
## Rank 5 - Rank 1  0.0366285195 0.03147462     0.2446030       0.5944
##                      Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 -0.0072620685 0.03136255     0.8168989       0.9603
## Rank 3 - Rank 1  0.0155922215 0.03129723     0.6183747       0.9212
## Rank 4 - Rank 1  0.0006401455 0.03136255     0.9837165       0.9845
## Rank 5 - Rank 1  0.0369325107 0.03125421     0.2374099       0.5741
## 
## Best linear projection of the conditional average treatment effect.
## Confidence intervals are cluster- and heteroskedasticity-robust (HC3):
## 
##                                                 Estimate  Std. Error t value
## (Intercept)                                   0.07314911  0.18285885  0.4000
## att_check_pre                                 0.02816037  0.00885232  3.1811
## base_rate_pre                                 0.00357326  0.03880605  0.0921
## misinfo_pre                                  -0.02004515  0.05396807 -0.3714
## misinfo_avg_acc_score_pre                     0.01396817  0.00897607  1.5562
## base_avg_acc_score_pre                        0.01552759  0.00678753  2.2877
## age                                          -0.00037239  0.00218477 -0.1704
## gender_Man                                   -0.02278712  0.01497173 -1.5220
## education_High_school_or_less                 0.01217396  0.13664784  0.0891
## education_Some_college                        0.00358591  0.12987712  0.0276
## education_Bachelor_degree                     0.00995033  0.12181495  0.0817
## marital_Married_or_in_a_domestic_partnership -0.03228318  0.02436402 -1.3250
## employment_Employed                           0.05134997  0.02515331  2.0415
## employment_Unemployed                         0.01001266  0.01766586  0.5668
## location_Mostly_urban                        -0.01394607  0.02188082 -0.6374
## location_Suburban                            -0.00827884  0.02542636 -0.3256
## religion_Christian                            0.01648292  0.04474104  0.3684
## religiosity_Attends                          -0.05000387  0.04525262 -1.1050
## social_media_bin_Yes                          0.01416157  0.15578232  0.0909
## social_media_hours                           -0.00189751  0.00266570 -0.7118
## social_media_share_80_100                    -0.10614748  0.03374149 -3.1459
## social_media_share_60_80                     -0.00752203  0.04440510 -0.1694
## social_media_share_40_60                     -0.03328738  0.02839044 -1.1725
## social_media_share_20_40                     -0.02958376  0.03898511 -0.7588
##                                              Pr(>|t|)   
## (Intercept)                                  0.689158   
## att_check_pre                                0.001479 **
## base_rate_pre                                0.926640   
## misinfo_pre                                  0.710342   
## misinfo_avg_acc_score_pre                    0.119759   
## base_avg_acc_score_pre                       0.022215 * 
## age                                          0.864667   
## gender_Man                                   0.128094   
## education_High_school_or_less                0.929015   
## education_Some_college                       0.977975   
## education_Bachelor_degree                    0.934903   
## marital_Married_or_in_a_domestic_partnership 0.185244   
## employment_Employed                          0.041276 * 
## employment_Unemployed                        0.570899   
## location_Mostly_urban                        0.523927   
## location_Suburban                            0.744746   
## religion_Christian                           0.712591   
## religiosity_Attends                          0.269236   
## social_media_bin_Yes                         0.927572   
## social_media_hours                           0.476620   
## social_media_share_80_100                    0.001669 **
## social_media_share_60_80                     0.865495   
## social_media_share_40_60                     0.241080   
## social_media_share_20_40                     0.447993   
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## Best linear fit using forest predictions (on held-out data)
## as well as the mean forest prediction as regressors, along
## with one-sided heteroskedasticity-robust (HC3) SEs:
## 
##                                Estimate Std. Error t value    Pr(>t)    
## mean.forest.prediction          1.00386    0.31166  3.2210 0.0006443 ***
## differential.forest.prediction  0.53578    0.14565  3.6787 0.0001189 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

## [1] "Outcome: new_disc_diff"
##   method ranking     estimate    std.err
## 1    ols      Q1  0.003311889 0.03135942
## 2    ols      Q2 -0.007383113 0.03204380
## 3    ols      Q3  0.027083507 0.02990321
## 4    ols      Q4  0.036717496 0.02957003
## 5    ols      Q5  0.046942357 0.02815543
##   method ranking    estimate    std.err
## 1   aipw      Q1 0.006694447 0.02249607
## 2   aipw      Q2 0.011228165 0.02327535
## 3   aipw      Q3 0.031400819 0.02239695
## 4   aipw      Q4 0.020413356 0.02098180
## 5   aipw      Q5 0.059609481 0.02230140

##                    Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 -0.01069500 0.04280700     0.8027234       0.8020
## Rank 3 - Rank 1  0.02377162 0.04271713     0.5779110       0.8005
## Rank 4 - Rank 1  0.03340561 0.04280625     0.4352119       0.7732
## Rank 5 - Rank 1  0.04363047 0.04267518     0.3066671       0.6912
##                    Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 0.004533718 0.03154828    0.88573959       0.8788
## Rank 3 - Rank 1 0.024706371 0.03148258    0.43264367       0.7690
## Rank 4 - Rank 1 0.013718908 0.03154828    0.66369399       0.8683
## Rank 5 - Rank 1 0.052915034 0.03143930    0.09244473       0.2692
## 
## Best linear projection of the conditional average treatment effect.
## Confidence intervals are cluster- and heteroskedasticity-robust (HC3):
## 
##                                                 Estimate  Std. Error t value
## (Intercept)                                   0.06768996  0.18620949  0.3635
## att_check_pre                                 0.02395363  0.00857923  2.7920
## base_rate_pre                                -0.00014222  0.04051385 -0.0035
## misinfo_pre                                  -0.01595397  0.05980408 -0.2668
## misinfo_avg_acc_score_pre                     0.01648375  0.00925613  1.7808
## base_avg_acc_score_pre                        0.01432579  0.00695863  2.0587
## age                                          -0.00021760  0.00224438 -0.0970
## gender_Man                                   -0.02372647  0.01428602 -1.6608
## education_High_school_or_less                 0.01035718  0.13788098  0.0751
## education_Some_college                        0.00364844  0.12978686  0.0281
## education_Bachelor_degree                     0.00988169  0.12330348  0.0801
## marital_Married_or_in_a_domestic_partnership -0.03368034  0.02417858 -1.3930
## employment_Employed                           0.05029545  0.02430615  2.0692
## employment_Unemployed                         0.00778417  0.01778913  0.4376
## location_Mostly_urban                        -0.01338650  0.02383250 -0.5617
## location_Suburban                            -0.00902592  0.02641142 -0.3417
## religion_Christian                            0.01281388  0.04186995  0.3060
## religiosity_Attends                          -0.04421752  0.04379650 -1.0096
## social_media_bin_Yes                          0.02283584  0.16096538  0.1419
## social_media_hours                           -0.00215913  0.00273695 -0.7889
## social_media_share_80_100                    -0.11098096  0.03439913 -3.2263
## social_media_share_60_80                     -0.00971633  0.04434434 -0.2191
## social_media_share_40_60                     -0.03196584  0.02750063 -1.1624
## social_media_share_20_40                     -0.03136535  0.03915523 -0.8011
##                                              Pr(>|t|)   
## (Intercept)                                  0.716241   
## att_check_pre                                0.005265 **
## base_rate_pre                                0.997199   
## misinfo_pre                                  0.789661   
## misinfo_avg_acc_score_pre                    0.075022 . 
## base_avg_acc_score_pre                       0.039594 * 
## age                                          0.922767   
## gender_Man                                   0.096837 . 
## education_High_school_or_less                0.940126   
## education_Some_college                       0.977575   
## education_Bachelor_degree                    0.936129   
## marital_Married_or_in_a_domestic_partnership 0.163711   
## employment_Employed                          0.038594 * 
## employment_Unemployed                        0.661717   
## location_Mostly_urban                        0.574362   
## location_Suburban                            0.732564   
## religion_Christian                           0.759592   
## religiosity_Attends                          0.312748   
## social_media_bin_Yes                         0.887192   
## social_media_hours                           0.430232   
## social_media_share_80_100                    0.001265 **
## social_media_share_60_80                     0.826576   
## social_media_share_40_60                     0.245163   
## social_media_share_20_40                     0.423155   
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## Best linear fit using forest predictions (on held-out data)
## as well as the mean forest prediction as regressors, along
## with one-sided heteroskedasticity-robust (HC3) SEs:
## 
##                                Estimate Std. Error t value    Pr(>t)    
## mean.forest.prediction          0.99343    0.29877  3.3250 0.0004464 ***
## differential.forest.prediction  0.69667    0.21708  3.2093 0.0006712 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

## [1] "Outcome: misinfo_avg_acc_score_post"
##   method ranking   estimate    std.err
## 1    ols      Q1 -0.2601795 0.10705576
## 2    ols      Q2 -0.2600762 0.09838730
## 3    ols      Q3 -0.2865519 0.09186719
## 4    ols      Q4 -0.4040318 0.08935891
## 5    ols      Q5 -0.1725169 0.08685344
##   method ranking   estimate    std.err
## 1   aipw      Q1 -0.2642129 0.09930527
## 2   aipw      Q2 -0.2524807 0.09389217
## 3   aipw      Q3 -0.2861132 0.08461158
## 4   aipw      Q4 -0.3967589 0.08231854
## 5   aipw      Q5 -0.1906100 0.07783597

##                      Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1  0.0001032168  0.1345186     0.9993878       0.9991
## Rank 3 - Rank 1 -0.0263724282  0.1341007     0.8441036       0.9710
## Rank 4 - Rank 1 -0.1438523467  0.1344493     0.2847189       0.6295
## Rank 5 - Rank 1  0.0876625686  0.1339725     0.5129392       0.8354
##                    Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1  0.01173219  0.1243996     0.9248677       0.9780
## Rank 3 - Rank 1 -0.02190032  0.1241405     0.8599773       0.9780
## Rank 4 - Rank 1 -0.13254599  0.1243996     0.2867272       0.6275
## Rank 5 - Rank 1  0.07360288  0.1239699     0.5527392       0.8735
## 
## Best linear projection of the conditional average treatment effect.
## Confidence intervals are cluster- and heteroskedasticity-robust (HC3):
## 
##                                                Estimate Std. Error t value
## (Intercept)                                   0.4004228  0.7440739  0.5381
## att_check_pre                                -0.0159636  0.0863852 -0.1848
## base_rate_pre                                 0.0577387  0.2081872  0.2773
## misinfo_pre                                  -0.1631112  0.2215349 -0.7363
## misinfo_avg_acc_score_pre                    -0.0208249  0.0555938 -0.3746
## base_avg_acc_score_pre                       -0.0614002  0.0422379 -1.4537
## age                                          -0.0038764  0.0051385 -0.7544
## gender_Man                                    0.1525246  0.0844218  1.8067
## education_High_school_or_less                -0.3738178  0.3568651 -1.0475
## education_Some_college                       -0.2349592  0.3014673 -0.7794
## education_Bachelor_degree                    -0.2906358  0.3277953 -0.8866
## marital_Married_or_in_a_domestic_partnership  0.0149716  0.1370129  0.1093
## employment_Employed                           0.0378187  0.1053124  0.3591
## employment_Unemployed                        -0.0209594  0.1194093 -0.1755
## location_Mostly_urban                         0.0215694  0.1139376  0.1893
## location_Suburban                            -0.1880316  0.0473831 -3.9683
## religion_Christian                            0.0300352  0.1932658  0.1554
## religiosity_Attends                          -0.0845727  0.1840329 -0.4596
## social_media_bin_Yes                         -0.0599461  0.5727537 -0.1047
## social_media_hours                           -0.0081038  0.0115471 -0.7018
## social_media_share_80_100                    -0.1406309  0.2087121 -0.6738
## social_media_share_60_80                     -0.1094176  0.0967895 -1.1305
## social_media_share_40_60                     -0.0522937  0.1735854 -0.3013
## social_media_share_20_40                     -0.0231427  0.1640726 -0.1411
##                                                Pr(>|t|)    
## (Intercept)                                     0.59051    
## att_check_pre                                   0.85340    
## base_rate_pre                                   0.78153    
## misinfo_pre                                     0.46161    
## misinfo_avg_acc_score_pre                       0.70799    
## base_avg_acc_score_pre                          0.14612    
## age                                             0.45066    
## gender_Man                                      0.07089 .  
## education_High_school_or_less                   0.29494    
## education_Some_college                          0.43580    
## education_Bachelor_degree                       0.37533    
## marital_Married_or_in_a_domestic_partnership    0.91299    
## employment_Employed                             0.71953    
## employment_Unemployed                           0.86068    
## location_Mostly_urban                           0.84986    
## location_Suburban                            0.00007379 ***
## religion_Christian                              0.87651    
## religiosity_Attends                             0.64587    
## social_media_bin_Yes                            0.91665    
## social_media_hours                              0.48284    
## social_media_share_80_100                       0.50048    
## social_media_share_60_80                        0.25835    
## social_media_share_40_60                        0.76324    
## social_media_share_20_40                        0.88784    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## Best linear fit using forest predictions (on held-out data)
## as well as the mean forest prediction as regressors, along
## with one-sided heteroskedasticity-robust (HC3) SEs:
## 
##                                Estimate Std. Error t value        Pr(>t)    
## mean.forest.prediction          1.00577    0.18912  5.3183 0.00000005555 ***
## differential.forest.prediction  0.11664    0.38295  0.3046        0.3804    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

## [1] "Outcome: misinfo_avg_acc_score_diff"
##   method ranking   estimate    std.err
## 1    ols      Q1 -0.2584789 0.10777539
## 2    ols      Q2 -0.2436207 0.10922999
## 3    ols      Q3 -0.2839151 0.10423435
## 4    ols      Q4 -0.2966437 0.09522594
## 5    ols      Q5 -0.1917808 0.09004132
##   method ranking   estimate    std.err
## 1   aipw      Q1 -0.3306438 0.09875264
## 2   aipw      Q2 -0.2193145 0.09300702
## 3   aipw      Q3 -0.2779625 0.08806774
## 4   aipw      Q4 -0.3218404 0.08268198
## 5   aipw      Q5 -0.2016284 0.07686564

##                    Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1  0.01485817  0.1437742     0.9176958       0.9859
## Rank 3 - Rank 1 -0.02543621  0.1435953     0.8594098       0.9859
## Rank 4 - Rank 1 -0.03816486  0.1437966     0.7907098       0.9859
## Rank 5 - Rank 1  0.06669805  0.1432775     0.6415896       0.9725
##                    Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 0.111329311  0.1247708     0.3723071       0.6818
## Rank 3 - Rank 1 0.052681290  0.1245109     0.6722432       0.8701
## Rank 4 - Rank 1 0.008803401  0.1247708     0.9437545       0.9454
## Rank 5 - Rank 1 0.129015382  0.1243398     0.2995240       0.6559
## 
## Best linear projection of the conditional average treatment effect.
## Confidence intervals are cluster- and heteroskedasticity-robust (HC3):
## 
##                                                Estimate Std. Error t value
## (Intercept)                                   0.3667037  0.7616837  0.4814
## att_check_pre                                -0.0169985  0.0813838 -0.2089
## base_rate_pre                                 0.1498500  0.2141830  0.6996
## misinfo_pre                                  -0.1513451  0.2175866 -0.6956
## misinfo_avg_acc_score_pre                    -0.0348778  0.0533151 -0.6542
## base_avg_acc_score_pre                       -0.0748199  0.0428156 -1.7475
## age                                          -0.0025267  0.0049779 -0.5076
## gender_Man                                    0.1441640  0.0852858  1.6904
## education_High_school_or_less                -0.4087203  0.3542326 -1.1538
## education_Some_college                       -0.2800682  0.2940709 -0.9524
## education_Bachelor_degree                    -0.3208575  0.3259427 -0.9844
## marital_Married_or_in_a_domestic_partnership  0.0029195  0.1378701  0.0212
## employment_Employed                           0.0452896  0.1136204  0.3986
## employment_Unemployed                        -0.0231225  0.1231979 -0.1877
## location_Mostly_urban                         0.0210819  0.1106196  0.1906
## location_Suburban                            -0.1668826  0.0448225 -3.7232
## religion_Christian                            0.0618074  0.2145702  0.2881
## religiosity_Attends                          -0.1283488  0.1820927 -0.7049
## social_media_bin_Yes                         -0.0667211  0.5702183 -0.1170
## social_media_hours                           -0.0080297  0.0104676 -0.7671
## social_media_share_80_100                    -0.1173380  0.1952343 -0.6010
## social_media_share_60_80                     -0.1022794  0.1005245 -1.0175
## social_media_share_40_60                     -0.0504455  0.1760148 -0.2866
## social_media_share_20_40                     -0.0232711  0.1648992 -0.1411
##                                               Pr(>|t|)    
## (Intercept)                                  0.6302343    
## att_check_pre                                0.8345628    
## base_rate_pre                                0.4842002    
## misinfo_pre                                  0.4867475    
## misinfo_avg_acc_score_pre                    0.5130366    
## base_avg_acc_score_pre                       0.0806375 .  
## age                                          0.6117781    
## gender_Man                                   0.0910451 .  
## education_High_school_or_less                0.2486509    
## education_Some_college                       0.3409667    
## education_Bachelor_degree                    0.3249858    
## marital_Married_or_in_a_domestic_partnership 0.9831069    
## employment_Employed                          0.6902084    
## employment_Unemployed                        0.8511336    
## location_Mostly_urban                        0.8488654    
## location_Suburban                            0.0001997 ***
## religion_Christian                           0.7733234    
## religiosity_Attends                          0.4809467    
## social_media_bin_Yes                         0.9068588    
## social_media_hours                           0.4430703    
## social_media_share_80_100                    0.5478702    
## social_media_share_60_80                     0.3090044    
## social_media_share_40_60                     0.7744368    
## social_media_share_20_40                     0.8877805    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## Best linear fit using forest predictions (on held-out data)
## as well as the mean forest prediction as regressors, along
## with one-sided heteroskedasticity-robust (HC3) SEs:
## 
##                                  Estimate Std. Error t value       Pr(>t)    
## mean.forest.prediction          0.9963519  0.1989549  5.0079 0.0000002882 ***
## differential.forest.prediction -0.0024261  0.4317034 -0.0056       0.5022    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

## [1] "Outcome: base_avg_acc_score_post"
##   method ranking   estimate   std.err
## 1    ols      Q1 -0.3269579 0.1141401
## 2    ols      Q2 -0.1614509 0.1092959
## 3    ols      Q3 -0.1608686 0.1041006
## 4    ols      Q4 -0.2871953 0.1028896
## 5    ols      Q5 -0.2259760 0.1117143
##   method ranking   estimate    std.err
## 1   aipw      Q1 -0.3015515 0.10409107
## 2   aipw      Q2 -0.1559882 0.10462557
## 3   aipw      Q3 -0.1875218 0.10198380
## 4   aipw      Q4 -0.2750272 0.09725772
## 5   aipw      Q5 -0.2504385 0.10228179

##                   Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 0.16550704  0.1537189     0.2816924       0.6389
## Rank 3 - Rank 1 0.16608935  0.1533927     0.2789817       0.6389
## Rank 4 - Rank 1 0.03976262  0.1537189     0.7959039       0.7985
## Rank 5 - Rank 1 0.10098197  0.1531894     0.5098113       0.7288
##                   Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 0.14556330  0.1444076     0.3135203       0.7038
## Rank 3 - Rank 1 0.11402967  0.1441069     0.4288292       0.7684
## Rank 4 - Rank 1 0.02652432  0.1444076     0.8542773       0.9151
## Rank 5 - Rank 1 0.05111300  0.1439088     0.7224781       0.9151
## 
## Best linear projection of the conditional average treatment effect.
## Confidence intervals are cluster- and heteroskedasticity-robust (HC3):
## 
##                                                Estimate Std. Error t value
## (Intercept)                                  -0.0887317  1.0019574 -0.0886
## att_check_pre                                 0.1124332  0.1005755  1.1179
## base_rate_pre                                 0.0792821  0.2401073  0.3302
## misinfo_pre                                  -0.3536884  0.3424416 -1.0328
## misinfo_avg_acc_score_pre                     0.0013579  0.0685345  0.0198
## base_avg_acc_score_pre                       -0.0047189  0.0419030 -0.1126
## age                                          -0.0086490  0.0068461 -1.2634
## gender_Man                                    0.1815443  0.1048173  1.7320
## education_High_school_or_less                 0.2845512  0.4853587  0.5863
## education_Some_college                        0.3117952  0.4714768  0.6613
## education_Bachelor_degree                     0.4028359  0.5092676  0.7910
## marital_Married_or_in_a_domestic_partnership -0.1769867  0.1329024 -1.3317
## employment_Employed                           0.3058429  0.0671033  4.5578
## employment_Unemployed                         0.1300449  0.1170221  1.1113
## location_Mostly_urban                        -0.0877929  0.1140014 -0.7701
## location_Suburban                            -0.2119582  0.1318325 -1.6078
## religion_Christian                            0.1603703  0.2462669  0.6512
## religiosity_Attends                          -0.4394316  0.2075371 -2.1174
## social_media_bin_Yes                          0.1819391  0.6574690  0.2767
## social_media_hours                           -0.0014220  0.0098732 -0.1440
## social_media_share_80_100                    -0.4359750  0.3100105 -1.4063
## social_media_share_60_80                     -0.1357178  0.1890983 -0.7177
## social_media_share_40_60                     -0.2233505  0.1656281 -1.3485
## social_media_share_20_40                     -0.1961308  0.2036519 -0.9631
##                                                 Pr(>|t|)    
## (Intercept)                                      0.92944    
## att_check_pre                                    0.26368    
## base_rate_pre                                    0.74127    
## misinfo_pre                                      0.30175    
## misinfo_avg_acc_score_pre                        0.98419    
## base_avg_acc_score_pre                           0.91034    
## age                                              0.20654    
## gender_Man                                       0.08336 .  
## education_High_school_or_less                    0.55773    
## education_Some_college                           0.50845    
## education_Bachelor_degree                        0.42899    
## marital_Married_or_in_a_domestic_partnership     0.18304    
## employment_Employed                          0.000005341 ***
## employment_Unemployed                            0.26652    
## location_Mostly_urban                            0.44129    
## location_Suburban                                0.10797    
## religion_Christian                               0.51496    
## religiosity_Attends                              0.03430 *  
## social_media_bin_Yes                             0.78201    
## social_media_hours                               0.88549    
## social_media_share_80_100                        0.15971    
## social_media_share_60_80                         0.47298    
## social_media_share_40_60                         0.17758    
## social_media_share_20_40                         0.33558    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## Best linear fit using forest predictions (on held-out data)
## as well as the mean forest prediction as regressors, along
## with one-sided heteroskedasticity-robust (HC3) SEs:
## 
##                                Estimate Std. Error t value       Pr(>t)    
## mean.forest.prediction          1.00886    0.19403  5.1995 0.0000001054 ***
## differential.forest.prediction  0.22926    0.35517  0.6455       0.2593    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

## [1] "Outcome: base_avg_acc_score_diff"
##   method ranking    estimate   std.err
## 1    ols      Q1 -0.34486216 0.1242633
## 2    ols      Q2 -0.16258280 0.1213812
## 3    ols      Q3 -0.31687455 0.1209796
## 4    ols      Q4  0.04509398 0.1291115
## 5    ols      Q5 -0.37966987 0.1283279
##   method ranking    estimate    std.err
## 1   aipw      Q1 -0.33408508 0.10595765
## 2   aipw      Q2 -0.15009392 0.10429009
## 3   aipw      Q3 -0.34474599 0.09885531
## 4   aipw      Q4 -0.06392723 0.10236002
## 5   aipw      Q5 -0.29550808 0.10202735

##                    Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1  0.18227935  0.1766614    0.30223462       0.6014
## Rank 3 - Rank 1  0.02798760  0.1763823    0.87393318       0.9711
## Rank 4 - Rank 1  0.38995614  0.1766542    0.02734492       0.0873
## Rank 5 - Rank 1 -0.03480771  0.1760445    0.84327438       0.9711
##                   Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1  0.1839912  0.1453050     0.2055082       0.4338
## Rank 3 - Rank 1 -0.0106609  0.1450024     0.9413946       0.9501
## Rank 4 - Rank 1  0.2701579  0.1453050     0.0630732       0.1847
## Rank 5 - Rank 1  0.0385770  0.1448031     0.7899385       0.9501
## 
## Best linear projection of the conditional average treatment effect.
## Confidence intervals are cluster- and heteroskedasticity-robust (HC3):
## 
##                                                   Estimate    Std. Error
## (Intercept)                                  -0.1419298812  1.0946307980
## att_check_pre                                 0.1159704840  0.1038611064
## base_rate_pre                                 0.1001001549  0.2297290025
## misinfo_pre                                  -0.3240195695  0.3370385736
## misinfo_avg_acc_score_pre                    -0.0000030224  0.0709572889
## base_avg_acc_score_pre                       -0.0029188315  0.0405111433
## age                                          -0.0068977504  0.0065610071
## gender_Man                                    0.1657769360  0.1059435293
## education_High_school_or_less                 0.2801385381  0.4966978237
## education_Some_college                        0.3350671958  0.4914674628
## education_Bachelor_degree                     0.4114890903  0.5206409515
## marital_Married_or_in_a_domestic_partnership -0.2095493284  0.1317032327
## employment_Employed                           0.3038223816  0.0658123502
## employment_Unemployed                         0.1154671989  0.1118632803
## location_Mostly_urban                        -0.0945538502  0.1091716074
## location_Suburban                            -0.1863160683  0.1314283233
## religion_Christian                            0.1835078136  0.2534799827
## religiosity_Attends                          -0.4485128401  0.2128925251
## social_media_bin_Yes                          0.1652227122  0.6911611343
## social_media_hours                           -0.0005773380  0.0098542433
## social_media_share_80_100                    -0.4439531500  0.3151520725
## social_media_share_60_80                     -0.1591869364  0.1981669740
## social_media_share_40_60                     -0.2578504298  0.1666149218
## social_media_share_20_40                     -0.2260741178  0.1977084054
##                                              t value    Pr(>|t|)    
## (Intercept)                                  -0.1297     0.89684    
## att_check_pre                                 1.1166     0.26424    
## base_rate_pre                                 0.4357     0.66306    
## misinfo_pre                                  -0.9614     0.33643    
## misinfo_avg_acc_score_pre                     0.0000     0.99997    
## base_avg_acc_score_pre                       -0.0721     0.94257    
## age                                          -1.0513     0.29318    
## gender_Man                                    1.5648     0.11773    
## education_High_school_or_less                 0.5640     0.57279    
## education_Some_college                        0.6818     0.49543    
## education_Bachelor_degree                     0.7904     0.42937    
## marital_Married_or_in_a_domestic_partnership -1.5911     0.11168    
## employment_Employed                           4.6165 0.000004039 ***
## employment_Unemployed                         1.0322     0.30204    
## location_Mostly_urban                        -0.8661     0.38649    
## location_Suburban                            -1.4176     0.15639    
## religion_Christian                            0.7240     0.46914    
## religiosity_Attends                          -2.1068     0.03521 *  
## social_media_bin_Yes                          0.2391     0.81108    
## social_media_hours                           -0.0586     0.95328    
## social_media_share_80_100                    -1.4087     0.15901    
## social_media_share_60_80                     -0.8033     0.42186    
## social_media_share_40_60                     -1.5476     0.12181    
## social_media_share_20_40                     -1.1435     0.25292    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## Best linear fit using forest predictions (on held-out data)
## as well as the mean forest prediction as regressors, along
## with one-sided heteroskedasticity-robust (HC3) SEs:
## 
##                                Estimate Std. Error t value       Pr(>t)    
## mean.forest.prediction          1.00219    0.19334  5.1836 0.0000001148 ***
## differential.forest.prediction  0.31195    0.42971  0.7259        0.234    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

## [1] "Outcome: new_acc_post"
##   method ranking    estimate    std.err
## 1    ols      Q1 -0.04709100 0.09849485
## 2    ols      Q2 -0.17630106 0.09186857
## 3    ols      Q3  0.19011754 0.09407689
## 4    ols      Q4  0.20096618 0.09550970
## 5    ols      Q5  0.06691396 0.10224697
##   method ranking    estimate    std.err
## 1   aipw      Q1 -0.04650658 0.09730117
## 2   aipw      Q2 -0.14629080 0.09042132
## 3   aipw      Q3  0.17105342 0.09287513
## 4   aipw      Q4  0.20063949 0.09468110
## 5   aipw      Q5  0.05769630 0.10123190

##                   Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 -0.1292101  0.1365844    0.34420709       0.5399
## Rank 3 - Rank 1  0.2372085  0.1363673    0.08203491       0.2034
## Rank 4 - Rank 1  0.2480572  0.1366271    0.06951795       0.2034
## Rank 5 - Rank 1  0.1140050  0.1361084    0.40230834       0.5399
##                    Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 -0.09978422  0.1349555    0.45972039       0.6506
## Rank 3 - Rank 1  0.21756000  0.1346744    0.10630005       0.2390
## Rank 4 - Rank 1  0.24714608  0.1349555    0.06713553       0.1945
## Rank 5 - Rank 1  0.10420288  0.1344893    0.43850590       0.6506
## 
## Best linear projection of the conditional average treatment effect.
## Confidence intervals are cluster- and heteroskedasticity-robust (HC3):
## 
##                                                Estimate Std. Error t value
## (Intercept)                                  -0.5525604  1.0939851 -0.5051
## att_check_pre                                 0.1135877  0.0767070  1.4808
## base_rate_pre                                -0.0035882  0.1609105 -0.0223
## misinfo_pre                                  -0.2012241  0.2498982 -0.8052
## misinfo_avg_acc_score_pre                     0.0275008  0.0438993  0.6265
## base_avg_acc_score_pre                        0.0612187  0.0243778  2.5113
## age                                          -0.0047431  0.0050831 -0.9331
## gender_Man                                    0.0305991  0.0836328  0.3659
## education_High_school_or_less                 0.6559169  0.4654327  1.4093
## education_Some_college                        0.5616221  0.4379696  1.2823
## education_Bachelor_degree                     0.7096892  0.4578111  1.5502
## marital_Married_or_in_a_domestic_partnership -0.1715625  0.1329637 -1.2903
## employment_Employed                           0.2572582  0.1350384  1.9051
## employment_Unemployed                         0.1276437  0.0985128  1.2957
## location_Mostly_urban                        -0.1264788  0.0642991 -1.9670
## location_Suburban                            -0.0372110  0.1117672 -0.3329
## religion_Christian                            0.1548299  0.2433307  0.6363
## religiosity_Attends                          -0.3737967  0.2075766 -1.8008
## social_media_bin_Yes                          0.3246278  0.7988601  0.4064
## social_media_hours                            0.0064867  0.0103098  0.6292
## social_media_share_80_100                    -0.2850155  0.2196759 -1.2974
## social_media_share_60_80                     -0.0142528  0.1808304 -0.0788
## social_media_share_40_60                     -0.1568226  0.1306773 -1.2001
## social_media_share_20_40                     -0.1635843  0.2191001 -0.7466
##                                              Pr(>|t|)  
## (Intercept)                                   0.61353  
## att_check_pre                                 0.13875  
## base_rate_pre                                 0.98221  
## misinfo_pre                                   0.42074  
## misinfo_avg_acc_score_pre                     0.53106  
## base_avg_acc_score_pre                        0.01207 *
## age                                           0.35083  
## gender_Man                                    0.71448  
## education_High_school_or_less                 0.15884  
## education_Some_college                        0.19981  
## education_Bachelor_degree                     0.12119  
## marital_Married_or_in_a_domestic_partnership  0.19703  
## employment_Employed                           0.05685 .
## employment_Unemployed                         0.19516  
## location_Mostly_urban                         0.04926 *
## location_Suburban                             0.73920  
## religion_Christian                            0.52463  
## religiosity_Attends                           0.07182 .
## social_media_bin_Yes                          0.68450  
## social_media_hours                            0.52927  
## social_media_share_80_100                     0.19456  
## social_media_share_60_80                      0.93718  
## social_media_share_40_60                      0.23019  
## social_media_share_20_40                      0.45534  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## Best linear fit using forest predictions (on held-out data)
## as well as the mean forest prediction as regressors, along
## with one-sided heteroskedasticity-robust (HC3) SEs:
## 
##                                Estimate Std. Error t value  Pr(>t)  
## mean.forest.prediction          0.99820    1.11338  0.8966 0.18501  
## differential.forest.prediction  0.65268    0.35637  1.8315 0.03355 *
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

## [1] "Outcome: new_acc_diff"
##   method ranking    estimate   std.err
## 1    ols      Q1 -0.21126370 0.1435095
## 2    ols      Q2  0.04025740 0.1325621
## 3    ols      Q3  0.09253187 0.1327346
## 4    ols      Q4  0.08668599 0.1285102
## 5    ols      Q5  0.12352480 0.1274207
##   method ranking     estimate    std.err
## 1   aipw      Q1 -0.088270271 0.09784004
## 2   aipw      Q2  0.002379657 0.09518002
## 3   aipw      Q3  0.018650838 0.09499503
## 4   aipw      Q4  0.099264429 0.09205516
## 5   aipw      Q5  0.142832080 0.10161822

##                  Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 0.2515211  0.1882119    0.18151174       0.2401
## Rank 3 - Rank 1 0.3037956  0.1877867    0.10579949       0.2401
## Rank 4 - Rank 1 0.2979497  0.1882576    0.11358420       0.2401
## Rank 5 - Rank 1 0.3347885  0.1876527    0.07449356       0.2121
##                   Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 0.09064993  0.1363893    0.50632245       0.6400
## Rank 3 - Rank 1 0.10692111  0.1361053    0.43216701       0.6400
## Rank 4 - Rank 1 0.18753470  0.1363893    0.16921787       0.3669
## Rank 5 - Rank 1 0.23110235  0.1359182    0.08915947       0.2526
## 
## Best linear projection of the conditional average treatment effect.
## Confidence intervals are cluster- and heteroskedasticity-robust (HC3):
## 
##                                                Estimate Std. Error t value
## (Intercept)                                  -0.6776062  1.1574739 -0.5854
## att_check_pre                                 0.1290602  0.0815149  1.5833
## base_rate_pre                                -0.0216109  0.1582381 -0.1366
## misinfo_pre                                  -0.2140417  0.2478091 -0.8637
## misinfo_avg_acc_score_pre                     0.0362670  0.0447066  0.8112
## base_avg_acc_score_pre                        0.0634147  0.0304084  2.0854
## age                                          -0.0044069  0.0050129 -0.8791
## gender_Man                                    0.0433297  0.0861710  0.5028
## education_High_school_or_less                 0.7210246  0.4945518  1.4579
## education_Some_college                        0.6272799  0.4714018  1.3307
## education_Bachelor_degree                     0.7770022  0.4864006  1.5975
## marital_Married_or_in_a_domestic_partnership -0.1689362  0.1372648 -1.2307
## employment_Employed                           0.2876601  0.1389480  2.0703
## employment_Unemployed                         0.1433652  0.0974318  1.4714
## location_Mostly_urban                        -0.1298080  0.0559078 -2.3218
## location_Suburban                            -0.0400793  0.1053688 -0.3804
## religion_Christian                            0.1278198  0.2339282  0.5464
## religiosity_Attends                          -0.3289945  0.2178474 -1.5102
## social_media_bin_Yes                          0.3396574  0.8125363  0.4180
## social_media_hours                            0.0068034  0.0104379  0.6518
## social_media_share_80_100                    -0.2954999  0.2105230 -1.4036
## social_media_share_60_80                     -0.0077632  0.1819758 -0.0427
## social_media_share_40_60                     -0.1891195  0.1298141 -1.4568
## social_media_share_20_40                     -0.1666253  0.2126076 -0.7837
##                                              Pr(>|t|)  
## (Intercept)                                    0.5583  
## att_check_pre                                  0.1134  
## base_rate_pre                                  0.8914  
## misinfo_pre                                    0.3878  
## misinfo_avg_acc_score_pre                      0.4173  
## base_avg_acc_score_pre                         0.0371 *
## age                                            0.3794  
## gender_Man                                     0.6151  
## education_High_school_or_less                  0.1449  
## education_Some_college                         0.1834  
## education_Bachelor_degree                      0.1103  
## marital_Married_or_in_a_domestic_partnership   0.2185  
## employment_Employed                            0.0385 *
## employment_Unemployed                          0.1413  
## location_Mostly_urban                          0.0203 *
## location_Suburban                              0.7037  
## religion_Christian                             0.5848  
## religiosity_Attends                            0.1311  
## social_media_bin_Yes                           0.6760  
## social_media_hours                             0.5146  
## social_media_share_80_100                      0.1605  
## social_media_share_60_80                       0.9660  
## social_media_share_40_60                       0.1452  
## social_media_share_20_40                       0.4333  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## Best linear fit using forest predictions (on held-out data)
## as well as the mean forest prediction as regressors, along
## with one-sided heteroskedasticity-robust (HC3) SEs:
## 
##                                Estimate Std. Error t value Pr(>t)
## mean.forest.prediction          1.00868    1.54937  0.6510 0.2575
## differential.forest.prediction  0.43297    0.39756  1.0891 0.1381

Subgroup Analysis

We conducted subgroup analysis with Romano-Wolf correction in separate scripts (see issue). In this script, we summarize the results by reporting the treatment effect (difference between Emotions course and Facts Baseline) on each outcome of interest for each subgroup.

The outcomes we evaluate are our two main outcomes (special outcome and sharing discernment), plus the outcomes that we expect to find heterogeneity based on tree analysis above. The outcomes are:

Pre-specified misinfo sharing rate (special outcome)
Post misinfo sharing rate
Post non-misinfo sharing rate
Post - pre misinfo sharing rate
Post - pre non-misinfo sharing rate
Post sharing discernment

The subgroups we evaluate are based on the results of the subgroup analysis with Romano-Wolf correction (so can use those p-values). The subgroups are:

Above/below median pre misinfo sharing rate
Above/below median pre non-misinfo sharing rate
Above/below median pre total misinfo accuracy score

# Create a pipe that takes the data, filters to the two groups of interest (Emotions, Facts Baseline), and redefines the treatment variable equal to 1 if treatment is Emotions, 0 if treatment is Facts Baseline; the pipe then computes the median of misinfo_pre, base_rate_pre, and misinfo_total_acc_score_pre and creates three new variables, where each variable = 1 if the value is above the median, 0 if the value is below the median, then output the number of missing observations and the sum of the new variables
df_subgroup <- data %>% 
  mutate(misinfo_pre_above_med = ifelse(misinfo_pre > median(misinfo_pre, na.rm = TRUE), 1, 0),
         base_rate_pre_above_med = ifelse(base_rate_pre > median(base_rate_pre, na.rm = TRUE), 1, 0),
         misinfo_total_acc_score_pre_above_med = ifelse(misinfo_total_acc_score_pre > median(misinfo_total_acc_score_pre, na.rm = TRUE), 1, 0)) 

# report facts about the subgroups to make sure everything looks ok
temp <- df_subgroup %>% 
  summarize(
    median(misinfo_pre, na.rm = TRUE),
    median(base_rate_pre, na.rm = TRUE),
    median(misinfo_total_acc_score_pre, na.rm = TRUE),
    missing_misinfo_pre = sum(is.na(misinfo_pre)),
    missing_base_rate_pre = sum(is.na(base_rate_pre)),
    missing_misinfo_total_acc_score_pre = sum(is.na(misinfo_total_acc_score_pre)),
    sum_misinfo_pre_above_med = sum(misinfo_pre_above_med),
    sum_base_rate_pre_above_med = sum(base_rate_pre_above_med),
    sum_misinfo_total_acc_score_pre_above_med = sum(misinfo_total_acc_score_pre_above_med)
  )

 
# temp <- df_subgroup %>%
#   summarize(
#     misinfo_pre_above_med = misinfo_pre_above_med,
#     base_rate_pre_above_med = base_rate_pre_above_med,
#     misinfo_total_acc_score_pre_above_med,
#     # Compute the medians for misinfo_pre, base_rate_pre, misinfo_total_acc_score_pre
#     across(c(misinfo_pre, base_rate_pre, misinfo_total_acc_score_pre),
#            ~median(.)),
#     # Calculate the sum of missing values for certain columns
#     across(c(misinfo_pre_above_med, base_rate_pre_above_med, misinfo_total_acc_score_pre_above_med),
#            ~sum(is.na(.)), .names = "missing_{.col}"),
#     # Calculate the sum of non-missing values for other columns
#     across(c(misinfo_pre_above_med, base_rate_pre_above_med, misinfo_total_acc_score_pre_above_med),
#            list(sum = ~sum(., na.rm = TRUE), count = ~sum(!is.na(.))), .names = "{.fn}_{.col}"),
#     # Calculate the proportion: sum of values / number of non-missing values
#     across(c(misinfo_pre_above_med, base_rate_pre_above_med, misinfo_total_acc_score_pre_above_med),
#            ~sum(., na.rm = TRUE) / sum(!is.na(.)), .names = "prop_{.col}")
#   )

#reformat temp such that the rows are each subgroup variable and the columns are the statistics
# temp <- temp %>%
#   pivot_longer(cols = everything(),
#                names_to = c("variable", ".value"),
#                names_pattern = "(.*_)(.*)") %>%
#   dplyr::select(variable, everything())

kable(temp, caption = "Check that medians make sense", digits = 3)%>%
  kable_styling(bootstrap_options = c("striped", "hover"))

Check that medians make sense
median(misinfo_pre, na.rm = TRUE)	median(base_rate_pre, na.rm = TRUE)	median(misinfo_total_acc_score_pre, na.rm = TRUE)	missing_misinfo_pre	missing_base_rate_pre	missing_misinfo_total_acc_score_pre	sum_misinfo_pre_above_med	sum_base_rate_pre_above_med	sum_misinfo_total_acc_score_pre_above_med
0.5	0.667	0	0	0	0	1672	1622	1786

# create a table that shows the number of users that falls into each possible combination of the "above_med" variables
temp <- df_subgroup %>%
  group_by(misinfo_pre_above_med, base_rate_pre_above_med, misinfo_total_acc_score_pre_above_med) %>%
  summarize(n = n(),
            prop = n / 3626)

kable(temp, caption = "Users by subgroup intersections", digits = 3)%>%
  kable_styling(bootstrap_options = c("striped", "hover"))

Users by subgroup intersections
misinfo_pre_above_med	base_rate_pre_above_med	misinfo_total_acc_score_pre_above_med	n	prop
0	0	0	1123	0.310
0	0	1	352	0.097
0	1	0	353	0.097
0	1	1	126	0.035
1	0	0	133	0.037
1	0	1	396	0.109
1	1	0	231	0.064
1	1	1	912	0.252

# Initialize an empty data frame or NULL object to store the combined results
combined_results <- NULL

# Loop over each grouping variable
for(grouping_var in c("misinfo_pre_above_med", "base_rate_pre_above_med", "misinfo_total_acc_score_pre_above_med")) {
  # Calculate the mean and standard error for each outcome by treatment group
  temp_results <- df_subgroup %>%
    group_by(!!sym(grouping_var), treatment) %>%
    summarize(across(all_of(outcomes_subgroup),
                     list(mean = ~mean(.x, na.rm = TRUE),
                          se = ~sd(.x, na.rm = TRUE) / sqrt(sum(!is.na(.x)))),
                     .names = "{.col}_{.fn}")) %>%
    pivot_longer(cols = ends_with(c("_mean", "_se")),
                 names_to = c("outcome", ".value"),
                 names_pattern = "(.*)_(mean|se)") %>%
    pivot_wider(names_from = treatment, values_from = c(mean, se), names_sep = "_") %>%
    mutate(diff_mean = mean_1 - mean_0,
           diff_se = sqrt(se_1^2 + se_0^2)) %>%
    # reorder rows by grouping first on outcome, then on grouping_var
    arrange(outcome, !!sym(grouping_var))

  # Append the current results as rows
  combined_results <- bind_rows(combined_results, temp_results)
}

combined_results <- combined_results %>% 
  mutate(
    # calculate the ratio of the treatment effect to the baseline group mean
    ratio = diff_mean / mean_0
  ) %>% 
dplyr::select(
  misinfo_pre_above_med,
  base_rate_pre_above_med,
  misinfo_total_acc_score_pre_above_med,
  mean_0,
  se_0,
  mean_1,
  se_1,
  diff_mean,
  diff_se,
  ratio
)

# make a kable table of combined_results
kable(combined_results, caption = "Subgroup analysis results", digits = 3)%>%
  kable_styling(bootstrap_options = c("striped", "hover"))

Subgroup analysis results
misinfo_pre_above_med	base_rate_pre_above_med	misinfo_total_acc_score_pre_above_med	mean_0	se_0	mean_1	se_1	diff_mean	diff_se	ratio
0	NA	NA	-0.052	0.013	-0.104	0.013	-0.052	0.019	0.991
1	NA	NA	-0.104	0.012	-0.244	0.014	-0.141	0.018	1.359
0	NA	NA	0.480	0.012	0.430	0.011	-0.050	0.017	-0.104
1	NA	NA	0.759	0.011	0.622	0.013	-0.137	0.017	-0.181
0	NA	NA	0.027	0.009	-0.028	0.009	-0.056	0.013	-2.035
1	NA	NA	-0.194	0.011	-0.370	0.011	-0.176	0.016	0.909
0	NA	NA	0.309	0.009	0.242	0.008	-0.067	0.012	-0.217
1	NA	NA	0.651	0.011	0.480	0.011	-0.170	0.016	-0.262
0	NA	NA	0.170	0.010	0.188	0.010	0.017	0.014	0.101
1	NA	NA	0.108	0.010	0.142	0.010	0.033	0.014	0.306
0	NA	NA	0.434	0.015	0.334	0.014	-0.100	0.020	-0.230
1	NA	NA	0.686	0.013	0.517	0.013	-0.169	0.019	-0.246
NA	0	NA	0.044	0.012	0.001	0.012	-0.043	0.017	-0.970
NA	1	NA	-0.231	0.011	-0.372	0.012	-0.142	0.017	0.614
NA	0	NA	0.481	0.012	0.428	0.011	-0.053	0.017	-0.111
NA	1	NA	0.769	0.011	0.628	0.012	-0.142	0.017	-0.184
NA	0	NA	-0.068	0.010	-0.128	0.010	-0.061	0.014	0.898
NA	1	NA	-0.081	0.011	-0.258	0.013	-0.176	0.017	2.168
NA	0	NA	0.324	0.010	0.257	0.008	-0.067	0.013	-0.207
NA	1	NA	0.647	0.012	0.467	0.012	-0.180	0.016	-0.278
NA	0	NA	0.157	0.010	0.171	0.010	0.014	0.014	0.087
NA	1	NA	0.123	0.010	0.161	0.011	0.038	0.014	0.311
NA	0	NA	0.459	0.016	0.355	0.015	-0.104	0.021	-0.227
NA	1	NA	0.671	0.012	0.500	0.013	-0.171	0.018	-0.254
NA	NA	0	-0.063	0.013	-0.150	0.014	-0.087	0.019	1.374
NA	NA	1	-0.089	0.012	-0.190	0.014	-0.101	0.018	1.131
NA	NA	0	0.509	0.013	0.449	0.012	-0.060	0.017	-0.118
NA	NA	1	0.709	0.012	0.590	0.012	-0.119	0.017	-0.168
NA	NA	0	-0.015	0.010	-0.086	0.010	-0.071	0.014	4.764
NA	NA	1	-0.135	0.011	-0.291	0.011	-0.156	0.016	1.156
NA	NA	0	0.342	0.010	0.270	0.009	-0.072	0.014	-0.211
NA	NA	1	0.594	0.011	0.438	0.011	-0.156	0.016	-0.263
NA	NA	0	0.168	0.010	0.180	0.010	0.012	0.015	0.072
NA	NA	1	0.115	0.009	0.153	0.010	0.037	0.014	0.321
NA	NA	0	0.454	0.015	0.361	0.014	-0.093	0.020	-0.204
NA	NA	1	0.662	0.013	0.492	0.014	-0.170	0.019	-0.257

Causal Tree

Function

This code is copy/paste from the HTE tutorial, including defining the minimum tree size as 1. It defines a function that takes outcome, covariates, and treatment as arguments and returns a causal tree diagram.

# define a function that takes outcome, covariates, and treatment as arguments and returns a causal tree diagram
causal_tree <- function(outcome, covariates, treatment, data) {

fmla <- paste(outcome, " ~", paste(covariates, collapse = " + "))

# Dividing data into three subsets
indices <- split(seq(nrow(data)), sort(seq(nrow(data)) %% 3))
names(indices) <- c('split', 'est', 'test')

# Fitting the forest
ct.unpruned <- honest.causalTree(
  formula=fmla,            # Define the model
  data=data[indices$split,],
  treatment=data[indices$split, treatment],
  est_data=data[indices$est,],
  est_treatment=data[indices$est, treatment],
  minsize=1,                 # Min. number of treatment and control cases in each leaf
  HonestSampleSize=length(indices$est), #  Num obs used in estimation after splitting
  # We recommend not changing the parameters below
  split.Rule="CT",            # Define the splitting option
  cv.option="TOT",            # Cross validation options
  cp=0,                       # Complexity parameter
  split.Honest=TRUE,          # Use honesty when splitting
  cv.Honest=TRUE              # Use honesty when performing cross-validation
)

# Table of cross-validated values by tuning parameter.
ct.cptable <- as.data.frame(ct.unpruned$cptable)

# Obtain optimal complexity parameter to prune tree.
cp.selected <- which.min(ct.cptable$xerror)
cp.optimal <- ct.cptable[cp.selected, "CP"]

# Prune the tree at optimal complexity parameter.
ct.pruned <- prune(tree=ct.unpruned, cp=cp.optimal)

# Predict point estimates (on estimation sample)
tau.hat.est <- predict(ct.pruned, newdata=data[indices$est,])

# Create a factor column 'leaf' indicating leaf assignment in the estimation set
num.leaves <- length(unique(tau.hat.est))
leaf <- factor(tau.hat.est, levels=sort(unique(tau.hat.est)), labels = seq(num.leaves))

# Output diagram with a caption that reads "outcome_covariates"
rpart.plot(
  x=ct.pruned,        # Pruned tree
  type=3,             # Draw separate split labels for the left and right directions
  fallen=TRUE,        # Position the leaf nodes at the bottom of the graph
  leaf.round=1,       # Rounding of the corners of the leaf node boxes
  extra=100,          # Display the percentage of observations in the node
  branch=.1,          # Shape of the branch lines
  box.palette="RdBu") # Palette for coloring the node

# Add caption using title function
title(paste("Tree for", outcome), line = -2, cex.main = 0.8)

fmla <- paste0(outcome, ' ~ ', paste0(treatment, '* leaf'))
if (num.leaves == 1) {
  print("Skipping since there's a single leaf.")

} else if (num.leaves == 2) {
  # if there are only two leaves, no need to correct for multiple hypotheses
  ols <- lm(fmla, data=transform(data[indices$est,], leaf=leaf))
  print(coeftest(ols, vcov=vcovHC(ols, 'HC2'))[4,,drop=F])

} else if (num.leaves > 5) {
  print("Skipping since it breaks with 6 leaves.")
}else {
  # if there are three or more leaves, use Romano-Wolf test correction
  ols <- lm(fmla, data=transform(data[indices$est,], leaf=leaf))
  interact <- which(sapply(names(coef(ols)), function(x) grepl(paste0(treatment, ":"), x)))
  print(summary_rw_lm(ols, indices=interact, cov.type = 'HC2'))
}

if (num.leaves == 1) {
  print("Skipping since there's a single leaf.")

} else {
  df <- mapply(function(covariate) {
      # Looping over covariate names
      # Compute average covariate value per leaf (with correct standard errors)
      fmla <- formula(paste0(covariate, "~ 0 + leaf"))
      ols <- lm(fmla, data=transform(data[indices$est,], leaf=leaf))
      ols.res <- coeftest(ols, vcov=vcovHC(ols, "HC2"))

      # Retrieve results
      avg <- ols.res[,1]
      stderr <- ols.res[,2]

      # Tally up results
      data.frame(covariate, avg, stderr, leaf=paste0("Leaf", seq(num.leaves)),
                 # Used for coloring
                 scaling=pnorm((avg - mean(avg))/sd(avg)),
                 # We will order based on how much variation is 'explain' by the averages
                 # relative to the total variation of the covariate in the data
                 variation=sd(avg) / sd(data[,covariate]),
                 # String to print in each cell in heatmap below
                 labels=paste0(signif(avg, 3), "\n", "(", signif(stderr, 3), ")"))
}, covariates, SIMPLIFY = FALSE)
df <- do.call(rbind, df)

# a small optional trick to ensure heatmap will be in decreasing order of 'variation'
df$covariate <- reorder(df$covariate, order(df$variation))

# plot heatmap
heatmap <- ggplot(df) +
    aes(leaf, covariate) +
    geom_tile(aes(fill = scaling)) + 
    geom_text(aes(label = labels)) +
    scale_fill_gradient(low = "#E1BE6A", high = "#40B0A6") +
    ggtitle(paste0("Average covariate values within each leaf")) +
    theme_minimal() + 
    ylab("") + xlab("") +
    theme(plot.title = element_text(size = 12, face = "bold"),
          axis.text=element_text(size=11))

print(heatmap)

}

}

Main outcomes & covariate sets

# Apply the causal tree function across interesting outcomes and covariate sets
for (outcome in outcomes_int) {
  for (covariates in list(covariates_1, covariates_5)) { # Define interesting covariate sets
    causal_tree(outcome, covariates, treatment, data)
  }
}

## [1] 2
## [1] "CT"

##                   Estimate Std. Error Orig. p-value Adj. p-value
## treatment:leaf2 0.09362637 0.04529014    0.03892339       0.0711
## treatment:leaf3 0.09783580 0.07318237    0.18151565       0.1797

## [1] 2
## [1] "CT"

##                  Estimate Std. Error  t value   Pr(>|t|)
## treatment:leaf2 0.1002619 0.04408018 2.274535 0.02310897

## [1] 2
## [1] "CT"

## [1] "Skipping since it breaks with 6 leaves."

## [1] 2
## [1] "CT"

## [1] "Skipping since there's a single leaf."
## [1] "Skipping since there's a single leaf."
## [1] 2
## [1] "CT"

##                  Estimate Std. Error t value    Pr(>|t|)
## treatment:leaf2 0.1029808 0.03926832 2.62249 0.008839064

## [1] 2
## [1] "CT"

##                  Estimate Std. Error t value    Pr(>|t|)
## treatment:leaf2 0.1029808 0.03926832 2.62249 0.008839064

Appendix

Causal forest for all outcomes & covariate sets

# Execute causal forest function on all outcomes & covariate sets
for (outcome in outcomes) {
  if (outcome == "pre_spec_outcome_rates") {
    print(paste("Outcome:", outcome))
    print("Outcome has missing values.")
  } else {
     for (covariate_set_name in names(covariate_sets)) {
      covariates <- covariate_sets[[covariate_set_name]]

      # Print Outcome and Covariates at the top of each loop
      print(paste("Outcome:", outcome))
      print(paste("Covariates:", covariate_set_name))

      file_name <- sprintf("~/GitHub/First_Draft/main_analysis/new_analyses_for_submission/generated_figures/%s_%s_heatmap_plot.png", outcome, covariate_set_name)
      causal_forest_analysis(XX, outcome, W, covariates, file_name, W.hat = 0.5)
     }
  }
}

## [1] "Outcome: pre_spec_outcome_rates"
## [1] "Outcome has missing values."
## [1] "Outcome: base_rate_post"
## [1] "Covariates: covariates_1"
##   method ranking    estimate    std.err
## 1    ols      Q1 -0.14853994 0.02355155
## 2    ols      Q2 -0.14204282 0.02822249
## 3    ols      Q3 -0.10560468 0.02601860
## 4    ols      Q4 -0.03381313 0.02775570
## 5    ols      Q5 -0.01564362 0.02750732
##   method ranking    estimate    std.err
## 1   aipw      Q1 -0.14790479 0.02273349
## 2   aipw      Q2 -0.14638157 0.02738873
## 3   aipw      Q3 -0.10826267 0.02571893
## 4   aipw      Q4 -0.02331118 0.02502218
## 5   aipw      Q5 -0.01731005 0.02535625

##                    Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 0.006497119 0.03725378  0.8615596230       0.8584
## Rank 3 - Rank 1 0.042935254 0.03618867  0.2355312085       0.4013
## Rank 4 - Rank 1 0.114726808 0.03618720  0.0015351743       0.0048
## Rank 5 - Rank 1 0.132896322 0.03677859  0.0003063023       0.0010
##                    Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 0.001523222 0.03526120  0.9655458888       0.9649
## Rank 3 - Rank 1 0.039642124 0.03425042  0.2471770578       0.4024
## Rank 4 - Rank 1 0.124593610 0.03423761  0.0002774259       0.0011
## Rank 5 - Rank 1 0.130594746 0.03480344  0.0001779290       0.0011
## 
## Best linear projection of the conditional average treatment effect.
## Confidence intervals are cluster- and heteroskedasticity-robust (HC3):
## 
##                Estimate Std. Error t value Pr(>|t|)   
## (Intercept)    0.016340   0.024123  0.6773 0.498233   
## att_check_pre  0.011430   0.013946  0.8196 0.412493   
## base_rate_pre -0.058408   0.034532 -1.6914 0.090841 . 
## misinfo_pre   -0.133648   0.050065 -2.6695 0.007631 **
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## Best linear fit using forest predictions (on held-out data)
## as well as the mean forest prediction as regressors, along
## with one-sided heteroskedasticity-robust (HC3) SEs:
## 
##                                Estimate Std. Error t value           Pr(>t)    
## mean.forest.prediction          0.96968    0.14552  6.6637 0.00000000001537 ***
## differential.forest.prediction  0.52661    0.17767  2.9639         0.001528 ** 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

## [1] "Outcome: base_rate_post"
## [1] "Covariates: covariates_2"
##   method ranking     estimate    std.err
## 1    ols      Q1 -0.098039216 0.02680256
## 2    ols      Q2 -0.100884525 0.02751469
## 3    ols      Q3 -0.108780217 0.02898114
## 4    ols      Q4 -0.130722233 0.02750314
## 5    ols      Q5 -0.003568803 0.02781857
##   method ranking     estimate    std.err
## 1   aipw      Q1 -0.099368306 0.02629478
## 2   aipw      Q2 -0.090271620 0.02708085
## 3   aipw      Q3 -0.106507599 0.02799189
## 4   aipw      Q4 -0.129801115 0.02644746
## 5   aipw      Q5 -0.006835109 0.02730416

##                     Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 -0.002845309 0.03907368     0.9419541       0.9490
## Rank 3 - Rank 1 -0.010741001 0.03899032     0.7829643       0.9490
## Rank 4 - Rank 1 -0.032683017 0.03898948     0.4019444       0.7420
## Rank 5 - Rank 1  0.094470413 0.03893278     0.0152938       0.0575
##                     Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1  0.009096685 0.03807536    0.81118683       0.9596
## Rank 3 - Rank 1 -0.007139293 0.03799411    0.85096142       0.9596
## Rank 4 - Rank 1 -0.030432810 0.03799411    0.42319134       0.7667
## Rank 5 - Rank 1  0.092533196 0.03794059    0.01477988       0.0537
## 
## Best linear projection of the conditional average treatment effect.
## Confidence intervals are cluster- and heteroskedasticity-robust (HC3):
## 
##                             Estimate Std. Error t value  Pr(>|t|)    
## (Intercept)               -0.0809624  0.0212013 -3.8187 0.0001364 ***
## att_check_pre              0.0122362  0.0191742  0.6382 0.5234086    
## misinfo_avg_acc_score_pre -0.0188651  0.0087466 -2.1569 0.0310824 *  
## base_avg_acc_score_pre    -0.0104634  0.0078389 -1.3348 0.1820225    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## Best linear fit using forest predictions (on held-out data)
## as well as the mean forest prediction as regressors, along
## with one-sided heteroskedasticity-robust (HC3) SEs:
## 
##                                Estimate Std. Error t value        Pr(>t)    
## mean.forest.prediction          0.96859    0.17740  5.4600 0.00000002541 ***
## differential.forest.prediction  0.22840    0.14341  1.5927       0.05566 .  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

## [1] "Outcome: base_rate_post"
## [1] "Covariates: covariates_3"
##   method ranking    estimate    std.err
## 1    ols      Q1 -0.14724281 0.02454503
## 2    ols      Q2 -0.12804353 0.02693084
## 3    ols      Q3 -0.05297672 0.02705781
## 4    ols      Q4 -0.04602843 0.02773135
## 5    ols      Q5 -0.07317636 0.02635142
##   method ranking    estimate    std.err
## 1   aipw      Q1 -0.13858700 0.02442435
## 2   aipw      Q2 -0.13779108 0.02608228
## 3   aipw      Q3 -0.05591138 0.02607381
## 4   aipw      Q4 -0.04826738 0.02596903
## 5   aipw      Q5 -0.05737567 0.02417197

##                   Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 0.01919928 0.03751004    0.60879158       0.6096
## Rank 3 - Rank 1 0.09426609 0.03747208    0.01192479       0.0342
## Rank 4 - Rank 1 0.10121438 0.03753763    0.00704300       0.0272
## Rank 5 - Rank 1 0.07406645 0.03740623    0.04777286       0.0937
##                     Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 0.0007959126 0.03581871    0.98227323       0.9821
## Rank 3 - Rank 1 0.0826756146 0.03576890    0.02086761       0.0594
## Rank 4 - Rank 1 0.0903196136 0.03584380    0.01178446       0.0458
## Rank 5 - Rank 1 0.0812113282 0.03571956    0.02304948       0.0594
## 
## Best linear projection of the conditional average treatment effect.
## Confidence intervals are cluster- and heteroskedasticity-robust (HC3):
## 
##                             Estimate Std. Error t value Pr(>|t|)   
## (Intercept)                0.0352079  0.0291687  1.2070 0.227493   
## att_check_pre              0.0140082  0.0130911  1.0701 0.284666   
## base_rate_pre             -0.0664040  0.0447190 -1.4849 0.137653   
## misinfo_pre               -0.1674219  0.0539436 -3.1036 0.001926 **
## misinfo_avg_acc_score_pre  0.0109447  0.0113493  0.9643 0.334936   
## base_avg_acc_score_pre     0.0056903  0.0077458  0.7346 0.462618   
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## Best linear fit using forest predictions (on held-out data)
## as well as the mean forest prediction as regressors, along
## with one-sided heteroskedasticity-robust (HC3) SEs:
## 
##                                Estimate Std. Error t value        Pr(>t)    
## mean.forest.prediction          0.98174    0.17489  5.6135 0.00000001066 ***
## differential.forest.prediction  0.51504    0.18970  2.7151      0.003329 ** 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

## [1] "Outcome: base_rate_post"
## [1] "Covariates: covariates_4"
##   method ranking    estimate    std.err
## 1    ols      Q1 -0.06394014 0.02785913
## 2    ols      Q2 -0.09516747 0.02829951
## 3    ols      Q3 -0.06847393 0.02816563
## 4    ols      Q4 -0.08379705 0.02800281
## 5    ols      Q5 -0.12390135 0.02772747
##   method ranking    estimate    std.err
## 1   aipw      Q1 -0.06459118 0.02744512
## 2   aipw      Q2 -0.10335809 0.02780019
## 3   aipw      Q3 -0.06908676 0.02734246
## 4   aipw      Q4 -0.08326081 0.02686723
## 5   aipw      Q5 -0.12506485 0.02735314

##                     Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 -0.031227327 0.03961047     0.4305376       0.7639
## Rank 3 - Rank 1 -0.004533783 0.03950873     0.9086465       0.9063
## Rank 4 - Rank 1 -0.019856907 0.03962950     0.6163567       0.8290
## Rank 5 - Rank 1 -0.059961207 0.03951290     0.1292255       0.3599
##                     Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 -0.038766909 0.03870541     0.3166092       0.6176
## Rank 3 - Rank 1 -0.004495574 0.03861149     0.9073174       0.9058
## Rank 4 - Rank 1 -0.018669632 0.03871896     0.6297050       0.8440
## Rank 5 - Rank 1 -0.060473672 0.03857172     0.1170095       0.3270
## 
## Best linear projection of the conditional average treatment effect.
## Confidence intervals are cluster- and heteroskedasticity-robust (HC3):
## 
##                                                Estimate Std. Error t value
## (Intercept)                                   0.0352611  0.2230270  0.1581
## age                                          -0.0018647  0.0014371 -1.2976
## gender_Man                                   -0.0080763  0.0209845 -0.3849
## education_High_school_or_less                -0.1802140  0.1289437 -1.3976
## education_Some_college                       -0.1821839  0.1255877 -1.4507
## education_Bachelor_degree                    -0.1656282  0.1294762 -1.2792
## marital_Married_or_in_a_domestic_partnership -0.0380763  0.0444338 -0.8569
## employment_Employed                           0.0134785  0.0407416  0.3308
## employment_Unemployed                         0.0015623  0.0344425  0.0454
## location_Mostly_urban                         0.0656664  0.0281620  2.3317
## location_Suburban                             0.0141845  0.0296079  0.4791
## religion_Christian                           -0.0373363  0.0467937 -0.7979
## religiosity_Attends                          -0.1197101  0.0695501 -1.7212
## social_media_bin_Yes                          0.3142159  0.2512566  1.2506
## social_media_hours                           -0.0041340  0.0036888 -1.1207
## social_media_share_80_100                    -0.1237790  0.0423610 -2.9220
## social_media_share_60_80                     -0.0361335  0.0371204 -0.9734
## social_media_share_40_60                     -0.0890112  0.0294763 -3.0198
## social_media_share_20_40                     -0.0377525  0.0524557 -0.7197
##                                              Pr(>|t|)   
## (Intercept)                                  0.874385   
## age                                          0.194510   
## gender_Man                                   0.700355   
## education_High_school_or_less                0.162314   
## education_Some_college                       0.146964   
## education_Bachelor_degree                    0.200903   
## marital_Married_or_in_a_domestic_partnership 0.391544   
## employment_Employed                          0.740793   
## employment_Unemployed                        0.963822   
## location_Mostly_urban                        0.019769 * 
## location_Suburban                            0.631911   
## religion_Christian                           0.424985   
## religiosity_Attends                          0.085299 . 
## social_media_bin_Yes                         0.211170   
## social_media_hours                           0.262491   
## social_media_share_80_100                    0.003499 **
## social_media_share_60_80                     0.330413   
## social_media_share_40_60                     0.002547 **
## social_media_share_20_40                     0.471755   
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## Best linear fit using forest predictions (on held-out data)
## as well as the mean forest prediction as regressors, along
## with one-sided heteroskedasticity-robust (HC3) SEs:
## 
##                                Estimate Std. Error t value              Pr(>t)
## mean.forest.prediction          1.00353    0.13363  7.5095 0.00000000000003714
## differential.forest.prediction -0.42772    0.36719 -1.1648              0.8779
##                                   
## mean.forest.prediction         ***
## differential.forest.prediction    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

## [1] "Outcome: base_rate_post"
## [1] "Covariates: covariates_5"
##   method ranking    estimate    std.err
## 1    ols      Q1 -0.17385151 0.02453046
## 2    ols      Q2 -0.09802466 0.02564743
## 3    ols      Q3 -0.08727766 0.02681808
## 4    ols      Q4 -0.08085317 0.02740247
## 5    ols      Q5 -0.02044403 0.02654639
##   method ranking    estimate    std.err
## 1   aipw      Q1 -0.16614765 0.02413100
## 2   aipw      Q2 -0.10428775 0.02468032
## 3   aipw      Q3 -0.08166610 0.02571829
## 4   aipw      Q4 -0.08613617 0.02553416
## 5   aipw      Q5 -0.01781877 0.02461450

##                   Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 0.07582686 0.03712201 0.04116056657       0.0432
## Rank 3 - Rank 1 0.08657386 0.03705506 0.01952699657       0.0374
## Rank 4 - Rank 1 0.09299834 0.03714280 0.01233028152       0.0323
## Rank 5 - Rank 1 0.15340748 0.03697868 0.00003423012       0.0000
##                   Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 0.06185990 0.03527575 0.07958200272       0.0822
## Rank 3 - Rank 1 0.08448155 0.03520229 0.01645030125       0.0463
## Rank 4 - Rank 1 0.08001148 0.03527575 0.02337737119       0.0463
## Rank 5 - Rank 1 0.14832888 0.03515390 0.00002509419       0.0000
## 
## Best linear projection of the conditional average treatment effect.
## Confidence intervals are cluster- and heteroskedasticity-robust (HC3):
## 
##                                                 Estimate  Std. Error t value
## (Intercept)                                   0.07694625  0.22734016  0.3385
## att_check_pre                                 0.01123651  0.01325661  0.8476
## base_rate_pre                                -0.05542657  0.04665348 -1.1880
## misinfo_pre                                  -0.14955601  0.05580200 -2.6801
## misinfo_avg_acc_score_pre                     0.01041952  0.01093895  0.9525
## base_avg_acc_score_pre                        0.00254069  0.00741353  0.3427
## age                                          -0.00092705  0.00186421 -0.4973
## gender_Man                                    0.01418264  0.01891239  0.7499
## education_High_school_or_less                -0.12719749  0.12123410 -1.0492
## education_Some_college                       -0.13105728  0.12394638 -1.0574
## education_Bachelor_degree                    -0.13819155  0.12331087 -1.1207
## marital_Married_or_in_a_domestic_partnership -0.03668158  0.03867500 -0.9485
## employment_Employed                           0.03103004  0.03552960  0.8734
## employment_Unemployed                        -0.00067696  0.02978021 -0.0227
## location_Mostly_urban                         0.02666252  0.02182090  1.2219
## location_Suburban                            -0.01633356  0.02235757 -0.7306
## religion_Christian                           -0.01343601  0.05539866 -0.2425
## religiosity_Attends                          -0.09130341  0.05034002 -1.8137
## social_media_bin_Yes                          0.20998147  0.23095675  0.9092
## social_media_hours                           -0.00186877  0.00324167 -0.5765
## social_media_share_80_100                    -0.05265757  0.03052577 -1.7250
## social_media_share_60_80                      0.01428201  0.03773469  0.3785
## social_media_share_40_60                     -0.02654009  0.03937958 -0.6740
## social_media_share_20_40                     -0.00276890  0.04434521 -0.0624
##                                              Pr(>|t|)   
## (Intercept)                                  0.735034   
## att_check_pre                                0.396709   
## base_rate_pre                                0.234893   
## misinfo_pre                                  0.007393 **
## misinfo_avg_acc_score_pre                    0.340900   
## base_avg_acc_score_pre                       0.731837   
## age                                          0.619017   
## gender_Man                                   0.453356   
## education_High_school_or_less                0.294162   
## education_Some_college                       0.290413   
## education_Bachelor_degree                    0.262500   
## marital_Married_or_in_a_domestic_partnership 0.342960   
## employment_Employed                          0.382526   
## employment_Unemployed                        0.981865   
## location_Mostly_urban                        0.221833   
## location_Suburban                            0.465095   
## religion_Christian                           0.808381   
## religiosity_Attends                          0.069802 . 
## social_media_bin_Yes                         0.363315   
## social_media_hours                           0.564324   
## social_media_share_80_100                    0.084610 . 
## social_media_share_60_80                     0.705093   
## social_media_share_40_60                     0.500383   
## social_media_share_20_40                     0.950216   
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## Best linear fit using forest predictions (on held-out data)
## as well as the mean forest prediction as regressors, along
## with one-sided heteroskedasticity-robust (HC3) SEs:
## 
##                                Estimate Std. Error t value          Pr(>t)    
## mean.forest.prediction          0.98867    0.15904  6.2165 0.0000000002831 ***
## differential.forest.prediction  0.81480    0.26483  3.0767        0.001054 ** 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

## [1] "Outcome: misinfo_post"
## [1] "Covariates: covariates_1"
##   method ranking    estimate    std.err
## 1    ols      Q1 -0.22302366 0.02224680
## 2    ols      Q2 -0.11438412 0.02538605
## 3    ols      Q3 -0.10144387 0.02091327
## 4    ols      Q4 -0.05811658 0.02016336
## 5    ols      Q5 -0.05388973 0.02140141
##   method ranking    estimate    std.err
## 1   aipw      Q1 -0.22873459 0.02165387
## 2   aipw      Q2 -0.10898275 0.02449954
## 3   aipw      Q3 -0.11127054 0.02017894
## 4   aipw      Q4 -0.05451900 0.01808849
## 5   aipw      Q5 -0.05356298 0.01993574

##                  Estimate Std. Error    Orig. p-value Adj. p-value
## Rank 2 - Rank 1 0.1086395 0.03141742 0.00055060597713       0.0002
## Rank 3 - Rank 1 0.1215798 0.02983050 0.00004686470287       0.0002
## Rank 4 - Rank 1 0.1649071 0.02965380 0.00000002875396       0.0000
## Rank 5 - Rank 1 0.1691339 0.03056259 0.00000003352146       0.0000
##                  Estimate Std. Error     Orig. p-value Adj. p-value
## Rank 2 - Rank 1 0.1197518 0.02981045 0.000060112667631       0.0001
## Rank 3 - Rank 1 0.1174641 0.02830368 0.000033992387694       0.0001
## Rank 4 - Rank 1 0.1742156 0.02813478 0.000000000659733       0.0000
## Rank 5 - Rank 1 0.1751716 0.02899880 0.000000001690052       0.0000
## 
## Best linear projection of the conditional average treatment effect.
## Confidence intervals are cluster- and heteroskedasticity-robust (HC3):
## 
##                Estimate Std. Error t value  Pr(>|t|)    
## (Intercept)    0.028355   0.016972  1.6707 0.0948678 .  
## att_check_pre -0.017279   0.018665 -0.9258 0.3546299    
## base_rate_pre -0.098186   0.039067 -2.5133 0.0120042 *  
## misinfo_pre   -0.130206   0.039462 -3.2996 0.0009777 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## Best linear fit using forest predictions (on held-out data)
## as well as the mean forest prediction as regressors, along
## with one-sided heteroskedasticity-robust (HC3) SEs:
## 
##                                Estimate Std. Error t value
## mean.forest.prediction         0.993403   0.115463  8.6037
## differential.forest.prediction 0.717095   0.054278 13.2115
##                                               Pr(>t)    
## mean.forest.prediction         < 0.00000000000000022 ***
## differential.forest.prediction < 0.00000000000000022 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

## [1] "Outcome: misinfo_post"
## [1] "Covariates: covariates_2"
##   method ranking    estimate    std.err
## 1    ols      Q1 -0.17372685 0.02428916
## 2    ols      Q2 -0.13489871 0.02441023
## 3    ols      Q3 -0.07320263 0.02342975
## 4    ols      Q4 -0.14175578 0.02311886
## 5    ols      Q5 -0.04217147 0.02295517
##   method ranking    estimate    std.err
## 1   aipw      Q1 -0.16868115 0.02426695
## 2   aipw      Q2 -0.13214880 0.02291394
## 3   aipw      Q3 -0.08093070 0.02306973
## 4   aipw      Q4 -0.14373451 0.02245151
## 5   aipw      Q5 -0.03533262 0.02285746

##                   Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 0.03882814 0.03324677 0.24293328499       0.3991
## Rank 3 - Rank 1 0.10052422 0.03322458 0.00249888446       0.0065
## Rank 4 - Rank 1 0.03197107 0.03345070 0.33925398859       0.3991
## Rank 5 - Rank 1 0.13155538 0.03323529 0.00007693498       0.0004
##                   Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 0.03653235 0.03253408 0.26155664528       0.4189
## Rank 3 - Rank 1 0.08775046 0.03251136 0.00698563753       0.0184
## Rank 4 - Rank 1 0.02494664 0.03273172 0.44601711694       0.4473
## Rank 5 - Rank 1 0.13334853 0.03252271 0.00004219184       0.0002
## 
## Best linear projection of the conditional average treatment effect.
## Confidence intervals are cluster- and heteroskedasticity-robust (HC3):
## 
##                            Estimate Std. Error t value  Pr(>|t|)    
## (Intercept)               -0.071969   0.021404 -3.3624 0.0007808 ***
## att_check_pre             -0.028890   0.020350 -1.4197 0.1557939    
## misinfo_avg_acc_score_pre -0.027279   0.010326 -2.6419 0.0082798 ** 
## base_avg_acc_score_pre    -0.026282   0.011339 -2.3179 0.0205080 *  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## Best linear fit using forest predictions (on held-out data)
## as well as the mean forest prediction as regressors, along
## with one-sided heteroskedasticity-robust (HC3) SEs:
## 
##                                Estimate Std. Error t value              Pr(>t)
## mean.forest.prediction          0.97178    0.13173  7.3772 0.00000000000009966
## differential.forest.prediction  0.41396    0.13840  2.9910              0.0014
##                                   
## mean.forest.prediction         ***
## differential.forest.prediction ** 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

## [1] "Outcome: misinfo_post"
## [1] "Covariates: covariates_3"
##   method ranking    estimate    std.err
## 1    ols      Q1 -0.19402598 0.02376845
## 2    ols      Q2 -0.14301416 0.02419614
## 3    ols      Q3 -0.13189343 0.02189488
## 4    ols      Q4 -0.07762070 0.02020002
## 5    ols      Q5 -0.03032567 0.01833515
##   method ranking    estimate    std.err
## 1   aipw      Q1 -0.20487707 0.02385977
## 2   aipw      Q2 -0.14416387 0.02307518
## 3   aipw      Q3 -0.13626987 0.02086025
## 4   aipw      Q4 -0.08250223 0.01951536
## 5   aipw      Q5 -0.02557793 0.01755453

##                   Estimate Std. Error    Orig. p-value Adj. p-value
## Rank 2 - Rank 1 0.05101182 0.03083854 0.09818225266912       0.0982
## Rank 3 - Rank 1 0.06213255 0.03065131 0.04272782300916       0.0735
## Rank 4 - Rank 1 0.11640528 0.03069551 0.00015172232021       0.0006
## Rank 5 - Rank 1 0.16370030 0.03063008 0.00000009630993       0.0000
##                  Estimate Std. Error     Orig. p-value Adj. p-value
## Rank 2 - Rank 1 0.0607132 0.02985500 0.042064464244374       0.0405
## Rank 3 - Rank 1 0.0686072 0.02970813 0.020978804888971       0.0379
## Rank 4 - Rank 1 0.1223748 0.02974958 0.000039837361825       0.0000
## Rank 5 - Rank 1 0.1792991 0.02968755 0.000000001701616       0.0000
## 
## Best linear projection of the conditional average treatment effect.
## Confidence intervals are cluster- and heteroskedasticity-robust (HC3):
## 
##                             Estimate Std. Error t value Pr(>|t|)  
## (Intercept)                0.0138125  0.0133868  1.0318  0.30223  
## att_check_pre             -0.0179519  0.0165650 -1.0837  0.27856  
## base_rate_pre             -0.0647426  0.0456945 -1.4169  0.15661  
## misinfo_pre               -0.1280021  0.0565745 -2.2625  0.02372 *
## misinfo_avg_acc_score_pre -0.0037441  0.0112324 -0.3333  0.73891  
## base_avg_acc_score_pre    -0.0114312  0.0102901 -1.1109  0.26669  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## Best linear fit using forest predictions (on held-out data)
## as well as the mean forest prediction as regressors, along
## with one-sided heteroskedasticity-robust (HC3) SEs:
## 
##                                Estimate Std. Error t value
## mean.forest.prediction         0.984448   0.114438  8.6024
## differential.forest.prediction 0.728009   0.079298  9.1807
##                                               Pr(>t)    
## mean.forest.prediction         < 0.00000000000000022 ***
## differential.forest.prediction < 0.00000000000000022 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

## [1] "Outcome: misinfo_post"
## [1] "Covariates: covariates_4"
##   method ranking    estimate    std.err
## 1    ols      Q1 -0.10884829 0.02484176
## 2    ols      Q2 -0.07152140 0.02480517
## 3    ols      Q3 -0.05756679 0.02466666
## 4    ols      Q4 -0.13957999 0.02488021
## 5    ols      Q5 -0.18343047 0.02503642
##   method ranking    estimate    std.err
## 1   aipw      Q1 -0.11052501 0.02441296
## 2   aipw      Q2 -0.07398465 0.02326563
## 3   aipw      Q3 -0.06977267 0.02421941
## 4   aipw      Q4 -0.13817436 0.02379809
## 5   aipw      Q5 -0.18364934 0.02369974

##                    Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1  0.03732689 0.03512487    0.28799293       0.4610
## Rank 3 - Rank 1  0.05128150 0.03503023    0.14330237       0.3252
## Rank 4 - Rank 1 -0.03073170 0.03510382    0.38138626       0.4610
## Rank 5 - Rank 1 -0.07458218 0.03498680    0.03309681       0.1080
##                    Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1  0.03654036 0.03378553    0.27952950       0.4831
## Rank 3 - Rank 1  0.04075234 0.03371517    0.22684770       0.4831
## Rank 4 - Rank 1 -0.02764935 0.03378553    0.41319488       0.4831
## Rank 5 - Rank 1 -0.07312433 0.03366882    0.02993012       0.0956
## 
## Best linear projection of the conditional average treatment effect.
## Confidence intervals are cluster- and heteroskedasticity-robust (HC3):
## 
##                                                Estimate Std. Error t value
## (Intercept)                                  -0.0542762  0.1473844 -0.3683
## age                                          -0.0013306  0.0017348 -0.7670
## gender_Man                                    0.0196964  0.0259250  0.7597
## education_High_school_or_less                -0.1640506  0.1099010 -1.4927
## education_Some_college                       -0.1673380  0.0998217 -1.6764
## education_Bachelor_degree                    -0.1539723  0.1037008 -1.4848
## marital_Married_or_in_a_domestic_partnership  0.0026166  0.0448836  0.0583
## employment_Employed                          -0.0342099  0.0407282 -0.8400
## employment_Unemployed                        -0.0071346  0.0387506 -0.1841
## location_Mostly_urban                         0.0728071  0.0290951  2.5024
## location_Suburban                             0.0230796  0.0255812  0.9022
## religion_Christian                           -0.0567273  0.0365927 -1.5502
## religiosity_Attends                          -0.0725851  0.0640186 -1.1338
## social_media_bin_Yes                          0.2683625  0.1049019  2.5582
## social_media_hours                           -0.0019591  0.0040391 -0.4850
## social_media_share_80_100                    -0.0200724  0.0521011 -0.3853
## social_media_share_60_80                     -0.0365937  0.0329528 -1.1105
## social_media_share_40_60                     -0.0552605  0.0281322 -1.9643
## social_media_share_20_40                     -0.0101003  0.0261955 -0.3856
##                                              Pr(>|t|)  
## (Intercept)                                   0.71270  
## age                                           0.44313  
## gender_Man                                    0.44746  
## education_High_school_or_less                 0.13560  
## education_Some_college                        0.09375 .
## education_Bachelor_degree                     0.13769  
## marital_Married_or_in_a_domestic_partnership  0.95351  
## employment_Employed                           0.40099  
## employment_Unemployed                         0.85393  
## location_Mostly_urban                         0.01238 *
## location_Suburban                             0.36700  
## religion_Christian                            0.12117  
## religiosity_Attends                           0.25695  
## social_media_bin_Yes                          0.01056 *
## social_media_hours                            0.62768  
## social_media_share_80_100                     0.70007  
## social_media_share_60_80                      0.26686  
## social_media_share_40_60                      0.04957 *
## social_media_share_20_40                      0.69984  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## Best linear fit using forest predictions (on held-out data)
## as well as the mean forest prediction as regressors, along
## with one-sided heteroskedasticity-robust (HC3) SEs:
## 
##                                Estimate Std. Error t value              Pr(>t)
## mean.forest.prediction          1.00550    0.09300 10.8118 <0.0000000000000002
## differential.forest.prediction -1.12029    0.25663 -4.3654                   1
##                                   
## mean.forest.prediction         ***
## differential.forest.prediction    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

## [1] "Outcome: misinfo_post"
## [1] "Covariates: covariates_5"
##   method ranking   estimate    std.err
## 1    ols      Q1 -0.2357322 0.02388802
## 2    ols      Q2 -0.1014277 0.02404293
## 3    ols      Q3 -0.1155080 0.02094814
## 4    ols      Q4 -0.1078261 0.02044220
## 5    ols      Q5 -0.0178041 0.01797493
##   method ranking    estimate    std.err
## 1   aipw      Q1 -0.23321361 0.02330761
## 2   aipw      Q2 -0.11788353 0.02264000
## 3   aipw      Q3 -0.11416797 0.02008387
## 4   aipw      Q4 -0.10519100 0.01939985
## 5   aipw      Q5 -0.01637678 0.01724433

##                  Estimate Std. Error         Orig. p-value Adj. p-value
## Rank 2 - Rank 1 0.1343045 0.03054853 0.0000113210186655012            0
## Rank 3 - Rank 1 0.1202243 0.03050670 0.0000827062532596802            0
## Rank 4 - Rank 1 0.1279062 0.03056041 0.0000291496033124105            0
## Rank 5 - Rank 1 0.2179281 0.03044247 0.0000000000009818758            0
##                  Estimate Std. Error         Orig. p-value Adj. p-value
## Rank 2 - Rank 1 0.1153301 0.02920917 0.0000801646566744891       0.0002
## Rank 3 - Rank 1 0.1190456 0.02914835 0.0000451992922418982       0.0002
## Rank 4 - Rank 1 0.1280226 0.02920917 0.0000120401201644828       0.0002
## Rank 5 - Rank 1 0.2168368 0.02910828 0.0000000000001166787       0.0000
## 
## Best linear projection of the conditional average treatment effect.
## Confidence intervals are cluster- and heteroskedasticity-robust (HC3):
## 
##                                                 Estimate  Std. Error t value
## (Intercept)                                  -0.02123783  0.13898304 -0.1528
## att_check_pre                                -0.01522139  0.01480233 -1.0283
## base_rate_pre                                -0.05806174  0.04594150 -1.2638
## misinfo_pre                                  -0.12815970  0.05617983 -2.2812
## misinfo_avg_acc_score_pre                    -0.00514533  0.01050047 -0.4900
## base_avg_acc_score_pre                       -0.01349562  0.00982300 -1.3739
## age                                          -0.00062082  0.00143664 -0.4321
## gender_Man                                    0.03186612  0.02122161  1.5016
## education_High_school_or_less                -0.12761767  0.08900998 -1.4337
## education_Some_college                       -0.12342736  0.08000333 -1.5428
## education_Bachelor_degree                    -0.13207465  0.09081061 -1.4544
## marital_Married_or_in_a_domestic_partnership -0.00563928  0.03805553 -0.1482
## employment_Employed                          -0.01348947  0.02916852 -0.4625
## employment_Unemployed                        -0.00469881  0.02864933 -0.1640
## location_Mostly_urban                         0.03947086  0.02625960  1.5031
## location_Suburban                            -0.00410656  0.01940117 -0.2117
## religion_Christian                           -0.02184068  0.03375534 -0.6470
## religiosity_Attends                          -0.04461841  0.03922605 -1.1375
## social_media_bin_Yes                          0.20063985  0.10440436  1.9218
## social_media_hours                            0.00014230  0.00271206  0.0525
## social_media_share_80_100                     0.05336544  0.03689768  1.4463
## social_media_share_60_80                      0.02448457  0.03026608  0.8090
## social_media_share_40_60                      0.00087466  0.03231937  0.0271
## social_media_share_20_40                      0.02787310  0.02772795  1.0052
##                                              Pr(>|t|)  
## (Intercept)                                   0.87856  
## att_check_pre                                 0.30387  
## base_rate_pre                                 0.20638  
## misinfo_pre                                   0.02259 *
## misinfo_avg_acc_score_pre                     0.62416  
## base_avg_acc_score_pre                        0.16956  
## age                                           0.66567  
## gender_Man                                    0.13329  
## education_High_school_or_less                 0.15173  
## education_Some_college                        0.12297  
## education_Bachelor_degree                     0.14592  
## marital_Married_or_in_a_domestic_partnership  0.88220  
## employment_Employed                           0.64377  
## employment_Unemployed                         0.86973  
## location_Mostly_urban                         0.13290  
## location_Suburban                             0.83238  
## religion_Christian                            0.51765  
## religiosity_Attends                           0.25542  
## social_media_bin_Yes                          0.05472 .
## social_media_hours                            0.95816  
## social_media_share_80_100                     0.14818  
## social_media_share_60_80                      0.41858  
## social_media_share_40_60                      0.97841  
## social_media_share_20_40                      0.31485  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## Best linear fit using forest predictions (on held-out data)
## as well as the mean forest prediction as regressors, along
## with one-sided heteroskedasticity-robust (HC3) SEs:
## 
##                                Estimate Std. Error t value
## mean.forest.prediction         0.994649   0.112808  8.8171
## differential.forest.prediction 0.994070   0.094529 10.5160
##                                               Pr(>t)    
## mean.forest.prediction         < 0.00000000000000022 ***
## differential.forest.prediction < 0.00000000000000022 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

## [1] "Outcome: base_rate_diff"
## [1] "Covariates: covariates_1"
##   method ranking    estimate    std.err
## 1    ols      Q1 -0.14519034 0.02435371
## 2    ols      Q2 -0.14630593 0.03159301
## 3    ols      Q3 -0.10401069 0.03000546
## 4    ols      Q4 -0.01418561 0.03075658
## 5    ols      Q5 -0.04430004 0.02938941
##   method ranking     estimate    std.err
## 1   aipw      Q1 -0.156121665 0.02241865
## 2   aipw      Q2 -0.139754248 0.02819840
## 3   aipw      Q3 -0.105180062 0.02519814
## 4   aipw      Q4 -0.007703605 0.02557123
## 5   aipw      Q5 -0.032469935 0.02513025

##                     Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 -0.001115596 0.04110942  0.9783517691       0.9807
## Rank 3 - Rank 1  0.041179650 0.03907318  0.2919941640       0.4782
## Rank 4 - Rank 1  0.131004726 0.03961699  0.0009529637       0.0033
## Rank 5 - Rank 1  0.100890301 0.03999599  0.0116946310       0.0322
##                   Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 0.01636742 0.03554294 0.64518716708       0.6482
## Rank 3 - Rank 1 0.05094160 0.03377890 0.13161889337       0.2342
## Rank 4 - Rank 1 0.14841806 0.03424034 0.00001499986       0.0000
## Rank 5 - Rank 1 0.12365173 0.03457114 0.00035246600       0.0015
## 
## Best linear projection of the conditional average treatment effect.
## Confidence intervals are cluster- and heteroskedasticity-robust (HC3):
## 
##                Estimate Std. Error t value Pr(>|t|)  
## (Intercept)    0.016227   0.023336  0.6954  0.48687  
## att_check_pre  0.011364   0.013609  0.8351  0.40374  
## base_rate_pre -0.061889   0.034268 -1.8061  0.07099 .
## misinfo_pre   -0.129488   0.050738 -2.5521  0.01075 *
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## Best linear fit using forest predictions (on held-out data)
## as well as the mean forest prediction as regressors, along
## with one-sided heteroskedasticity-robust (HC3) SEs:
## 
##                                Estimate Std. Error t value           Pr(>t)    
## mean.forest.prediction          0.98341    0.14848  6.6231 0.00000000002018 ***
## differential.forest.prediction  0.53305    0.17563  3.0351         0.001211 ** 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

## [1] "Outcome: base_rate_diff"
## [1] "Covariates: covariates_2"
##   method ranking    estimate    std.err
## 1    ols      Q1 -0.07523186 0.02818666
## 2    ols      Q2 -0.16017068 0.03024591
## 3    ols      Q3 -0.11765946 0.02887452
## 4    ols      Q4 -0.07530646 0.03012587
## 5    ols      Q5 -0.03794385 0.03001894
##   method ranking    estimate    std.err
## 1   aipw      Q1 -0.06602621 0.02829491
## 2   aipw      Q2 -0.13909015 0.02983567
## 3   aipw      Q3 -0.11582087 0.02813657
## 4   aipw      Q4 -0.07723254 0.02912690
## 5   aipw      Q5 -0.03981884 0.02900978

##                       Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 -0.08493882201 0.04163146    0.04139737       0.1318
## Rank 3 - Rank 1 -0.04242759734 0.04137624    0.30523849       0.6217
## Rank 4 - Rank 1 -0.00007459999 0.04171139    0.99857310       0.9987
## Rank 5 - Rank 1  0.03728801011 0.04144449    0.36833463       0.6217
##                    Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 -0.07306394 0.04072308    0.07287017       0.2066
## Rank 3 - Rank 1 -0.04979466 0.04045268    0.21842675       0.4557
## Rank 4 - Rank 1 -0.01120633 0.04081109    0.78364672       0.7748
## Rank 5 - Rank 1  0.02620737 0.04055086    0.51813636       0.7384
## 
## Best linear projection of the conditional average treatment effect.
## Confidence intervals are cluster- and heteroskedasticity-robust (HC3):
## 
##                             Estimate Std. Error t value     Pr(>|t|)    
## (Intercept)               -0.0951764  0.0186394 -5.1062 0.0000003457 ***
## att_check_pre              0.0333060  0.0234858  1.4181       0.1562    
## misinfo_avg_acc_score_pre -0.0025875  0.0142940 -0.1810       0.8564    
## base_avg_acc_score_pre    -0.0089120  0.0080193 -1.1113       0.2665    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## Best linear fit using forest predictions (on held-out data)
## as well as the mean forest prediction as regressors, along
## with one-sided heteroskedasticity-robust (HC3) SEs:
## 
##                                Estimate Std. Error t value       Pr(>t)    
## mean.forest.prediction         0.961491   0.188051  5.1129 0.0000001668 ***
## differential.forest.prediction 0.272043   0.069789  3.8981 0.0000493602 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

## [1] "Outcome: base_rate_diff"
## [1] "Covariates: covariates_3"
##   method ranking    estimate    std.err
## 1    ols      Q1 -0.15979916 0.02504710
## 2    ols      Q2 -0.11783989 0.02775198
## 3    ols      Q3 -0.07444824 0.03093999
## 4    ols      Q4 -0.04919132 0.03173967
## 5    ols      Q5 -0.06474379 0.02834796
##   method ranking    estimate    std.err
## 1   aipw      Q1 -0.15150969 0.02439102
## 2   aipw      Q2 -0.12663029 0.02562975
## 3   aipw      Q3 -0.07338555 0.02637858
## 4   aipw      Q4 -0.03392738 0.02609334
## 5   aipw      Q5 -0.05224743 0.02412852

##                   Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 0.04195927 0.04083348   0.304220766       0.3046
## Rank 3 - Rank 1 0.08535092 0.04078250   0.036433675       0.0684
## Rank 4 - Rank 1 0.11060784 0.04082842   0.006778607       0.0223
## Rank 5 - Rank 1 0.09505537 0.04070945   0.019599388       0.0526
##                   Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 0.02487941 0.03581942   0.487363958       0.4770
## Rank 3 - Rank 1 0.07812415 0.03575714   0.028963421       0.0516
## Rank 4 - Rank 1 0.11758232 0.03580690   0.001033951       0.0037
## Rank 5 - Rank 1 0.09926227 0.03570786   0.005466587       0.0154
## 
## Best linear projection of the conditional average treatment effect.
## Confidence intervals are cluster- and heteroskedasticity-robust (HC3):
## 
##                             Estimate Std. Error t value Pr(>|t|)   
## (Intercept)                0.0391802  0.0268171  1.4610  0.14410   
## att_check_pre              0.0120636  0.0129488  0.9316  0.35159   
## base_rate_pre             -0.0794407  0.0469841 -1.6908  0.09096 . 
## misinfo_pre               -0.1581163  0.0528649 -2.9910  0.00280 **
## misinfo_avg_acc_score_pre  0.0110696  0.0109473  1.0112  0.31200   
## base_avg_acc_score_pre     0.0067203  0.0077464  0.8675  0.38571   
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## Best linear fit using forest predictions (on held-out data)
## as well as the mean forest prediction as regressors, along
## with one-sided heteroskedasticity-robust (HC3) SEs:
## 
##                                Estimate Std. Error t value         Pr(>t)    
## mean.forest.prediction          0.99719    0.17473  5.7071 0.000000006205 ***
## differential.forest.prediction  0.53734    0.20280  2.6496       0.004047 ** 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

## [1] "Outcome: base_rate_diff"
## [1] "Covariates: covariates_4"
##   method ranking    estimate    std.err
## 1    ols      Q1 -0.15530219 0.03000776
## 2    ols      Q2 -0.04792326 0.02930988
## 3    ols      Q3 -0.05750145 0.02980913
## 4    ols      Q4 -0.14632304 0.03004508
## 5    ols      Q5 -0.06082603 0.02826342
##   method ranking    estimate    std.err
## 1   aipw      Q1 -0.15973712 0.03029015
## 2   aipw      Q2 -0.04716455 0.02958081
## 3   aipw      Q3 -0.06056357 0.02981956
## 4   aipw      Q4 -0.14448193 0.03015301
## 5   aipw      Q5 -0.06125897 0.02855252

##                    Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 0.107378927 0.04175613    0.01016338       0.0353
## Rank 3 - Rank 1 0.097800732 0.04166972    0.01897659       0.0496
## Rank 4 - Rank 1 0.008979142 0.04176709    0.82979406       0.8306
## Rank 5 - Rank 1 0.094476151 0.04161500    0.02325146       0.0496
##                   Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 0.11257257 0.04198609   0.007369417       0.0237
## Rank 3 - Rank 1 0.09917354 0.04189866   0.017985907       0.0470
## Rank 4 - Rank 1 0.01525518 0.04198609   0.716372885       0.7144
## Rank 5 - Rank 1 0.09847815 0.04184106   0.018644555       0.0470
## 
## Best linear projection of the conditional average treatment effect.
## Confidence intervals are cluster- and heteroskedasticity-robust (HC3):
## 
##                                                Estimate Std. Error t value
## (Intercept)                                  -0.0697207  0.2657287 -0.2624
## age                                          -0.0007477  0.0028195 -0.2652
## gender_Man                                    0.0293139  0.0284400  1.0307
## education_High_school_or_less                -0.0979896  0.1484461 -0.6601
## education_Some_college                       -0.0925942  0.1524025 -0.6076
## education_Bachelor_degree                    -0.1097285  0.1389977 -0.7894
## marital_Married_or_in_a_domestic_partnership -0.0663700  0.0545072 -1.2176
## employment_Employed                           0.0663135  0.0444113  1.4932
## employment_Unemployed                        -0.0005022  0.0295670 -0.0170
## location_Mostly_urban                         0.0169366  0.0374762  0.4519
## location_Suburban                            -0.0410265  0.0233745 -1.7552
## religion_Christian                           -0.0877088  0.0720961 -1.2166
## religiosity_Attends                          -0.0453343  0.0407896 -1.1114
## social_media_bin_Yes                          0.2326393  0.2295051  1.0137
## social_media_hours                            0.0030858  0.0043335  0.7121
## social_media_share_80_100                    -0.1241057  0.0344319 -3.6044
## social_media_share_60_80                     -0.0263685  0.0347021 -0.7599
## social_media_share_40_60                     -0.0645305  0.0386882 -1.6680
## social_media_share_20_40                     -0.0017332  0.0449421 -0.0386
##                                               Pr(>|t|)    
## (Intercept)                                  0.7930468    
## age                                          0.7908803    
## gender_Man                                   0.3027381    
## education_High_school_or_less                0.5092304    
## education_Some_college                       0.5435153    
## education_Bachelor_degree                    0.4299145    
## marital_Married_or_in_a_domestic_partnership 0.2234414    
## employment_Employed                          0.1354807    
## employment_Unemployed                        0.9864495    
## location_Mostly_urban                        0.6513461    
## location_Suburban                            0.0793133 .  
## religion_Christian                           0.2238536    
## religiosity_Attends                          0.2664629    
## social_media_bin_Yes                         0.3108148    
## social_media_hours                           0.4764687    
## social_media_share_80_100                    0.0003171 ***
## social_media_share_60_80                     0.4473923    
## social_media_share_40_60                     0.0954097 .  
## social_media_share_20_40                     0.9692394    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## Best linear fit using forest predictions (on held-out data)
## as well as the mean forest prediction as regressors, along
## with one-sided heteroskedasticity-robust (HC3) SEs:
## 
##                                Estimate Std. Error t value         Pr(>t)    
## mean.forest.prediction          1.01892    0.17946  5.6777 0.000000007362 ***
## differential.forest.prediction  0.41210    0.34615  1.1905          0.117    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

## [1] "Outcome: base_rate_diff"
## [1] "Covariates: covariates_5"
##   method ranking    estimate    std.err
## 1    ols      Q1 -0.18583756 0.02529253
## 2    ols      Q2 -0.08316921 0.02825853
## 3    ols      Q3 -0.03523420 0.03096610
## 4    ols      Q4 -0.13279353 0.02981503
## 5    ols      Q5 -0.02375616 0.02845861
##   method ranking    estimate    std.err
## 1   aipw      Q1 -0.18902167 0.02392828
## 2   aipw      Q2 -0.09051187 0.02450155
## 3   aipw      Q3 -0.04394950 0.02588343
## 4   aipw      Q4 -0.11100924 0.02566070
## 5   aipw      Q5 -0.02228136 0.02464789

##                   Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 0.10266835 0.04055365 0.01139418811       0.0224
## Rank 3 - Rank 1 0.15060336 0.04046917 0.00020110745       0.0009
## Rank 4 - Rank 1 0.05304403 0.04055090 0.19092757460       0.1979
## Rank 5 - Rank 1 0.16208139 0.04041080 0.00006172518       0.0004
##                   Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 0.09850980 0.03526471 0.00524250506       0.0078
## Rank 3 - Rank 1 0.14507217 0.03519127 0.00003833669       0.0003
## Rank 4 - Rank 1 0.07801242 0.03526471 0.02701537334       0.0244
## Rank 5 - Rank 1 0.16674031 0.03514290 0.00000216947       0.0001
## 
## Best linear projection of the conditional average treatment effect.
## Confidence intervals are cluster- and heteroskedasticity-robust (HC3):
## 
##                                                 Estimate  Std. Error t value
## (Intercept)                                   0.10964337  0.23046362  0.4758
## att_check_pre                                 0.00877059  0.01144883  0.7661
## base_rate_pre                                -0.07397743  0.04657681 -1.5883
## misinfo_pre                                  -0.14080140  0.05693791 -2.4729
## misinfo_avg_acc_score_pre                     0.01003202  0.01076369  0.9320
## base_avg_acc_score_pre                        0.00471078  0.00793775  0.5935
## age                                          -0.00103451  0.00190761 -0.5423
## gender_Man                                    0.01627083  0.02002149  0.8127
## education_High_school_or_less                -0.12820659  0.12584312 -1.0188
## education_Some_college                       -0.13152425  0.12614430 -1.0426
## education_Bachelor_degree                    -0.14104459  0.12436485 -1.1341
## marital_Married_or_in_a_domestic_partnership -0.03793005  0.04033458 -0.9404
## employment_Employed                           0.03102082  0.03604996  0.8605
## employment_Unemployed                        -0.00157506  0.02994929 -0.0526
## location_Mostly_urban                         0.02212920  0.02326783  0.9511
## location_Suburban                            -0.01715764  0.02216399 -0.7741
## religion_Christian                           -0.02172608  0.05473980 -0.3969
## religiosity_Attends                          -0.08199778  0.04815423 -1.7028
## social_media_bin_Yes                          0.18431753  0.21644307  0.8516
## social_media_hours                           -0.00141882  0.00340170 -0.4171
## social_media_share_80_100                    -0.05621429  0.03049999 -1.8431
## social_media_share_60_80                      0.01830819  0.03667069  0.4993
## social_media_share_40_60                     -0.02462262  0.03943420 -0.6244
## social_media_share_20_40                     -0.00025181  0.04311969 -0.0058
##                                              Pr(>|t|)  
## (Intercept)                                   0.63428  
## att_check_pre                                 0.44369  
## base_rate_pre                                 0.11231  
## misinfo_pre                                   0.01345 *
## misinfo_avg_acc_score_pre                     0.35139  
## base_avg_acc_score_pre                        0.55291  
## age                                           0.58764  
## gender_Man                                    0.41646  
## education_High_school_or_less                 0.30838  
## education_Some_college                        0.29718  
## education_Bachelor_degree                     0.25682  
## marital_Married_or_in_a_domestic_partnership  0.34708  
## employment_Employed                           0.38957  
## employment_Unemployed                         0.95806  
## location_Mostly_urban                         0.34164  
## location_Suburban                             0.43891  
## religion_Christian                            0.69147  
## religiosity_Attends                           0.08869 .
## social_media_bin_Yes                          0.39451  
## social_media_hours                            0.67664  
## social_media_share_80_100                     0.06540 .
## social_media_share_60_80                      0.61763  
## social_media_share_40_60                      0.53241  
## social_media_share_20_40                      0.99534  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## Best linear fit using forest predictions (on held-out data)
## as well as the mean forest prediction as regressors, along
## with one-sided heteroskedasticity-robust (HC3) SEs:
## 
##                                Estimate Std. Error t value          Pr(>t)    
## mean.forest.prediction          0.99051    0.15625  6.3392 0.0000000001297 ***
## differential.forest.prediction  0.85471    0.25666  3.3302       0.0004383 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

## [1] "Outcome: misinfo_diff"
## [1] "Covariates: covariates_1"
##   method ranking    estimate    std.err
## 1    ols      Q1 -0.23250052 0.02258938
## 2    ols      Q2 -0.08236709 0.02774147
## 3    ols      Q3 -0.08672257 0.02369244
## 4    ols      Q4 -0.05575939 0.02139145
## 5    ols      Q5 -0.06533373 0.02155175
##   method ranking    estimate    std.err
## 1   aipw      Q1 -0.23618785 0.02155262
## 2   aipw      Q2 -0.09371408 0.02455564
## 3   aipw      Q3 -0.10967228 0.02019269
## 4   aipw      Q4 -0.05962663 0.01844326
## 5   aipw      Q5 -0.05447515 0.01960552

##                  Estimate Std. Error    Orig. p-value Adj. p-value
## Rank 2 - Rank 1 0.1501334 0.03315608 0.00000614405970            0
## Rank 3 - Rank 1 0.1457779 0.03175159 0.00000455696875            0
## Rank 4 - Rank 1 0.1767411 0.03154705 0.00000002271028            0
## Rank 5 - Rank 1 0.1671668 0.03213535 0.00000020814932            0
##                  Estimate Std. Error      Orig. p-value Adj. p-value
## Rank 2 - Rank 1 0.1424738 0.02970670 0.0000016837279186            0
## Rank 3 - Rank 1 0.1265156 0.02844816 0.0000089595767651            0
## Rank 4 - Rank 1 0.1765612 0.02826393 0.0000000004672965            0
## Rank 5 - Rank 1 0.1817127 0.02879244 0.0000000003105099            0
## 
## Best linear projection of the conditional average treatment effect.
## Confidence intervals are cluster- and heteroskedasticity-robust (HC3):
## 
##                Estimate Std. Error t value  Pr(>|t|)    
## (Intercept)    0.029022   0.017191  1.6882 0.0914564 .  
## att_check_pre -0.017463   0.019021 -0.9181 0.3586163    
## base_rate_pre -0.097543   0.039132 -2.4927 0.0127229 *  
## misinfo_pre   -0.131932   0.039690 -3.3240 0.0008961 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## Best linear fit using forest predictions (on held-out data)
## as well as the mean forest prediction as regressors, along
## with one-sided heteroskedasticity-robust (HC3) SEs:
## 
##                                Estimate Std. Error t value
## mean.forest.prediction         0.989614   0.115847  8.5424
## differential.forest.prediction 0.714227   0.056504 12.6402
##                                               Pr(>t)    
## mean.forest.prediction         < 0.00000000000000022 ***
## differential.forest.prediction < 0.00000000000000022 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

## [1] "Outcome: misinfo_diff"
## [1] "Covariates: covariates_2"
##   method ranking    estimate    std.err
## 1    ols      Q1 -0.10721326 0.02468015
## 2    ols      Q2 -0.15000830 0.02436551
## 3    ols      Q3 -0.14160334 0.02443631
## 4    ols      Q4 -0.12175300 0.02489809
## 5    ols      Q5 -0.04613694 0.02274786
##   method ranking    estimate    std.err
## 1   aipw      Q1 -0.10855026 0.02495935
## 2   aipw      Q2 -0.13928563 0.02368977
## 3   aipw      Q3 -0.14496784 0.02401047
## 4   aipw      Q4 -0.12827779 0.02433468
## 5   aipw      Q5 -0.02691665 0.02219992

##                    Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 -0.04279503 0.03424100    0.21144676       0.4535
## Rank 3 - Rank 1 -0.03439007 0.03410951    0.31341379       0.5018
## Rank 4 - Rank 1 -0.01453973 0.03429958    0.67166098       0.6780
## Rank 5 - Rank 1  0.06107633 0.03417757    0.07401647       0.2148
##                    Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 -0.03073537 0.03366682     0.3613414       0.5522
## Rank 3 - Rank 1 -0.03641758 0.03353801     0.2776133       0.5522
## Rank 4 - Rank 1 -0.01972753 0.03372652     0.5586336       0.5522
## Rank 5 - Rank 1  0.08163361 0.03360785     0.0151887       0.0463
## 
## Best linear projection of the conditional average treatment effect.
## Confidence intervals are cluster- and heteroskedasticity-robust (HC3):
## 
##                             Estimate Std. Error t value         Pr(>|t|)    
## (Intercept)               -0.0916431  0.0136082 -6.7344 0.00000000001907 ***
## att_check_pre              0.0015795  0.0226144  0.0698        0.9443226    
## misinfo_avg_acc_score_pre -0.0328552  0.0092733 -3.5430        0.0004006 ***
## base_avg_acc_score_pre    -0.0167686  0.0075618 -2.2175        0.0266481 *  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## Best linear fit using forest predictions (on held-out data)
## as well as the mean forest prediction as regressors, along
## with one-sided heteroskedasticity-robust (HC3) SEs:
## 
##                                Estimate Std. Error t value
## mean.forest.prediction          0.96306    0.11463  8.4018
## differential.forest.prediction  0.35631    0.12049  2.9572
##                                               Pr(>t)    
## mean.forest.prediction         < 0.00000000000000022 ***
## differential.forest.prediction              0.001562 ** 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

## [1] "Outcome: misinfo_diff"
## [1] "Covariates: covariates_3"
##   method ranking    estimate    std.err
## 1    ols      Q1 -0.19154798 0.02362632
## 2    ols      Q2 -0.15314066 0.02571720
## 3    ols      Q3 -0.13009245 0.02487258
## 4    ols      Q4 -0.07695378 0.02323198
## 5    ols      Q5 -0.01009733 0.01914302
##   method ranking    estimate    std.err
## 1   aipw      Q1 -0.19195613 0.02374742
## 2   aipw      Q2 -0.15636645 0.02336448
## 3   aipw      Q3 -0.14770329 0.02097334
## 4   aipw      Q4 -0.07761715 0.01921163
## 5   aipw      Q5 -0.01768842 0.01761164

##                   Estimate Std. Error    Orig. p-value Adj. p-value
## Rank 2 - Rank 1 0.03840732 0.03318795 0.24723972257704       0.2535
## Rank 3 - Rank 1 0.06145554 0.03307644 0.06325236212823       0.1144
## Rank 4 - Rank 1 0.11459420 0.03308500 0.00053912652678       0.0013
## Rank 5 - Rank 1 0.18145065 0.03299315 0.00000004069277       0.0000
##                   Estimate Std. Error     Orig. p-value Adj. p-value
## Rank 2 - Rank 1 0.03558968 0.02984188 0.233100997841537       0.2318
## Rank 3 - Rank 1 0.04425284 0.02977910 0.137356428543774       0.2318
## Rank 4 - Rank 1 0.11433898 0.02979992 0.000126727798297       0.0005
## Rank 5 - Rank 1 0.17426771 0.02971721 0.000000004917343       0.0000
## 
## Best linear projection of the conditional average treatment effect.
## Confidence intervals are cluster- and heteroskedasticity-robust (HC3):
## 
##                             Estimate Std. Error t value Pr(>|t|)  
## (Intercept)                0.0157281  0.0140564  1.1189   0.2632  
## att_check_pre             -0.0189935  0.0164738 -1.1529   0.2490  
## base_rate_pre             -0.0650845  0.0469061 -1.3875   0.1654  
## misinfo_pre               -0.1301329  0.0587800 -2.2139   0.0269 *
## misinfo_avg_acc_score_pre -0.0054108  0.0111089 -0.4871   0.6262  
## base_avg_acc_score_pre    -0.0106529  0.0099286 -1.0730   0.2834  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## Best linear fit using forest predictions (on held-out data)
## as well as the mean forest prediction as regressors, along
## with one-sided heteroskedasticity-robust (HC3) SEs:
## 
##                                Estimate Std. Error t value
## mean.forest.prediction         0.993123   0.119283  8.3258
## differential.forest.prediction 0.734242   0.088884  8.2607
##                                               Pr(>t)    
## mean.forest.prediction         < 0.00000000000000022 ***
## differential.forest.prediction < 0.00000000000000022 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

## [1] "Outcome: misinfo_diff"
## [1] "Covariates: covariates_4"
##   method ranking    estimate    std.err
## 1    ols      Q1 -0.13485300 0.02473016
## 2    ols      Q2 -0.11979453 0.02444925
## 3    ols      Q3 -0.12236178 0.02427152
## 4    ols      Q4 -0.06968941 0.02468508
## 5    ols      Q5 -0.12284329 0.02469981
##   method ranking    estimate    std.err
## 1   aipw      Q1 -0.13848299 0.02487631
## 2   aipw      Q2 -0.11583835 0.02441552
## 3   aipw      Q3 -0.11276268 0.02410872
## 4   aipw      Q4 -0.07146327 0.02458693
## 5   aipw      Q5 -0.12595513 0.02445153

##                   Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 0.01505847 0.03480754    0.66531601       0.9465
## Rank 3 - Rank 1 0.01249122 0.03473203    0.71913323       0.9465
## Rank 4 - Rank 1 0.06516359 0.03479323    0.06116543       0.1859
## Rank 5 - Rank 1 0.01200970 0.03467191    0.72907626       0.9465
##                   Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 0.02264465 0.03464090    0.51334868       0.7917
## Rank 3 - Rank 1 0.02572032 0.03455661    0.45674627       0.7917
## Rank 4 - Rank 1 0.06701972 0.03462878    0.05302187       0.1690
## Rank 5 - Rank 1 0.01252786 0.03450908    0.71660295       0.7917
## 
## Best linear projection of the conditional average treatment effect.
## Confidence intervals are cluster- and heteroskedasticity-robust (HC3):
## 
##                                                  Estimate   Std. Error t value
## (Intercept)                                  -0.170268186  0.122230540 -1.3930
## age                                          -0.000317041  0.001862981 -0.1702
## gender_Man                                    0.029056771  0.019257340  1.5089
## education_High_school_or_less                -0.215920215  0.114130592 -1.8919
## education_Some_college                       -0.202563539  0.117585022 -1.7227
## education_Bachelor_degree                    -0.235980519  0.119972749 -1.9670
## marital_Married_or_in_a_domestic_partnership -0.029155155  0.037942468 -0.7684
## employment_Employed                           0.039381730  0.019859199  1.9830
## employment_Unemployed                         0.016037564  0.031299828  0.5124
## location_Mostly_urban                         0.018950528  0.032915161  0.5757
## location_Suburban                            -0.012491442  0.019907553 -0.6275
## religion_Christian                            0.007965847  0.055611959  0.1432
## religiosity_Attends                           0.000663218  0.029791994  0.0223
## social_media_bin_Yes                          0.267324835  0.124139022  2.1534
## social_media_hours                           -0.000066908  0.002695039 -0.0248
## social_media_share_80_100                    -0.020747155  0.036704656 -0.5652
## social_media_share_60_80                     -0.042319439  0.041604743 -1.0172
## social_media_share_40_60                     -0.042170128  0.039761118 -1.0606
## social_media_share_20_40                      0.018298727  0.029016371  0.6306
##                                              Pr(>|t|)  
## (Intercept)                                   0.16370  
## age                                           0.86488  
## gender_Man                                    0.13142  
## education_High_school_or_less                 0.05859 .
## education_Some_college                        0.08503 .
## education_Bachelor_degree                     0.04927 *
## marital_Married_or_in_a_domestic_partnership  0.44230  
## employment_Employed                           0.04744 *
## employment_Unemployed                         0.60841  
## location_Mostly_urban                         0.56483  
## location_Suburban                             0.53039  
## religion_Christian                            0.88611  
## religiosity_Attends                           0.98224  
## social_media_bin_Yes                          0.03135 *
## social_media_hours                            0.98019  
## social_media_share_80_100                     0.57194  
## social_media_share_60_80                      0.30914  
## social_media_share_40_60                      0.28895  
## social_media_share_20_40                      0.52832  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## Best linear fit using forest predictions (on held-out data)
## as well as the mean forest prediction as regressors, along
## with one-sided heteroskedasticity-robust (HC3) SEs:
## 
##                                Estimate Std. Error t value              Pr(>t)
## mean.forest.prediction          0.99956    0.11136  8.9762 <0.0000000000000002
## differential.forest.prediction -0.11118    0.47354 -0.2348              0.5928
##                                   
## mean.forest.prediction         ***
## differential.forest.prediction    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

## [1] "Outcome: misinfo_diff"
## [1] "Covariates: covariates_5"
##   method ranking    estimate    std.err
## 1    ols      Q1 -0.23433920 0.02417633
## 2    ols      Q2 -0.10995112 0.02408321
## 3    ols      Q3 -0.12567052 0.02487327
## 4    ols      Q4 -0.06667293 0.02248622
## 5    ols      Q5 -0.02341137 0.01932993
##   method ranking    estimate    std.err
## 1   aipw      Q1 -0.23117020 0.02359676
## 2   aipw      Q2 -0.11468452 0.02183122
## 3   aipw      Q3 -0.13509570 0.02088499
## 4   aipw      Q4 -0.08665641 0.01913033
## 5   aipw      Q5 -0.02074329 0.01740418

##                  Estimate Std. Error      Orig. p-value Adj. p-value
## Rank 2 - Rank 1 0.1243881 0.03268275 0.0001436150608373       0.0002
## Rank 3 - Rank 1 0.1086687 0.03262957 0.0008759914210504       0.0009
## Rank 4 - Rank 1 0.1676663 0.03270025 0.0000003092295419       0.0000
## Rank 5 - Rank 1 0.2109278 0.03257071 0.0000000001068752       0.0000
##                   Estimate Std. Error         Orig. p-value Adj. p-value
## Rank 2 - Rank 1 0.11648568 0.02925406 0.0000697196603983841       0.0002
## Rank 3 - Rank 1 0.09607451 0.02919314 0.0010079002093236204       0.0012
## Rank 4 - Rank 1 0.14451379 0.02925406 0.0000008167441802844       0.0000
## Rank 5 - Rank 1 0.21042692 0.02915301 0.0000000000006396269       0.0000
## 
## Best linear projection of the conditional average treatment effect.
## Confidence intervals are cluster- and heteroskedasticity-robust (HC3):
## 
##                                                  Estimate   Std. Error t value
## (Intercept)                                  -0.030658595  0.139971173 -0.2190
## att_check_pre                                -0.014003198  0.015101220 -0.9273
## base_rate_pre                                -0.056567588  0.046839648 -1.2077
## misinfo_pre                                  -0.138139915  0.060930152 -2.2672
## misinfo_avg_acc_score_pre                    -0.004816969  0.011162920 -0.4315
## base_avg_acc_score_pre                       -0.012499465  0.009602022 -1.3018
## age                                          -0.000599286  0.001417420 -0.4228
## gender_Man                                    0.033716162  0.020327890  1.6586
## education_High_school_or_less                -0.141725762  0.092140706 -1.5381
## education_Some_college                       -0.137860990  0.085618435 -1.6102
## education_Bachelor_degree                    -0.148960390  0.094726867 -1.5725
## marital_Married_or_in_a_domestic_partnership -0.005063339  0.037245356 -0.1359
## employment_Employed                          -0.008833309  0.025736540 -0.3432
## employment_Unemployed                        -0.005309472  0.027704747 -0.1916
## location_Mostly_urban                         0.036874640  0.026207819  1.4070
## location_Suburban                            -0.007260683  0.020196446 -0.3595
## religion_Christian                           -0.014404291  0.035380142 -0.4071
## religiosity_Attends                          -0.035592095  0.036136463 -0.9849
## social_media_bin_Yes                          0.209795763  0.106688550  1.9664
## social_media_hours                            0.000018007  0.002711722  0.0066
## social_media_share_80_100                     0.046834789  0.033968825  1.3788
## social_media_share_60_80                      0.026052055  0.031315773  0.8319
## social_media_share_40_60                      0.003812970  0.033240163  0.1147
## social_media_share_20_40                      0.032931773  0.027093526  1.2155
##                                              Pr(>|t|)  
## (Intercept)                                   0.82664  
## att_check_pre                                 0.35384  
## base_rate_pre                                 0.22725  
## misinfo_pre                                   0.02344 *
## misinfo_avg_acc_score_pre                     0.66612  
## base_avg_acc_score_pre                        0.19308  
## age                                           0.67247  
## gender_Man                                    0.09728 .
## education_High_school_or_less                 0.12410  
## education_Some_college                        0.10745  
## education_Bachelor_degree                     0.11592  
## marital_Married_or_in_a_domestic_partnership  0.89187  
## employment_Employed                           0.73145  
## employment_Unemployed                         0.84803  
## location_Mostly_urban                         0.15951  
## location_Suburban                             0.71924  
## religion_Christian                            0.68394  
## religiosity_Attends                           0.32472  
## social_media_bin_Yes                          0.04933 *
## social_media_hours                            0.99470  
## social_media_share_80_100                     0.16805  
## social_media_share_60_80                      0.40551  
## social_media_share_40_60                      0.90868  
## social_media_share_20_40                      0.22426  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## Best linear fit using forest predictions (on held-out data)
## as well as the mean forest prediction as regressors, along
## with one-sided heteroskedasticity-robust (HC3) SEs:
## 
##                                Estimate Std. Error t value
## mean.forest.prediction         0.987884   0.114881  8.5992
## differential.forest.prediction 1.028534   0.094341 10.9023
##                                               Pr(>t)    
## mean.forest.prediction         < 0.00000000000000022 ***
## differential.forest.prediction < 0.00000000000000022 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

## [1] "Outcome: new_disc_post"
## [1] "Covariates: covariates_1"
##   method ranking     estimate    std.err
## 1    ols      Q1  0.060875917 0.02225619
## 2    ols      Q2  0.006115566 0.02367999
## 3    ols      Q3  0.025876894 0.01977426
## 4    ols      Q4 -0.017287638 0.02118042
## 5    ols      Q5  0.043242990 0.02488410
##   method ranking     estimate    std.err
## 1   aipw      Q1  0.061746983 0.02246153
## 2   aipw      Q2  0.009858703 0.02380181
## 3   aipw      Q3  0.028585880 0.01936533
## 4   aipw      Q4 -0.012964180 0.02088506
## 5   aipw      Q5  0.043021880 0.02496235

##                    Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 -0.05476035 0.03092368    0.07667440       0.1898
## Rank 3 - Rank 1 -0.03499902 0.02999730    0.24339268       0.3997
## Rank 4 - Rank 1 -0.07816356 0.03113686    0.01210543       0.0423
## Rank 5 - Rank 1 -0.01763293 0.03190642    0.58054060       0.5807
##                    Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 -0.05188828 0.03087664    0.09294575       0.2193
## Rank 3 - Rank 1 -0.03316110 0.02995010    0.26827593       0.4333
## Rank 4 - Rank 1 -0.07471116 0.03107914    0.01627115       0.0516
## Rank 5 - Rank 1 -0.01872510 0.03185451    0.55668170       0.5591
## 
## Best linear projection of the conditional average treatment effect.
## Confidence intervals are cluster- and heteroskedasticity-robust (HC3):
## 
##                 Estimate Std. Error t value Pr(>|t|)  
## (Intercept)   -0.0126603  0.0248654 -0.5092  0.61068  
## att_check_pre  0.0289343  0.0117863  2.4549  0.01414 *
## base_rate_pre  0.0418229  0.0453466  0.9223  0.35644  
## misinfo_pre   -0.0054867  0.0505254 -0.1086  0.91353  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## Best linear fit using forest predictions (on held-out data)
## as well as the mean forest prediction as regressors, along
## with one-sided heteroskedasticity-robust (HC3) SEs:
## 
##                                Estimate Std. Error t value    Pr(>t)    
## mean.forest.prediction          1.01948    0.25556  3.9892 0.0000338 ***
## differential.forest.prediction -0.11950    0.16254 -0.7352    0.7689    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

## [1] "Outcome: new_disc_post"
## [1] "Covariates: covariates_2"
##   method ranking    estimate    std.err
## 1    ols      Q1 0.004554928 0.02261970
## 2    ols      Q2 0.004646258 0.02273406
## 3    ols      Q3 0.025744417 0.02219161
## 4    ols      Q4 0.030431088 0.02130467
## 5    ols      Q5 0.057363465 0.02293832
##   method ranking     estimate    std.err
## 1   aipw      Q1  0.005624201 0.02274881
## 2   aipw      Q2 -0.001903795 0.02293227
## 3   aipw      Q3  0.030712595 0.02265135
## 4   aipw      Q4  0.032253035 0.02186064
## 5   aipw      Q5  0.061852731 0.02338307

##                      Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 0.00009133053 0.03155284    0.99769067       0.9979
## Rank 3 - Rank 1 0.02118948906 0.03146413    0.50070467       0.7451
## Rank 4 - Rank 1 0.02587616085 0.03137842    0.40962575       0.7451
## Rank 5 - Rank 1 0.05280853709 0.03158071    0.09457594       0.2710
##                     Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 -0.007527996 0.03204284    0.81427219       0.8201
## Rank 3 - Rank 1  0.025088394 0.03197519    0.43272740       0.7270
## Rank 4 - Rank 1  0.026628834 0.03186461    0.40338619       0.7270
## Rank 5 - Rank 1  0.056228530 0.03209997    0.07991534       0.2369
## 
## Best linear projection of the conditional average treatment effect.
## Confidence intervals are cluster- and heteroskedasticity-robust (HC3):
## 
##                             Estimate Std. Error t value  Pr(>|t|)    
## (Intercept)               -0.0098231  0.0165735 -0.5927 0.5534169    
## att_check_pre              0.0428379  0.0129914  3.2974 0.0009853 ***
## misinfo_avg_acc_score_pre  0.0086980  0.0062862  1.3837 0.1665440    
## base_avg_acc_score_pre     0.0156038  0.0090090  1.7320 0.0833551 .  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## Best linear fit using forest predictions (on held-out data)
## as well as the mean forest prediction as regressors, along
## with one-sided heteroskedasticity-robust (HC3) SEs:
## 
##                                Estimate Std. Error t value   Pr(>t)   
## mean.forest.prediction          1.00358    0.37554  2.6723 0.003783 **
## differential.forest.prediction  0.20559    0.21487  0.9568 0.169354   
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

## [1] "Outcome: new_disc_post"
## [1] "Covariates: covariates_3"
##   method ranking    estimate    std.err
## 1    ols      Q1 0.005655654 0.02303009
## 2    ols      Q2 0.027417818 0.02279565
## 3    ols      Q3 0.025197181 0.02175374
## 4    ols      Q4 0.045941512 0.02146527
## 5    ols      Q5 0.019526208 0.02262696
##   method ranking    estimate    std.err
## 1   aipw      Q1 0.007362701 0.02277412
## 2   aipw      Q2 0.029270975 0.02261887
## 3   aipw      Q3 0.039589441 0.02174562
## 4   aipw      Q4 0.045197487 0.02191365
## 5   aipw      Q5 0.026050142 0.02272828

##                   Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 0.02176216 0.03158454     0.4908591       0.8296
## Rank 3 - Rank 1 0.01954153 0.03155935     0.5358241       0.8296
## Rank 4 - Rank 1 0.04028586 0.03163817     0.2029825       0.5176
## Rank 5 - Rank 1 0.01387055 0.03149546     0.6596748       0.8296
##                   Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 0.02190827 0.03159948     0.4881588       0.7119
## Rank 3 - Rank 1 0.03222674 0.03159948     0.3078676       0.6142
## Rank 4 - Rank 1 0.03783479 0.03162150     0.2315840       0.5713
## Rank 5 - Rank 1 0.01868744 0.03153408     0.5534786       0.7119
## 
## Best linear projection of the conditional average treatment effect.
## Confidence intervals are cluster- and heteroskedasticity-robust (HC3):
## 
##                             Estimate Std. Error t value  Pr(>|t|)    
## (Intercept)                0.0159849  0.0271186  0.5894 0.5555992    
## att_check_pre              0.0345066  0.0097631  3.5344 0.0004139 ***
## base_rate_pre              0.0019963  0.0389327  0.0513 0.9591095    
## misinfo_pre               -0.0360581  0.0565383 -0.6378 0.5236672    
## misinfo_avg_acc_score_pre  0.0145293  0.0096930  1.4990 0.1339713    
## base_avg_acc_score_pre     0.0150147  0.0071173  2.1096 0.0349611 *  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## Best linear fit using forest predictions (on held-out data)
## as well as the mean forest prediction as regressors, along
## with one-sided heteroskedasticity-robust (HC3) SEs:
## 
##                                Estimate Std. Error t value    Pr(>t)    
## mean.forest.prediction          0.96308    0.26417  3.6456 0.0001352 ***
## differential.forest.prediction  0.13845    0.24270  0.5705 0.2842034    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

## [1] "Outcome: new_disc_post"
## [1] "Covariates: covariates_4"
##   method ranking    estimate    std.err
## 1    ols      Q1 0.007576951 0.02213580
## 2    ols      Q2 0.019353854 0.02208793
## 3    ols      Q3 0.010826076 0.02301274
## 4    ols      Q4 0.053240738 0.02288704
## 5    ols      Q5 0.031334839 0.02156669
##   method ranking   estimate    std.err
## 1   aipw      Q1 0.01191222 0.02225924
## 2   aipw      Q2 0.01521870 0.02215363
## 3   aipw      Q3 0.01359508 0.02301947
## 4   aipw      Q4 0.05781789 0.02305447
## 5   aipw      Q5 0.03176672 0.02163186

##                    Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 0.011776903 0.03159430     0.7093533       0.9055
## Rank 3 - Rank 1 0.003249125 0.03156243     0.9180141       0.9108
## Rank 4 - Rank 1 0.045663787 0.03160091     0.1485403       0.4091
## Rank 5 - Rank 1 0.023757888 0.03152885     0.4511814       0.7958
##                    Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 0.003306478 0.03171054     0.9169604       0.9923
## Rank 3 - Rank 1 0.001682866 0.03166650     0.9576206       0.9923
## Rank 4 - Rank 1 0.045905669 0.03172161     0.1479438       0.3954
## Rank 5 - Rank 1 0.019854505 0.03161204     0.5299991       0.8656
## 
## Best linear projection of the conditional average treatment effect.
## Confidence intervals are cluster- and heteroskedasticity-robust (HC3):
## 
##                                                 Estimate  Std. Error t value
## (Intercept)                                   0.10146942  0.18359845  0.5527
## age                                          -0.00042318  0.00230644 -0.1835
## gender_Man                                   -0.02652833  0.01839236 -1.4424
## education_High_school_or_less                -0.01540929  0.13350240 -0.1154
## education_Some_college                       -0.01729512  0.12554672 -0.1378
## education_Bachelor_degree                    -0.01105387  0.11456598 -0.0965
## marital_Married_or_in_a_domestic_partnership -0.04009522  0.02518865 -1.5918
## employment_Employed                           0.04975043  0.02443392  2.0361
## employment_Unemployed                         0.00825294  0.01824829  0.4523
## location_Mostly_urban                        -0.00801404  0.02455063 -0.3264
## location_Suburban                            -0.01081385  0.02560341 -0.4224
## religion_Christian                            0.02026996  0.04092289  0.4953
## religiosity_Attends                          -0.05281629  0.05032395 -1.0495
## social_media_bin_Yes                          0.03185186  0.17156963  0.1856
## social_media_hours                           -0.00203208  0.00278135 -0.7306
## social_media_share_80_100                    -0.09492511  0.03674979 -2.5830
## social_media_share_60_80                      0.00373929  0.04581998  0.0816
## social_media_share_40_60                     -0.02956226  0.02447848 -1.2077
## social_media_share_20_40                     -0.02877974  0.04051383 -0.7104
##                                              Pr(>|t|)   
## (Intercept)                                  0.580523   
## age                                          0.854434   
## gender_Man                                   0.149289   
## education_High_school_or_less                0.908116   
## education_Some_college                       0.890439   
## education_Bachelor_degree                    0.923141   
## marital_Married_or_in_a_domestic_partnership 0.111518   
## employment_Employed                          0.041811 * 
## employment_Unemployed                        0.651110   
## location_Mostly_urban                        0.744119   
## location_Suburban                            0.672788   
## religion_Christian                           0.620404   
## religiosity_Attends                          0.294007   
## social_media_bin_Yes                         0.852730   
## social_media_hours                           0.465065   
## social_media_share_80_100                    0.009833 **
## social_media_share_60_80                     0.934963   
## social_media_share_40_60                     0.227248   
## social_media_share_20_40                     0.477522   
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## Best linear fit using forest predictions (on held-out data)
## as well as the mean forest prediction as regressors, along
## with one-sided heteroskedasticity-robust (HC3) SEs:
## 
##                                Estimate Std. Error t value    Pr(>t)    
## mean.forest.prediction          0.97209    0.29652  3.2783 0.0005271 ***
## differential.forest.prediction  0.26574    0.20299  1.3091 0.0952893 .  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

## [1] "Outcome: new_disc_post"
## [1] "Covariates: covariates_5"
##   method ranking      estimate    std.err
## 1    ols      Q1 -0.0009355343 0.02170321
## 2    ols      Q2  0.0146880271 0.02288212
## 3    ols      Q3  0.0216758752 0.02345264
## 4    ols      Q4  0.0045139225 0.02208064
## 5    ols      Q5  0.0775814513 0.02125646
##   method ranking    estimate    std.err
## 1   aipw      Q1 0.003373699 0.02160903
## 2   aipw      Q2 0.012498373 0.02273870
## 3   aipw      Q3 0.025896622 0.02331288
## 4   aipw      Q4 0.008452919 0.02174994
## 5   aipw      Q5 0.080446549 0.02127618

##                    Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 0.015623561 0.03157067    0.62071780       0.8359
## Rank 3 - Rank 1 0.022611410 0.03150552    0.47299061       0.8169
## Rank 4 - Rank 1 0.005449457 0.03156975    0.86296260       0.8654
## Rank 5 - Rank 1 0.078516986 0.03147406    0.01265217       0.0416
##                    Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 0.009124674 0.03132592    0.77085240       0.9432
## Rank 3 - Rank 1 0.022522923 0.03126068    0.47127157       0.8152
## Rank 4 - Rank 1 0.005079220 0.03132592    0.87120378       0.9432
## Rank 5 - Rank 1 0.077072850 0.03121771    0.01359943       0.0469
## 
## Best linear projection of the conditional average treatment effect.
## Confidence intervals are cluster- and heteroskedasticity-robust (HC3):
## 
##                                                 Estimate  Std. Error t value
## (Intercept)                                   0.09465512  0.17810026  0.5315
## att_check_pre                                 0.02793932  0.00953373  2.9306
## base_rate_pre                                 0.00466216  0.03836612  0.1215
## misinfo_pre                                  -0.02028509  0.05412917 -0.3748
## misinfo_avg_acc_score_pre                     0.01409069  0.00885996  1.5904
## base_avg_acc_score_pre                        0.01577323  0.00676173  2.3327
## age                                          -0.00049431  0.00216553 -0.2283
## gender_Man                                   -0.02309852  0.01445876 -1.5975
## education_High_school_or_less                 0.00025745  0.13110853  0.0020
## education_Some_college                       -0.00675832  0.12394194 -0.0545
## education_Bachelor_degree                     0.00061851  0.11627296  0.0053
## marital_Married_or_in_a_domestic_partnership -0.03473668  0.02468693 -1.4071
## employment_Employed                           0.05053388  0.02496691  2.0240
## employment_Unemployed                         0.00972110  0.01818421  0.5346
## location_Mostly_urban                        -0.01352069  0.02126602 -0.6358
## location_Suburban                            -0.00854401  0.02556800 -0.3342
## religion_Christian                            0.01408814  0.04398401  0.3203
## religiosity_Attends                          -0.04701051  0.04687303 -1.0029
## social_media_bin_Yes                          0.00885633  0.15723861  0.0563
## social_media_hours                           -0.00190050  0.00260171 -0.7305
## social_media_share_80_100                    -0.10937805  0.03500182 -3.1249
## social_media_share_60_80                     -0.01067905  0.04427605 -0.2412
## social_media_share_40_60                     -0.03408377  0.02850684 -1.1956
## social_media_share_20_40                     -0.03075021  0.03854208 -0.7978
##                                              Pr(>|t|)   
## (Intercept)                                  0.595125   
## att_check_pre                                0.003405 **
## base_rate_pre                                0.903288   
## misinfo_pre                                  0.707866   
## misinfo_avg_acc_score_pre                    0.111837   
## base_avg_acc_score_pre                       0.019718 * 
## age                                          0.819455   
## gender_Man                                   0.110232   
## education_High_school_or_less                0.998433   
## education_Some_college                       0.956517   
## education_Bachelor_degree                    0.995756   
## marital_Married_or_in_a_domestic_partnership 0.159488   
## employment_Employed                          0.043040 * 
## employment_Unemployed                        0.592966   
## location_Mostly_urban                        0.524955   
## location_Suburban                            0.738272   
## religion_Christian                           0.748758   
## religiosity_Attends                          0.315961   
## social_media_bin_Yes                         0.955087   
## social_media_hours                           0.465144   
## social_media_share_80_100                    0.001793 **
## social_media_share_60_80                     0.809420   
## social_media_share_40_60                     0.231918   
## social_media_share_20_40                     0.425019   
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## Best linear fit using forest predictions (on held-out data)
## as well as the mean forest prediction as regressors, along
## with one-sided heteroskedasticity-robust (HC3) SEs:
## 
##                                Estimate Std. Error t value           Pr(>t)    
## mean.forest.prediction          1.00271    0.30585  3.2784        0.0005269 ***
## differential.forest.prediction  0.77650    0.11989  6.4769 0.00000000005311 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

## [1] "Outcome: new_disc_diff"
## [1] "Covariates: covariates_1"
##   method ranking     estimate    std.err
## 1    ols      Q1  0.080161208 0.03025836
## 2    ols      Q2  0.003561884 0.03196454
## 3    ols      Q3  0.015522805 0.02632077
## 4    ols      Q4 -0.034493285 0.02661354
## 5    ols      Q5  0.022568137 0.03484219
##   method ranking     estimate    std.err
## 1   aipw      Q1  0.076280090 0.02343006
## 2   aipw      Q2 -0.006655928 0.02282483
## 3   aipw      Q3  0.038118386 0.01923605
## 4   aipw      Q4 -0.049268196 0.02205102
## 5   aipw      Q5  0.063121620 0.02342254

##                    Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 -0.07659932 0.04206606   0.068700180       0.1668
## Rank 3 - Rank 1 -0.06463840 0.04089019   0.114015733       0.1917
## Rank 4 - Rank 1 -0.11465449 0.04436190   0.009790119       0.0332
## Rank 5 - Rank 1 -0.05759307 0.04281468   0.178654356       0.1917
##                    Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 -0.08293602 0.03089693  0.0073017219       0.0206
## Rank 3 - Rank 1 -0.03816170 0.03003678  0.2039884704       0.3365
## Rank 4 - Rank 1 -0.12554829 0.03258070  0.0001184811       0.0004
## Rank 5 - Rank 1 -0.01315847 0.03144672  0.6756525401       0.6712
## 
## Best linear projection of the conditional average treatment effect.
## Confidence intervals are cluster- and heteroskedasticity-robust (HC3):
## 
##                  Estimate  Std. Error t value Pr(>|t|)  
## (Intercept)   -0.01113527  0.02340139 -0.4758  0.63422  
## att_check_pre  0.02763787  0.01158704  2.3852  0.01712 *
## base_rate_pre  0.03593382  0.04503761  0.7979  0.42500  
## misinfo_pre    0.00068705  0.05242701  0.0131  0.98954  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## Best linear fit using forest predictions (on held-out data)
## as well as the mean forest prediction as regressors, along
## with one-sided heteroskedasticity-robust (HC3) SEs:
## 
##                                Estimate Std. Error t value     Pr(>t)    
## mean.forest.prediction          1.03709    0.25264  4.1050 0.00002066 ***
## differential.forest.prediction -0.12872    0.15343 -0.8389     0.7992    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

## [1] "Outcome: new_disc_diff"
## [1] "Covariates: covariates_2"
##   method ranking     estimate    std.err
## 1    ols      Q1  0.027592147 0.03294309
## 2    ols      Q2  0.007665021 0.03118976
## 3    ols      Q3 -0.005677652 0.03004717
## 4    ols      Q4 -0.015552665 0.03067332
## 5    ols      Q5  0.091365508 0.02599655
##   method ranking    estimate    std.err
## 1   aipw      Q1  0.01616993 0.02982685
## 2   aipw      Q2  0.03690513 0.02929269
## 3   aipw      Q3 -0.01587730 0.02879492
## 4   aipw      Q4 -0.01970551 0.02958018
## 5   aipw      Q5  0.09369476 0.02547779

##                    Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 -0.01992713 0.04273016     0.6409945       0.6458
## Rank 3 - Rank 1 -0.03326980 0.04274285     0.4364012       0.6456
## Rank 4 - Rank 1 -0.04314481 0.04276334     0.3130803       0.6142
## Rank 5 - Rank 1  0.06377336 0.04282327     0.1365161       0.3541
##                    Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1  0.02073520 0.04039204    0.60773885       0.7013
## Rank 3 - Rank 1 -0.03204723 0.04040608    0.42775478       0.7013
## Rank 4 - Rank 1 -0.03587543 0.04042016    0.37483485       0.7013
## Rank 5 - Rank 1  0.07752483 0.04047680    0.05553416       0.1676
## 
## Best linear projection of the conditional average treatment effect.
## Confidence intervals are cluster- and heteroskedasticity-robust (HC3):
## 
##                             Estimate Std. Error t value   Pr(>|t|)    
## (Intercept)               -0.0039749  0.0156430 -0.2541     0.7994    
## att_check_pre              0.0331788  0.0171779  1.9315     0.0535 .  
## misinfo_avg_acc_score_pre  0.0302165  0.0076965  3.9260 0.00008797 ***
## base_avg_acc_score_pre     0.0075709  0.0077617  0.9754     0.3294    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## Best linear fit using forest predictions (on held-out data)
## as well as the mean forest prediction as regressors, along
## with one-sided heteroskedasticity-robust (HC3) SEs:
## 
##                                Estimate Std. Error t value   Pr(>t)   
## mean.forest.prediction          0.98533    0.42035  2.3441 0.009564 **
## differential.forest.prediction  0.27829    0.13897  2.0025 0.022651 * 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

## [1] "Outcome: new_disc_diff"
## [1] "Covariates: covariates_3"
##   method ranking    estimate    std.err
## 1    ols      Q1 0.021452730 0.03041625
## 2    ols      Q2 0.025429490 0.03199676
## 3    ols      Q3 0.010916523 0.02956156
## 4    ols      Q4 0.002515343 0.03038819
## 5    ols      Q5 0.043729526 0.02902261
##   method ranking     estimate    std.err
## 1   aipw      Q1 0.0001085265 0.02244769
## 2   aipw      Q2 0.0450227441 0.02295911
## 3   aipw      Q3 0.0128031778 0.02211896
## 4   aipw      Q4 0.0327503797 0.02160050
## 5   aipw      Q5 0.0543479265 0.02273399

##                    Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1  0.00397676 0.04289488     0.9261395       0.9592
## Rank 3 - Rank 1 -0.01053621 0.04276202     0.8053930       0.9592
## Rank 4 - Rank 1 -0.01893739 0.04293731     0.6592051       0.9592
## Rank 5 - Rank 1  0.02227680 0.04273607     0.6022145       0.9592
##                   Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 0.04491422 0.03165544    0.15602981       0.3435
## Rank 3 - Rank 1 0.01269465 0.03155691    0.68750321       0.6837
## Rank 4 - Rank 1 0.03264185 0.03168877    0.30304290       0.4726
## Rank 5 - Rank 1 0.05423940 0.03154610    0.08563256       0.2438
## 
## Best linear projection of the conditional average treatment effect.
## Confidence intervals are cluster- and heteroskedasticity-robust (HC3):
## 
##                               Estimate   Std. Error t value  Pr(>|t|)    
## (Intercept)                0.016994747  0.023520219  0.7226 0.4699975    
## att_check_pre              0.033157852  0.009139930  3.6278 0.0002898 ***
## base_rate_pre             -0.000095001  0.042940465 -0.0022 0.9982349    
## misinfo_pre               -0.035500838  0.063649354 -0.5578 0.5770452    
## misinfo_avg_acc_score_pre  0.015579508  0.010164934  1.5327 0.1254441    
## base_avg_acc_score_pre     0.015200963  0.007105178  2.1394 0.0324683 *  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## Best linear fit using forest predictions (on held-out data)
## as well as the mean forest prediction as regressors, along
## with one-sided heteroskedasticity-robust (HC3) SEs:
## 
##                                Estimate Std. Error t value    Pr(>t)    
## mean.forest.prediction          0.96403    0.26401  3.6515 0.0001322 ***
## differential.forest.prediction  0.16088    0.20241  0.7948 0.2133806    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

## [1] "Outcome: new_disc_diff"
## [1] "Covariates: covariates_4"
##   method ranking    estimate    std.err
## 1    ols      Q1 -0.01731527 0.03029737
## 2    ols      Q2  0.05841572 0.03184304
## 3    ols      Q3  0.01315989 0.02901626
## 4    ols      Q4  0.01607082 0.03035338
## 5    ols      Q5  0.03194855 0.03059041
##   method ranking    estimate    std.err
## 1   aipw      Q1 -0.01824086 0.03049209
## 2   aipw      Q2  0.05639501 0.03171418
## 3   aipw      Q3  0.01715282 0.02921011
## 4   aipw      Q4  0.01648694 0.03058247
## 5   aipw      Q5  0.03239119 0.03091462

##                   Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 0.07573099 0.04301105     0.0783681       0.2330
## Rank 3 - Rank 1 0.03047516 0.04287590     0.4772689       0.6519
## Rank 4 - Rank 1 0.03338609 0.04296881     0.4372189       0.6519
## Rank 5 - Rank 1 0.04926382 0.04283099     0.2501413       0.5159
##                   Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 0.07463587 0.04326888    0.08462507       0.2519
## Rank 3 - Rank 1 0.03539368 0.04317877    0.41244056       0.6225
## Rank 4 - Rank 1 0.03472779 0.04326888    0.42225607       0.6225
## Rank 5 - Rank 1 0.05063205 0.04311942    0.24038072       0.5005
## 
## Best linear projection of the conditional average treatment effect.
## Confidence intervals are cluster- and heteroskedasticity-robust (HC3):
## 
##                                                 Estimate  Std. Error t value
## (Intercept)                                   0.09632028  0.26314695  0.3660
## age                                          -0.00062920  0.00198062 -0.3177
## gender_Man                                    0.00127464  0.02177649  0.0585
## education_High_school_or_less                 0.11457437  0.18317160  0.6255
## education_Some_college                        0.10563991  0.16480648  0.6410
## education_Bachelor_degree                     0.12342868  0.15959262  0.7734
## marital_Married_or_in_a_domestic_partnership -0.03381702  0.03902644 -0.8665
## employment_Employed                           0.03233867  0.03378372  0.9572
## employment_Unemployed                        -0.01162518  0.03185181 -0.3650
## location_Mostly_urban                        -0.00025535  0.05217707 -0.0049
## location_Suburban                            -0.02634886  0.02998614 -0.8787
## religion_Christian                           -0.08896157  0.05547799 -1.6035
## religiosity_Attends                          -0.05440132  0.04638278 -1.1729
## social_media_bin_Yes                         -0.02550523  0.15732477 -0.1621
## social_media_hours                            0.00289968  0.00370011  0.7837
## social_media_share_80_100                    -0.09641440  0.05678919 -1.6978
## social_media_share_60_80                      0.01645856  0.04769792  0.3451
## social_media_share_40_60                     -0.01723081  0.03710829 -0.4643
## social_media_share_20_40                     -0.01919522  0.04503253 -0.4263
##                                              Pr(>|t|)  
## (Intercept)                                   0.71436  
## age                                           0.75075  
## gender_Man                                    0.95333  
## education_High_school_or_less                 0.53168  
## education_Some_college                        0.52157  
## education_Bachelor_degree                     0.43934  
## marital_Married_or_in_a_domestic_partnership  0.38627  
## employment_Employed                           0.33852  
## employment_Unemployed                         0.71515  
## location_Mostly_urban                         0.99610  
## location_Suburban                             0.37962  
## religion_Christian                            0.10890  
## religiosity_Attends                           0.24092  
## social_media_bin_Yes                          0.87122  
## social_media_hours                            0.43328  
## social_media_share_80_100                     0.08964 .
## social_media_share_60_80                      0.73007  
## social_media_share_40_60                      0.64243  
## social_media_share_20_40                      0.66995  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## Best linear fit using forest predictions (on held-out data)
## as well as the mean forest prediction as regressors, along
## with one-sided heteroskedasticity-robust (HC3) SEs:
## 
##                                 Estimate Std. Error t value  Pr(>t)  
## mean.forest.prediction         0.9756668  0.5192491  1.8790 0.03016 *
## differential.forest.prediction 0.0037572  0.2649161  0.0142 0.49434  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

## [1] "Outcome: new_disc_diff"
## [1] "Covariates: covariates_5"
##   method ranking    estimate    std.err
## 1    ols      Q1  0.02687057 0.03133932
## 2    ols      Q2 -0.04108708 0.03151702
## 3    ols      Q3  0.02779107 0.03109643
## 4    ols      Q4  0.03321001 0.02998539
## 5    ols      Q5  0.06191451 0.02738131
##   method ranking     estimate    std.err
## 1   aipw      Q1  0.012199773 0.02244057
## 2   aipw      Q2 -0.003432361 0.02277372
## 3   aipw      Q3  0.023274072 0.02231780
## 4   aipw      Q4  0.026613230 0.02253882
## 5   aipw      Q5  0.068057901 0.02134657

##                      Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 -0.0679576501 0.04292056     0.1134324       0.3134
## Rank 3 - Rank 1  0.0009205056 0.04280285     0.9828434       0.9839
## Rank 4 - Rank 1  0.0063394481 0.04289028     0.8825040       0.9839
## Rank 5 - Rank 1  0.0350439438 0.04275844     0.4125099       0.7321
##                    Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 -0.01563213 0.03152112    0.61997693       0.9264
## Rank 3 - Rank 1  0.01107430 0.03145548    0.72481179       0.9264
## Rank 4 - Rank 1  0.01441346 0.03152112    0.64750913       0.9264
## Rank 5 - Rank 1  0.05585813 0.03141224    0.07545029       0.2290
## 
## Best linear projection of the conditional average treatment effect.
## Confidence intervals are cluster- and heteroskedasticity-robust (HC3):
## 
##                                                   Estimate    Std. Error
## (Intercept)                                   0.0695006957  0.1878937635
## att_check_pre                                 0.0245625758  0.0084130237
## base_rate_pre                                -0.0095703050  0.0410600287
## misinfo_pre                                  -0.0089237385  0.0590866712
## misinfo_avg_acc_score_pre                     0.0160966902  0.0089641107
## base_avg_acc_score_pre                        0.0155892908  0.0072373576
## age                                          -0.0002094036  0.0022502354
## gender_Man                                   -0.0250422813  0.0155509666
## education_High_school_or_less                 0.0091115798  0.1378460561
## education_Some_college                       -0.0000012283  0.1291251243
## education_Bachelor_degree                     0.0060330305  0.1228513242
## marital_Married_or_in_a_domestic_partnership -0.0350647326  0.0247162267
## employment_Employed                           0.0489830392  0.0247844111
## employment_Unemployed                         0.0086736861  0.0177534059
## location_Mostly_urban                        -0.0131338739  0.0233383801
## location_Suburban                            -0.0067470963  0.0265140056
## religion_Christian                            0.0129705985  0.0428608729
## religiosity_Attends                          -0.0401641712  0.0447638257
## social_media_bin_Yes                          0.0234761000  0.1639934700
## social_media_hours                           -0.0020004227  0.0028025535
## social_media_share_80_100                    -0.1160161989  0.0341162556
## social_media_share_60_80                     -0.0124117162  0.0441875603
## social_media_share_40_60                     -0.0373786849  0.0277845700
## social_media_share_20_40                     -0.0342699073  0.0393784632
##                                              t value  Pr(>|t|)    
## (Intercept)                                   0.3699 0.7114835    
## att_check_pre                                 2.9196 0.0035267 ** 
## base_rate_pre                                -0.2331 0.8157119    
## misinfo_pre                                  -0.1510 0.8799621    
## misinfo_avg_acc_score_pre                     1.7957 0.0726289 .  
## base_avg_acc_score_pre                        2.1540 0.0313061 *  
## age                                          -0.0931 0.9258622    
## gender_Man                                   -1.6103 0.1074121    
## education_High_school_or_less                 0.0661 0.9473021    
## education_Some_college                        0.0000 0.9999924    
## education_Bachelor_degree                     0.0491 0.9608356    
## marital_Married_or_in_a_domestic_partnership -1.4187 0.1560751    
## employment_Employed                           1.9764 0.0481896 *  
## employment_Unemployed                         0.4886 0.6251797    
## location_Mostly_urban                        -0.5628 0.5736343    
## location_Suburban                            -0.2545 0.7991448    
## religion_Christian                            0.3026 0.7621962    
## religiosity_Attends                          -0.8972 0.3696475    
## social_media_bin_Yes                          0.1432 0.8861776    
## social_media_hours                           -0.7138 0.4754059    
## social_media_share_80_100                    -3.4006 0.0006797 ***
## social_media_share_60_80                     -0.2809 0.7788131    
## social_media_share_40_60                     -1.3453 0.1786119    
## social_media_share_20_40                     -0.8703 0.3842107    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## Best linear fit using forest predictions (on held-out data)
## as well as the mean forest prediction as regressors, along
## with one-sided heteroskedasticity-robust (HC3) SEs:
## 
##                                Estimate Std. Error t value     Pr(>t)    
## mean.forest.prediction          0.97427    0.30878  3.1552  0.0008084 ***
## differential.forest.prediction  0.66324    0.15691  4.2268 0.00001214 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

## [1] "Outcome: misinfo_avg_acc_score_post"
## [1] "Covariates: covariates_1"
##   method ranking    estimate    std.err
## 1    ols      Q1 -0.12446281 0.09542310
## 2    ols      Q2 -0.48187912 0.11067988
## 3    ols      Q3 -0.53520251 0.10014685
## 4    ols      Q4 -0.05109785 0.08696611
## 5    ols      Q5 -0.24642342 0.09666644
##   method ranking    estimate    std.err
## 1   aipw      Q1 -0.12491503 0.09028697
## 2   aipw      Q2 -0.49604069 0.10275389
## 3   aipw      Q3 -0.54775206 0.09563293
## 4   aipw      Q4 -0.05783658 0.08447146
## 5   aipw      Q5 -0.23363165 0.09449546

##                    Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 -0.35741631  0.1368866   0.009064046       0.0251
## Rank 3 - Rank 1 -0.41073970  0.1345741   0.002288634       0.0091
## Rank 4 - Rank 1  0.07336497  0.1332480   0.581949049       0.5781
## Rank 5 - Rank 1 -0.12196060  0.1365704   0.371903805       0.5694
##                    Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 -0.37112566  0.1305189   0.004487628       0.0115
## Rank 3 - Rank 1 -0.42283703  0.1284074   0.001000998       0.0031
## Rank 4 - Rank 1  0.06707845  0.1271284   0.597779272       0.6047
## Rank 5 - Rank 1 -0.10871662  0.1303031   0.404146642       0.6047
## 
## Best linear projection of the conditional average treatment effect.
## Confidence intervals are cluster- and heteroskedasticity-robust (HC3):
## 
##               Estimate Std. Error t value Pr(>|t|)
## (Intercept)   -0.03393    0.14975 -0.2266   0.8208
## att_check_pre  0.01950    0.11177  0.1745   0.8615
## base_rate_pre -0.21550    0.18709 -1.1519   0.2494
## misinfo_pre   -0.20445    0.17856 -1.1450   0.2523
## 
## 
## Best linear fit using forest predictions (on held-out data)
## as well as the mean forest prediction as regressors, along
## with one-sided heteroskedasticity-robust (HC3) SEs:
## 
##                                Estimate Std. Error t value        Pr(>t)    
## mean.forest.prediction         0.972804   0.181821  5.3503 0.00000004662 ***
## differential.forest.prediction 0.096358   0.162703  0.5922        0.2769    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

## [1] "Outcome: misinfo_avg_acc_score_post"
## [1] "Covariates: covariates_2"
##   method ranking    estimate    std.err
## 1    ols      Q1 -0.47687776 0.10694654
## 2    ols      Q2  0.01112038 0.10052937
## 3    ols      Q3 -0.44111500 0.09185121
## 4    ols      Q4 -0.21633354 0.09174166
## 5    ols      Q5 -0.30001517 0.09196726
##   method ranking     estimate    std.err
## 1   aipw      Q1 -0.461455099 0.10012296
## 2   aipw      Q2 -0.006062796 0.09480928
## 3   aipw      Q3 -0.399218074 0.08572348
## 4   aipw      Q4 -0.197564570 0.08626720
## 5   aipw      Q5 -0.259588852 0.08427206

##                   Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 0.48799814  0.1363774   0.000350386       0.0012
## Rank 3 - Rank 1 0.03576276  0.1365887   0.793468843       0.7915
## Rank 4 - Rank 1 0.26054422  0.1369816   0.057245596       0.1352
## Rank 5 - Rank 1 0.17686259  0.1363562   0.194692379       0.3126
##                   Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 0.45539230  0.1274421  0.0003570695       0.0011
## Rank 3 - Rank 1 0.06223702  0.1276643  0.6259296584       0.6263
## Rank 4 - Rank 1 0.26389053  0.1280709  0.0394216955       0.0962
## Rank 5 - Rank 1 0.20186625  0.1274864  0.1134098701       0.1844
## 
## Best linear projection of the conditional average treatment effect.
## Confidence intervals are cluster- and heteroskedasticity-robust (HC3):
## 
##                            Estimate Std. Error t value Pr(>|t|)  
## (Intercept)               -0.190735   0.074869 -2.5476  0.01089 *
## att_check_pre             -0.034076   0.076532 -0.4453  0.65616  
## misinfo_avg_acc_score_pre -0.067094   0.046544 -1.4415  0.14953  
## base_avg_acc_score_pre    -0.057056   0.036418 -1.5667  0.11728  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## Best linear fit using forest predictions (on held-out data)
## as well as the mean forest prediction as regressors, along
## with one-sided heteroskedasticity-robust (HC3) SEs:
## 
##                                Estimate Std. Error t value      Pr(>t)    
## mean.forest.prediction         0.985380   0.212159  4.6445 0.000001765 ***
## differential.forest.prediction 0.051801   0.244425  0.2119      0.4161    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

## [1] "Outcome: misinfo_avg_acc_score_post"
## [1] "Covariates: covariates_3"
##   method ranking   estimate    std.err
## 1    ols      Q1 -0.3390046 0.10814527
## 2    ols      Q2 -0.1902809 0.09897731
## 3    ols      Q3 -0.3751200 0.09395838
## 4    ols      Q4 -0.2006548 0.08776129
## 5    ols      Q5 -0.3357436 0.08838625
##   method ranking   estimate    std.err
## 1   aipw      Q1 -0.3000006 0.10267565
## 2   aipw      Q2 -0.1802461 0.09314262
## 3   aipw      Q3 -0.3600497 0.08667052
## 4   aipw      Q4 -0.2199937 0.08381102
## 5   aipw      Q5 -0.3023253 0.07938279

##                     Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1  0.148723715  0.1354077     0.2721291       0.6143
## Rank 3 - Rank 1 -0.036115461  0.1354891     0.7898261       0.9478
## Rank 4 - Rank 1  0.138349778  0.1354518     0.3071356       0.6143
## Rank 5 - Rank 1  0.003260938  0.1351735     0.9807550       0.9796
##                     Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1  0.119754524  0.1264832     0.3438030       0.7255
## Rank 3 - Rank 1 -0.060049032  0.1265272     0.6351052       0.8484
## Rank 4 - Rank 1  0.080006982  0.1265272     0.5272111       0.8484
## Rank 5 - Rank 1 -0.002324683  0.1262646     0.9853118       0.9862
## 
## Best linear projection of the conditional average treatment effect.
## Confidence intervals are cluster- and heteroskedasticity-robust (HC3):
## 
##                             Estimate Std. Error t value Pr(>|t|)
## (Intercept)               -0.1413648  0.1091300 -1.2954   0.1953
## att_check_pre              0.0022794  0.0804808  0.0283   0.9774
## base_rate_pre              0.0160246  0.2055676  0.0780   0.9379
## misinfo_pre               -0.1803972  0.2130285 -0.8468   0.3972
## misinfo_avg_acc_score_pre -0.0329430  0.0580864 -0.5671   0.5707
## base_avg_acc_score_pre    -0.0475230  0.0405191 -1.1729   0.2409
## 
## 
## Best linear fit using forest predictions (on held-out data)
## as well as the mean forest prediction as regressors, along
## with one-sided heteroskedasticity-robust (HC3) SEs:
## 
##                                Estimate Std. Error t value       Pr(>t)    
## mean.forest.prediction          1.01159    0.19969  5.0658 0.0000002135 ***
## differential.forest.prediction -0.17763    0.40251 -0.4413       0.6705    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

## [1] "Outcome: misinfo_avg_acc_score_post"
## [1] "Covariates: covariates_4"
##   method ranking   estimate    std.err
## 1    ols      Q1 -0.2717959 0.10312676
## 2    ols      Q2 -0.2439232 0.09623867
## 3    ols      Q3 -0.2005908 0.10278187
## 4    ols      Q4 -0.2485250 0.09796804
## 5    ols      Q5 -0.3960293 0.10200412
##   method ranking   estimate    std.err
## 1   aipw      Q1 -0.2641778 0.10104683
## 2   aipw      Q2 -0.2712184 0.09335445
## 3   aipw      Q3 -0.1937210 0.10130254
## 4   aipw      Q4 -0.2649666 0.09407291
## 5   aipw      Q5 -0.4095116 0.10044972

##                    Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1  0.02787271  0.1423450     0.8447691       0.9720
## Rank 3 - Rank 1  0.07120515  0.1419287     0.6159115       0.9233
## Rank 4 - Rank 1  0.02327091  0.1421853     0.8700031       0.9720
## Rank 5 - Rank 1 -0.12423335  0.1416850     0.3806385       0.7877
##                      Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 -0.0070406669  0.1388301     0.9595560       0.9982
## Rank 3 - Rank 1  0.0704567298  0.1385410     0.6110899       0.9217
## Rank 4 - Rank 1 -0.0007888565  0.1388301     0.9954666       0.9982
## Rank 5 - Rank 1 -0.1453338644  0.1383505     0.2935695       0.6659
## 
## Best linear projection of the conditional average treatment effect.
## Confidence intervals are cluster- and heteroskedasticity-robust (HC3):
## 
##                                                Estimate Std. Error t value
## (Intercept)                                   0.6134930  0.8301851  0.7390
## age                                          -0.0076316  0.0071841 -1.0623
## gender_Man                                    0.1237256  0.0945491  1.3086
## education_High_school_or_less                -0.4118917  0.4227430 -0.9743
## education_Some_college                       -0.2527238  0.3853157 -0.6559
## education_Bachelor_degree                    -0.3013874  0.3952850 -0.7625
## marital_Married_or_in_a_domestic_partnership  0.0467056  0.1449637  0.3222
## employment_Employed                          -0.0286389  0.1516932 -0.1888
## employment_Unemployed                        -0.0211956  0.1241142 -0.1708
## location_Mostly_urban                         0.1345911  0.1354474  0.9937
## location_Suburban                            -0.1260170  0.0626330 -2.0120
## religion_Christian                           -0.1746575  0.1812443 -0.9637
## religiosity_Attends                          -0.0649326  0.2150190 -0.3020
## social_media_bin_Yes                          0.0068021  0.7679496  0.0089
## social_media_hours                           -0.0126901  0.0163487 -0.7762
## social_media_share_80_100                    -0.3253806  0.2854510 -1.1399
## social_media_share_60_80                     -0.2406353  0.0943826 -2.5496
## social_media_share_40_60                     -0.1856174  0.1580764 -1.1742
## social_media_share_20_40                     -0.1036536  0.1475984 -0.7023
##                                              Pr(>|t|)  
## (Intercept)                                   0.45997  
## age                                           0.28818  
## gender_Man                                    0.19076  
## education_High_school_or_less                 0.32996  
## education_Some_college                        0.51194  
## education_Bachelor_degree                     0.44584  
## marital_Married_or_in_a_domestic_partnership  0.74733  
## employment_Employed                           0.85026  
## employment_Unemployed                         0.86441  
## location_Mostly_urban                         0.32045  
## location_Suburban                             0.04430 *
## religion_Christian                            0.33528  
## religiosity_Attends                           0.76268  
## social_media_bin_Yes                          0.99293  
## social_media_hours                            0.43767  
## social_media_share_80_100                     0.25441  
## social_media_share_60_80                      0.01083 *
## social_media_share_40_60                      0.24038  
## social_media_share_20_40                      0.48256  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## Best linear fit using forest predictions (on held-out data)
## as well as the mean forest prediction as regressors, along
## with one-sided heteroskedasticity-robust (HC3) SEs:
## 
##                                Estimate Std. Error t value         Pr(>t)    
## mean.forest.prediction          1.02329    0.16960  6.0336 0.000000000882 ***
## differential.forest.prediction -0.42298    0.20455 -2.0679         0.9806    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

## [1] "Outcome: misinfo_avg_acc_score_post"
## [1] "Covariates: covariates_5"
##   method ranking   estimate    std.err
## 1    ols      Q1 -0.2470861 0.11011451
## 2    ols      Q2 -0.3929732 0.09409176
## 3    ols      Q3 -0.3552789 0.09140147
## 4    ols      Q4 -0.2240773 0.09085611
## 5    ols      Q5 -0.1663758 0.08931954
##   method ranking   estimate    std.err
## 1   aipw      Q1 -0.2016096 0.10094026
## 2   aipw      Q2 -0.4247141 0.08941362
## 3   aipw      Q3 -0.3410410 0.08596132
## 4   aipw      Q4 -0.2334163 0.08360629
## 5   aipw      Q5 -0.1885362 0.07770244

##                    Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 -0.14588710  0.1352138     0.2806881       0.6267
## Rank 3 - Rank 1 -0.10819280  0.1348599     0.4224549       0.7488
## Rank 4 - Rank 1  0.02300878  0.1351878     0.8648634       0.8674
## Rank 5 - Rank 1  0.08071027  0.1347352     0.5491911       0.7681
##                    Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 -0.22310450  0.1243076    0.07277258       0.2030
## Rank 3 - Rank 1 -0.13943141  0.1240488    0.26108544       0.5099
## Rank 4 - Rank 1 -0.03180672  0.1243076    0.79806500       0.9517
## Rank 5 - Rank 1  0.01307343  0.1238782    0.91595751       0.9517
## 
## Best linear projection of the conditional average treatment effect.
## Confidence intervals are cluster- and heteroskedasticity-robust (HC3):
## 
##                                                Estimate Std. Error t value
## (Intercept)                                   0.4299504  0.7446643  0.5774
## att_check_pre                                -0.0115342  0.0822532 -0.1402
## base_rate_pre                                 0.0728223  0.2070427  0.3517
## misinfo_pre                                  -0.1513205  0.2219239 -0.6819
## misinfo_avg_acc_score_pre                    -0.0238919  0.0557649 -0.4284
## base_avg_acc_score_pre                       -0.0616185  0.0426710 -1.4440
## age                                          -0.0030888  0.0050236 -0.6149
## gender_Man                                    0.1509125  0.0846103  1.7836
## education_High_school_or_less                -0.3770205  0.3591838 -1.0497
## education_Some_college                       -0.2374654  0.3046334 -0.7795
## education_Bachelor_degree                    -0.2950461  0.3320374 -0.8886
## marital_Married_or_in_a_domestic_partnership  0.0087745  0.1359741  0.0645
## employment_Employed                           0.0262099  0.1035763  0.2530
## employment_Unemployed                        -0.0179403  0.1194477 -0.1502
## location_Mostly_urban                         0.0201670  0.1143238  0.1764
## location_Suburban                            -0.1870347  0.0462993 -4.0397
## religion_Christian                            0.0193248  0.1899536  0.1017
## religiosity_Attends                          -0.0978658  0.1779273 -0.5500
## social_media_bin_Yes                         -0.0967522  0.5807863 -0.1666
## social_media_hours                           -0.0077241  0.0116012 -0.6658
## social_media_share_80_100                    -0.1321438  0.2005811 -0.6588
## social_media_share_60_80                     -0.1109525  0.1059114 -1.0476
## social_media_share_40_60                     -0.0561761  0.1780428 -0.3155
## social_media_share_20_40                     -0.0297136  0.1656791 -0.1793
##                                                Pr(>|t|)    
## (Intercept)                                     0.56372    
## att_check_pre                                   0.88849    
## base_rate_pre                                   0.72506    
## misinfo_pre                                     0.49537    
## misinfo_avg_acc_score_pre                       0.66836    
## base_avg_acc_score_pre                          0.14882    
## age                                             0.53869    
## gender_Man                                      0.07457 .  
## education_High_school_or_less                   0.29395    
## education_Some_college                          0.43573    
## education_Bachelor_degree                       0.37428    
## marital_Married_or_in_a_domestic_partnership    0.94855    
## employment_Employed                             0.80024    
## employment_Unemployed                           0.88062    
## location_Mostly_urban                           0.85999    
## location_Suburban                            0.00005464 ***
## religion_Christian                              0.91897    
## religiosity_Attends                             0.58233    
## social_media_bin_Yes                            0.86770    
## social_media_hours                              0.50558    
## social_media_share_80_100                       0.51006    
## social_media_share_60_80                        0.29489    
## social_media_share_40_60                        0.75239    
## social_media_share_20_40                        0.85768    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## Best linear fit using forest predictions (on held-out data)
## as well as the mean forest prediction as regressors, along
## with one-sided heteroskedasticity-robust (HC3) SEs:
## 
##                                Estimate Std. Error t value        Pr(>t)    
## mean.forest.prediction          0.99192    0.18587  5.3367 0.00000005024 ***
## differential.forest.prediction  0.27477    0.41064  0.6691        0.2517    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

## [1] "Outcome: misinfo_avg_acc_score_diff"
## [1] "Covariates: covariates_1"
##   method ranking     estimate   std.err
## 1    ols      Q1 -0.198161076 0.1016369
## 2    ols      Q2 -0.327154266 0.1037869
## 3    ols      Q3 -0.662877565 0.1103637
## 4    ols      Q4 -0.121200646 0.1001174
## 5    ols      Q5 -0.004547106 0.1026694
##   method ranking     estimate    std.err
## 1   aipw      Q1 -0.202870189 0.09572296
## 2   aipw      Q2 -0.339347060 0.10103715
## 3   aipw      Q3 -0.679550797 0.10836737
## 4   aipw      Q4 -0.177109381 0.09347978
## 5   aipw      Q5  0.002468616 0.09961353

##                    Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 -0.12899319  0.1409717    0.36023742       0.5550
## Rank 3 - Rank 1 -0.46471649  0.1497956    0.00193481       0.0066
## Rank 4 - Rank 1  0.07696043  0.1456127    0.59716523       0.5921
## Rank 5 - Rank 1  0.19361397  0.1465378    0.18649953       0.4067
##                    Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 -0.13647687  0.1352421  0.3129795155       0.4999
## Rank 3 - Rank 1 -0.47668061  0.1438363  0.0009285995       0.0039
## Rank 4 - Rank 1  0.02576081  0.1398180  0.8538313811       0.8542
## Rank 5 - Rank 1  0.20533880  0.1407015  0.1445436224       0.3253
## 
## Best linear projection of the conditional average treatment effect.
## Confidence intervals are cluster- and heteroskedasticity-robust (HC3):
## 
##                Estimate Std. Error t value Pr(>|t|)
## (Intercept)   -0.119130   0.172867 -0.6891   0.4908
## att_check_pre -0.096511   0.090087 -1.0713   0.2841
## base_rate_pre  0.096865   0.224375  0.4317   0.6660
## misinfo_pre   -0.322862   0.204424 -1.5794   0.1143
## 
## 
## Best linear fit using forest predictions (on held-out data)
## as well as the mean forest prediction as regressors, along
## with one-sided heteroskedasticity-robust (HC3) SEs:
## 
##                                Estimate Std. Error t value     Pr(>t)    
## mean.forest.prediction          0.99105    0.23614  4.1969 0.00001385 ***
## differential.forest.prediction  0.24410    0.23879  1.0223     0.1534    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

## [1] "Outcome: misinfo_avg_acc_score_diff"
## [1] "Covariates: covariates_2"
##   method ranking    estimate    std.err
## 1    ols      Q1 -0.38982212 0.11018369
## 2    ols      Q2 -0.03203267 0.10708520
## 3    ols      Q3 -0.40482956 0.09799230
## 4    ols      Q4 -0.13583837 0.09970650
## 5    ols      Q5 -0.26133804 0.09771639
##   method ranking    estimate    std.err
## 1   aipw      Q1 -0.45400166 0.10072469
## 2   aipw      Q2 -0.01489905 0.09178962
## 3   aipw      Q3 -0.40469268 0.08833917
## 4   aipw      Q4 -0.15925134 0.08596559
## 5   aipw      Q5 -0.29326524 0.08462599

##                    Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1  0.35778945  0.1445768    0.01337898       0.0403
## Rank 3 - Rank 1 -0.01500744  0.1451319    0.91764688       0.9174
## Rank 4 - Rank 1  0.25398375  0.1447607    0.07942833       0.1788
## Rank 5 - Rank 1  0.12848408  0.1448305    0.37506583       0.5694
##                   Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 0.43910261  0.1274559  0.0005772632       0.0021
## Rank 3 - Rank 1 0.04930898  0.1279921  0.7000746354       0.6988
## Rank 4 - Rank 1 0.29475032  0.1276774  0.0210243470       0.0521
## Rank 5 - Rank 1 0.16073642  0.1277220  0.2082971043       0.3375
## 
## Best linear projection of the conditional average treatment effect.
## Confidence intervals are cluster- and heteroskedasticity-robust (HC3):
## 
##                            Estimate Std. Error t value Pr(>|t|)   
## (Intercept)               -0.192850   0.074075 -2.6034 0.009267 **
## att_check_pre             -0.028097   0.077949 -0.3605 0.718527   
## misinfo_avg_acc_score_pre -0.067863   0.047144 -1.4395 0.150101   
## base_avg_acc_score_pre    -0.057664   0.037422 -1.5409 0.123421   
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## Best linear fit using forest predictions (on held-out data)
## as well as the mean forest prediction as regressors, along
## with one-sided heteroskedasticity-robust (HC3) SEs:
## 
##                                Estimate Std. Error t value      Pr(>t)    
## mean.forest.prediction         0.990617   0.215202  4.6032 0.000002152 ***
## differential.forest.prediction 0.065659   0.234480  0.2800      0.3897    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

## [1] "Outcome: misinfo_avg_acc_score_diff"
## [1] "Covariates: covariates_3"
##   method ranking   estimate    std.err
## 1    ols      Q1 -0.2333918 0.11689909
## 2    ols      Q2 -0.3637848 0.10396557
## 3    ols      Q3 -0.2346555 0.09954634
## 4    ols      Q4 -0.0924921 0.09810162
## 5    ols      Q5 -0.3037087 0.09403426
##   method ranking   estimate    std.err
## 1   aipw      Q1 -0.2850668 0.10479693
## 2   aipw      Q2 -0.3894918 0.08900391
## 3   aipw      Q3 -0.2301252 0.08793196
## 4   aipw      Q4 -0.1454553 0.08324793
## 5   aipw      Q5 -0.3254465 0.08173409

##                     Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 -0.130393006  0.1455316     0.3703244       0.7012
## Rank 3 - Rank 1 -0.001263658  0.1453586     0.9930643       0.9926
## Rank 4 - Rank 1  0.140899704  0.1455190     0.3329810       0.7012
## Rank 5 - Rank 1 -0.070316948  0.1452614     0.6283642       0.8329
##                    Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 -0.10442492  0.1268821     0.4105580       0.7274
## Rank 3 - Rank 1  0.05494163  0.1267501     0.6647033       0.8680
## Rank 4 - Rank 1  0.13961152  0.1268821     0.2712638       0.6095
## Rank 5 - Rank 1 -0.04037969  0.1266626     0.7498986       0.8680
## 
## Best linear projection of the conditional average treatment effect.
## Confidence intervals are cluster- and heteroskedasticity-robust (HC3):
## 
##                             Estimate Std. Error t value Pr(>|t|)  
## (Intercept)               -0.1847189  0.1078653 -1.7125  0.08689 .
## att_check_pre             -0.0026121  0.0785567 -0.0333  0.97348  
## base_rate_pre              0.0840285  0.2042687  0.4114  0.68083  
## misinfo_pre               -0.1705080  0.2037324 -0.8369  0.40269  
## misinfo_avg_acc_score_pre -0.0442951  0.0577131 -0.7675  0.44283  
## base_avg_acc_score_pre    -0.0560033  0.0418623 -1.3378  0.18105  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## Best linear fit using forest predictions (on held-out data)
## as well as the mean forest prediction as regressors, along
## with one-sided heteroskedasticity-robust (HC3) SEs:
## 
##                                 Estimate Std. Error t value       Pr(>t)    
## mean.forest.prediction          0.982893   0.193346  5.0836 0.0000001945 ***
## differential.forest.prediction -0.091707   0.400640 -0.2289       0.5905    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

## [1] "Outcome: misinfo_avg_acc_score_diff"
## [1] "Covariates: covariates_4"
##   method ranking   estimate   std.err
## 1    ols      Q1 -0.1482951 0.1031413
## 2    ols      Q2 -0.2588065 0.1091433
## 3    ols      Q3 -0.3204657 0.1022182
## 4    ols      Q4 -0.4357049 0.1073206
## 5    ols      Q5 -0.1376011 0.1045696
##   method ranking   estimate   std.err
## 1   aipw      Q1 -0.1385429 0.1025320
## 2   aipw      Q2 -0.2679802 0.1089032
## 3   aipw      Q3 -0.3074576 0.1039297
## 4   aipw      Q4 -0.4336703 0.1087047
## 5   aipw      Q5 -0.1431971 0.1048334

##                    Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 -0.11051149  0.1489250    0.45809846       0.6841
## Rank 3 - Rank 1 -0.17217062  0.1486467    0.24683679       0.5179
## Rank 4 - Rank 1 -0.28740980  0.1489366    0.05371647       0.1644
## Rank 5 - Rank 1  0.01069397  0.1484108    0.94256085       0.9441
##                   Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 -0.1294373  0.1496263    0.38705752       0.5955
## Rank 3 - Rank 1 -0.1689146  0.1493147    0.25801809       0.5248
## Rank 4 - Rank 1 -0.2951273  0.1496263    0.04863656       0.1438
## Rank 5 - Rank 1 -0.0046542  0.1491094    0.97510114       0.9743
## 
## Best linear projection of the conditional average treatment effect.
## Confidence intervals are cluster- and heteroskedasticity-robust (HC3):
## 
##                                                 Estimate  Std. Error t value
## (Intercept)                                   0.13936575  0.82241169  0.1695
## age                                          -0.00046904  0.00713788 -0.0657
## gender_Man                                    0.16582582  0.08608147  1.9264
## education_High_school_or_less                -0.63573533  0.32412220 -1.9614
## education_Some_college                       -0.52610936  0.35990180 -1.4618
## education_Bachelor_degree                    -0.58229311  0.31697555 -1.8370
## marital_Married_or_in_a_domestic_partnership -0.04281358  0.17236832 -0.2484
## employment_Employed                           0.18914254  0.12860107  1.4708
## employment_Unemployed                         0.00715210  0.14957617  0.0478
## location_Mostly_urban                        -0.04294770  0.12431738 -0.3455
## location_Suburban                            -0.15635861  0.04799414 -3.2579
## religion_Christian                            0.27958975  0.34899097  0.8011
## religiosity_Attends                          -0.15437620  0.17322910 -0.8912
## social_media_bin_Yes                          0.11816658  0.52318975  0.2259
## social_media_hours                           -0.01085752  0.00633878 -1.7129
## social_media_share_80_100                    -0.14902989  0.19612120 -0.7599
## social_media_share_60_80                     -0.14630152  0.14143206 -1.0344
## social_media_share_40_60                     -0.06117955  0.19812847 -0.3088
## social_media_share_20_40                     -0.04320198  0.19939634 -0.2167
##                                              Pr(>|t|)   
## (Intercept)                                  0.865444   
## age                                          0.947611   
## gender_Man                                   0.054135 . 
## education_High_school_or_less                0.049908 * 
## education_Some_college                       0.143879   
## education_Bachelor_degree                    0.066288 . 
## marital_Married_or_in_a_domestic_partnership 0.803851   
## employment_Employed                          0.141441   
## employment_Unemployed                        0.961866   
## location_Mostly_urban                        0.729763   
## location_Suburban                            0.001133 **
## religion_Christian                           0.423105   
## religiosity_Attends                          0.372899   
## social_media_bin_Yes                         0.821325   
## social_media_hours                           0.086822 . 
## social_media_share_80_100                    0.447372   
## social_media_share_60_80                     0.301005   
## social_media_share_40_60                     0.757501   
## social_media_share_20_40                     0.828483   
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## Best linear fit using forest predictions (on held-out data)
## as well as the mean forest prediction as regressors, along
## with one-sided heteroskedasticity-robust (HC3) SEs:
## 
##                                Estimate Std. Error t value     Pr(>t)    
## mean.forest.prediction          1.01638    0.24060  4.2243 0.00001228 ***
## differential.forest.prediction -0.28005    0.16855 -1.6615     0.9517    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

## [1] "Outcome: misinfo_avg_acc_score_diff"
## [1] "Covariates: covariates_5"
##   method ranking   estimate    std.err
## 1    ols      Q1 -0.2787206 0.10789127
## 2    ols      Q2 -0.1867269 0.10748514
## 3    ols      Q3 -0.3523658 0.10462211
## 4    ols      Q4 -0.3444530 0.09796250
## 5    ols      Q5 -0.1314301 0.09244138
##   method ranking   estimate    std.err
## 1   aipw      Q1 -0.3272551 0.09846430
## 2   aipw      Q2 -0.1847723 0.09144773
## 3   aipw      Q3 -0.3512161 0.08597988
## 4   aipw      Q4 -0.3285391 0.08570407
## 5   aipw      Q5 -0.1610171 0.07794393

##                    Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1  0.09199372  0.1445176     0.5244536       0.8559
## Rank 3 - Rank 1 -0.07364526  0.1443172     0.6098714       0.8559
## Rank 4 - Rank 1 -0.06573244  0.1444738     0.6491522       0.8559
## Rank 5 - Rank 1  0.14729050  0.1439605     0.3063144       0.6846
##                     Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1  0.142482780  0.1247153     0.2533367       0.5057
## Rank 3 - Rank 1 -0.023960989  0.1244556     0.8473407       0.9731
## Rank 4 - Rank 1 -0.001284002  0.1247153     0.9917861       0.9918
## Rank 5 - Rank 1  0.166237945  0.1242845     0.1811241       0.4536
## 
## Best linear projection of the conditional average treatment effect.
## Confidence intervals are cluster- and heteroskedasticity-robust (HC3):
## 
##                                                Estimate Std. Error t value
## (Intercept)                                   0.3814426  0.7544006  0.5056
## att_check_pre                                -0.0155143  0.0817230 -0.1898
## base_rate_pre                                 0.1487682  0.2091686  0.7112
## misinfo_pre                                  -0.1553214  0.2112822 -0.7351
## misinfo_avg_acc_score_pre                    -0.0360641  0.0539420 -0.6686
## base_avg_acc_score_pre                       -0.0737079  0.0410272 -1.7966
## age                                          -0.0023820  0.0050840 -0.4685
## gender_Man                                    0.1359519  0.0866285  1.5694
## education_High_school_or_less                -0.3928146  0.3642971 -1.0783
## education_Some_college                       -0.2783596  0.3076140 -0.9049
## education_Bachelor_degree                    -0.3214310  0.3357672 -0.9573
## marital_Married_or_in_a_domestic_partnership  0.0018244  0.1391002  0.0131
## employment_Employed                           0.0378890  0.1167335  0.3246
## employment_Unemployed                        -0.0368091  0.1236122 -0.2978
## location_Mostly_urban                         0.0190520  0.1083510  0.1758
## location_Suburban                            -0.1690893  0.0433031 -3.9048
## religion_Christian                            0.0616012  0.2095067  0.2940
## religiosity_Attends                          -0.1296948  0.1889504 -0.6864
## social_media_bin_Yes                         -0.0615468  0.5712401 -0.1077
## social_media_hours                           -0.0083970  0.0102886 -0.8161
## social_media_share_80_100                    -0.1199040  0.2009987 -0.5965
## social_media_share_60_80                     -0.1120278  0.1026871 -1.0910
## social_media_share_40_60                     -0.0620262  0.1785013 -0.3475
## social_media_share_20_40                     -0.0357456  0.1683009 -0.2124
##                                                Pr(>|t|)    
## (Intercept)                                     0.61315    
## att_check_pre                                   0.84945    
## base_rate_pre                                   0.47698    
## misinfo_pre                                     0.46230    
## misinfo_avg_acc_score_pre                       0.50381    
## base_avg_acc_score_pre                          0.07249 .  
## age                                             0.63943    
## gender_Man                                      0.11665    
## education_High_school_or_less                   0.28098    
## education_Some_college                          0.36558    
## education_Bachelor_degree                       0.33848    
## marital_Married_or_in_a_domestic_partnership    0.98954    
## employment_Employed                             0.74552    
## employment_Unemployed                           0.76589    
## location_Mostly_urban                           0.86043    
## location_Suburban                            0.00009604 ***
## religion_Christian                              0.76875    
## religiosity_Attends                             0.49251    
## social_media_bin_Yes                            0.91421    
## social_media_hours                              0.41447    
## social_media_share_80_100                       0.55085    
## social_media_share_60_80                        0.27536    
## social_media_share_40_60                        0.72825    
## social_media_share_20_40                        0.83181    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## Best linear fit using forest predictions (on held-out data)
## as well as the mean forest prediction as regressors, along
## with one-sided heteroskedasticity-robust (HC3) SEs:
## 
##                                Estimate Std. Error t value       Pr(>t)    
## mean.forest.prediction          0.99279    0.19732  5.0314 0.0000002553 ***
## differential.forest.prediction  0.16506    0.35448  0.4656       0.3208    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

## [1] "Outcome: base_avg_acc_score_post"
## [1] "Covariates: covariates_1"
##   method ranking     estimate    std.err
## 1    ols      Q1 -0.156344462 0.09831204
## 2    ols      Q2 -0.607215813 0.11516712
## 3    ols      Q3 -0.350771489 0.11053732
## 4    ols      Q4 -0.008655337 0.11161414
## 5    ols      Q5 -0.077852970 0.10838528
##   method ranking    estimate    std.err
## 1   aipw      Q1 -0.16096560 0.09569419
## 2   aipw      Q2 -0.60142144 0.11547772
## 3   aipw      Q3 -0.37030997 0.10812675
## 4   aipw      Q4  0.00504691 0.11000331
## 5   aipw      Q5 -0.09121877 0.10522513

##                    Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 -0.45087135  0.1553793   0.003733302       0.0133
## Rank 3 - Rank 1 -0.19442703  0.1441387   0.177457252       0.4033
## Rank 4 - Rank 1  0.14768912  0.1493877   0.322910598       0.5126
## Rank 5 - Rank 1  0.07849149  0.1486236   0.597446604       0.5989
##                    Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 -0.44045585  0.1524072   0.003875408       0.0146
## Rank 3 - Rank 1 -0.20934437  0.1413797   0.138766914       0.3308
## Rank 4 - Rank 1  0.16601250  0.1464707   0.257113941       0.4251
## Rank 5 - Rank 1  0.06974682  0.1457617   0.632324119       0.6330
## 
## Best linear projection of the conditional average treatment effect.
## Confidence intervals are cluster- and heteroskedasticity-robust (HC3):
## 
##                Estimate Std. Error t value Pr(>|t|)  
## (Intercept)   -0.052797   0.132736 -0.3978  0.69083  
## att_check_pre  0.132458   0.117188  1.1303  0.25843  
## base_rate_pre -0.048612   0.116634 -0.4168  0.67686  
## misinfo_pre   -0.390952   0.237225 -1.6480  0.09943 .
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## Best linear fit using forest predictions (on held-out data)
## as well as the mean forest prediction as regressors, along
## with one-sided heteroskedasticity-robust (HC3) SEs:
## 
##                                Estimate Std. Error t value        Pr(>t)    
## mean.forest.prediction          0.97688    0.18717  5.2194 0.00000009482 ***
## differential.forest.prediction  0.23991    0.21583  1.1116        0.1332    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

## [1] "Outcome: base_avg_acc_score_post"
## [1] "Covariates: covariates_2"
##   method ranking    estimate   std.err
## 1    ols      Q1 -0.37577691 0.1043115
## 2    ols      Q2 -0.27301617 0.1153343
## 3    ols      Q3 -0.35075026 0.1058990
## 4    ols      Q4 -0.07033348 0.1084337
## 5    ols      Q5 -0.04249536 0.1113810
##   method ranking    estimate    std.err
## 1   aipw      Q1 -0.41295767 0.10100891
## 2   aipw      Q2 -0.20579983 0.11212072
## 3   aipw      Q3 -0.35895861 0.09795956
## 4   aipw      Q4 -0.11342206 0.10314327
## 5   aipw      Q5 -0.02265971 0.10557271

##                   Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 0.10276074  0.1536173    0.50357665       0.7357
## Rank 3 - Rank 1 0.02502664  0.1531837    0.87023099       0.8712
## Rank 4 - Rank 1 0.30544342  0.1536103    0.04683939       0.1211
## Rank 5 - Rank 1 0.33328155  0.1537244    0.03022018       0.1036
##                   Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 0.20715784  0.1465296   0.157518111       0.2661
## Rank 3 - Rank 1 0.05399906  0.1461193   0.711736052       0.7017
## Rank 4 - Rank 1 0.29953561  0.1464779   0.040934883       0.0991
## Rank 5 - Rank 1 0.39029796  0.1466334   0.007808475       0.0273
## 
## Best linear projection of the conditional average treatment effect.
## Confidence intervals are cluster- and heteroskedasticity-robust (HC3):
## 
##                            Estimate Std. Error t value Pr(>|t|)   
## (Intercept)               -0.278480   0.085673 -3.2505 0.001163 **
## att_check_pre              0.132340   0.094246  1.4042 0.160344   
## misinfo_avg_acc_score_pre -0.063671   0.050512 -1.2605 0.207563   
## base_avg_acc_score_pre    -0.002254   0.031571 -0.0714 0.943088   
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## Best linear fit using forest predictions (on held-out data)
## as well as the mean forest prediction as regressors, along
## with one-sided heteroskedasticity-robust (HC3) SEs:
## 
##                                Estimate Std. Error t value      Pr(>t)    
## mean.forest.prediction          0.97558    0.21853  4.4643 0.000004139 ***
## differential.forest.prediction  0.37154    0.18082  2.0547     0.01999 *  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

## [1] "Outcome: base_avg_acc_score_post"
## [1] "Covariates: covariates_3"
##   method ranking    estimate   std.err
## 1    ols      Q1 -0.20663945 0.1092672
## 2    ols      Q2 -0.41861743 0.1117360
## 3    ols      Q3 -0.26647416 0.1044964
## 4    ols      Q4 -0.26623088 0.1065898
## 5    ols      Q5  0.05942363 0.1132379
##   method ranking    estimate    std.err
## 1   aipw      Q1 -0.23835939 0.10546958
## 2   aipw      Q2 -0.44380964 0.10799365
## 3   aipw      Q3 -0.19417366 0.09909287
## 4   aipw      Q4 -0.30185300 0.10146002
## 5   aipw      Q5  0.09682842 0.10067850

##                    Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 -0.21197799  0.1538760    0.16841516       0.3681
## Rank 3 - Rank 1 -0.05983472  0.1536043    0.69690119       0.8920
## Rank 4 - Rank 1 -0.05959144  0.1540205    0.69884865       0.8920
## Rank 5 - Rank 1  0.26606308  0.1533982    0.08292173       0.2421
##                    Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 -0.20545025  0.1456165    0.15836005       0.3458
## Rank 3 - Rank 1  0.04418573  0.1453640    0.76117116       0.8737
## Rank 4 - Rank 1 -0.06349361  0.1456673    0.66294910       0.8737
## Rank 5 - Rank 1  0.33518781  0.1452139    0.02104218       0.0697
## 
## Best linear projection of the conditional average treatment effect.
## Confidence intervals are cluster- and heteroskedasticity-robust (HC3):
## 
##                               Estimate   Std. Error t value Pr(>|t|)
## (Intercept)               -0.069173021  0.146550816 -0.4720   0.6370
## att_check_pre              0.127340900  0.100865857  1.2625   0.2069
## base_rate_pre              0.014372232  0.207701454  0.0692   0.9448
## misinfo_pre               -0.430687123  0.317840446 -1.3550   0.1755
## misinfo_avg_acc_score_pre  0.000091116  0.069460617  0.0013   0.9990
## base_avg_acc_score_pre     0.017028741  0.044767545  0.3804   0.7037
## 
## 
## Best linear fit using forest predictions (on held-out data)
## as well as the mean forest prediction as regressors, along
## with one-sided heteroskedasticity-robust (HC3) SEs:
## 
##                                Estimate Std. Error t value     Pr(>t)    
## mean.forest.prediction          1.01415    0.25393  3.9939 0.00003315 ***
## differential.forest.prediction  0.53029    0.22092  2.4004   0.008214 ** 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

## [1] "Outcome: base_avg_acc_score_post"
## [1] "Covariates: covariates_4"
##   method ranking   estimate   std.err
## 1    ols      Q1 -0.2686269 0.1125817
## 2    ols      Q2 -0.1301686 0.1103209
## 3    ols      Q3 -0.3317427 0.1084277
## 4    ols      Q4 -0.1911442 0.1068102
## 5    ols      Q5 -0.2009593 0.1098250
##   method ranking   estimate   std.err
## 1   aipw      Q1 -0.2744643 0.1106019
## 2   aipw      Q2 -0.1555857 0.1098859
## 3   aipw      Q3 -0.3324959 0.1051930
## 4   aipw      Q4 -0.2089315 0.1068278
## 5   aipw      Q5 -0.2125422 0.1092493

##                    Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1  0.13845836  0.1551568     0.3722504       0.7823
## Rank 3 - Rank 1 -0.06311573  0.1547963     0.6834930       0.9201
## Rank 4 - Rank 1  0.07748268  0.1552024     0.6176432       0.9201
## Rank 5 - Rank 1  0.06766762  0.1545351     0.6615004       0.9201
##                    Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1  0.11887859  0.1532995     0.4381147       0.8475
## Rank 3 - Rank 1 -0.05803166  0.1529264     0.7043579       0.9513
## Rank 4 - Rank 1  0.06553277  0.1532458     0.6689447       0.9513
## Rank 5 - Rank 1  0.06192205  0.1527161     0.6851547       0.9513
## 
## Best linear projection of the conditional average treatment effect.
## Confidence intervals are cluster- and heteroskedasticity-robust (HC3):
## 
##                                                Estimate Std. Error t value
## (Intercept)                                   0.1098168  0.8002855  0.1372
## age                                          -0.0131738  0.0068629 -1.9196
## gender_Man                                    0.1387285  0.1131433  1.2261
## education_High_school_or_less                 0.1427697  0.4433724  0.3220
## education_Some_college                        0.2545271  0.4305708  0.5911
## education_Bachelor_degree                     0.3406800  0.4786542  0.7117
## marital_Married_or_in_a_domestic_partnership -0.1408116  0.1376481 -1.0230
## employment_Employed                           0.2585267  0.0836453  3.0907
## employment_Unemployed                         0.0990441  0.1201051  0.8246
## location_Mostly_urban                        -0.0126475  0.1361117 -0.0929
## location_Suburban                            -0.1718983  0.1326852 -1.2955
## religion_Christian                            0.0299096  0.2188881  0.1366
## religiosity_Attends                          -0.4382893  0.2035258 -2.1535
## social_media_bin_Yes                          0.3599272  0.7063425  0.5096
## social_media_hours                           -0.0057194  0.0118979 -0.4807
## social_media_share_80_100                    -0.6426736  0.3381886 -1.9003
## social_media_share_60_80                     -0.2766608  0.2160548 -1.2805
## social_media_share_40_60                     -0.3784279  0.1626633 -2.3264
## social_media_share_20_40                     -0.2735952  0.2187977 -1.2504
##                                              Pr(>|t|)   
## (Intercept)                                  0.890863   
## age                                          0.054991 . 
## gender_Man                                   0.220229   
## education_High_school_or_less                0.747465   
## education_Some_college                       0.554464   
## education_Bachelor_degree                    0.476668   
## marital_Married_or_in_a_domestic_partnership 0.306385   
## employment_Employed                          0.002012 **
## employment_Unemployed                        0.409628   
## location_Mostly_urban                        0.925972   
## location_Suburban                            0.195219   
## religion_Christian                           0.891321   
## religiosity_Attends                          0.031347 * 
## social_media_bin_Yes                         0.610388   
## social_media_hours                           0.630756   
## social_media_share_80_100                    0.057468 . 
## social_media_share_60_80                     0.200447   
## social_media_share_40_60                     0.020050 * 
## social_media_share_20_40                     0.211217   
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## Best linear fit using forest predictions (on held-out data)
## as well as the mean forest prediction as regressors, along
## with one-sided heteroskedasticity-robust (HC3) SEs:
## 
##                                Estimate Std. Error t value              Pr(>t)
## mean.forest.prediction          1.02471    0.13563  7.5551 0.00000000000002634
## differential.forest.prediction  0.10028    0.28093  0.3570              0.3606
##                                   
## mean.forest.prediction         ***
## differential.forest.prediction    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

## [1] "Outcome: base_avg_acc_score_post"
## [1] "Covariates: covariates_5"
##   method ranking    estimate   std.err
## 1    ols      Q1 -0.28260242 0.1147716
## 2    ols      Q2 -0.23138999 0.1062496
## 3    ols      Q3 -0.37425374 0.1021851
## 4    ols      Q4 -0.20288509 0.1054207
## 5    ols      Q5 -0.04050125 0.1132276
##   method ranking    estimate   std.err
## 1   aipw      Q1 -0.27007632 0.1046628
## 2   aipw      Q2 -0.23453326 0.1027899
## 3   aipw      Q3 -0.37248067 0.0975461
## 4   aipw      Q4 -0.20160530 0.1009394
## 5   aipw      Q5 -0.09157136 0.1035129

##                    Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1  0.05121244  0.1533649     0.7384552       0.8712
## Rank 3 - Rank 1 -0.09165132  0.1530273     0.5492628       0.8712
## Rank 4 - Rank 1  0.07971733  0.1534535     0.6034519       0.8712
## Rank 5 - Rank 1  0.24210117  0.1528815     0.1133751       0.3070
##                    Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1  0.03554306  0.1441777     0.8052917       0.8527
## Rank 3 - Rank 1 -0.10240435  0.1438775     0.4766673       0.8195
## Rank 4 - Rank 1  0.06847102  0.1441777     0.6348818       0.8527
## Rank 5 - Rank 1  0.17850495  0.1436797     0.2141764       0.5337
## 
## Best linear projection of the conditional average treatment effect.
## Confidence intervals are cluster- and heteroskedasticity-robust (HC3):
## 
##                                                Estimate Std. Error t value
## (Intercept)                                  -0.0344412  0.9942499 -0.0346
## att_check_pre                                 0.1097550  0.1027718  1.0679
## base_rate_pre                                 0.0557639  0.2358494  0.2364
## misinfo_pre                                  -0.3352202  0.3423922 -0.9791
## misinfo_avg_acc_score_pre                    -0.0049514  0.0682710 -0.0725
## base_avg_acc_score_pre                        0.0011565  0.0426565  0.0271
## age                                          -0.0079383  0.0067112 -1.1828
## gender_Man                                    0.1750576  0.1089050  1.6074
## education_High_school_or_less                 0.2527367  0.4726904  0.5347
## education_Some_college                        0.2847787  0.4625021  0.6157
## education_Bachelor_degree                     0.3705926  0.4956195  0.7477
## marital_Married_or_in_a_domestic_partnership -0.1819815  0.1308952 -1.3903
## employment_Employed                           0.3086180  0.0655292  4.7096
## employment_Unemployed                         0.1197164  0.1137293  1.0526
## location_Mostly_urban                        -0.0931188  0.1106570 -0.8415
## location_Suburban                            -0.2179062  0.1292505 -1.6859
## religion_Christian                            0.1342213  0.2483560  0.5404
## religiosity_Attends                          -0.4195998  0.2036889 -2.0600
## social_media_bin_Yes                          0.1652150  0.6769726  0.2440
## social_media_hours                           -0.0020510  0.0099836 -0.2054
## social_media_share_80_100                    -0.4455152  0.3040792 -1.4651
## social_media_share_60_80                     -0.1287652  0.1881109 -0.6845
## social_media_share_40_60                     -0.2245805  0.1657968 -1.3546
## social_media_share_20_40                     -0.2036276  0.1960060 -1.0389
##                                                 Pr(>|t|)    
## (Intercept)                                      0.97237    
## att_check_pre                                    0.28562    
## base_rate_pre                                    0.81311    
## misinfo_pre                                      0.32762    
## misinfo_avg_acc_score_pre                        0.94219    
## base_avg_acc_score_pre                           0.97837    
## age                                              0.23695    
## gender_Man                                       0.10805    
## education_High_school_or_less                    0.59291    
## education_Some_college                           0.53811    
## education_Bachelor_degree                        0.45467    
## marital_Married_or_in_a_domestic_partnership     0.16453    
## employment_Employed                          0.000002575 ***
## employment_Unemployed                            0.29257    
## location_Mostly_urban                            0.40012    
## location_Suburban                                0.09190 .  
## religion_Christian                               0.58893    
## religiosity_Attends                              0.03947 *  
## social_media_bin_Yes                             0.80721    
## social_media_hours                               0.83724    
## social_media_share_80_100                        0.14297    
## social_media_share_60_80                         0.49369    
## social_media_share_40_60                         0.17565    
## social_media_share_20_40                         0.29893    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## Best linear fit using forest predictions (on held-out data)
## as well as the mean forest prediction as regressors, along
## with one-sided heteroskedasticity-robust (HC3) SEs:
## 
##                                Estimate Std. Error t value       Pr(>t)    
## mean.forest.prediction          1.02598    0.20772  4.9392 0.0000004099 ***
## differential.forest.prediction  0.44691    0.38636  1.1567       0.1237    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

## [1] "Outcome: base_avg_acc_score_diff"
## [1] "Covariates: covariates_1"
##   method ranking     estimate   std.err
## 1    ols      Q1 -0.475974753 0.1216390
## 2    ols      Q2  0.001025919 0.1197662
## 3    ols      Q3 -0.428318687 0.1349359
## 4    ols      Q4 -0.237257705 0.1252538
## 5    ols      Q5 -0.038251913 0.1235154
##   method ranking     estimate   std.err
## 1   aipw      Q1 -0.528139389 0.1174458
## 2   aipw      Q2  0.048095196 0.1164961
## 3   aipw      Q3 -0.458158350 0.1319925
## 4   aipw      Q4 -0.229498942 0.1219248
## 5   aipw      Q5  0.006914535 0.1163285

##                   Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 0.47700067  0.1699318   0.005027102       0.0192
## Rank 3 - Rank 1 0.04765607  0.1773870   0.788209829       0.7888
## Rank 4 - Rank 1 0.23871705  0.1753883   0.173574043       0.3024
## Rank 5 - Rank 1 0.43772284  0.1756044   0.012723263       0.0392
##                   Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 0.57623459  0.1642343  0.0004559315       0.0025
## Rank 3 - Rank 1 0.06998104  0.1715588  0.6833617624       0.6918
## Rank 4 - Rank 1 0.29864045  0.1696300  0.0784006124       0.1423
## Rank 5 - Rank 1 0.53505392  0.1698230  0.0016423279       0.0055
## 
## Best linear projection of the conditional average treatment effect.
## Confidence intervals are cluster- and heteroskedasticity-robust (HC3):
## 
##                Estimate Std. Error t value Pr(>|t|)
## (Intercept)   -0.163655   0.141293 -1.1583   0.2468
## att_check_pre  0.094802   0.101988  0.9295   0.3527
## base_rate_pre  0.119287   0.153220  0.7785   0.4363
## misinfo_pre   -0.358244   0.264687 -1.3535   0.1760
## 
## 
## Best linear fit using forest predictions (on held-out data)
## as well as the mean forest prediction as regressors, along
## with one-sided heteroskedasticity-robust (HC3) SEs:
## 
##                                Estimate Std. Error t value    Pr(>t)    
## mean.forest.prediction          0.98123    0.28865  3.3993 0.0003414 ***
## differential.forest.prediction  0.21639    0.25607  0.8451 0.1990691    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

## [1] "Outcome: base_avg_acc_score_diff"
## [1] "Covariates: covariates_2"
##   method ranking   estimate   std.err
## 1    ols      Q1 -0.4682114 0.1162732
## 2    ols      Q2 -0.2982771 0.1230200
## 3    ols      Q3 -0.3413551 0.1125399
## 4    ols      Q4 -0.2000772 0.1359668
## 5    ols      Q5  0.1641402 0.1320551
##   method ranking    estimate    std.err
## 1   aipw      Q1 -0.38310264 0.10092503
## 2   aipw      Q2 -0.25930170 0.11399967
## 3   aipw      Q3 -0.33030759 0.09793411
## 4   aipw      Q4 -0.20609408 0.10245988
## 5   aipw      Q5  0.06738508 0.10494345

##                  Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 0.1699344  0.1752239  0.3322047105       0.5232
## Rank 3 - Rank 1 0.1268563  0.1738683  0.4656747581       0.5232
## Rank 4 - Rank 1 0.2681342  0.1750869  0.1257494557       0.2967
## Rank 5 - Rank 1 0.6323516  0.1747123  0.0002993449       0.0014
##                   Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 0.12380094  0.1469996   0.399739635       0.6115
## Rank 3 - Rank 1 0.05279505  0.1459262   0.717527920       0.7213
## Rank 4 - Rank 1 0.17700856  0.1469472   0.228447140       0.4794
## Rank 5 - Rank 1 0.45048772  0.1466357   0.002140985       0.0092
## 
## Best linear projection of the conditional average treatment effect.
## Confidence intervals are cluster- and heteroskedasticity-robust (HC3):
## 
##                              Estimate  Std. Error t value Pr(>|t|)   
## (Intercept)               -0.27879133  0.08653276 -3.2218 0.001285 **
## att_check_pre              0.13148146  0.09524749  1.3804 0.167543   
## misinfo_avg_acc_score_pre -0.06361082  0.05170798 -1.2302 0.218704   
## base_avg_acc_score_pre    -0.00048318  0.03202866 -0.0151 0.987965   
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## Best linear fit using forest predictions (on held-out data)
## as well as the mean forest prediction as regressors, along
## with one-sided heteroskedasticity-robust (HC3) SEs:
## 
##                                Estimate Std. Error t value      Pr(>t)    
## mean.forest.prediction          0.95784    0.21739  4.4062 0.000005413 ***
## differential.forest.prediction  0.36966    0.17645  2.0950     0.01812 *  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

## [1] "Outcome: base_avg_acc_score_diff"
## [1] "Covariates: covariates_3"
##   method ranking    estimate   std.err
## 1    ols      Q1 -0.23388081 0.1216451
## 2    ols      Q2 -0.44262741 0.1198506
## 3    ols      Q3 -0.43165876 0.1235464
## 4    ols      Q4 -0.15693477 0.1309852
## 5    ols      Q5  0.09420466 0.1280226
##   method ranking     estimate   std.err
## 1   aipw      Q1 -0.208939283 0.1067366
## 2   aipw      Q2 -0.455198912 0.1039547
## 3   aipw      Q3 -0.304062802 0.1021031
## 4   aipw      Q4 -0.122372355 0.1043356
## 5   aipw      Q5 -0.001302975 0.1007874

##                    Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 -0.20874659  0.1765223    0.23706502       0.5021
## Rank 3 - Rank 1 -0.19777795  0.1763081    0.26203277       0.5021
## Rank 4 - Rank 1  0.07694604  0.1761826    0.66232585       0.6634
## Rank 5 - Rank 1  0.32808548  0.1760516    0.06246239       0.1952
##                    Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 -0.24625963  0.1463766    0.09258337       0.2654
## Rank 3 - Rank 1 -0.09512352  0.1461202    0.51509104       0.7322
## Rank 4 - Rank 1  0.08656693  0.1461202    0.55359539       0.7322
## Rank 5 - Rank 1  0.20763631  0.1459678    0.15497311       0.3479
## 
## Best linear projection of the conditional average treatment effect.
## Confidence intervals are cluster- and heteroskedasticity-robust (HC3):
## 
##                             Estimate Std. Error t value Pr(>|t|)
## (Intercept)               -0.0830433  0.1372526 -0.6050   0.5452
## att_check_pre              0.1157518  0.0941662  1.2292   0.2191
## base_rate_pre              0.0692846  0.2138776  0.3239   0.7460
## misinfo_pre               -0.4561994  0.3119581 -1.4624   0.1437
## misinfo_avg_acc_score_pre  0.0023611  0.0724740  0.0326   0.9740
## base_avg_acc_score_pre     0.0091592  0.0478352  0.1915   0.8482
## 
## 
## Best linear fit using forest predictions (on held-out data)
## as well as the mean forest prediction as regressors, along
## with one-sided heteroskedasticity-robust (HC3) SEs:
## 
##                                Estimate Std. Error t value     Pr(>t)    
## mean.forest.prediction          0.99907    0.23529  4.2461 0.00001115 ***
## differential.forest.prediction  0.50654    0.23758  2.1321    0.01653 *  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

## [1] "Outcome: base_avg_acc_score_diff"
## [1] "Covariates: covariates_4"
##   method ranking    estimate   std.err
## 1    ols      Q1 -0.06116930 0.1249143
## 2    ols      Q2 -0.38167373 0.1191319
## 3    ols      Q3 -0.33988446 0.1302489
## 4    ols      Q4 -0.32441957 0.1285879
## 5    ols      Q5 -0.05721553 0.1238264
##   method ranking    estimate   std.err
## 1   aipw      Q1 -0.07152127 0.1230580
## 2   aipw      Q2 -0.40379793 0.1197034
## 3   aipw      Q3 -0.34744332 0.1300497
## 4   aipw      Q4 -0.33994488 0.1295342
## 5   aipw      Q5 -0.07484224 0.1248680

##                     Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 -0.320504435  0.1772316    0.07062838       0.2185
## Rank 3 - Rank 1 -0.278715164  0.1768484    0.11511051       0.2710
## Rank 4 - Rank 1 -0.263250278  0.1772124    0.13749673       0.2710
## Rank 5 - Rank 1  0.003953771  0.1767014    0.98214971       0.9809
##                     Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 -0.332276656  0.1775193    0.06131788       0.1879
## Rank 3 - Rank 1 -0.275922045  0.1771496    0.11942370       0.2746
## Rank 4 - Rank 1 -0.268423612  0.1775193    0.13060037       0.2746
## Rank 5 - Rank 1 -0.003320966  0.1769061    0.98502364       0.9830
## 
## Best linear projection of the conditional average treatment effect.
## Confidence intervals are cluster- and heteroskedasticity-robust (HC3):
## 
##                                                Estimate Std. Error t value
## (Intercept)                                   0.1147741  1.5613383  0.0735
## age                                          -0.0081031  0.0108129 -0.7494
## gender_Man                                    0.1063258  0.1432906  0.7420
## education_High_school_or_less                -0.0178978  0.6248963 -0.0286
## education_Some_college                       -0.0107945  0.6239641 -0.0173
## education_Bachelor_degree                     0.1645116  0.6335735  0.2597
## marital_Married_or_in_a_domestic_partnership -0.3486349  0.1214569 -2.8704
## employment_Employed                           0.5360710  0.0848355  6.3189
## employment_Unemployed                         0.2490548  0.1492140  1.6691
## location_Mostly_urban                        -0.0957587  0.1747378 -0.5480
## location_Suburban                            -0.1495454  0.1280981 -1.1674
## religion_Christian                            0.1107813  0.2796199  0.3962
## religiosity_Attends                          -0.4118541  0.3103146 -1.3272
## social_media_bin_Yes                          0.1855852  1.0947166  0.1695
## social_media_hours                           -0.0024643  0.0157729 -0.1562
## social_media_share_80_100                    -0.5509431  0.4058062 -1.3577
## social_media_share_60_80                     -0.1422134  0.2882036 -0.4934
## social_media_share_40_60                     -0.2883522  0.2094457 -1.3767
## social_media_share_20_40                     -0.2924142  0.2392515 -1.2222
##                                                     Pr(>|t|)    
## (Intercept)                                         0.941404    
## age                                                 0.453670    
## gender_Man                                          0.458118    
## education_High_school_or_less                       0.977152    
## education_Some_college                              0.986198    
## education_Bachelor_degree                           0.795143    
## marital_Married_or_in_a_domestic_partnership        0.004123 ** 
## employment_Employed                          0.0000000002955 ***
## employment_Unemployed                               0.095182 .  
## location_Mostly_urban                               0.583716    
## location_Suburban                                   0.243115    
## religion_Christian                                  0.691992    
## religiosity_Attends                                 0.184522    
## social_media_bin_Yes                                0.865391    
## social_media_hours                                  0.875854    
## social_media_share_80_100                           0.174659    
## social_media_share_60_80                            0.621726    
## social_media_share_40_60                            0.168678    
## social_media_share_20_40                            0.221710    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## Best linear fit using forest predictions (on held-out data)
## as well as the mean forest prediction as regressors, along
## with one-sided heteroskedasticity-robust (HC3) SEs:
## 
##                                Estimate Std. Error t value    Pr(>t)    
## mean.forest.prediction         1.021149   0.305983  3.3373 0.0004273 ***
## differential.forest.prediction 0.036168   0.342665  0.1055 0.4579727    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

## [1] "Outcome: base_avg_acc_score_diff"
## [1] "Covariates: covariates_5"
##   method ranking   estimate   std.err
## 1    ols      Q1 -0.3774760 0.1213719
## 2    ols      Q2 -0.0177305 0.1222472
## 3    ols      Q3 -0.3173456 0.1243382
## 4    ols      Q4 -0.3302064 0.1236126
## 5    ols      Q5 -0.1106574 0.1316710
##   method ranking    estimate    std.err
## 1   aipw      Q1 -0.35025145 0.10529077
## 2   aipw      Q2 -0.05204375 0.10451873
## 3   aipw      Q3 -0.40323218 0.10132726
## 4   aipw      Q4 -0.31241935 0.09872365
## 5   aipw      Q5 -0.07649221 0.10239539

##                   Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 0.35974552  0.1765819    0.04169542       0.1337
## Rank 3 - Rank 1 0.06013044  0.1761481    0.73285012       0.9196
## Rank 4 - Rank 1 0.04726964  0.1765218    0.78888121       0.9196
## Rank 5 - Rank 1 0.26681864  0.1758876    0.12935809       0.3093
##                    Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1  0.29820770  0.1449683    0.03975240       0.1249
## Rank 3 - Rank 1 -0.05298074  0.1446664    0.71421704       0.9023
## Rank 4 - Rank 1  0.03783210  0.1449683    0.79413190       0.9023
## Rank 5 - Rank 1  0.27375924  0.1444675    0.05817811       0.1404
## 
## Best linear projection of the conditional average treatment effect.
## Confidence intervals are cluster- and heteroskedasticity-robust (HC3):
## 
##                                                 Estimate  Std. Error t value
## (Intercept)                                  -0.12638467  1.08461361 -0.1165
## att_check_pre                                 0.12207528  0.10133464  1.2047
## base_rate_pre                                 0.10679351  0.23507826  0.4543
## misinfo_pre                                  -0.33164075  0.34925677 -0.9496
## misinfo_avg_acc_score_pre                    -0.00189357  0.07291019 -0.0260
## base_avg_acc_score_pre                       -0.00202466  0.04032786 -0.0502
## age                                          -0.00753523  0.00645622 -1.1671
## gender_Man                                    0.17123346  0.10700316  1.6003
## education_High_school_or_less                 0.26132706  0.48907625  0.5343
## education_Some_college                        0.30573467  0.48318743  0.6327
## education_Bachelor_degree                     0.37791150  0.51168150  0.7386
## marital_Married_or_in_a_domestic_partnership -0.21358628  0.13468245 -1.5859
## employment_Employed                           0.32334851  0.06561598  4.9279
## employment_Unemployed                         0.13630052  0.11157717  1.2216
## location_Mostly_urban                        -0.11291146  0.10697070 -1.0555
## location_Suburban                            -0.19850011  0.12693466 -1.5638
## religion_Christian                            0.19819613  0.25182971  0.7870
## religiosity_Attends                          -0.45741451  0.21646141 -2.1131
## social_media_bin_Yes                          0.17735351  0.68255704  0.2598
## social_media_hours                           -0.00066227  0.00974503 -0.0680
## social_media_share_80_100                    -0.46352424  0.31495630 -1.4717
## social_media_share_60_80                     -0.14530736  0.19727049 -0.7366
## social_media_share_40_60                     -0.25441674  0.16546704 -1.5376
## social_media_share_20_40                     -0.23760993  0.19487991 -1.2193
##                                                  Pr(>|t|)    
## (Intercept)                                       0.90724    
## att_check_pre                                     0.22841    
## base_rate_pre                                     0.64965    
## misinfo_pre                                       0.34240    
## misinfo_avg_acc_score_pre                         0.97928    
## base_avg_acc_score_pre                            0.95996    
## age                                               0.24324    
## gender_Man                                        0.10963    
## education_High_school_or_less                     0.59315    
## education_Some_college                            0.52694    
## education_Bachelor_degree                         0.46022    
## marital_Married_or_in_a_domestic_partnership      0.11286    
## employment_Employed                          0.0000008687 ***
## employment_Unemployed                             0.22195    
## location_Mostly_urban                             0.29125    
## location_Suburban                                 0.11795    
## religion_Christian                                0.43132    
## religiosity_Attends                               0.03466 *  
## social_media_bin_Yes                              0.79500    
## social_media_hours                                0.94582    
## social_media_share_80_100                         0.14119    
## social_media_share_60_80                          0.46142    
## social_media_share_40_60                          0.12424    
## social_media_share_20_40                          0.22282    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## Best linear fit using forest predictions (on held-out data)
## as well as the mean forest prediction as regressors, along
## with one-sided heteroskedasticity-robust (HC3) SEs:
## 
##                                Estimate Std. Error t value       Pr(>t)    
## mean.forest.prediction          1.00553    0.19874  5.0595 0.0000002206 ***
## differential.forest.prediction  0.42170    0.41934  1.0056       0.1573    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

## [1] "Outcome: new_acc_post"
## [1] "Covariates: covariates_1"
##   method ranking    estimate    std.err
## 1    ols      Q1 -0.05021097 0.10014525
## 2    ols      Q2 -0.02103178 0.08953972
## 3    ols      Q3  0.09487193 0.09838262
## 4    ols      Q4  0.04437422 0.09731614
## 5    ols      Q5  0.17440620 0.09762454
##   method ranking     estimate    std.err
## 1   aipw      Q1 -0.057719235 0.09997407
## 2   aipw      Q2 -0.005169919 0.08905108
## 3   aipw      Q3  0.118865806 0.09723328
## 4   aipw      Q4  0.044743502 0.09718803
## 5   aipw      Q5  0.171419074 0.09824455

##                   Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 0.02917919  0.1324390     0.8256330       0.8270
## Rank 3 - Rank 1 0.14508290  0.1372162     0.2904311       0.5745
## Rank 4 - Rank 1 0.09458519  0.1339810     0.4802588       0.6985
## Rank 5 - Rank 1 0.22461717  0.1372162     0.1017261       0.2830
##                   Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 0.05254932  0.1322094    0.69104413       0.6889
## Rank 3 - Rank 1 0.17658504  0.1370234    0.19757723       0.4279
## Rank 4 - Rank 1 0.10246274  0.1338111    0.44388874       0.6553
## Rank 5 - Rank 1 0.22913831  0.1370234    0.09456015       0.2744
## 
## Best linear projection of the conditional average treatment effect.
## Confidence intervals are cluster- and heteroskedasticity-robust (HC3):
## 
##                Estimate Std. Error t value Pr(>|t|)
## (Intercept)   -0.020618   0.078135 -0.2639   0.7919
## att_check_pre  0.113043   0.082510  1.3700   0.1708
## base_rate_pre  0.181728   0.144607  1.2567   0.2089
## misinfo_pre   -0.201226   0.196700 -1.0230   0.3064
## 
## 
## Best linear fit using forest predictions (on held-out data)
## as well as the mean forest prediction as regressors, along
## with one-sided heteroskedasticity-robust (HC3) SEs:
## 
##                                Estimate Std. Error t value Pr(>t)
## mean.forest.prediction          0.99429    1.08192  0.9190 0.1791
## differential.forest.prediction  0.20283    0.21249  0.9545 0.1699

## [1] "Outcome: new_acc_post"
## [1] "Covariates: covariates_2"
##   method ranking    estimate    std.err
## 1    ols      Q1 -0.16687065 0.09600259
## 2    ols      Q2 -0.08101805 0.09944335
## 3    ols      Q3  0.04017043 0.09397158
## 4    ols      Q4  0.32139081 0.09333389
## 5    ols      Q5  0.13933218 0.09962347
##   method ranking    estimate    std.err
## 1   aipw      Q1 -0.18716898 0.09744819
## 2   aipw      Q2 -0.05049713 0.09789579
## 3   aipw      Q3  0.03850712 0.09356619
## 4   aipw      Q4  0.30036877 0.09472870
## 5   aipw      Q5  0.12850951 0.10122222

##                  Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 0.0858526  0.1356841  0.5269442644       0.5294
## Rank 3 - Rank 1 0.2070411  0.1357284  0.1272451137       0.2176
## Rank 4 - Rank 1 0.4882615  0.1360710  0.0003372727       0.0009
## Rank 5 - Rank 1 0.3062028  0.1362732  0.0247015228       0.0637
##                  Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 0.1366719  0.1364046  0.3164301325       0.3235
## Rank 3 - Rank 1 0.2256761  0.1364521  0.0982374958       0.1789
## Rank 4 - Rank 1 0.4875378  0.1366433  0.0003644336       0.0011
## Rank 5 - Rank 1 0.3156785  0.1370313  0.0212960347       0.0590
## 
## Best linear projection of the conditional average treatment effect.
## Confidence intervals are cluster- and heteroskedasticity-robust (HC3):
## 
##                              Estimate  Std. Error t value Pr(>|t|)   
## (Intercept)               -0.08349257  0.06528817 -1.2788 0.201038   
## att_check_pre              0.15826328  0.06496001  2.4363 0.014885 * 
## misinfo_avg_acc_score_pre -0.00090502  0.02576597 -0.0351 0.971982   
## base_avg_acc_score_pre     0.05867591  0.02109003  2.7822 0.005428 **
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## Best linear fit using forest predictions (on held-out data)
## as well as the mean forest prediction as regressors, along
## with one-sided heteroskedasticity-robust (HC3) SEs:
## 
##                                Estimate Std. Error t value     Pr(>t)    
## mean.forest.prediction          1.03409    1.19734  0.8637     0.1939    
## differential.forest.prediction  0.46552    0.12109  3.8445 0.00006145 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

## [1] "Outcome: new_acc_post"
## [1] "Covariates: covariates_3"
##   method ranking    estimate    std.err
## 1    ols      Q1 -0.09309585 0.09789299
## 2    ols      Q2 -0.02981800 0.09566205
## 3    ols      Q3  0.08170086 0.09420811
## 4    ols      Q4  0.09788518 0.10369534
## 5    ols      Q5  0.19940247 0.09132459
##   method ranking   estimate    std.err
## 1   aipw      Q1 -0.1125933 0.09808577
## 2   aipw      Q2 -0.0458247 0.09377710
## 3   aipw      Q3  0.1084676 0.09281959
## 4   aipw      Q4  0.0921863 0.10329327
## 5   aipw      Q5  0.2353064 0.09172730

##                   Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 0.06327785  0.1366623    0.64337541       0.6341
## Rank 3 - Rank 1 0.17479671  0.1362026    0.19944896       0.3592
## Rank 4 - Rank 1 0.19098104  0.1364989    0.16185675       0.3592
## Rank 5 - Rank 1 0.29249833  0.1360809    0.03166538       0.0979
##                   Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 0.06676859  0.1358333    0.62306861       0.6234
## Rank 3 - Rank 1 0.22106094  0.1354070    0.10264709       0.2404
## Rank 4 - Rank 1 0.20477959  0.1356427    0.13120748       0.2404
## Rank 5 - Rank 1 0.34789964  0.1352669    0.01015231       0.0334
## 
## Best linear projection of the conditional average treatment effect.
## Confidence intervals are cluster- and heteroskedasticity-robust (HC3):
## 
##                             Estimate Std. Error t value  Pr(>|t|)    
## (Intercept)                0.0803801  0.0957086  0.8398 0.4010522    
## att_check_pre              0.1164538  0.0726187  1.6036 0.1088817    
## base_rate_pre             -0.0075429  0.1462826 -0.0516 0.9588789    
## misinfo_pre               -0.2469489  0.2628822 -0.9394 0.3475933    
## misinfo_avg_acc_score_pre  0.0350544  0.0456281  0.7683 0.4423808    
## base_avg_acc_score_pre     0.0605441  0.0178736  3.3874 0.0007132 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## Best linear fit using forest predictions (on held-out data)
## as well as the mean forest prediction as regressors, along
## with one-sided heteroskedasticity-robust (HC3) SEs:
## 
##                                Estimate Std. Error t value      Pr(>t)    
## mean.forest.prediction          0.95133    0.92789  1.0253      0.1527    
## differential.forest.prediction  0.59257    0.13595  4.3587 0.000006722 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

## [1] "Outcome: new_acc_post"
## [1] "Covariates: covariates_4"
##   method ranking     estimate    std.err
## 1    ols      Q1  0.159817349 0.09710494
## 2    ols      Q2  0.135242517 0.09616436
## 3    ols      Q3  0.004858308 0.09154957
## 4    ols      Q4  0.067911927 0.10105641
## 5    ols      Q5 -0.138173047 0.09794718
##   method ranking     estimate    std.err
## 1   aipw      Q1  0.155712691 0.09710276
## 2   aipw      Q2  0.142155071 0.09604429
## 3   aipw      Q3 -0.003608882 0.09099923
## 4   aipw      Q4  0.086836165 0.09976879
## 5   aipw      Q5 -0.148704544 0.09843539

##                    Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 -0.02457483  0.1368207    0.85746611       0.8579
## Rank 3 - Rank 1 -0.15495904  0.1364388    0.25614053       0.5315
## Rank 4 - Rank 1 -0.09190542  0.1367823    0.50168366       0.7304
## Rank 5 - Rank 1 -0.29799040  0.1362544    0.02880535       0.0921
##                    Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 -0.01355762  0.1365150    0.92089581       0.9234
## Rank 3 - Rank 1 -0.15932157  0.1362307    0.24227995       0.5007
## Rank 4 - Rank 1 -0.06887653  0.1365150    0.61391656       0.8253
## Rank 5 - Rank 1 -0.30441723  0.1360435    0.02530467       0.0842
## 
## Best linear projection of the conditional average treatment effect.
## Confidence intervals are cluster- and heteroskedasticity-robust (HC3):
## 
##                                                Estimate Std. Error t value
## (Intercept)                                  -0.3999956  1.0464766 -0.3822
## age                                          -0.0059220  0.0054065 -1.0953
## gender_Man                                    0.0179342  0.0891396  0.2012
## education_High_school_or_less                 0.5278968  0.4707097  1.1215
## education_Some_college                        0.4434110  0.4431834  1.0005
## education_Bachelor_degree                     0.5950685  0.4498186  1.3229
## marital_Married_or_in_a_domestic_partnership -0.1903392  0.1299204 -1.4650
## employment_Employed                           0.2934942  0.1435857  2.0440
## employment_Unemployed                         0.1236204  0.1037249  1.1918
## location_Mostly_urban                        -0.1299760  0.0671508 -1.9356
## location_Suburban                            -0.0569142  0.1089234 -0.5225
## religion_Christian                            0.1766395  0.2287391  0.7722
## religiosity_Attends                          -0.3630977  0.2242166 -1.6194
## social_media_bin_Yes                          0.3161918  0.8381898  0.3772
## social_media_hours                            0.0071984  0.0106836  0.6738
## social_media_share_80_100                    -0.3199256  0.2037452 -1.5702
## social_media_share_60_80                     -0.0316706  0.1709749 -0.1852
## social_media_share_40_60                     -0.1922559  0.1353458 -1.4205
## social_media_share_20_40                     -0.1601785  0.2182877 -0.7338
##                                              Pr(>|t|)  
## (Intercept)                                   0.70231  
## age                                           0.27344  
## gender_Man                                    0.84056  
## education_High_school_or_less                 0.26215  
## education_Some_college                        0.31713  
## education_Bachelor_degree                     0.18595  
## marital_Married_or_in_a_domestic_partnership  0.14300  
## employment_Employed                           0.04102 *
## employment_Unemployed                         0.23341  
## location_Mostly_urban                         0.05300 .
## location_Suburban                             0.60134  
## religion_Christian                            0.44003  
## religiosity_Attends                           0.10545  
## social_media_bin_Yes                          0.70602  
## social_media_hours                            0.50049  
## social_media_share_80_100                     0.11645  
## social_media_share_60_80                      0.85305  
## social_media_share_40_60                      0.15555  
## social_media_share_20_40                      0.46312  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## Best linear fit using forest predictions (on held-out data)
## as well as the mean forest prediction as regressors, along
## with one-sided heteroskedasticity-robust (HC3) SEs:
## 
##                                Estimate Std. Error t value Pr(>t)
## mean.forest.prediction          0.88214    0.94143  0.9370 0.1744
## differential.forest.prediction -0.54132    0.35233 -1.5364 0.9377

## [1] "Outcome: new_acc_post"
## [1] "Covariates: covariates_5"
##   method ranking   estimate    std.err
## 1    ols      Q1 -0.1178592 0.09796242
## 2    ols      Q2 -0.1015579 0.09302514
## 3    ols      Q3  0.1956749 0.09050484
## 4    ols      Q4  0.1601512 0.09777336
## 5    ols      Q5  0.1008326 0.10273691
##   method ranking    estimate    std.err
## 1   aipw      Q1 -0.09892542 0.09698223
## 2   aipw      Q2 -0.10964787 0.09095590
## 3   aipw      Q3  0.20688701 0.08932577
## 4   aipw      Q4  0.13682119 0.09633110
## 5   aipw      Q5  0.08775350 0.10288064

##                   Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 0.01630124  0.1365895    0.90500885       0.8996
## Rank 3 - Rank 1 0.31353404  0.1363283    0.02151363       0.0702
## Rank 4 - Rank 1 0.27801035  0.1367175    0.04207836       0.1028
## Rank 5 - Rank 1 0.21869171  0.1360915    0.10815362       0.1908
##                    Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 -0.01072245  0.1350093    0.93670289       0.9359
## Rank 3 - Rank 1  0.30581244  0.1347281    0.02327533       0.0769
## Rank 4 - Rank 1  0.23574661  0.1350093    0.08086955       0.1980
## Rank 5 - Rank 1  0.18667893  0.1345430    0.16537341       0.2883
## 
## Best linear projection of the conditional average treatment effect.
## Confidence intervals are cluster- and heteroskedasticity-robust (HC3):
## 
##                                                Estimate Std. Error t value
## (Intercept)                                  -0.5301972  1.0968579 -0.4834
## att_check_pre                                 0.1180758  0.0764296  1.5449
## base_rate_pre                                -0.0011700  0.1602586 -0.0073
## misinfo_pre                                  -0.2055351  0.2511393 -0.8184
## misinfo_avg_acc_score_pre                     0.0260691  0.0447832  0.5821
## base_avg_acc_score_pre                        0.0599875  0.0237690  2.5238
## age                                          -0.0051353  0.0048658 -1.0554
## gender_Man                                    0.0260824  0.0839197  0.3108
## education_High_school_or_less                 0.6205024  0.4707789  1.3180
## education_Some_college                        0.5158962  0.4402303  1.1719
## education_Bachelor_degree                     0.6662660  0.4570913  1.4576
## marital_Married_or_in_a_domestic_partnership -0.1714445  0.1360744 -1.2599
## employment_Employed                           0.2612497  0.1350375  1.9346
## employment_Unemployed                         0.1272868  0.0997332  1.2763
## location_Mostly_urban                        -0.1207110  0.0621061 -1.9436
## location_Suburban                            -0.0388146  0.1095414 -0.3543
## religion_Christian                            0.1514598  0.2453519  0.6173
## religiosity_Attends                          -0.3535283  0.2149539 -1.6447
## social_media_bin_Yes                          0.3387192  0.7981908  0.4244
## social_media_hours                            0.0059130  0.0100714  0.5871
## social_media_share_80_100                    -0.2784038  0.2162750 -1.2873
## social_media_share_60_80                     -0.0112094  0.1795785 -0.0624
## social_media_share_40_60                     -0.1578305  0.1298530 -1.2155
## social_media_share_20_40                     -0.1648349  0.2195284 -0.7509
##                                              Pr(>|t|)  
## (Intercept)                                   0.62886  
## att_check_pre                                 0.12246  
## base_rate_pre                                 0.99418  
## misinfo_pre                                   0.41318  
## misinfo_avg_acc_score_pre                     0.56052  
## base_avg_acc_score_pre                        0.01165 *
## age                                           0.29132  
## gender_Man                                    0.75597  
## education_High_school_or_less                 0.18758  
## education_Some_college                        0.24132  
## education_Bachelor_degree                     0.14503  
## marital_Married_or_in_a_domestic_partnership  0.20778  
## employment_Employed                           0.05311 .
## employment_Unemployed                         0.20194  
## location_Mostly_urban                         0.05202 .
## location_Suburban                             0.72311  
## religion_Christian                            0.53706  
## religiosity_Attends                           0.10013  
## social_media_bin_Yes                          0.67133  
## social_media_hours                            0.55717  
## social_media_share_80_100                     0.19808  
## social_media_share_60_80                      0.95023  
## social_media_share_40_60                      0.22427  
## social_media_share_20_40                      0.45279  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## Best linear fit using forest predictions (on held-out data)
## as well as the mean forest prediction as regressors, along
## with one-sided heteroskedasticity-robust (HC3) SEs:
## 
##                                Estimate Std. Error t value  Pr(>t)  
## mean.forest.prediction          0.90818    1.08023  0.8407 0.20028  
## differential.forest.prediction  0.76928    0.39675  1.9390 0.02629 *
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

## [1] "Outcome: new_acc_diff"
## [1] "Covariates: covariates_1"
##   method ranking    estimate   std.err
## 1    ols      Q1 -0.21131944 0.1310504
## 2    ols      Q2  0.07511444 0.1378094
## 3    ols      Q3  0.14804823 0.1220128
## 4    ols      Q4 -0.04251724 0.1369799
## 5    ols      Q5  0.14372082 0.1484703
##   method ranking    estimate   std.err
## 1   aipw      Q1 -0.19392061 0.1200110
## 2   aipw      Q2  0.09043185 0.1318566
## 3   aipw      Q3  0.13285814 0.1145899
## 4   aipw      Q4 -0.01885869 0.1347037
## 5   aipw      Q5  0.22686523 0.1276068

##                  Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 0.2864339  0.1884306    0.12857248       0.2257
## Rank 3 - Rank 1 0.3593677  0.1810510    0.04723138       0.1503
## Rank 4 - Rank 1 0.1688022  0.1938767    0.38399349       0.3773
## Rank 5 - Rank 1 0.3550403  0.1925453    0.06527496       0.1656
##                  Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 0.2843525  0.1753200    0.10491067       0.1862
## Rank 3 - Rank 1 0.3267787  0.1684511    0.05246905       0.1294
## Rank 4 - Rank 1 0.1750619  0.1803906    0.33188190       0.3185
## Rank 5 - Rank 1 0.4207858  0.1791231    0.01887014       0.0618
## 
## Best linear projection of the conditional average treatment effect.
## Confidence intervals are cluster- and heteroskedasticity-robust (HC3):
## 
##                Estimate Std. Error t value Pr(>|t|)
## (Intercept)   -0.040929   0.150570 -0.2718   0.7858
## att_check_pre  0.185725   0.124964  1.4862   0.1373
## base_rate_pre  0.030054   0.215608  0.1394   0.8891
## misinfo_pre   -0.042711   0.218748 -0.1953   0.8452
## 
## 
## Best linear fit using forest predictions (on held-out data)
## as well as the mean forest prediction as regressors, along
## with one-sided heteroskedasticity-robust (HC3) SEs:
## 
##                                Estimate Std. Error t value Pr(>t)
## mean.forest.prediction          0.96050    1.54483  0.6218 0.2671
## differential.forest.prediction  0.17846    0.23379  0.7633 0.2227

## [1] "Outcome: new_acc_diff"
## [1] "Covariates: covariates_2"
##   method ranking    estimate   std.err
## 1    ols      Q1 -0.28629554 0.1253189
## 2    ols      Q2 -0.15587845 0.1522165
## 3    ols      Q3  0.06917144 0.1353265
## 4    ols      Q4  0.23580240 0.1359834
## 5    ols      Q5  0.30054666 0.1216104
##   method ranking     estimate    std.err
## 1   aipw      Q1 -0.186727270 0.09615918
## 2   aipw      Q2 -0.076014716 0.10124720
## 3   aipw      Q3  0.002700817 0.09350253
## 4   aipw      Q4  0.226063395 0.09595463
## 5   aipw      Q5  0.234716626 0.09883416

##                  Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 0.1304171  0.1888781   0.489934941       0.4855
## Rank 3 - Rank 1 0.3554670  0.1892500   0.060421410       0.1043
## Rank 4 - Rank 1 0.5220979  0.1891215   0.005797784       0.0147
## Rank 5 - Rank 1 0.5868422  0.1891851   0.001937348       0.0073
##                  Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 0.1107126  0.1368467   0.418553364       0.4217
## Rank 3 - Rank 1 0.1894281  0.1371345   0.167262407       0.2862
## Rank 4 - Rank 1 0.4127907  0.1368944   0.002584262       0.0090
## Rank 5 - Rank 1 0.4214439  0.1370862   0.002125706       0.0090
## 
## Best linear projection of the conditional average treatment effect.
## Confidence intervals are cluster- and heteroskedasticity-robust (HC3):
## 
##                             Estimate Std. Error t value Pr(>|t|)  
## (Intercept)               -0.0922295  0.0678197 -1.3599  0.17394  
## att_check_pre              0.1585169  0.0699286  2.2668  0.02346 *
## misinfo_avg_acc_score_pre -0.0026078  0.0254578 -0.1024  0.91842  
## base_avg_acc_score_pre     0.0626805  0.0253321  2.4744  0.01339 *
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## Best linear fit using forest predictions (on held-out data)
## as well as the mean forest prediction as regressors, along
## with one-sided heteroskedasticity-robust (HC3) SEs:
## 
##                                Estimate Std. Error t value     Pr(>t)    
## mean.forest.prediction          1.07744    1.43169  0.7526     0.2259    
## differential.forest.prediction  0.49801    0.11745  4.2400 0.00001145 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

## [1] "Outcome: new_acc_diff"
## [1] "Covariates: covariates_3"
##   method ranking     estimate   std.err
## 1    ols      Q1 -0.005984217 0.1359426
## 2    ols      Q2 -0.041318677 0.1433721
## 3    ols      Q3 -0.175198542 0.1295874
## 4    ols      Q4  0.188713599 0.1386254
## 5    ols      Q5  0.230267179 0.1175956
##   method ranking    estimate    std.err
## 1   aipw      Q1 -0.13136016 0.09900152
## 2   aipw      Q2  0.02891580 0.09528455
## 3   aipw      Q3 -0.03309044 0.09215369
## 4   aipw      Q4  0.18236802 0.10165462
## 5   aipw      Q5  0.19106996 0.09475169

##                    Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 -0.03533446  0.1881461     0.8510413       0.8480
## Rank 3 - Rank 1 -0.16921433  0.1878027     0.3676367       0.5980
## Rank 4 - Rank 1  0.19469782  0.1883926     0.3014537       0.5980
## Rank 5 - Rank 1  0.23625140  0.1878419     0.2085764       0.5205
##                   Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 0.16027596  0.1365151    0.24045091       0.3950
## Rank 3 - Rank 1 0.09826972  0.1362787    0.47089836       0.4733
## Rank 4 - Rank 1 0.31372819  0.1366583    0.02174918       0.0608
## Rank 5 - Rank 1 0.32243012  0.1362787    0.01803569       0.0608
## 
## Best linear projection of the conditional average treatment effect.
## Confidence intervals are cluster- and heteroskedasticity-robust (HC3):
## 
##                            Estimate Std. Error t value Pr(>|t|)  
## (Intercept)                0.074259   0.096317  0.7710  0.44077  
## att_check_pre              0.114528   0.072486  1.5800  0.11420  
## base_rate_pre              0.003890   0.160264  0.0243  0.98064  
## misinfo_pre               -0.267131   0.273660 -0.9761  0.32906  
## misinfo_avg_acc_score_pre  0.041447   0.049092  0.8443  0.39858  
## base_avg_acc_score_pre     0.061597   0.027654  2.2274  0.02598 *
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## Best linear fit using forest predictions (on held-out data)
## as well as the mean forest prediction as regressors, along
## with one-sided heteroskedasticity-robust (HC3) SEs:
## 
##                                Estimate Std. Error t value       Pr(>t)    
## mean.forest.prediction          0.97709    1.15966  0.8426       0.1998    
## differential.forest.prediction  0.61722    0.12338  5.0026 0.0000002962 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

## [1] "Outcome: new_acc_diff"
## [1] "Covariates: covariates_4"
##   method ranking    estimate   std.err
## 1    ols      Q1 -0.18451485 0.1376132
## 2    ols      Q2  0.11944444 0.1303331
## 3    ols      Q3  0.21858093 0.1374063
## 4    ols      Q4  0.07761796 0.1308731
## 5    ols      Q5 -0.09719721 0.1409199
##   method ranking    estimate   std.err
## 1   aipw      Q1 -0.19755330 0.1381242
## 2   aipw      Q2  0.11783189 0.1301989
## 3   aipw      Q3  0.20605173 0.1391223
## 4   aipw      Q4  0.05756257 0.1315342
## 5   aipw      Q5 -0.10338794 0.1413729

##                   Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 0.30395929  0.1917225    0.11296004       0.2649
## Rank 3 - Rank 1 0.40309578  0.1913601    0.03523156       0.1173
## Rank 4 - Rank 1 0.26213281  0.1917527    0.17169895       0.2999
## Rank 5 - Rank 1 0.08731764  0.1910699    0.64770343       0.6485
##                   Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 0.31538519  0.1926393    0.10167980       0.2384
## Rank 3 - Rank 1 0.40360502  0.1922381    0.03584141       0.1103
## Rank 4 - Rank 1 0.25511587  0.1926393    0.18548075       0.3095
## Rank 5 - Rank 1 0.09416536  0.1919739    0.62380186       0.6238
## 
## Best linear projection of the conditional average treatment effect.
## Confidence intervals are cluster- and heteroskedasticity-robust (HC3):
## 
##                                                Estimate Std. Error t value
## (Intercept)                                  -0.1521178  1.5447617 -0.0985
## age                                          -0.0072581  0.0073472 -0.9879
## gender_Man                                   -0.0622510  0.1565182 -0.3977
## education_High_school_or_less                 0.7184267  0.6169661  1.1645
## education_Some_college                        0.6031879  0.5631061  1.0712
## education_Bachelor_degree                     0.8483835  0.6392079  1.3272
## marital_Married_or_in_a_domestic_partnership -0.3075157  0.1741616 -1.7657
## employment_Employed                           0.3503745  0.1580739  2.2165
## employment_Unemployed                         0.2326439  0.1547149  1.5037
## location_Mostly_urban                        -0.0581717  0.2031240 -0.2864
## location_Suburban                             0.0038001  0.1228002  0.0309
## religion_Christian                           -0.1644395  0.2293729 -0.7169
## religiosity_Attends                          -0.2398076  0.3177681 -0.7547
## social_media_bin_Yes                          0.0850133  1.1507633  0.0739
## social_media_hours                            0.0079682  0.0165193  0.4824
## social_media_share_80_100                    -0.4064488  0.3534265 -1.1500
## social_media_share_60_80                      0.0078602  0.2605458  0.0302
## social_media_share_40_60                     -0.2359007  0.1904725 -1.2385
## social_media_share_20_40                     -0.2441817  0.2414797 -1.0112
##                                              Pr(>|t|)  
## (Intercept)                                   0.92156  
## age                                           0.32328  
## gender_Man                                    0.69086  
## education_High_school_or_less                 0.24432  
## education_Some_college                        0.28416  
## education_Bachelor_degree                     0.18451  
## marital_Married_or_in_a_domestic_partnership  0.07753 .
## employment_Employed                           0.02672 *
## employment_Unemployed                         0.13275  
## location_Mostly_urban                         0.77460  
## location_Suburban                             0.97531  
## religion_Christian                            0.47348  
## religiosity_Attends                           0.45050  
## social_media_bin_Yes                          0.94111  
## social_media_hours                            0.62958  
## social_media_share_80_100                     0.25021  
## social_media_share_60_80                      0.97593  
## social_media_share_40_60                      0.21561  
## social_media_share_20_40                      0.31199  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## Best linear fit using forest predictions (on held-out data)
## as well as the mean forest prediction as regressors, along
## with one-sided heteroskedasticity-robust (HC3) SEs:
## 
##                                Estimate Std. Error t value Pr(>t)
## mean.forest.prediction         0.858117   4.109961  0.2088 0.4173
## differential.forest.prediction 0.081305   0.191652  0.4242 0.3357

## [1] "Outcome: new_acc_diff"
## [1] "Covariates: covariates_5"
##   method ranking    estimate   std.err
## 1    ols      Q1 -0.23258312 0.1476138
## 2    ols      Q2  0.12632895 0.1314107
## 3    ols      Q3  0.06357556 0.1283038
## 4    ols      Q4  0.26846936 0.1285579
## 5    ols      Q5 -0.12251925 0.1276903
##   method ranking    estimate    std.err
## 1   aipw      Q1 -0.13865998 0.09986154
## 2   aipw      Q2  0.04513082 0.09367490
## 3   aipw      Q3  0.07290893 0.09279627
## 4   aipw      Q4  0.22970905 0.09273417
## 5   aipw      Q5 -0.01509418 0.10253986

##                  Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 0.3589121  0.1881972   0.056586009       0.1303
## Rank 3 - Rank 1 0.2961587  0.1877887   0.114863503       0.1886
## Rank 4 - Rank 1 0.5010525  0.1881734   0.007785615       0.0225
## Rank 5 - Rank 1 0.1100639  0.1875234   0.557284693       0.5559
##                  Estimate Std. Error Orig. p-value Adj. p-value
## Rank 2 - Rank 1 0.1837908  0.1364264   0.178006876       0.2950
## Rank 3 - Rank 1 0.2115689  0.1361423   0.120265191       0.2732
## Rank 4 - Rank 1 0.3683690  0.1364264   0.006963476       0.0236
## Rank 5 - Rank 1 0.1235658  0.1359552   0.363478341       0.3605
## 
## Best linear projection of the conditional average treatment effect.
## Confidence intervals are cluster- and heteroskedasticity-robust (HC3):
## 
##                                                Estimate Std. Error t value
## (Intercept)                                  -0.7161048  1.1551358 -0.6199
## att_check_pre                                 0.1270101  0.0806558  1.5747
## base_rate_pre                                -0.0271523  0.1625721 -0.1670
## misinfo_pre                                  -0.2135676  0.2511200 -0.8505
## misinfo_avg_acc_score_pre                     0.0350907  0.0450904  0.7782
## base_avg_acc_score_pre                        0.0649492  0.0319371  2.0337
## age                                          -0.0048109  0.0048260 -0.9969
## gender_Man                                    0.0404921  0.0853783  0.4743
## education_High_school_or_less                 0.7466395  0.4986510  1.4973
## education_Some_college                        0.6581935  0.4733419  1.3905
## education_Bachelor_degree                     0.8041810  0.4873126  1.6502
## marital_Married_or_in_a_domestic_partnership -0.1737619  0.1421076 -1.2227
## employment_Employed                           0.3001944  0.1408170  2.1318
## employment_Unemployed                         0.1509794  0.1002568  1.5059
## location_Mostly_urban                        -0.1238781  0.0518825 -2.3877
## location_Suburban                            -0.0376609  0.1043666 -0.3609
## religion_Christian                            0.1354804  0.2332502  0.5808
## religiosity_Attends                          -0.3301576  0.2183251 -1.5122
## social_media_bin_Yes                          0.3658916  0.8119931  0.4506
## social_media_hours                            0.0064430  0.0106862  0.6029
## social_media_share_80_100                    -0.3177753  0.2123032 -1.4968
## social_media_share_60_80                     -0.0132224  0.1804859 -0.0733
## social_media_share_40_60                     -0.1984216  0.1309920 -1.5148
## social_media_share_20_40                     -0.1794030  0.2122123 -0.8454
##                                              Pr(>|t|)  
## (Intercept)                                   0.53534  
## att_check_pre                                 0.11541  
## base_rate_pre                                 0.86737  
## misinfo_pre                                   0.39513  
## misinfo_avg_acc_score_pre                     0.43648  
## base_avg_acc_score_pre                        0.04206 *
## age                                           0.31890  
## gender_Man                                    0.63534  
## education_High_school_or_less                 0.13440  
## education_Some_college                        0.16446  
## education_Bachelor_degree                     0.09898 .
## marital_Married_or_in_a_domestic_partnership  0.22150  
## employment_Employed                           0.03309 *
## employment_Unemployed                         0.13217  
## location_Mostly_urban                         0.01701 *
## location_Suburban                             0.71823  
## religion_Christian                            0.56139  
## religiosity_Attends                           0.13056  
## social_media_bin_Yes                          0.65230  
## social_media_hours                            0.54660  
## social_media_share_80_100                     0.13453  
## social_media_share_60_80                      0.94160  
## social_media_share_40_60                      0.12992  
## social_media_share_20_40                      0.39795  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## Best linear fit using forest predictions (on held-out data)
## as well as the mean forest prediction as regressors, along
## with one-sided heteroskedasticity-robust (HC3) SEs:
## 
##                                Estimate Std. Error t value  Pr(>t)  
## mean.forest.prediction          0.89010    1.22479  0.7267 0.23372  
## differential.forest.prediction  0.56420    0.43633  1.2931 0.09804 .
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Heterogeneity by post type

To evaluate whether each treatment course changed behavior on certain types of misinformation more than others, I want to take the post - pre difference for each post type, and compare treatment groups.

# For each user in wide_df, create a new variable called diff_(type) for each type of misinfo post that takes the misinfo_post_(type) - misinfo_pre_(type)
result <- df_wide %>% 
  dplyr::select(treatment,misinfo_pre_Emotions,misinfo_pre_Reasoning,misinfo_pre_Combo,misinfo_post_Emotions,misinfo_post_Reasoning,misinfo_post_Combo) %>%
  mutate(
    misinfo_diff_Emotions = misinfo_post_Emotions - misinfo_pre_Emotions,
    misinfo_diff_Reasoning = misinfo_post_Reasoning - misinfo_pre_Reasoning,
    misinfo_diff_Combo = misinfo_post_Combo - misinfo_pre_Combo
  ) %>% 
  group_by(treatment) %>% 
  summarise(
    mean_diff_Emotions = mean(misinfo_diff_Emotions, na.rm = TRUE),
    mean_diff_Reasoning = mean(misinfo_diff_Reasoning, na.rm = TRUE),
    mean_diff_Combo = mean(misinfo_diff_Combo, na.rm = TRUE),
    se_diff_Emotions = se_cont(misinfo_diff_Emotions, na.rm = TRUE),
    se_diff_Reasoning = se_cont(misinfo_diff_Reasoning, na.rm = TRUE),
    se_diff_Combo = se_cont(misinfo_diff_Combo, na.rm = TRUE)
  )

print(result)

## # A tibble: 5 × 7
##   treatment          mean_diff_Emotions mean_diff_Reasoning mean_diff_Combo
##   <chr>                           <dbl>               <dbl>           <dbl>
## 1 Combo                         -0.166              -0.187          -0.172 
## 2 Emotions                      -0.197              -0.176          -0.190 
## 3 Facts Baseline                -0.0564             -0.0807         -0.0838
## 4 No-course Baseline            -0.0120             -0.0203         -0.0246
## 5 Reasoning                     -0.142              -0.122          -0.144 
## # ℹ 3 more variables: se_diff_Emotions <dbl>, se_diff_Reasoning <dbl>,
## #   se_diff_Combo <dbl>

HTE Tutorial for Inoculation against Misinformation

Kristine Koutout

2024-04-11

Goal

Summary

Key Takeaways

RATE

Causal Forest

Subgroup Analysis

Causal Tree

Experimental Design

Data

Samples

Outcomes and Covariates

All Outcomes

All Covariates

Demographics

Pre-treatment variables

Setup

Load packages

Load data

Helper Functions

Define input for script

RATE

Causal Forest

Define function

All outcomes with all covariates (covariates_5)

Subgroup Analysis

Causal Tree

Function

Main outcomes & covariate sets

Appendix

Causal forest for all outcomes & covariate sets

Heterogeneity by post type

Description	Variable Construction
Pre misinfo sharing rate	\(\frac{E[Y^{M, pre}]}{6}\)
Pre non-misinfo sharing rate	\(\frac{E[Y^{N, pre}]}{3}\)
Pre sharing discernment	\(\frac{E[Y^{N, pre}(1)]}{3} - \frac{E[Y^{M, pre}(1)]}{6}\)
Pre misinfo accuracy score	\(\frac{E[A^{M, pre}]}{6}\)
Pre non-misinfo accuracy score	\(\frac{E[A^{N, pre}]}{3}\)
Pre accuracy discernment	\(\frac{E[A^{N, pre}(1)]}{3} - \frac{E[A^{M, pre}(1)]}{6}\)
Pre attention check passing	\(1[\text{passed attention check}]\)