IAM: Subgroup analysis - Nonmisinfo Rate

This script takes the main dataset df_final.rds and produces the results of selected tests conducted for different covariate-defined subgroups and the differences of the estimates across subgroups. It also implements Romano-Wolf corrections.

Test Dictionary

Test 6D: Reasoning course vs Facts Baseline
Test 7D: Emotions course vs Facts Baseline
Test 8D: Combo course vs Facts Baseline
Test 9D: Facts Baseline vs No-course Baseline
Test 10D: Reasoning vs Emotions
Test 11D: Reasoning vs Combo
Test 12D: Emotions vs Combo

Subgroup Dictionary

Note: Pre-survey outcomes are post-treatment in the case of accuracy nudge tests, so those subgroups are not included.

Gender: Man; Woman
Education: High school or less; Over high school or less
Marital: Married; Not married
Location: Mostly rural; Not mostly rural
Religion: Christian; Non-Christian
Religiosity: Attends religious services; Does not attend religious services
Proportion of content shared in social media: 0-40%; 40-100%
Age: Below median; Above median
Hrs/day on social media: Below median; Above median;
Pre-survey Sharing: Below median; Above median;
Misinfo-survey Sharing: Below median; Above median;
Accuracy Discernment Score: Below median; Above median;

Setup

Package loading

options(warn=-1)
options(scipen=999)
suppressMessages(library(estimatr))
suppressMessages(library(stats))
suppressMessages(library(tidyverse))
suppressMessages(library(rmarkdown))
suppressMessages(library(tictoc))
suppressMessages(library(stargazer))
suppressMessages(library(ggpattern))
suppressMessages(library(doParallel))
suppressMessages(library(mltools))
suppressMessages(library(data.table))
suppressMessages(library(RItools))
suppressMessages(library(stringr))
suppressMessages(library(grf))
suppressMessages(library(multcomp))
nrCores = detectCores()
cl = makeCluster(nrCores)
registerDoParallel(cl)

Bring necessary packages to clusters for parallel computing

clusterEvalQ(cl, {suppressMessages(library(estimatr))
suppressMessages(library(stats))
suppressMessages(library(tidyverse))
suppressMessages(library(rmarkdown))
suppressMessages(library(multcomp))})

## [[1]]
##  [1] "multcomp"  "TH.data"   "MASS"      "survival"  "mvtnorm"   "rmarkdown"
##  [7] "lubridate" "forcats"   "stringr"   "dplyr"     "purrr"     "readr"    
## [13] "tidyr"     "tibble"    "ggplot2"   "tidyverse" "estimatr"  "stats"    
## [19] "graphics"  "grDevices" "utils"     "datasets"  "methods"   "base"     
## 
## [[2]]
##  [1] "multcomp"  "TH.data"   "MASS"      "survival"  "mvtnorm"   "rmarkdown"
##  [7] "lubridate" "forcats"   "stringr"   "dplyr"     "purrr"     "readr"    
## [13] "tidyr"     "tibble"    "ggplot2"   "tidyverse" "estimatr"  "stats"    
## [19] "graphics"  "grDevices" "utils"     "datasets"  "methods"   "base"     
## 
## [[3]]
##  [1] "multcomp"  "TH.data"   "MASS"      "survival"  "mvtnorm"   "rmarkdown"
##  [7] "lubridate" "forcats"   "stringr"   "dplyr"     "purrr"     "readr"    
## [13] "tidyr"     "tibble"    "ggplot2"   "tidyverse" "estimatr"  "stats"    
## [19] "graphics"  "grDevices" "utils"     "datasets"  "methods"   "base"     
## 
## [[4]]
##  [1] "multcomp"  "TH.data"   "MASS"      "survival"  "mvtnorm"   "rmarkdown"
##  [7] "lubridate" "forcats"   "stringr"   "dplyr"     "purrr"     "readr"    
## [13] "tidyr"     "tibble"    "ggplot2"   "tidyverse" "estimatr"  "stats"    
## [19] "graphics"  "grDevices" "utils"     "datasets"  "methods"   "base"     
## 
## [[5]]
##  [1] "multcomp"  "TH.data"   "MASS"      "survival"  "mvtnorm"   "rmarkdown"
##  [7] "lubridate" "forcats"   "stringr"   "dplyr"     "purrr"     "readr"    
## [13] "tidyr"     "tibble"    "ggplot2"   "tidyverse" "estimatr"  "stats"    
## [19] "graphics"  "grDevices" "utils"     "datasets"  "methods"   "base"     
## 
## [[6]]
##  [1] "multcomp"  "TH.data"   "MASS"      "survival"  "mvtnorm"   "rmarkdown"
##  [7] "lubridate" "forcats"   "stringr"   "dplyr"     "purrr"     "readr"    
## [13] "tidyr"     "tibble"    "ggplot2"   "tidyverse" "estimatr"  "stats"    
## [19] "graphics"  "grDevices" "utils"     "datasets"  "methods"   "base"     
## 
## [[7]]
##  [1] "multcomp"  "TH.data"   "MASS"      "survival"  "mvtnorm"   "rmarkdown"
##  [7] "lubridate" "forcats"   "stringr"   "dplyr"     "purrr"     "readr"    
## [13] "tidyr"     "tibble"    "ggplot2"   "tidyverse" "estimatr"  "stats"    
## [19] "graphics"  "grDevices" "utils"     "datasets"  "methods"   "base"     
## 
## [[8]]
##  [1] "multcomp"  "TH.data"   "MASS"      "survival"  "mvtnorm"   "rmarkdown"
##  [7] "lubridate" "forcats"   "stringr"   "dplyr"     "purrr"     "readr"    
## [13] "tidyr"     "tibble"    "ggplot2"   "tidyverse" "estimatr"  "stats"    
## [19] "graphics"  "grDevices" "utils"     "datasets"  "methods"   "base"     
## 
## [[9]]
##  [1] "multcomp"  "TH.data"   "MASS"      "survival"  "mvtnorm"   "rmarkdown"
##  [7] "lubridate" "forcats"   "stringr"   "dplyr"     "purrr"     "readr"    
## [13] "tidyr"     "tibble"    "ggplot2"   "tidyverse" "estimatr"  "stats"    
## [19] "graphics"  "grDevices" "utils"     "datasets"  "methods"   "base"     
## 
## [[10]]
##  [1] "multcomp"  "TH.data"   "MASS"      "survival"  "mvtnorm"   "rmarkdown"
##  [7] "lubridate" "forcats"   "stringr"   "dplyr"     "purrr"     "readr"    
## [13] "tidyr"     "tibble"    "ggplot2"   "tidyverse" "estimatr"  "stats"    
## [19] "graphics"  "grDevices" "utils"     "datasets"  "methods"   "base"     
## 
## [[11]]
##  [1] "multcomp"  "TH.data"   "MASS"      "survival"  "mvtnorm"   "rmarkdown"
##  [7] "lubridate" "forcats"   "stringr"   "dplyr"     "purrr"     "readr"    
## [13] "tidyr"     "tibble"    "ggplot2"   "tidyverse" "estimatr"  "stats"    
## [19] "graphics"  "grDevices" "utils"     "datasets"  "methods"   "base"

Useful functions

# Computes standard errors of a given continuous variable or one-hot binary variable, respectively #
se_cont = function(x, na.rm=FALSE) {
  if (na.rm) x <- na.omit(x)
  sqrt(var(x)/length(x))}

se_binary = function(x, na.rm=FALSE) {
  if (na.rm) x <- na.omit(x)
  sqrt(mean(x)*(1-mean(x))/length(x))}

# Adds the level 'missing' to factor variables #
addmissing = function(x){
  if(is.factor(x)) return(factor(x, levels=c(levels(x), "missing")))
  return(x)}

Data pre-processing and outcome construction

# This function takes the main csv as input and outputs a list of dataframes used to conduct the different tests in the analysis #
# It also has the option to apply filtering by attention check status #
pre_processing = function(data,att_check="all"){  
    
# Family 1 subsets to posts that were shared in the pre, then looks at the misinfo counterparts in post #
course_tests_data = data[data$type == "Base posts" & data$pre_post == "pre",]
course_tests_data$share_post = NA
course_tests_data$type_post = NA
#course_tests_data$accuracy_score_post = NA  
    
# non_counterparts subsets to all misinfo posts in the pre and misinfo posts that are not counterparts in the post
non_counterparts_data = data[data$type != "Base posts" & data$pre_post == "pre",]

# Identify who shared 0, 1, 2 or 3 base posts in the pre #
groups = aggregate(share~user,course_tests_data,sum)
colnames(groups) = c("user","group")
course_tests_data = left_join(course_tests_data,groups,by="user")
    
# Followup preprocessing #
followup_course_tests_data = data[!is.na(data$att_check_followup) & data$type == "Base posts" & data$pre_post == "post",]
followup_course_tests_data$share_followup = NA
followup_course_tests_data$type_followup = NA
followup_course_tests_data$Q4 = NA
followup_course_tests_data$Q5 = NA    

# Identify who shared 0, 1, 2 or 3 base posts in the post #
groups_followup = aggregate(share~user,followup_course_tests_data,sum)
colnames(groups_followup) = c("user","group")
followup_course_tests_data = left_join(followup_course_tests_data,groups_followup,by="user")    

# For each user #
for (u in unique(course_tests_data$user)){
    # Take the vector of facts that were seen in the pre as base # 
    F = course_tests_data$fact[course_tests_data$user == u]
    # Add the non-counterpart posts to that data #
    non_counterparts_data = rbind(non_counterparts_data,data[data$user == u & data$type != "Base posts" & data$pre_post == "post" & !data$fact %in% F,])
    # For each of these facts #
    for (f in F){
        # Take the sharing decision of the misinfo counterpart in the post # 
        course_tests_data$share_post[course_tests_data$user == u & course_tests_data$fact == f] = data$share[data$user == u & data$fact == f & data$pre_post == "post"]
        # Take the post type of the misinfo counterpart in the post #
        course_tests_data$type_post[course_tests_data$user == u & course_tests_data$fact == f] = data$type[data$user == u & data$fact == f & data$pre_post == "post"]    
        # Take the post type of the misinfo counterpart in the post #
        #course_tests_data$accuracy_score_post[course_tests_data$user == u & course_tests_data$fact == f] = data$accuracy_score[data$user == u & data$fact == f & data$pre_post == "post"]
        }
    
    F = followup_course_tests_data$fact[followup_course_tests_data$user == u]
    # Extract reflective questions #
    followup_course_tests_data$Q4[followup_course_tests_data$user == u] = main_data$reflective_4.contain_course_keyword[main_data$user == u & main_data$order == 1]
    followup_course_tests_data$Q5[followup_course_tests_data$user == u] = main_data$reflective_5.contain_course_keyword[main_data$user == u & main_data$order == 1]
    # For each of these facts #
    for (f in F){
        # Take the sharing decision of the misinfo counterpart in the followup # 
        followup_course_tests_data$share_followup[followup_course_tests_data$user == u & followup_course_tests_data$fact == f] = data$share[data$user == u & data$fact == f & data$pre_post == "followup"]
        # Take the post type of the misinfo counterpart in the post #
        followup_course_tests_data$type_followup[followup_course_tests_data$user == u & followup_course_tests_data$fact == f] = data$type[data$user == u & data$fact == f & data$pre_post == "followup"]}}

    # Main Outcome # 
    
    course_tests_data_final = aggregate(share_post~user*group*treatment*accuracy*att_check_pre*att_check_post,course_tests_data[course_tests_data$share == 1,],mean)
    #course_tests_data_final$accuracy_score_post = aggregate(accuracy_score_post~user)[,2]
    course_tests_data_by_type_final = aggregate(share_post~user*type_post*treatment*accuracy*att_check_pre*att_check_post,course_tests_data[course_tests_data$share == 1,],mean)
    #course_tests_data_by_type_final$accuracy_score_post = aggregate(accuracy_score_post~user*type_post,course_tests_data[course_tests_data$share == 1,],mean)[,3]
    
    # Followup Outcome #
    followup_course_tests_data_final = aggregate(share_followup~user*group*treatment*priming*att_check_pre*att_check_post*att_check_followup,followup_course_tests_data[followup_course_tests_data$share == 1,],mean)
    followup_course_tests_data_by_type_final = aggregate(share_followup~user*treatment*type_followup*att_check_pre*att_check_post*att_check_followup,followup_course_tests_data[followup_course_tests_data$share == 1,],mean)

    # Followup text outcomes #

    followup_course_text_tests_data_final = aggregate(Q4~user*group*treatment*priming*att_check_pre*att_check_post*att_check_followup,followup_course_tests_data,mean)
    colnames(followup_course_text_tests_data_final)[8] = "Q4"
    followup_course_text_tests_data_final$Q5 = aggregate(Q5~user*group*treatment*priming*att_check_pre*att_check_post*att_check_followup,followup_course_tests_data,mean)[,8]
    
    # Opposite Outcome #

    # Aggregate at the user level by computing averages #
    opposite_tests_data_final = aggregate(share_post~user*group*treatment*accuracy*att_check_pre*att_check_post,course_tests_data[course_tests_data$share == 0,],mean)  # Subset to those facts that were not shared in the pre (as base) #
    #opposite_tests_data_final$accuracy_score_post = aggregate(accuracy_score~user,course_tests_data[course_tests_data$share == 0,],mean)[,2]

    # Non-corresponding Outcome #
    
    non_counterparts_data_final = aggregate(share~user*treatment*accuracy*att_check_pre*att_check_post,non_counterparts_data[non_counterparts_data$pre_post=="pre",],mean)
    colnames(non_counterparts_data_final)[6] = "pre"
    non_counterparts_data_final$post = aggregate(share~user*treatment*accuracy*att_check_pre*att_check_post,non_counterparts_data[non_counterparts_data$pre_post=="post",],mean)[,6]
    non_counterparts_data_final$diff = non_counterparts_data_final$post - non_counterparts_data_final$pre

    # Outcomes using all misinfo posts # 
    
    # Sharing #
    all_posts_data_final = aggregate(share~user*treatment*accuracy*att_check_pre*att_check_post,data[data$type!="Base posts" & data$pre_post=="pre",],mean)    
    colnames(all_posts_data_final)[6] = "share_pre"
    all_posts_data_final$share_post = aggregate(share~user*treatment*accuracy*att_check_pre*att_check_post,data[data$type!="Base posts" & data$pre_post=="post",],mean)[,6] 
    all_posts_data_final$share_diff = all_posts_data_final$share_post - all_posts_data_final$share_pre
    
    # Discernment #
    all_posts_data_final$disc_pre = aggregate(disc_score~user*treatment*accuracy*att_check_pre*att_check_post,data[data$pre_post=="pre",],sum)[,6]
    all_posts_data_final$disc_post = aggregate(disc_score~user*treatment*accuracy*att_check_pre*att_check_post,data[data$pre_post=="post",],sum)[,6]
    all_posts_data_final$disc_diff = all_posts_data_final$disc_post - all_posts_data_final$disc_pre
    
    # Accuracy Discernment #
    all_posts_data_final$acc_disc_pre = aggregate(acc_disc_score~user*treatment*accuracy*att_check_pre*att_check_post,data[data$pre_post=="pre",],sum)[,6]
    all_posts_data_final$acc_disc_post = aggregate(acc_disc_score~user*treatment*accuracy*att_check_pre*att_check_post,data[data$pre_post=="post",],sum)[,6]
    all_posts_data_final$acc_disc_diff = all_posts_data_final$acc_disc_post - all_posts_data_final$acc_disc_pre

    # Adding 'course' variable #
    
    course_tests_data_final$course = ifelse(course_tests_data_final$treatment == "No-course Baseline","control",
                                      ifelse(course_tests_data_final$treatment %in% c("Reasoning","Emotions","Combo"),"treatment",NA))
    
    course_tests_data_by_type_final$course = ifelse(course_tests_data_by_type_final$treatment == "No-course Baseline","control",
                                      ifelse(course_tests_data_by_type_final$treatment %in% c("Reasoning","Emotions","Combo"),"treatment",NA))
    
    followup_course_tests_data_final$course = ifelse(followup_course_tests_data_final$treatment == "No-course Baseline","control",
                                      ifelse(followup_course_tests_data_final$treatment %in% c("Reasoning","Emotions","Combo"),"treatment",NA))
    
    followup_course_tests_data_by_type_final$course = ifelse(followup_course_tests_data_by_type_final$treatment == "No-course Baseline","control",
                                      ifelse(followup_course_tests_data_by_type_final$treatment %in% c("Reasoning","Emotions","Combo"),"treatment",NA))

    followup_course_text_tests_data_final$course = ifelse(followup_course_text_tests_data_final$treatment == "No-course Baseline","control",
                                      ifelse(followup_course_text_tests_data_final$treatment %in% c("Reasoning","Emotions","Combo"),"treatment",NA))
    
    opposite_tests_data_final$course = ifelse(opposite_tests_data_final$treatment == "No-course Baseline","control",
                                      ifelse(opposite_tests_data_final$treatment %in% c("Reasoning","Emotions","Combo"),"treatment",NA))
    
    non_counterparts_data_final$course = ifelse(non_counterparts_data_final$treatment == "No-course Baseline","control",
                                      ifelse(non_counterparts_data_final$treatment %in% c("Reasoning","Emotions","Combo"),"treatment",NA))
    
    all_posts_data_final$course = ifelse(all_posts_data_final$treatment == "No-course Baseline","control",
                                      ifelse(all_posts_data_final$treatment %in% c("Reasoning","Emotions","Combo"),"treatment",NA))
    # Apply attention check filter as specified #      
    if(att_check == "passed"){
        return(list("course_tests_data_final"=course_tests_data_final[course_tests_data_final$att_check_pre=="1" & course_tests_data_final$att_check_post=="1",],
            "course_tests_data_by_type_final" = course_tests_data_by_type_final[course_tests_data_by_type_final$att_check_pre=="1" & course_tests_data_by_type_final$att_check_post=="1",],
            "followup_course_tests_data_final" = followup_course_tests_data_final[followup_course_tests_data_final$att_check_pre=="1" & followup_course_tests_data_final$att_check_post=="1" &followup_course_tests_data_final$att_check_followup=="1",],
            "followup_course_tests_data_by_type_final" = followup_course_tests_data_by_type_final[followup_course_tests_data_by_type_final$att_check_pre=="1" & followup_course_tests_data_by_type_final$att_check_post=="1" &followup_course_tests_data_by_type_final$att_check_followup=="1",],
            "followup_course_text_tests_data_final" = followup_course_text_tests_data_final[followup_course_tests_data_final$att_check_pre=="1" & followup_course_tests_data_final$att_check_post=="1" &followup_course_tests_data_final$att_check_followup=="1",],
            "opposite_tests_data_final" = opposite_tests_data_final[opposite_tests_data_final$att_check_pre=="1" & opposite_tests_data_final$att_check_post=="1",],
            "non_counterparts_data_final" = non_counterparts_data_final[non_counterparts_data_final$att_check_pre=="1" & non_counterparts_data_final$att_check_post=="1",],
            "all_posts_data_final" = all_posts_data_final[all_posts_data_final$att_check_pre=="1" & all_posts_data_final$att_check_post=="1",]))}

    if(att_check == "not_passed"){
        return(list("course_tests_data_final"=course_tests_data_final[course_tests_data_final$att_check_pre=="0" & course_tests_data_final$att_check_post=="0",],
            "course_tests_data_by_type_final" = course_tests_data_by_type_final[course_tests_data_by_type_final$att_check_pre=="0" & course_tests_data_by_type_final$att_check_post=="0",],
            "followup_course_tests_data_final" = followup_course_tests_data_final[followup_course_tests_data_final$att_check_pre=="0" & followup_course_tests_data_final$att_check_post=="0" &followup_course_tests_data_final$att_check_followup=="0",],
            "followup_course_tests_data_by_type_final" = followup_course_tests_data_by_type_final[followup_course_tests_data_by_type_final$att_check_pre=="0" & followup_course_tests_data_by_type_final$att_check_post=="0" &followup_course_tests_data_by_type_final$att_check_followup=="0",],
            "followup_course_text_tests_data_final" = followup_course_text_tests_data_final[followup_course_tests_data_final$att_check_pre=="0" & followup_course_tests_data_final$att_check_post=="0" &followup_course_tests_data_final$att_check_followup=="0",],
            "opposite_tests_data_final" = opposite_tests_data_final[opposite_tests_data_final$att_check_pre=="0" & opposite_tests_data_final$att_check_post=="0",],
            "non_counterparts_data_final" = non_counterparts_data_final[non_counterparts_data_final$att_check_pre=="0" & non_counterparts_data_final$att_check_post=="0",],
            "all_posts_data_final" = all_posts_data_final[all_posts_data_final$att_check_pre=="0" & all_posts_data_final$att_check_post=="0",]))}

    if(att_check == "all"){
        return(list("course_tests_data_final"=course_tests_data_final,
            "course_tests_data_by_type_final" = course_tests_data_by_type_final,
            "followup_course_tests_data_final" = followup_course_tests_data_final,
            "followup_course_text_tests_data_final" = followup_course_text_tests_data_final,
            "followup_course_tests_data_by_type_final" = followup_course_tests_data_by_type_final,
            "opposite_tests_data_final" = opposite_tests_data_final,
            "non_counterparts_data_final" = non_counterparts_data_final,
            "all_posts_data_final" = all_posts_data_final))}
}

Data loading and cleaning

data = read_rds("../../intermediate_outcomes/intermediate_data_wide.rds")

data$social_media_share_above_40 = data$social_media_share_40_60 + data$social_media_share_60_80 + data$social_media_share_80_100

# Create vectors with names and display names of covariates by type to use later #
covariates_continuous = c("age","social_media_hours","base_rate_pre","base_total_acc_score_pre",
                          "misinfo_pre","misinfo_total_acc_score_pre")
covariates_categorical = c("gender_Man","education_High_school_or_less","marital_Married_or_in_a_domestic_partnership",
                           "location_Mostly_rural","religion_Christian","religiosity_Attends",
                           "social_media_share_above_40","att_check_pre")

subgroups_all = covariates_categorical

# Add percentile based subgroups (for continuous covariates #
for (cov in covariates_continuous){
    
    percs = ntile(data[,cov],4)
    data$med = factor(ifelse(!is.na(percs) & (percs == 1 | percs == 2),"M1",
                 ifelse(!is.na(percs) & (percs == 3 | percs == 4),"M2",NA)))
#     data_covs$quart = factor(ifelse(!is.na(percs) & percs == 1,"Q1",
#              ifelse(!is.na(percs) & percs == 2,NA, # Replace NA with Q2 as desired #
#              ifelse(!is.na(percs) & percs == 3,NA, # Replace NA with Q3 as desired #          
#              ifelse(!is.na(percs) & percs == 4,"Q4",NA)))))
    
    colnames(data)[ncol(data)] = paste(cov,c("_med"),sep="")
    subgroups_all = c(subgroups_all,paste(cov,c("_med"),sep=""))}

# Reorder subgroups #
subgroups_all = c(subgroups_all[startsWith(subgroups_all,"age")],subgroups_all[!startsWith(subgroups_all,"age")])
subgroups_all = c(subgroups_all[!startsWith(subgroups_all,"social_media") & !startsWith(subgroups_all,"share_pre") & !startsWith(subgroups_all,"acc_disc_score_pre") & !startsWith(subgroups_all,"att_check")],
              subgroups_all[startsWith(subgroups_all,"social_media_hours")],subgroups_all[startsWith(subgroups_all,"social_media_share")],
              subgroups_all[startsWith(subgroups_all,"share_pre")], subgroups_all[startsWith(subgroups_all,"acc_disc")],subgroups_all[startsWith(subgroups_all,"att_check")])

# Mutate subgroup variables to factor #
data[,subgroups_all] = lapply(data[,subgroups_all], as.factor)

# Add treatment indicators for each test #
data$treatment_6 = recode(data$treatment,"Reasoning"="1","Emotions" = "no","Combo" = "no","No-course Baseline" = "no","Facts Baseline" = "0")
data$treatment_7 = recode(data$treatment,"Reasoning"="no","Emotions" = "1","Combo" = "no","No-course Baseline" = "no","Facts Baseline" = "0")
data$treatment_8 = recode(data$treatment,"Reasoning"="no","Emotions" = "no","Combo" = "1","No-course Baseline" = "no","Facts Baseline" = "0")
data$treatment_9 = recode(data$treatment,"Reasoning"="no","Emotions" = "no","Combo" = "no","No-course Baseline" = "0","Facts Baseline" = "1")
data$treatment_10 = recode(data$treatment,"Reasoning"="1","Emotions" = "0","Combo" = "no","No-course Baseline" = "no","Facts Baseline" = "no")
data$treatment_11 = recode(data$treatment,"Reasoning"="1","Emotions" = "no","Combo" = "0","No-course Baseline" = "no","Facts Baseline" = "no")
data$treatment_12 = recode(data$treatment,"Reasoning"="no","Emotions" = "1","Combo" = "0","No-course Baseline" = "no","Facts Baseline" = "no")

data[,startsWith(colnames(data),"treatment_") & !startsWith(colnames(data),"treatment_end")] = sapply(data[,startsWith(colnames(data),"treatment_") & !startsWith(colnames(data),"treatment_end")],function(x){na_if(x,"no")})

Subgroup analysis

Base rate post (nonmisinfo post)

# Pre-allocate dataframe #
subgroup_results = data.frame(matrix(NA,0,6))
#pre_subgroup_results = data.frame(matrix(NA,0,6))
tests_of_interest = c(7, 8, 12)

# Compute each test for each of the defined subgroups #
# Iterate over desired tests #
for (test in tests_of_interest){
    # Create a temporary dataframe #
    by_test_subgroup_results = data.frame(matrix(NA,0,6))
    # Iterate over subgroups #
    for (cov in subgroups_all){
        # Create the formula so that the treatment effects for each subgroup are included in the regression, and not the diff-in-diff #
        form = formula(paste("base_rate_post ~ 0 + treatment_",test,"*", cov," - treatment_",test,sep=""))
        # Linear model to estimate treatment effects by subgroup #
        model = lm_robust(form,data)
        # Quasipoisson model to estimate treatment effects as a percentage of the baseline by subgroup (Poisson model assumes equality of mean and variance, this breaks it) # 
        model_perc = glm(form, family = 'quasipoisson', data)
        # Extract output from the regressions #
        baselines = coef(summary(model))[!startsWith(rownames(coef(summary(model))),"treatment"),][,1]
        coefs = coef(summary(model))[startsWith(rownames(coef(summary(model))),"treatment"),][,1:4]
        # Add number of observations by subgroup #
        obs = table(data[cov])
        # Define the diff-in-diff coefficients as required by multcomp package #
        coefs = cbind(baselines,coefs,obs)
        rownames(coefs) = paste("treatment_",test,"1:",rownames(coefs),sep="")
        colnames(coefs) = NULL
        hyp_names = combn(rownames(coefs),2)
        # Iterate over differences in subgroups #
        hyps = c()
        for (h in 1:ncol(hyp_names)){
            hyps = c(hyps,paste(hyp_names[1,h],"-",hyp_names[2,h],"= 0"))}
        # Extract results and fill the dataframe #
        comps = summary(glht(model, hyps),test=adjusted("none"))
        comps2 = summary(glht(model_perc, hyps),test=adjusted("none"))
        comps_results = cbind(NA,comps$test$coefficients,comps$test$sigma,comps$test$tstat,comps$test$pvalues,NA)
        rownames(comps_results) = paste(rownames(comps_results),"_lev",sep="")
        comps2_results = cbind(NA,comps2$test$coefficients,comps2$test$sigma,comps2$test$tstat,comps2$test$pvalues,NA)
        rownames(comps2_results) = paste(rownames(comps2_results),"_perc",sep="")
        by_test_subgroup_results = rbind(by_test_subgroup_results, coefs, comps_results,comps2_results
                                        )}
        rownames(by_test_subgroup_results) = paste("test_",test,str_replace_all(rownames(by_test_subgroup_results),paste("treatment_",test,"1",sep=""),""),sep="")
        subgroup_results  = rbind(subgroup_results ,by_test_subgroup_results)}

#    by_test_subgroup_results = data.frame(matrix(NA,0,6))
#    for (cov in pre_subgroups){
#        form = formula(paste("share_pre ~ 0+",cov,sep=""))
#        model = lm_robust(form,pre_data)
#        hyp_names = combn(rownames(coef(summary(model))),2)
#        hyps = c()
#        for (h in 1:ncol(hyp_names)){
#            hyps = c(hyps,paste(hyp_names[1,h],"-",hyp_names[2,h],"= 0"))}
#        comps = summary(glht(model, hyps),test=adjusted("none"))
#        comps_results = cbind(NA,comps$test$coefficients,comps$test$sigma,comps$test$tstat,comps$test$pvalues,NA)
#        by_test_subgroup_results = rbind(by_test_subgroup_results, comps_results)
#        rownames(by_test_subgroup_results) = paste("test_",test,":",rownames(by_test_subgroup_results),sep="")
#    pre_subgroup_results = rbind(pre_subgroup_results,by_test_subgroup_results)}

# Rename columns after stacking #
colnames(subgroup_results) = c("baseline","estimate","std.err","ts","p_val","obs")
#colnames(pre_subgroup_results) = c("baseline","estimate","std.err","ts","p_val","obs")

# Construct confidence intervals taking into account one and two-sided tests #
subgroup_results$CI_low =  ifelse(is.na(subgroup_results$obs),subgroup_results$estimate - 1.96*subgroup_results$std.err,-Inf)
subgroup_results$CI_upp =  ifelse(is.na(subgroup_results$obs),subgroup_results$estimate + 1.96*subgroup_results$std.err,subgroup_results$estimate + 1.64*subgroup_results$std.err)

subgroup_results$CI_low[startsWith(rownames(subgroup_results),"test_10")] = subgroup_results$estimate[startsWith(rownames(subgroup_results),"test_10")] - 1.96*subgroup_results$std.err[startsWith(rownames(subgroup_results),"test_10")]
subgroup_results$CI_low[startsWith(rownames(subgroup_results),"test_11")] = subgroup_results$estimate[startsWith(rownames(subgroup_results),"test_11")] - 1.96*subgroup_results$std.err[startsWith(rownames(subgroup_results),"test_11")]
subgroup_results$CI_low[startsWith(rownames(subgroup_results),"test_12")] = subgroup_results$estimate[startsWith(rownames(subgroup_results),"test_12")] - 1.96*subgroup_results$std.err[startsWith(rownames(subgroup_results),"test_12")]

#pre_subgroup_results$CI_low =  pre_subgroup_results$estimate - 1.96*pre_subgroup_results$std.err
#pre_subgroup_results$CI_upp =  pre_subgroup_results$estimate + 1.96*pre_subgroup_results$std.err

# if contain string
subgroup_results$test_info <- ifelse(str_detect(row.names(subgroup_results), "test_6") ,"Reasoning v Facts Baseline",
                              ifelse(str_detect(row.names(subgroup_results), "test_7") ,"Emotions v Facts Baseline",
                              ifelse(str_detect(row.names(subgroup_results), "test_8") ,"Combo v Facts Baseline",
                              ifelse(str_detect(row.names(subgroup_results), "test_9") ,"Facts Baseline v No-course Baseline",
                              ifelse(str_detect(row.names(subgroup_results), "test_10") ,"Reasoning v Emotixwons ",
                              ifelse(str_detect(row.names(subgroup_results), "test_11") ,"Reasoning v Combo",
                              ifelse(str_detect(row.names(subgroup_results), "test_12") ,"Emotions v Combo",NA)))))))

write.csv(subgroup_results, "tables/nonmisinfo_post_subgroup.csv")

RW correction

This part of the script applies the Romano-Wolf correction. Even though it is parallelized, it takes a long time. Please look at https://docs.iza.org/dp12845.pdf for references.

# Since we have two-sided tests along with one-sided tests, we need to adapt the two-sided statistics accordingly: #
# Identify the tests that are two sided #
two_sided = rownames(subgroup_results)[!is.infinite(subgroup_results$CI_low)]

# We create the dataframes with the results separately by hypothesis involving differences in levels, differences in percentage changes, and treatment effect by subgroups #
sorted_orig_lev = subgroup_results[endsWith(rownames(subgroup_results),"lev"),]
sorted_orig_perc = subgroup_results[endsWith(rownames(subgroup_results),"perc"),]
sorted_orig_single = subgroup_results[!endsWith(rownames(subgroup_results),"lev") & ! endsWith(rownames(subgroup_results),"perc"),]
# We check if we reject the null hypothesis using the unadjusted p-values
alpha = 0.05
sorted_orig_lev$reject = sorted_orig_lev$p_val < alpha
sorted_orig_perc$reject = sorted_orig_perc$p_val < alpha
sorted_orig_single$reject = sorted_orig_single$p_val < alpha
# We adapt the t-statistic of the two-sided tests:
sorted_orig_lev[rownames(sorted_orig_lev) %in% two_sided,"ts"] = -abs(sorted_orig_lev[rownames(sorted_orig_lev) %in% two_sided,"ts"])
sorted_orig_perc[rownames(sorted_orig_perc) %in% two_sided,"ts"] = -abs(sorted_orig_perc[rownames(sorted_orig_perc) %in% two_sided,"ts"])
sorted_orig_single[rownames(sorted_orig_single) %in% two_sided,"ts"] = -abs(sorted_orig_single[rownames(sorted_orig_single) %in% two_sided,"ts"])
# We sort the hypotheses by the size of the original t-statistic #
sorted_orig_lev = sorted_orig_lev[order(sorted_orig_lev$ts),]
sorted_orig_perc = sorted_orig_perc[order(sorted_orig_perc$ts),]
sorted_orig_single = sorted_orig_single[order(sorted_orig_single$ts),]

# Now that we have what we need, we can implement Romano-Wolf correction #
# We need to bootstrap the sample B times (this essentially means sample with replacement keeping n constant) and compute  #
# the "null statistics" of each hypothesis we are testing for each bootstrapped sample. These require three elements: #
# 1) the original estimates --we already got those-- #
# 2) the same estimates but computed from the bootstrapped sample #
# 3) the standard errors from 2 #
B = 1000
num_hyp_lev = nrow(sorted_orig_lev)
num_hyp_perc = nrow(sorted_orig_perc)
num_hyp_single = nrow(sorted_orig_single)

RW = function(b){
    # We sample with replacement the indices which will make up our bootstrap sample #
    idx = sample(1:nrow(data),nrow(data),TRUE)
    boot_data = data[idx,]
    # We repeat exactly the same procedure as before to get the bootstrap estimates and standard errors #
    boot_subgroup_results = data.frame(matrix(NA,nrow=0,ncol=2))
    for (test in tests_of_interest){
        boot_by_test_subgroup_results = data.frame(matrix(NA,0,2))
        for (cov in subgroups_all){
        form = formula(paste("base_rate_post ~ 0 + treatment_",test,"*", cov," - treatment_",test,sep=""))
        model = lm_robust(form,boot_data)
        model_perc = glm(form, family = 'quasipoisson', boot_data)
        coefs = coef(summary(model))[startsWith(rownames(coef(summary(model))),"treatment"),][,1:2]
        colnames(coefs) = NULL
        hyp_names = combn(rownames(coefs),2)
        hyps = c()
        for (h in 1:ncol(hyp_names)){
            hyps = c(hyps,paste(hyp_names[1,h],"-",hyp_names[2,h],"= 0"))}
        comps = summary(glht(model, hyps),test=adjusted("none"))
        comps2 = summary(glht(model_perc, hyps),test=adjusted("none"))
        comps_results = cbind(comps$test$coefficients,comps$test$sigma)
        rownames(comps_results) = paste(rownames(comps_results),"_lev",sep="")
        comps2_results = cbind(comps2$test$coefficients,comps2$test$sigma)
        rownames(comps2_results) = paste(rownames(comps2_results),"_perc",sep="")
        boot_by_test_subgroup_results = rbind(boot_by_test_subgroup_results, coefs, comps_results, comps2_results
                                             )}
        rownames(boot_by_test_subgroup_results) = paste("test_",test,str_replace_all(rownames(boot_by_test_subgroup_results),paste("treatment_",test,"1",sep=""),""),sep="")
    boot_subgroup_results = rbind(boot_subgroup_results,boot_by_test_subgroup_results)}
    
    colnames(boot_subgroup_results) = c("estimate","std.err")
    # The null statistic is computed by subtracting the original estimate from the bootstrap sample and dividing #
    # by the standard error of the bootstraped estimate #
    # Remember the absolute value since we are testing a two-sided hypothesis #
    # Also, note that we sort the columns of the bootstrapped statistics by the order of significance as required by RW #
    Bs_lev = (boot_subgroup_results[rownames(sorted_orig_lev),"estimate"] - sorted_orig_lev$estimate)/boot_subgroup_results[rownames(sorted_orig_lev),"std.err"]
    Bs_perc = (boot_subgroup_results[rownames(sorted_orig_perc),"estimate"] - sorted_orig_perc$estimate)/boot_subgroup_results[rownames(sorted_orig_perc),"std.err"]
    Bs_single = (boot_subgroup_results[rownames(sorted_orig_single),"estimate"] - sorted_orig_single$estimate)/boot_subgroup_results[rownames(sorted_orig_single),"std.err"]
    
    return(rbind(Bs_lev,
                 Bs_perc,
                 Bs_single))}

# Run the RW function using parallelization #
start_time <- Sys.time()
set.seed(777)
boot_output = foreach(b = 1:B) %dopar% RW(b)
bootstrapped_statistics = do.call(rbind.data.frame, boot_output)
end_time <- Sys.time()
end_time - start_time

## Time difference of 1.271531 mins

# Convert the output to dataframes #
bootstrapped_statistics_lev = bootstrapped_statistics[startsWith(rownames(bootstrapped_statistics),"Bs_lev"),1:nrow(sorted_orig_lev)]
bootstrapped_statistics_perc = bootstrapped_statistics[startsWith(rownames(bootstrapped_statistics),"Bs_perc"),1:nrow(sorted_orig_perc)]
bootstrapped_statistics_single = bootstrapped_statistics[startsWith(rownames(bootstrapped_statistics),"Bs_single"),1:nrow(sorted_orig_single)]
# Important, we keep the order of the sorted set of hypotheses #
colnames(bootstrapped_statistics_lev) = rownames(sorted_orig_lev)
colnames(bootstrapped_statistics_perc) = rownames(sorted_orig_perc)
colnames(bootstrapped_statistics_single) = rownames(sorted_orig_single)
# We adapt the t-statistics of the two-sided tests:
bootstrapped_statistics_lev[rownames(sorted_orig_lev) %in% two_sided] = -abs(bootstrapped_statistics_lev[rownames(sorted_orig_lev) %in% two_sided])
bootstrapped_statistics_perc[rownames(sorted_orig_perc) %in% two_sided] = -abs(bootstrapped_statistics_perc[rownames(sorted_orig_perc) %in% two_sided])
bootstrapped_statistics_single[rownames(sorted_orig_single) %in% two_sided] = -abs(bootstrapped_statistics_single[rownames(sorted_orig_single) %in% two_sided])

# Now we compute the "max" statistics and the empirical quantiles #
min_stats_lev = data.frame(matrix(NA,B,num_hyp_lev))
colnames(min_stats_lev) = colnames(bootstrapped_statistics_lev)
for (h in 1:(num_hyp_lev-1)){
    min_stats_lev[,h] = apply(bootstrapped_statistics_lev[,h:num_hyp_lev], MARGIN=1, FUN=min)}
min_stats_lev[,num_hyp_lev] = bootstrapped_statistics_lev[,num_hyp_lev]

# Now we compute the "max" statistics and the empirical quantiles #
min_stats_perc = data.frame(matrix(NA,B,num_hyp_perc))
colnames(min_stats_perc) = colnames(bootstrapped_statistics_perc)
for (h in 1:(num_hyp_perc-1)){
    min_stats_perc[,h] = apply(bootstrapped_statistics_perc[,h:num_hyp_perc], MARGIN=1, FUN=min)}
min_stats_perc[,num_hyp_perc] = bootstrapped_statistics_perc[,num_hyp_perc]

# Now we compute the "max" statistics and the empirical quantiles #
min_stats_single = data.frame(matrix(NA,B,num_hyp_single))
colnames(min_stats_single) = colnames(bootstrapped_statistics_single)
for (h in 1:(num_hyp_single-1)){
    min_stats_single[,h] = apply(bootstrapped_statistics_single[,h:num_hyp_single], MARGIN=1, FUN=min)}
min_stats_single[,num_hyp_single] = bootstrapped_statistics_single[,num_hyp_single]

# And the empirical quantiles given the significance level #
Cs_lev = unname(apply(min_stats_lev, MARGIN=2, FUN=quantile, probs=c(alpha)))

# And the empirical quantiles given the significance level #
Cs_perc = unname(apply(min_stats_perc, MARGIN=2, FUN=quantile, probs=c(alpha)))

# And the empirical quantiles given the significance level #
Cs_single = unname(apply(min_stats_single, MARGIN=2, FUN=quantile, probs=c(alpha)))

# And finally we run the rejection algorithm
sorted_orig_lev$adjust_rejected = NA
# R will index the hypothesis we need to test at each round
R = 1
# r will be the counter of rejected hypotheses at each round
r = Inf
# The while loop stops if no hypotheses are rejected in a round or if we reject all.
while (r > 0 & R <= num_hyp_lev){
    # We compare the original t-statistics of the hypotheses we haven't tested at each round with the quantiles
    sorted_orig_lev$adjust_rejected[R:num_hyp_lev] = (sorted_orig_lev$ts[R:num_hyp_lev] < Cs_lev[R])
    r = sum(sorted_orig_lev$adjust_rejected[R:num_hyp_lev])
    R = r + R}

# And finally we run the rejection algorithm
sorted_orig_perc$adjust_rejected = NA
# R will index the hypothesis we need to test at each round
R = 1
# r will be the counter of rejected hypotheses at each round
r = Inf
# The while loop stops if no hypotheses are rejected in a round or if we reject all.
while (r > 0 & R <= num_hyp_perc){
    # We compare the original t-statistics of the hypotheses we haven't tested at each round with the quantiles
    sorted_orig_perc$adjust_rejected[R:num_hyp_perc] = (sorted_orig_perc$ts[R:num_hyp_perc] < Cs_perc[R])
    r = sum(sorted_orig_perc$adjust_rejected[R:num_hyp_perc])
    R = r + R}

# And finally we run the rejection algorithm
sorted_orig_single$adjust_rejected = NA
# R will index the hypothesis we need to test at each round
R = 1
# r will be the counter of rejected hypotheses at each round
r = Inf
# The while loop stops if no hypotheses are rejected in a round or if we reject all.
while (r > 0 & R <= num_hyp_single){
    # We compare the original t-statistics of the hypotheses we haven't tested at each round with the quantiles
    sorted_orig_single$adjust_rejected[R:num_hyp_single] = (sorted_orig_single$ts[R:num_hyp_single] < Cs_single[R])
    r = sum(sorted_orig_single$adjust_rejected[R:num_hyp_single])
    R = r + R}

# We can also compute the adjusted p-values
sorted_orig_lev$p_val_RW = NA
sorted_orig_lev$p_val_RW[1] = (sum(min_stats_lev[,1]<=sorted_orig_lev$ts[1])+1)/(B+1)
for (h in 2:num_hyp_lev){
    sorted_orig_lev$p_val_RW[h] = max((sum(min_stats_lev[,h]<=sorted_orig_lev$ts[h])+1)/(B+1), sorted_orig_lev$p_val_RW[h-1])}

# We can also compute the adjusted p-values
sorted_orig_perc$p_val_RW = NA
sorted_orig_perc$p_val_RW[1] = (sum(min_stats_perc[,1]<=sorted_orig_perc$ts[1])+1)/(B+1)
for (h in 2:num_hyp_perc){
    sorted_orig_perc$p_val_RW[h] = max((sum(min_stats_perc[,h]<=sorted_orig_perc$ts[h])+1)/(B+1), sorted_orig_perc$p_val_RW[h-1])}

# We can also compute the adjusted p-values
sorted_orig_single$p_val_RW = NA
sorted_orig_single$p_val_RW[1] = (sum(min_stats_single[,1]<=sorted_orig_single$ts[1])+1)/(B+1)
for (h in 2:num_hyp_single){
    sorted_orig_single$p_val_RW[h] = max((sum(min_stats_single[,h]<=sorted_orig_single$ts[h])+1)/(B+1), sorted_orig_single$p_val_RW[h-1])}

#Compare them to Bonferroni's and Holm's p-values (note that the minimum p-value in RW is 1/(B+1))
sorted_orig_lev$p_val_bonf = p.adjust(sorted_orig_lev$p_val,"bonferroni")
sorted_orig_lev$p_val_holm = p.adjust(sorted_orig_lev$p_val,"holm")

# Compare them to Bonferroni's p-values (note that the minimum p-value in RW is 1/(B+1))
sorted_orig_perc$p_val_bonf = p.adjust(sorted_orig_perc$p_val,"bonferroni")
sorted_orig_perc$p_val_holm = p.adjust(sorted_orig_perc$p_val,"holm")

# Compare them to Bonferroni's p-values (note that the minimum p-value in RW is 1/(B+1))
sorted_orig_single$p_val_bonf = p.adjust(sorted_orig_single$p_val,"bonferroni")
sorted_orig_single$p_val_holm = p.adjust(sorted_orig_single$p_val,"holm")

# Stack results #
final = rbind(sorted_orig_lev,
              sorted_orig_perc,
              sorted_orig_single)

# Filter columns #
final_subgroup_results = final[rownames(final),c("obs","baseline","estimate","std.err","CI_low","CI_upp","p_val","p_val_holm","p_val_RW")]

Visualization

# Extract the name of the tests #
tests = rownames(final_subgroup_results)

head(tests)

## [1] "test_7:base_rate_pre_medM1 - :base_rate_pre_medM2_lev"                            
## [2] "test_7:misinfo_pre_medM1 - :misinfo_pre_medM2_lev"                                
## [3] "test_12:misinfo_pre_medM1 - :misinfo_pre_medM2_lev"                               
## [4] "test_7:religiosity_Attends0 - :religiosity_Attends1_lev"                          
## [5] "test_7:misinfo_total_acc_score_pre_medM1 - :misinfo_total_acc_score_pre_medM2_lev"
## [6] "test_7:social_media_share_above_400 - :social_media_share_above_401_lev"

# Adjust the name of the tests for display purposes #
tests = str_replace_all(tests, ":", "")

tests = str_replace_all(tests, "test_10", "Test 10: ")
tests = str_replace_all(tests, "test_11", "Test 11: ")
tests = str_replace_all(tests, "test_12", "Test 12: ")
tests = str_replace_all(tests, "test_6", "Test 6: ")
tests = str_replace_all(tests, "test_7", "Test 7: ")
tests = str_replace_all(tests, "test_8", "Test 8: ")
tests = str_replace_all(tests, "test_9", "Test 9: ")

tests = str_replace_all(tests, "_medM1", "Below median")
tests = str_replace_all(tests, "_medM2", "Above median")

tests = str_replace_all(tests, "att_check_pre0", "Fail AC")
tests = str_replace_all(tests, "att_check_pre1", "Pass AC")
tests = str_replace_all(tests, "gender_Man0", "Gender: Woman")
tests = str_replace_all(tests, "gender_Man1", "Gender: Man")
tests = str_replace_all(tests, "education_High_school_or_less0", "Education: More than High School")
tests = str_replace_all(tests, "education_High_school_or_less1", "Education: High School or Less")
tests = str_replace_all(tests, "base_rate_pre", "Base Rate Pre ")
tests = str_replace_all(tests, "misinfo_pre", "Misinfo Rate Pre ")
tests = str_replace_all(tests, "base_total_acc_score_pre", "Base Acc Score Pre ")
tests = str_replace_all(tests, "misinfo_total_acc_score_pre", "Misinfo Acc Score Pre ")
tests = str_replace_all(tests, "High school or less1", "High School or Less")

tests = str_replace_all(tests, "marital_Married_or_in_a_domestic_partnership1", "Married ")
tests = str_replace_all(tests, "marital_Married_or_in_a_domestic_partnership0", "Non-Married ")
tests = str_replace_all(tests, "location_Mostly_rural0", "Location: Non Rural ")
tests = str_replace_all(tests, "location_Mostly_rural1", "Location: Mostly Rural ")
tests = str_replace_all(tests, "religion_Christian0", "Religion: Non-Christian ")
tests = str_replace_all(tests, "religion_Christian1", "Religion: Christian ")
tests = str_replace_all(tests, "religiosity_Attends0", "Religiosity: Does not attend ")
tests = str_replace_all(tests, "religiosity_Attends1", "Religiosity: Attends ")
tests = str_replace_all(tests, "social_media_share_above_400", "Prop. of content shared below 40% ")
tests = str_replace_all(tests, "social_media_share_above_401", "Prop. of content shared above 40% ")
tests = str_replace_all(tests, "age", "Age: ")



tests = str_replace_all(tests, "social_media_hours", "Hrs/day on social media: ")
tests = str_replace_all(tests, "share_pre_all", "Pre-survey sharing: ")
tests = str_replace_all(tests, "share_pre_misinfo", "Pre-survey misinfo sharing: ")
tests = str_replace_all(tests, "acc_disc_score_pre", "Pre-survey accuracy disc. score: ")
tests = str_replace_all(tests, "att_check_group", "Pre-survey attention checks: ")
tests = str_replace_all(tests, "zero", "0-20%")
tests = str_replace_all(tests, "twenty", "20-40%")
tests = str_replace_all(tests, "forty", "40-60%")
tests = str_replace_all(tests, "sixty", "60-80%")
tests = str_replace_all(tests, "eighty", "80-100%")
tests = str_replace_all(tests, "no", "None")
tests = str_replace_all(tests, "one", "One")
tests = str_replace_all(tests, "all", "All")
tests = str_replace_all(tests, "NOnet", "not")
tests = str_replace_all(tests, "_", " ")

head(tests)

## [1] "Test 7: Base Rate Pre Below median - Base Rate Pre Above median lev"                
## [2] "Test 7: Misinfo Rate Pre Below median - Misinfo Rate Pre Above median lev"          
## [3] "Test 12: Misinfo Rate Pre Below median - Misinfo Rate Pre Above median lev"         
## [4] "Test 7: Religiosity: Does not attend  - Religiosity: Attends  lev"                  
## [5] "Test 7: Misinfo Acc Score Pre Below median - Misinfo Acc Score Pre Above median lev"
## [6] "Test 7: Prop. of content shared below 40%  - Prop. of content shared above 40%  lev"

# Rename the dataframe with the results #
rownames(final_subgroup_results) = tests

# Store the results #
saveRDS(final_subgroup_results,"subgroup_analysis_base_rate_post.RDS")

write.csv(final_subgroup_results, "tables/nonmisinfo_post_subgroup_RW.csv")

# Snapshot of diff-in-diff by increasing p-value #
diff = final_subgroup_results[is.na(final_subgroup_results$obs),]
round(diff[order(diff$p_val_RW),],4)

##                                                                                             obs
## Test 7: Base Rate Pre Below median - Base Rate Pre Above median lev                          NA
## Test 7: Misinfo Rate Pre Below median - Misinfo Rate Pre Above median lev                    NA
## Test 12: Misinfo Rate Pre Below median - Misinfo Rate Pre Above median lev                   NA
## Test 12: Misinfo Rate Pre Below median - Misinfo Rate Pre Above median perc                  NA
## Test 7: Religiosity: Does not attend  - Religiosity: Attends  lev                            NA
## Test 7: Religiosity: Does not attend  - Religiosity: Attends  perc                           NA
## Test 7: Misinfo Acc Score Pre Below median - Misinfo Acc Score Pre Above median lev          NA
## Test 7: Base Rate Pre Below median - Base Rate Pre Above median perc                         NA
## Test 7: Misinfo Rate Pre Below median - Misinfo Rate Pre Above median perc                   NA
## Test 7: Prop. of content shared below 40%  - Prop. of content shared above 40%  lev          NA
## Test 8: Base Rate Pre Below median - Base Rate Pre Above median lev                          NA
## Test 12: Base Rate Pre Below median - Base Rate Pre Above median lev                         NA
## Test 12: Base Rate Pre Below median - Base Rate Pre Above median perc                        NA
## Test 8: Religiosity: Does not attend  - Religiosity: Attends  lev                            NA
## Test 8: Religiosity: Does not attend  - Religiosity: Attends  perc                           NA
## Test 8: Location: Non Rural  - Location: Mostly Rural  lev                                   NA
## Test 8: Location: Non Rural  - Location: Mostly Rural  perc                                  NA
## Test 8: Base Acc Score Pre Below median - Base Acc Score Pre Above median lev                NA
## Test 12: Hrs/day on social media: Below median - Hrs/day on social media: Above median lev   NA
## Test 12: Hrs/day on social media: Below median - Hrs/day on social media: Above median perc  NA
## Test 7: Misinfo Acc Score Pre Below median - Misinfo Acc Score Pre Above median perc         NA
## Test 7: Prop. of content shared below 40%  - Prop. of content shared above 40%  perc         NA
## Test 7: Non-Married  - Married  perc                                                         NA
## Test 12: Prop. of content shared below 40%  - Prop. of content shared above 40%  perc        NA
## Test 8: Misinfo Rate Pre Below median - Misinfo Rate Pre Above median perc                   NA
## Test 12: Misinfo Acc Score Pre Below median - Misinfo Acc Score Pre Above median lev         NA
## Test 8: Hrs/day on social media: Below median - Hrs/day on social media: Above median lev    NA
## Test 12: Prop. of content shared below 40%  - Prop. of content shared above 40%  lev         NA
## Test 8: Misinfo Acc Score Pre Below median - Misinfo Acc Score Pre Above median lev          NA
## Test 7: Religion: Non-Christian  - Religion: Christian  lev                                  NA
## Test 12: Religiosity: Does not attend  - Religiosity: Attends  lev                           NA
## Test 7: Fail AC - Pass AC lev                                                                NA
## Test 7: Base Acc Score Pre Below median - Base Acc Score Pre Above median lev                NA
## Test 8: Prop. of content shared below 40%  - Prop. of content shared above 40%  lev          NA
## Test 7: Non-Married  - Married  lev                                                          NA
## Test 12: Location: Non Rural  - Location: Mostly Rural  lev                                  NA
## Test 12: Non-Married  - Married  lev                                                         NA
## Test 8: Misinfo Rate Pre Below median - Misinfo Rate Pre Above median lev                    NA
## Test 8: Fail AC - Pass AC lev                                                                NA
## Test 8: Religion: Non-Christian  - Religion: Christian  lev                                  NA
## Test 7: Location: Non Rural  - Location: Mostly Rural  lev                                   NA
## Test 8: Age: Below median - Age: Above median lev                                            NA
## Test 12: Base Acc Score Pre Below median - Base Acc Score Pre Above median lev               NA
## Test 12: Religion: Non-Christian  - Religion: Christian  lev                                 NA
## Test 7: Age: Below median - Age: Above median lev                                            NA
## Test 12: Gender: Woman - Gender: Man lev                                                     NA
## Test 12: Education: More than High School - Education: High School or Less lev               NA
## Test 12: Fail AC - Pass AC lev                                                               NA
## Test 7: Education: More than High School - Education: High School or Less lev                NA
## Test 7: Gender: Woman - Gender: Man lev                                                      NA
## Test 8: Gender: Woman - Gender: Man lev                                                      NA
## Test 8: Non-Married  - Married  lev                                                          NA
## Test 12: Age: Below median - Age: Above median lev                                           NA
## Test 7: Hrs/day on social media: Below median - Hrs/day on social media: Above median lev    NA
## Test 8: Education: More than High School - Education: High School or Less lev                NA
## Test 12: Misinfo Acc Score Pre Below median - Misinfo Acc Score Pre Above median perc        NA
## Test 12: Religiosity: Does not attend  - Religiosity: Attends  perc                          NA
## Test 8: Hrs/day on social media: Below median - Hrs/day on social media: Above median perc   NA
## Test 7: Religion: Non-Christian  - Religion: Christian  perc                                 NA
## Test 12: Location: Non Rural  - Location: Mostly Rural  perc                                 NA
## Test 12: Non-Married  - Married  perc                                                        NA
## Test 7: Fail AC - Pass AC perc                                                               NA
## Test 8: Religion: Non-Christian  - Religion: Christian  perc                                 NA
## Test 7: Location: Non Rural  - Location: Mostly Rural  perc                                  NA
## Test 12: Base Acc Score Pre Below median - Base Acc Score Pre Above median perc              NA
## Test 7: Hrs/day on social media: Below median - Hrs/day on social media: Above median perc   NA
## Test 7: Gender: Woman - Gender: Man perc                                                     NA
## Test 12: Religion: Non-Christian  - Religion: Christian  perc                                NA
## Test 12: Gender: Woman - Gender: Man perc                                                    NA
## Test 8: Base Acc Score Pre Below median - Base Acc Score Pre Above median perc               NA
## Test 8: Non-Married  - Married  perc                                                         NA
## Test 8: Fail AC - Pass AC perc                                                               NA
## Test 12: Education: More than High School - Education: High School or Less perc              NA
## Test 8: Base Rate Pre Below median - Base Rate Pre Above median perc                         NA
## Test 12: Fail AC - Pass AC perc                                                              NA
## Test 7: Education: More than High School - Education: High School or Less perc               NA
## Test 12: Age: Below median - Age: Above median perc                                          NA
## Test 8: Education: More than High School - Education: High School or Less perc               NA
## Test 7: Base Acc Score Pre Below median - Base Acc Score Pre Above median perc               NA
## Test 8: Age: Below median - Age: Above median perc                                           NA
## Test 8: Prop. of content shared below 40%  - Prop. of content shared above 40%  perc         NA
## Test 7: Age: Below median - Age: Above median perc                                           NA
## Test 8: Misinfo Acc Score Pre Below median - Misinfo Acc Score Pre Above median perc         NA
## Test 8: Gender: Woman - Gender: Man perc                                                     NA
##                                                                                             baseline
## Test 7: Base Rate Pre Below median - Base Rate Pre Above median lev                               NA
## Test 7: Misinfo Rate Pre Below median - Misinfo Rate Pre Above median lev                         NA
## Test 12: Misinfo Rate Pre Below median - Misinfo Rate Pre Above median lev                        NA
## Test 12: Misinfo Rate Pre Below median - Misinfo Rate Pre Above median perc                       NA
## Test 7: Religiosity: Does not attend  - Religiosity: Attends  lev                                 NA
## Test 7: Religiosity: Does not attend  - Religiosity: Attends  perc                                NA
## Test 7: Misinfo Acc Score Pre Below median - Misinfo Acc Score Pre Above median lev               NA
## Test 7: Base Rate Pre Below median - Base Rate Pre Above median perc                              NA
## Test 7: Misinfo Rate Pre Below median - Misinfo Rate Pre Above median perc                        NA
## Test 7: Prop. of content shared below 40%  - Prop. of content shared above 40%  lev               NA
## Test 8: Base Rate Pre Below median - Base Rate Pre Above median lev                               NA
## Test 12: Base Rate Pre Below median - Base Rate Pre Above median lev                              NA
## Test 12: Base Rate Pre Below median - Base Rate Pre Above median perc                             NA
## Test 8: Religiosity: Does not attend  - Religiosity: Attends  lev                                 NA
## Test 8: Religiosity: Does not attend  - Religiosity: Attends  perc                                NA
## Test 8: Location: Non Rural  - Location: Mostly Rural  lev                                        NA
## Test 8: Location: Non Rural  - Location: Mostly Rural  perc                                       NA
## Test 8: Base Acc Score Pre Below median - Base Acc Score Pre Above median lev                     NA
## Test 12: Hrs/day on social media: Below median - Hrs/day on social media: Above median lev        NA
## Test 12: Hrs/day on social media: Below median - Hrs/day on social media: Above median perc       NA
## Test 7: Misinfo Acc Score Pre Below median - Misinfo Acc Score Pre Above median perc              NA
## Test 7: Prop. of content shared below 40%  - Prop. of content shared above 40%  perc              NA
## Test 7: Non-Married  - Married  perc                                                              NA
## Test 12: Prop. of content shared below 40%  - Prop. of content shared above 40%  perc             NA
## Test 8: Misinfo Rate Pre Below median - Misinfo Rate Pre Above median perc                        NA
## Test 12: Misinfo Acc Score Pre Below median - Misinfo Acc Score Pre Above median lev              NA
## Test 8: Hrs/day on social media: Below median - Hrs/day on social media: Above median lev         NA
## Test 12: Prop. of content shared below 40%  - Prop. of content shared above 40%  lev              NA
## Test 8: Misinfo Acc Score Pre Below median - Misinfo Acc Score Pre Above median lev               NA
## Test 7: Religion: Non-Christian  - Religion: Christian  lev                                       NA
## Test 12: Religiosity: Does not attend  - Religiosity: Attends  lev                                NA
## Test 7: Fail AC - Pass AC lev                                                                     NA
## Test 7: Base Acc Score Pre Below median - Base Acc Score Pre Above median lev                     NA
## Test 8: Prop. of content shared below 40%  - Prop. of content shared above 40%  lev               NA
## Test 7: Non-Married  - Married  lev                                                               NA
## Test 12: Location: Non Rural  - Location: Mostly Rural  lev                                       NA
## Test 12: Non-Married  - Married  lev                                                              NA
## Test 8: Misinfo Rate Pre Below median - Misinfo Rate Pre Above median lev                         NA
## Test 8: Fail AC - Pass AC lev                                                                     NA
## Test 8: Religion: Non-Christian  - Religion: Christian  lev                                       NA
## Test 7: Location: Non Rural  - Location: Mostly Rural  lev                                        NA
## Test 8: Age: Below median - Age: Above median lev                                                 NA
## Test 12: Base Acc Score Pre Below median - Base Acc Score Pre Above median lev                    NA
## Test 12: Religion: Non-Christian  - Religion: Christian  lev                                      NA
## Test 7: Age: Below median - Age: Above median lev                                                 NA
## Test 12: Gender: Woman - Gender: Man lev                                                          NA
## Test 12: Education: More than High School - Education: High School or Less lev                    NA
## Test 12: Fail AC - Pass AC lev                                                                    NA
## Test 7: Education: More than High School - Education: High School or Less lev                     NA
## Test 7: Gender: Woman - Gender: Man lev                                                           NA
## Test 8: Gender: Woman - Gender: Man lev                                                           NA
## Test 8: Non-Married  - Married  lev                                                               NA
## Test 12: Age: Below median - Age: Above median lev                                                NA
## Test 7: Hrs/day on social media: Below median - Hrs/day on social media: Above median lev         NA
## Test 8: Education: More than High School - Education: High School or Less lev                     NA
## Test 12: Misinfo Acc Score Pre Below median - Misinfo Acc Score Pre Above median perc             NA
## Test 12: Religiosity: Does not attend  - Religiosity: Attends  perc                               NA
## Test 8: Hrs/day on social media: Below median - Hrs/day on social media: Above median perc        NA
## Test 7: Religion: Non-Christian  - Religion: Christian  perc                                      NA
## Test 12: Location: Non Rural  - Location: Mostly Rural  perc                                      NA
## Test 12: Non-Married  - Married  perc                                                             NA
## Test 7: Fail AC - Pass AC perc                                                                    NA
## Test 8: Religion: Non-Christian  - Religion: Christian  perc                                      NA
## Test 7: Location: Non Rural  - Location: Mostly Rural  perc                                       NA
## Test 12: Base Acc Score Pre Below median - Base Acc Score Pre Above median perc                   NA
## Test 7: Hrs/day on social media: Below median - Hrs/day on social media: Above median perc        NA
## Test 7: Gender: Woman - Gender: Man perc                                                          NA
## Test 12: Religion: Non-Christian  - Religion: Christian  perc                                     NA
## Test 12: Gender: Woman - Gender: Man perc                                                         NA
## Test 8: Base Acc Score Pre Below median - Base Acc Score Pre Above median perc                    NA
## Test 8: Non-Married  - Married  perc                                                              NA
## Test 8: Fail AC - Pass AC perc                                                                    NA
## Test 12: Education: More than High School - Education: High School or Less perc                   NA
## Test 8: Base Rate Pre Below median - Base Rate Pre Above median perc                              NA
## Test 12: Fail AC - Pass AC perc                                                                   NA
## Test 7: Education: More than High School - Education: High School or Less perc                    NA
## Test 12: Age: Below median - Age: Above median perc                                               NA
## Test 8: Education: More than High School - Education: High School or Less perc                    NA
## Test 7: Base Acc Score Pre Below median - Base Acc Score Pre Above median perc                    NA
## Test 8: Age: Below median - Age: Above median perc                                                NA
## Test 8: Prop. of content shared below 40%  - Prop. of content shared above 40%  perc              NA
## Test 7: Age: Below median - Age: Above median perc                                                NA
## Test 8: Misinfo Acc Score Pre Below median - Misinfo Acc Score Pre Above median perc              NA
## Test 8: Gender: Woman - Gender: Man perc                                                          NA
##                                                                                             estimate
## Test 7: Base Rate Pre Below median - Base Rate Pre Above median lev                           0.0941
## Test 7: Misinfo Rate Pre Below median - Misinfo Rate Pre Above median lev                     0.0908
## Test 12: Misinfo Rate Pre Below median - Misinfo Rate Pre Above median lev                    0.0751
## Test 12: Misinfo Rate Pre Below median - Misinfo Rate Pre Above median perc                   0.1483
## Test 7: Religiosity: Does not attend  - Religiosity: Attends  lev                             0.1484
## Test 7: Religiosity: Does not attend  - Religiosity: Attends  perc                            0.2656
## Test 7: Misinfo Acc Score Pre Below median - Misinfo Acc Score Pre Above median lev           0.0563
## Test 7: Base Rate Pre Below median - Base Rate Pre Above median perc                          0.0982
## Test 7: Misinfo Rate Pre Below median - Misinfo Rate Pre Above median perc                    0.0960
## Test 7: Prop. of content shared below 40%  - Prop. of content shared above 40%  lev           0.0525
## Test 8: Base Rate Pre Below median - Base Rate Pre Above median lev                           0.0484
## Test 12: Base Rate Pre Below median - Base Rate Pre Above median lev                          0.0458
## Test 12: Base Rate Pre Below median - Base Rate Pre Above median perc                         0.0878
## Test 8: Religiosity: Does not attend  - Religiosity: Attends  lev                             0.0910
## Test 8: Religiosity: Does not attend  - Religiosity: Attends  perc                            0.1609
## Test 8: Location: Non Rural  - Location: Mostly Rural  lev                                    0.0386
## Test 8: Location: Non Rural  - Location: Mostly Rural  perc                                   0.0668
## Test 8: Base Acc Score Pre Below median - Base Acc Score Pre Above median lev                 0.0325
## Test 12: Hrs/day on social media: Below median - Hrs/day on social media: Above median lev   -0.0314
## Test 12: Hrs/day on social media: Below median - Hrs/day on social media: Above median perc  -0.0607
## Test 7: Misinfo Acc Score Pre Below median - Misinfo Acc Score Pre Above median perc          0.0545
## Test 7: Prop. of content shared below 40%  - Prop. of content shared above 40%  perc          0.0575
## Test 7: Non-Married  - Married  perc                                                          0.0561
## Test 12: Prop. of content shared below 40%  - Prop. of content shared above 40%  perc         0.0601
## Test 8: Misinfo Rate Pre Below median - Misinfo Rate Pre Above median perc                   -0.0523
## Test 12: Misinfo Acc Score Pre Below median - Misinfo Acc Score Pre Above median lev          0.0284
## Test 8: Hrs/day on social media: Below median - Hrs/day on social media: Above median lev     0.0292
## Test 12: Prop. of content shared below 40%  - Prop. of content shared above 40%  lev          0.0293
## Test 8: Misinfo Acc Score Pre Below median - Misinfo Acc Score Pre Above median lev           0.0280
## Test 7: Religion: Non-Christian  - Religion: Christian  lev                                   0.0543
## Test 12: Religiosity: Does not attend  - Religiosity: Attends  lev                            0.0574
## Test 7: Fail AC - Pass AC lev                                                                -0.0231
## Test 7: Base Acc Score Pre Below median - Base Acc Score Pre Above median lev                 0.0224
## Test 8: Prop. of content shared below 40%  - Prop. of content shared above 40%  lev           0.0232
## Test 7: Non-Married  - Married  lev                                                           0.0231
## Test 12: Location: Non Rural  - Location: Mostly Rural  lev                                  -0.0225
## Test 12: Non-Married  - Married  lev                                                          0.0199
## Test 8: Misinfo Rate Pre Below median - Misinfo Rate Pre Above median lev                     0.0157
## Test 8: Fail AC - Pass AC lev                                                                -0.0168
## Test 8: Religion: Non-Christian  - Religion: Christian  lev                                   0.0337
## Test 7: Location: Non Rural  - Location: Mostly Rural  lev                                    0.0160
## Test 8: Age: Below median - Age: Above median lev                                            -0.0116
## Test 12: Base Acc Score Pre Below median - Base Acc Score Pre Above median lev               -0.0101
## Test 12: Religion: Non-Christian  - Religion: Christian  lev                                  0.0205
## Test 7: Age: Below median - Age: Above median lev                                            -0.0091
## Test 12: Gender: Woman - Gender: Man lev                                                     -0.0090
## Test 12: Education: More than High School - Education: High School or Less lev                0.0076
## Test 12: Fail AC - Pass AC lev                                                               -0.0063
## Test 7: Education: More than High School - Education: High School or Less lev                 0.0065
## Test 7: Gender: Woman - Gender: Man lev                                                      -0.0049
## Test 8: Gender: Woman - Gender: Man lev                                                       0.0041
## Test 8: Non-Married  - Married  lev                                                           0.0032
## Test 12: Age: Below median - Age: Above median lev                                            0.0025
## Test 7: Hrs/day on social media: Below median - Hrs/day on social media: Above median lev    -0.0022
## Test 8: Education: More than High School - Education: High School or Less lev                -0.0011
## Test 12: Misinfo Acc Score Pre Below median - Misinfo Acc Score Pre Above median perc         0.0535
## Test 12: Religiosity: Does not attend  - Religiosity: Attends  perc                           0.1047
## Test 8: Hrs/day on social media: Below median - Hrs/day on social media: Above median perc    0.0427
## Test 7: Religion: Non-Christian  - Religion: Christian  perc                                  0.0919
## Test 12: Location: Non Rural  - Location: Mostly Rural  perc                                 -0.0433
## Test 12: Non-Married  - Married  perc                                                         0.0399
## Test 7: Fail AC - Pass AC perc                                                               -0.0239
## Test 8: Religion: Non-Christian  - Religion: Christian  perc                                  0.0514
## Test 7: Location: Non Rural  - Location: Mostly Rural  perc                                   0.0235
## Test 12: Base Acc Score Pre Below median - Base Acc Score Pre Above median perc              -0.0202
## Test 7: Hrs/day on social media: Below median - Hrs/day on social media: Above median perc   -0.0179
## Test 7: Gender: Woman - Gender: Man perc                                                     -0.0180
## Test 12: Religion: Non-Christian  - Religion: Christian  perc                                 0.0405
## Test 12: Gender: Woman - Gender: Man perc                                                    -0.0180
## Test 8: Base Acc Score Pre Below median - Base Acc Score Pre Above median perc                0.0155
## Test 8: Non-Married  - Married  perc                                                          0.0162
## Test 8: Fail AC - Pass AC perc                                                               -0.0126
## Test 12: Education: More than High School - Education: High School or Less perc               0.0143
## Test 8: Base Rate Pre Below median - Base Rate Pre Above median perc                          0.0105
## Test 12: Fail AC - Pass AC perc                                                              -0.0112
## Test 7: Education: More than High School - Education: High School or Less perc                0.0088
## Test 12: Age: Below median - Age: Above median perc                                           0.0054
## Test 8: Education: More than High School - Education: High School or Less perc               -0.0055
## Test 7: Base Acc Score Pre Below median - Base Acc Score Pre Above median perc               -0.0047
## Test 8: Age: Below median - Age: Above median perc                                           -0.0044
## Test 8: Prop. of content shared below 40%  - Prop. of content shared above 40%  perc         -0.0026
## Test 7: Age: Below median - Age: Above median perc                                            0.0011
## Test 8: Misinfo Acc Score Pre Below median - Misinfo Acc Score Pre Above median perc          0.0010
## Test 8: Gender: Woman - Gender: Man perc                                                     -0.0001
##                                                                                             std.err
## Test 7: Base Rate Pre Below median - Base Rate Pre Above median lev                          0.0235
## Test 7: Misinfo Rate Pre Below median - Misinfo Rate Pre Above median lev                    0.0236
## Test 12: Misinfo Rate Pre Below median - Misinfo Rate Pre Above median lev                   0.0239
## Test 12: Misinfo Rate Pre Below median - Misinfo Rate Pre Above median perc                  0.0484
## Test 7: Religiosity: Does not attend  - Religiosity: Attends  lev                            0.0568
## Test 7: Religiosity: Does not attend  - Religiosity: Attends  perc                           0.1004
## Test 7: Misinfo Acc Score Pre Below median - Misinfo Acc Score Pre Above median lev          0.0244
## Test 7: Base Rate Pre Below median - Base Rate Pre Above median perc                         0.0445
## Test 7: Misinfo Rate Pre Below median - Misinfo Rate Pre Above median perc                   0.0445
## Test 7: Prop. of content shared below 40%  - Prop. of content shared above 40%  lev          0.0254
## Test 8: Base Rate Pre Below median - Base Rate Pre Above median lev                          0.0238
## Test 12: Base Rate Pre Below median - Base Rate Pre Above median lev                         0.0241
## Test 12: Base Rate Pre Below median - Base Rate Pre Above median perc                        0.0484
## Test 8: Religiosity: Does not attend  - Religiosity: Attends  lev                            0.0595
## Test 8: Religiosity: Does not attend  - Religiosity: Attends  perc                           0.1047
## Test 8: Location: Non Rural  - Location: Mostly Rural  lev                                   0.0276
## Test 8: Location: Non Rural  - Location: Mostly Rural  perc                                  0.0487
## Test 8: Base Acc Score Pre Below median - Base Acc Score Pre Above median lev                0.0250
## Test 12: Hrs/day on social media: Below median - Hrs/day on social media: Above median lev   0.0252
## Test 12: Hrs/day on social media: Below median - Hrs/day on social media: Above median perc  0.0484
## Test 7: Misinfo Acc Score Pre Below median - Misinfo Acc Score Pre Above median perc         0.0446
## Test 7: Prop. of content shared below 40%  - Prop. of content shared above 40%  perc         0.0475
## Test 7: Non-Married  - Married  perc                                                         0.0477
## Test 12: Prop. of content shared below 40%  - Prop. of content shared above 40%  perc        0.0519
## Test 8: Misinfo Rate Pre Below median - Misinfo Rate Pre Above median perc                   0.0452
## Test 12: Misinfo Acc Score Pre Below median - Misinfo Acc Score Pre Above median lev         0.0247
## Test 8: Hrs/day on social media: Below median - Hrs/day on social media: Above median lev    0.0255
## Test 12: Prop. of content shared below 40%  - Prop. of content shared above 40%  lev         0.0256
## Test 8: Misinfo Acc Score Pre Below median - Misinfo Acc Score Pre Above median lev          0.0248
## Test 7: Religion: Non-Christian  - Religion: Christian  lev                                  0.0543
## Test 12: Religiosity: Does not attend  - Religiosity: Attends  lev                           0.0579
## Test 7: Fail AC - Pass AC lev                                                                0.0250
## Test 7: Base Acc Score Pre Below median - Base Acc Score Pre Above median lev                0.0244
## Test 8: Prop. of content shared below 40%  - Prop. of content shared above 40%  lev          0.0259
## Test 7: Non-Married  - Married  lev                                                          0.0263
## Test 12: Location: Non Rural  - Location: Mostly Rural  lev                                  0.0270
## Test 12: Non-Married  - Married  lev                                                         0.0269
## Test 8: Misinfo Rate Pre Below median - Misinfo Rate Pre Above median lev                    0.0237
## Test 8: Fail AC - Pass AC lev                                                                0.0255
## Test 8: Religion: Non-Christian  - Religion: Christian  lev                                  0.0551
## Test 7: Location: Non Rural  - Location: Mostly Rural  lev                                   0.0270
## Test 8: Age: Below median - Age: Above median lev                                            0.0254
## Test 12: Base Acc Score Pre Below median - Base Acc Score Pre Above median lev               0.0248
## Test 12: Religion: Non-Christian  - Religion: Christian  lev                                 0.0559
## Test 7: Age: Below median - Age: Above median lev                                            0.0249
## Test 12: Gender: Woman - Gender: Man lev                                                     0.0264
## Test 12: Education: More than High School - Education: High School or Less lev               0.0288
## Test 12: Fail AC - Pass AC lev                                                               0.0252
## Test 7: Education: More than High School - Education: High School or Less lev                0.0291
## Test 7: Gender: Woman - Gender: Man lev                                                      0.0263
## Test 8: Gender: Woman - Gender: Man lev                                                      0.0269
## Test 8: Non-Married  - Married  lev                                                          0.0274
## Test 12: Age: Below median - Age: Above median lev                                           0.0252
## Test 7: Hrs/day on social media: Below median - Hrs/day on social media: Above median lev    0.0250
## Test 8: Education: More than High School - Education: High School or Less lev                0.0297
## Test 12: Misinfo Acc Score Pre Below median - Misinfo Acc Score Pre Above median perc        0.0483
## Test 12: Religiosity: Does not attend  - Religiosity: Attends  perc                          0.1087
## Test 8: Hrs/day on social media: Below median - Hrs/day on social media: Above median perc   0.0455
## Test 7: Religion: Non-Christian  - Religion: Christian  perc                                 0.0990
## Test 12: Location: Non Rural  - Location: Mostly Rural  perc                                 0.0518
## Test 12: Non-Married  - Married  perc                                                        0.0524
## Test 7: Fail AC - Pass AC perc                                                               0.0449
## Test 8: Religion: Non-Christian  - Religion: Christian  perc                                 0.1013
## Test 7: Location: Non Rural  - Location: Mostly Rural  perc                                  0.0476
## Test 12: Base Acc Score Pre Below median - Base Acc Score Pre Above median perc              0.0485
## Test 7: Hrs/day on social media: Below median - Hrs/day on social media: Above median perc   0.0447
## Test 7: Gender: Woman - Gender: Man perc                                                     0.0472
## Test 12: Religion: Non-Christian  - Religion: Christian  perc                                0.1067
## Test 12: Gender: Woman - Gender: Man perc                                                    0.0513
## Test 8: Base Acc Score Pre Below median - Base Acc Score Pre Above median perc               0.0454
## Test 8: Non-Married  - Married  perc                                                         0.0490
## Test 8: Fail AC - Pass AC perc                                                               0.0456
## Test 12: Education: More than High School - Education: High School or Less perc              0.0543
## Test 8: Base Rate Pre Below median - Base Rate Pre Above median perc                         0.0452
## Test 12: Fail AC - Pass AC perc                                                              0.0486
## Test 7: Education: More than High School - Education: High School or Less perc               0.0509
## Test 12: Age: Below median - Age: Above median perc                                          0.0484
## Test 8: Education: More than High School - Education: High School or Less perc               0.0517
## Test 7: Base Acc Score Pre Below median - Base Acc Score Pre Above median perc               0.0446
## Test 8: Age: Below median - Age: Above median perc                                           0.0455
## Test 8: Prop. of content shared below 40%  - Prop. of content shared above 40%  perc         0.0487
## Test 7: Age: Below median - Age: Above median perc                                           0.0447
## Test 8: Misinfo Acc Score Pre Below median - Misinfo Acc Score Pre Above median perc         0.0453
## Test 8: Gender: Woman - Gender: Man perc                                                     0.0483
##                                                                                              CI_low
## Test 7: Base Rate Pre Below median - Base Rate Pre Above median lev                          0.0480
## Test 7: Misinfo Rate Pre Below median - Misinfo Rate Pre Above median lev                    0.0445
## Test 12: Misinfo Rate Pre Below median - Misinfo Rate Pre Above median lev                   0.0282
## Test 12: Misinfo Rate Pre Below median - Misinfo Rate Pre Above median perc                  0.0535
## Test 7: Religiosity: Does not attend  - Religiosity: Attends  lev                            0.0371
## Test 7: Religiosity: Does not attend  - Religiosity: Attends  perc                           0.0689
## Test 7: Misinfo Acc Score Pre Below median - Misinfo Acc Score Pre Above median lev          0.0086
## Test 7: Base Rate Pre Below median - Base Rate Pre Above median perc                         0.0110
## Test 7: Misinfo Rate Pre Below median - Misinfo Rate Pre Above median perc                   0.0089
## Test 7: Prop. of content shared below 40%  - Prop. of content shared above 40%  lev          0.0026
## Test 8: Base Rate Pre Below median - Base Rate Pre Above median lev                          0.0018
## Test 12: Base Rate Pre Below median - Base Rate Pre Above median lev                        -0.0015
## Test 12: Base Rate Pre Below median - Base Rate Pre Above median perc                       -0.0071
## Test 8: Religiosity: Does not attend  - Religiosity: Attends  lev                           -0.0257
## Test 8: Religiosity: Does not attend  - Religiosity: Attends  perc                          -0.0443
## Test 8: Location: Non Rural  - Location: Mostly Rural  lev                                  -0.0156
## Test 8: Location: Non Rural  - Location: Mostly Rural  perc                                 -0.0286
## Test 8: Base Acc Score Pre Below median - Base Acc Score Pre Above median lev               -0.0165
## Test 12: Hrs/day on social media: Below median - Hrs/day on social media: Above median lev  -0.0808
## Test 12: Hrs/day on social media: Below median - Hrs/day on social media: Above median perc -0.1556
## Test 7: Misinfo Acc Score Pre Below median - Misinfo Acc Score Pre Above median perc        -0.0328
## Test 7: Prop. of content shared below 40%  - Prop. of content shared above 40%  perc        -0.0355
## Test 7: Non-Married  - Married  perc                                                        -0.0374
## Test 12: Prop. of content shared below 40%  - Prop. of content shared above 40%  perc       -0.0416
## Test 8: Misinfo Rate Pre Below median - Misinfo Rate Pre Above median perc                  -0.1409
## Test 12: Misinfo Acc Score Pre Below median - Misinfo Acc Score Pre Above median lev        -0.0201
## Test 8: Hrs/day on social media: Below median - Hrs/day on social media: Above median lev   -0.0208
## Test 12: Prop. of content shared below 40%  - Prop. of content shared above 40%  lev        -0.0209
## Test 8: Misinfo Acc Score Pre Below median - Misinfo Acc Score Pre Above median lev         -0.0206
## Test 7: Religion: Non-Christian  - Religion: Christian  lev                                 -0.0522
## Test 12: Religiosity: Does not attend  - Religiosity: Attends  lev                          -0.0560
## Test 7: Fail AC - Pass AC lev                                                               -0.0721
## Test 7: Base Acc Score Pre Below median - Base Acc Score Pre Above median lev               -0.0255
## Test 8: Prop. of content shared below 40%  - Prop. of content shared above 40%  lev         -0.0275
## Test 7: Non-Married  - Married  lev                                                         -0.0285
## Test 12: Location: Non Rural  - Location: Mostly Rural  lev                                 -0.0755
## Test 12: Non-Married  - Married  lev                                                        -0.0328
## Test 8: Misinfo Rate Pre Below median - Misinfo Rate Pre Above median lev                   -0.0308
## Test 8: Fail AC - Pass AC lev                                                               -0.0667
## Test 8: Religion: Non-Christian  - Religion: Christian  lev                                 -0.0743
## Test 7: Location: Non Rural  - Location: Mostly Rural  lev                                  -0.0369
## Test 8: Age: Below median - Age: Above median lev                                           -0.0614
## Test 12: Base Acc Score Pre Below median - Base Acc Score Pre Above median lev              -0.0587
## Test 12: Religion: Non-Christian  - Religion: Christian  lev                                -0.0890
## Test 7: Age: Below median - Age: Above median lev                                           -0.0580
## Test 12: Gender: Woman - Gender: Man lev                                                    -0.0608
## Test 12: Education: More than High School - Education: High School or Less lev              -0.0488
## Test 12: Fail AC - Pass AC lev                                                              -0.0557
## Test 7: Education: More than High School - Education: High School or Less lev               -0.0505
## Test 7: Gender: Woman - Gender: Man lev                                                     -0.0565
## Test 8: Gender: Woman - Gender: Man lev                                                     -0.0485
## Test 8: Non-Married  - Married  lev                                                         -0.0505
## Test 12: Age: Below median - Age: Above median lev                                          -0.0469
## Test 7: Hrs/day on social media: Below median - Hrs/day on social media: Above median lev   -0.0511
## Test 8: Education: More than High School - Education: High School or Less lev               -0.0593
## Test 12: Misinfo Acc Score Pre Below median - Misinfo Acc Score Pre Above median perc       -0.0413
## Test 12: Religiosity: Does not attend  - Religiosity: Attends  perc                         -0.1085
## Test 8: Hrs/day on social media: Below median - Hrs/day on social media: Above median perc  -0.0465
## Test 7: Religion: Non-Christian  - Religion: Christian  perc                                -0.1021
## Test 12: Location: Non Rural  - Location: Mostly Rural  perc                                -0.1448
## Test 12: Non-Married  - Married  perc                                                       -0.0627
## Test 7: Fail AC - Pass AC perc                                                              -0.1118
## Test 8: Religion: Non-Christian  - Religion: Christian  perc                                -0.1471
## Test 7: Location: Non Rural  - Location: Mostly Rural  perc                                 -0.0697
## Test 12: Base Acc Score Pre Below median - Base Acc Score Pre Above median perc             -0.1152
## Test 7: Hrs/day on social media: Below median - Hrs/day on social media: Above median perc  -0.1055
## Test 7: Gender: Woman - Gender: Man perc                                                    -0.1106
## Test 12: Religion: Non-Christian  - Religion: Christian  perc                               -0.1685
## Test 12: Gender: Woman - Gender: Man perc                                                   -0.1185
## Test 8: Base Acc Score Pre Below median - Base Acc Score Pre Above median perc              -0.0735
## Test 8: Non-Married  - Married  perc                                                        -0.0798
## Test 8: Fail AC - Pass AC perc                                                              -0.1019
## Test 12: Education: More than High School - Education: High School or Less perc             -0.0921
## Test 8: Base Rate Pre Below median - Base Rate Pre Above median perc                        -0.0781
## Test 12: Fail AC - Pass AC perc                                                             -0.1065
## Test 7: Education: More than High School - Education: High School or Less perc              -0.0909
## Test 12: Age: Below median - Age: Above median perc                                         -0.0895
## Test 8: Education: More than High School - Education: High School or Less perc              -0.1068
## Test 7: Base Acc Score Pre Below median - Base Acc Score Pre Above median perc              -0.0921
## Test 8: Age: Below median - Age: Above median perc                                          -0.0935
## Test 8: Prop. of content shared below 40%  - Prop. of content shared above 40%  perc        -0.0980
## Test 7: Age: Below median - Age: Above median perc                                          -0.0865
## Test 8: Misinfo Acc Score Pre Below median - Misinfo Acc Score Pre Above median perc        -0.0879
## Test 8: Gender: Woman - Gender: Man perc                                                    -0.0947
##                                                                                             CI_upp
## Test 7: Base Rate Pre Below median - Base Rate Pre Above median lev                         0.1402
## Test 7: Misinfo Rate Pre Below median - Misinfo Rate Pre Above median lev                   0.1371
## Test 12: Misinfo Rate Pre Below median - Misinfo Rate Pre Above median lev                  0.1220
## Test 12: Misinfo Rate Pre Below median - Misinfo Rate Pre Above median perc                 0.2431
## Test 7: Religiosity: Does not attend  - Religiosity: Attends  lev                           0.2598
## Test 7: Religiosity: Does not attend  - Religiosity: Attends  perc                          0.4623
## Test 7: Misinfo Acc Score Pre Below median - Misinfo Acc Score Pre Above median lev         0.1041
## Test 7: Base Rate Pre Below median - Base Rate Pre Above median perc                        0.1855
## Test 7: Misinfo Rate Pre Below median - Misinfo Rate Pre Above median perc                  0.1832
## Test 7: Prop. of content shared below 40%  - Prop. of content shared above 40%  lev         0.1023
## Test 8: Base Rate Pre Below median - Base Rate Pre Above median lev                         0.0950
## Test 12: Base Rate Pre Below median - Base Rate Pre Above median lev                        0.0930
## Test 12: Base Rate Pre Below median - Base Rate Pre Above median perc                       0.1826
## Test 8: Religiosity: Does not attend  - Religiosity: Attends  lev                           0.2077
## Test 8: Religiosity: Does not attend  - Religiosity: Attends  perc                          0.3662
## Test 8: Location: Non Rural  - Location: Mostly Rural  lev                                  0.0927
## Test 8: Location: Non Rural  - Location: Mostly Rural  perc                                 0.1623
## Test 8: Base Acc Score Pre Below median - Base Acc Score Pre Above median lev               0.0815
## Test 12: Hrs/day on social media: Below median - Hrs/day on social media: Above median lev  0.0180
## Test 12: Hrs/day on social media: Below median - Hrs/day on social media: Above median perc 0.0343
## Test 7: Misinfo Acc Score Pre Below median - Misinfo Acc Score Pre Above median perc        0.1418
## Test 7: Prop. of content shared below 40%  - Prop. of content shared above 40%  perc        0.1505
## Test 7: Non-Married  - Married  perc                                                        0.1496
## Test 12: Prop. of content shared below 40%  - Prop. of content shared above 40%  perc       0.1618
## Test 8: Misinfo Rate Pre Below median - Misinfo Rate Pre Above median perc                  0.0363
## Test 12: Misinfo Acc Score Pre Below median - Misinfo Acc Score Pre Above median lev        0.0768
## Test 8: Hrs/day on social media: Below median - Hrs/day on social media: Above median lev   0.0792
## Test 12: Prop. of content shared below 40%  - Prop. of content shared above 40%  lev        0.0794
## Test 8: Misinfo Acc Score Pre Below median - Misinfo Acc Score Pre Above median lev         0.0766
## Test 7: Religion: Non-Christian  - Religion: Christian  lev                                 0.1607
## Test 12: Religiosity: Does not attend  - Religiosity: Attends  lev                          0.1709
## Test 7: Fail AC - Pass AC lev                                                               0.0259
## Test 7: Base Acc Score Pre Below median - Base Acc Score Pre Above median lev               0.0703
## Test 8: Prop. of content shared below 40%  - Prop. of content shared above 40%  lev         0.0739
## Test 7: Non-Married  - Married  lev                                                         0.0747
## Test 12: Location: Non Rural  - Location: Mostly Rural  lev                                 0.0304
## Test 12: Non-Married  - Married  lev                                                        0.0726
## Test 8: Misinfo Rate Pre Below median - Misinfo Rate Pre Above median lev                   0.0622
## Test 8: Fail AC - Pass AC lev                                                               0.0331
## Test 8: Religion: Non-Christian  - Religion: Christian  lev                                 0.1418
## Test 7: Location: Non Rural  - Location: Mostly Rural  lev                                  0.0690
## Test 8: Age: Below median - Age: Above median lev                                           0.0383
## Test 12: Base Acc Score Pre Below median - Base Acc Score Pre Above median lev              0.0384
## Test 12: Religion: Non-Christian  - Religion: Christian  lev                                0.1301
## Test 7: Age: Below median - Age: Above median lev                                           0.0398
## Test 12: Gender: Woman - Gender: Man lev                                                    0.0427
## Test 12: Education: More than High School - Education: High School or Less lev              0.0640
## Test 12: Fail AC - Pass AC lev                                                              0.0431
## Test 7: Education: More than High School - Education: High School or Less lev               0.0634
## Test 7: Gender: Woman - Gender: Man lev                                                     0.0467
## Test 8: Gender: Woman - Gender: Man lev                                                     0.0568
## Test 8: Non-Married  - Married  lev                                                         0.0568
## Test 12: Age: Below median - Age: Above median lev                                          0.0518
## Test 7: Hrs/day on social media: Below median - Hrs/day on social media: Above median lev   0.0468
## Test 8: Education: More than High School - Education: High School or Less lev               0.0570
## Test 12: Misinfo Acc Score Pre Below median - Misinfo Acc Score Pre Above median perc       0.1483
## Test 12: Religiosity: Does not attend  - Religiosity: Attends  perc                         0.3178
## Test 8: Hrs/day on social media: Below median - Hrs/day on social media: Above median perc  0.1319
## Test 7: Religion: Non-Christian  - Religion: Christian  perc                                0.2859
## Test 12: Location: Non Rural  - Location: Mostly Rural  perc                                0.0581
## Test 12: Non-Married  - Married  perc                                                       0.1425
## Test 7: Fail AC - Pass AC perc                                                              0.0641
## Test 8: Religion: Non-Christian  - Religion: Christian  perc                                0.2500
## Test 7: Location: Non Rural  - Location: Mostly Rural  perc                                 0.1168
## Test 12: Base Acc Score Pre Below median - Base Acc Score Pre Above median perc             0.0748
## Test 7: Hrs/day on social media: Below median - Hrs/day on social media: Above median perc  0.0697
## Test 7: Gender: Woman - Gender: Man perc                                                    0.0746
## Test 12: Religion: Non-Christian  - Religion: Christian  perc                               0.2496
## Test 12: Gender: Woman - Gender: Man perc                                                   0.0826
## Test 8: Base Acc Score Pre Below median - Base Acc Score Pre Above median perc              0.1046
## Test 8: Non-Married  - Married  perc                                                        0.1123
## Test 8: Fail AC - Pass AC perc                                                              0.0766
## Test 12: Education: More than High School - Education: High School or Less perc             0.1208
## Test 8: Base Rate Pre Below median - Base Rate Pre Above median perc                        0.0990
## Test 12: Fail AC - Pass AC perc                                                             0.0840
## Test 7: Education: More than High School - Education: High School or Less perc              0.1085
## Test 12: Age: Below median - Age: Above median perc                                         0.1003
## Test 8: Education: More than High School - Education: High School or Less perc              0.0958
## Test 7: Base Acc Score Pre Below median - Base Acc Score Pre Above median perc              0.0827
## Test 8: Age: Below median - Age: Above median perc                                          0.0848
## Test 8: Prop. of content shared below 40%  - Prop. of content shared above 40%  perc        0.0928
## Test 7: Age: Below median - Age: Above median perc                                          0.0886
## Test 8: Misinfo Acc Score Pre Below median - Misinfo Acc Score Pre Above median perc        0.0898
## Test 8: Gender: Woman - Gender: Man perc                                                    0.0946
##                                                                                              p_val
## Test 7: Base Rate Pre Below median - Base Rate Pre Above median lev                         0.0001
## Test 7: Misinfo Rate Pre Below median - Misinfo Rate Pre Above median lev                   0.0001
## Test 12: Misinfo Rate Pre Below median - Misinfo Rate Pre Above median lev                  0.0017
## Test 12: Misinfo Rate Pre Below median - Misinfo Rate Pre Above median perc                 0.0022
## Test 7: Religiosity: Does not attend  - Religiosity: Attends  lev                           0.0090
## Test 7: Religiosity: Does not attend  - Religiosity: Attends  perc                          0.0081
## Test 7: Misinfo Acc Score Pre Below median - Misinfo Acc Score Pre Above median lev         0.0208
## Test 7: Base Rate Pre Below median - Base Rate Pre Above median perc                        0.0273
## Test 7: Misinfo Rate Pre Below median - Misinfo Rate Pre Above median perc                  0.0308
## Test 7: Prop. of content shared below 40%  - Prop. of content shared above 40%  lev         0.0391
## Test 8: Base Rate Pre Below median - Base Rate Pre Above median lev                         0.0419
## Test 12: Base Rate Pre Below median - Base Rate Pre Above median lev                        0.0578
## Test 12: Base Rate Pre Below median - Base Rate Pre Above median perc                       0.0697
## Test 8: Religiosity: Does not attend  - Religiosity: Attends  lev                           0.1265
## Test 8: Religiosity: Does not attend  - Religiosity: Attends  perc                          0.1243
## Test 8: Location: Non Rural  - Location: Mostly Rural  lev                                  0.1628
## Test 8: Location: Non Rural  - Location: Mostly Rural  perc                                 0.1698
## Test 8: Base Acc Score Pre Below median - Base Acc Score Pre Above median lev               0.1933
## Test 12: Hrs/day on social media: Below median - Hrs/day on social media: Above median lev  0.2134
## Test 12: Hrs/day on social media: Below median - Hrs/day on social media: Above median perc 0.2104
## Test 7: Misinfo Acc Score Pre Below median - Misinfo Acc Score Pre Above median perc        0.2213
## Test 7: Prop. of content shared below 40%  - Prop. of content shared above 40%  perc        0.2256
## Test 7: Non-Married  - Married  perc                                                        0.2396
## Test 12: Prop. of content shared below 40%  - Prop. of content shared above 40%  perc       0.2469
## Test 8: Misinfo Rate Pre Below median - Misinfo Rate Pre Above median perc                  0.2470
## Test 12: Misinfo Acc Score Pre Below median - Misinfo Acc Score Pre Above median lev        0.2509
## Test 8: Hrs/day on social media: Below median - Hrs/day on social media: Above median lev   0.2527
## Test 12: Prop. of content shared below 40%  - Prop. of content shared above 40%  lev        0.2528
## Test 8: Misinfo Acc Score Pre Below median - Misinfo Acc Score Pre Above median lev         0.2588
## Test 7: Religion: Non-Christian  - Religion: Christian  lev                                 0.3177
## Test 12: Religiosity: Does not attend  - Religiosity: Attends  lev                          0.3208
## Test 7: Fail AC - Pass AC lev                                                               0.3560
## Test 7: Base Acc Score Pre Below median - Base Acc Score Pre Above median lev               0.3601
## Test 8: Prop. of content shared below 40%  - Prop. of content shared above 40%  lev         0.3695
## Test 7: Non-Married  - Married  lev                                                         0.3807
## Test 12: Location: Non Rural  - Location: Mostly Rural  lev                                 0.4045
## Test 12: Non-Married  - Married  lev                                                        0.4593
## Test 8: Misinfo Rate Pre Below median - Misinfo Rate Pre Above median lev                   0.5079
## Test 8: Fail AC - Pass AC lev                                                               0.5097
## Test 8: Religion: Non-Christian  - Religion: Christian  lev                                 0.5408
## Test 7: Location: Non Rural  - Location: Mostly Rural  lev                                  0.5533
## Test 8: Age: Below median - Age: Above median lev                                           0.6496
## Test 12: Base Acc Score Pre Below median - Base Acc Score Pre Above median lev              0.6827
## Test 12: Religion: Non-Christian  - Religion: Christian  lev                                0.7133
## Test 7: Age: Below median - Age: Above median lev                                           0.7154
## Test 12: Gender: Woman - Gender: Man lev                                                    0.7324
## Test 12: Education: More than High School - Education: High School or Less lev              0.7920
## Test 12: Fail AC - Pass AC lev                                                              0.8030
## Test 7: Education: More than High School - Education: High School or Less lev               0.8241
## Test 7: Gender: Woman - Gender: Man lev                                                     0.8523
## Test 8: Gender: Woman - Gender: Man lev                                                     0.8778
## Test 8: Non-Married  - Married  lev                                                         0.9077
## Test 12: Age: Below median - Age: Above median lev                                          0.9220
## Test 7: Hrs/day on social media: Below median - Hrs/day on social media: Above median lev   0.9305
## Test 8: Education: More than High School - Education: High School or Less lev               0.9695
## Test 12: Misinfo Acc Score Pre Below median - Misinfo Acc Score Pre Above median perc       0.2684
## Test 12: Religiosity: Does not attend  - Religiosity: Attends  perc                         0.3358
## Test 8: Hrs/day on social media: Below median - Hrs/day on social media: Above median perc  0.3477
## Test 7: Religion: Non-Christian  - Religion: Christian  perc                                0.3529
## Test 12: Location: Non Rural  - Location: Mostly Rural  perc                                0.4027
## Test 12: Non-Married  - Married  perc                                                       0.4464
## Test 7: Fail AC - Pass AC perc                                                              0.5950
## Test 8: Religion: Non-Christian  - Religion: Christian  perc                                0.6116
## Test 7: Location: Non Rural  - Location: Mostly Rural  perc                                 0.6210
## Test 12: Base Acc Score Pre Below median - Base Acc Score Pre Above median perc             0.6770
## Test 7: Hrs/day on social media: Below median - Hrs/day on social media: Above median perc  0.6882
## Test 7: Gender: Woman - Gender: Man perc                                                    0.7030
## Test 12: Religion: Non-Christian  - Religion: Christian  perc                               0.7041
## Test 12: Gender: Woman - Gender: Man perc                                                   0.7264
## Test 8: Base Acc Score Pre Below median - Base Acc Score Pre Above median perc              0.7325
## Test 8: Non-Married  - Married  perc                                                        0.7405
## Test 8: Fail AC - Pass AC perc                                                              0.7813
## Test 12: Education: More than High School - Education: High School or Less perc             0.7918
## Test 8: Base Rate Pre Below median - Base Rate Pre Above median perc                        0.8170
## Test 12: Fail AC - Pass AC perc                                                             0.8175
## Test 7: Education: More than High School - Education: High School or Less perc              0.8624
## Test 12: Age: Below median - Age: Above median perc                                         0.9108
## Test 8: Education: More than High School - Education: High School or Less perc              0.9149
## Test 7: Base Acc Score Pre Below median - Base Acc Score Pre Above median perc              0.9168
## Test 8: Age: Below median - Age: Above median perc                                          0.9237
## Test 8: Prop. of content shared below 40%  - Prop. of content shared above 40%  perc        0.9579
## Test 7: Age: Below median - Age: Above median perc                                          0.9809
## Test 8: Misinfo Acc Score Pre Below median - Misinfo Acc Score Pre Above median perc        0.9825
## Test 8: Gender: Woman - Gender: Man perc                                                    0.9990
##                                                                                             p_val_holm
## Test 7: Base Rate Pre Below median - Base Rate Pre Above median lev                             0.0026
## Test 7: Misinfo Rate Pre Below median - Misinfo Rate Pre Above median lev                       0.0050
## Test 12: Misinfo Rate Pre Below median - Misinfo Rate Pre Above median lev                      0.0686
## Test 12: Misinfo Rate Pre Below median - Misinfo Rate Pre Above median perc                     0.0908
## Test 7: Religiosity: Does not attend  - Religiosity: Attends  lev                               0.3495
## Test 7: Religiosity: Does not attend  - Religiosity: Attends  perc                              0.3339
## Test 7: Misinfo Acc Score Pre Below median - Misinfo Acc Score Pre Above median lev             0.7918
## Test 7: Base Rate Pre Below median - Base Rate Pre Above median perc                            1.0000
## Test 7: Misinfo Rate Pre Below median - Misinfo Rate Pre Above median perc                      1.0000
## Test 7: Prop. of content shared below 40%  - Prop. of content shared above 40%  lev             1.0000
## Test 8: Base Rate Pre Below median - Base Rate Pre Above median lev                             1.0000
## Test 12: Base Rate Pre Below median - Base Rate Pre Above median lev                            1.0000
## Test 12: Base Rate Pre Below median - Base Rate Pre Above median perc                           1.0000
## Test 8: Religiosity: Does not attend  - Religiosity: Attends  lev                               1.0000
## Test 8: Religiosity: Does not attend  - Religiosity: Attends  perc                              1.0000
## Test 8: Location: Non Rural  - Location: Mostly Rural  lev                                      1.0000
## Test 8: Location: Non Rural  - Location: Mostly Rural  perc                                     1.0000
## Test 8: Base Acc Score Pre Below median - Base Acc Score Pre Above median lev                   1.0000
## Test 12: Hrs/day on social media: Below median - Hrs/day on social media: Above median lev      1.0000
## Test 12: Hrs/day on social media: Below median - Hrs/day on social media: Above median perc     1.0000
## Test 7: Misinfo Acc Score Pre Below median - Misinfo Acc Score Pre Above median perc            1.0000
## Test 7: Prop. of content shared below 40%  - Prop. of content shared above 40%  perc            1.0000
## Test 7: Non-Married  - Married  perc                                                            1.0000
## Test 12: Prop. of content shared below 40%  - Prop. of content shared above 40%  perc           1.0000
## Test 8: Misinfo Rate Pre Below median - Misinfo Rate Pre Above median perc                      1.0000
## Test 12: Misinfo Acc Score Pre Below median - Misinfo Acc Score Pre Above median lev            1.0000
## Test 8: Hrs/day on social media: Below median - Hrs/day on social media: Above median lev       1.0000
## Test 12: Prop. of content shared below 40%  - Prop. of content shared above 40%  lev            1.0000
## Test 8: Misinfo Acc Score Pre Below median - Misinfo Acc Score Pre Above median lev             1.0000
## Test 7: Religion: Non-Christian  - Religion: Christian  lev                                     1.0000
## Test 12: Religiosity: Does not attend  - Religiosity: Attends  lev                              1.0000
## Test 7: Fail AC - Pass AC lev                                                                   1.0000
## Test 7: Base Acc Score Pre Below median - Base Acc Score Pre Above median lev                   1.0000
## Test 8: Prop. of content shared below 40%  - Prop. of content shared above 40%  lev             1.0000
## Test 7: Non-Married  - Married  lev                                                             1.0000
## Test 12: Location: Non Rural  - Location: Mostly Rural  lev                                     1.0000
## Test 12: Non-Married  - Married  lev                                                            1.0000
## Test 8: Misinfo Rate Pre Below median - Misinfo Rate Pre Above median lev                       1.0000
## Test 8: Fail AC - Pass AC lev                                                                   1.0000
## Test 8: Religion: Non-Christian  - Religion: Christian  lev                                     1.0000
## Test 7: Location: Non Rural  - Location: Mostly Rural  lev                                      1.0000
## Test 8: Age: Below median - Age: Above median lev                                               1.0000
## Test 12: Base Acc Score Pre Below median - Base Acc Score Pre Above median lev                  1.0000
## Test 12: Religion: Non-Christian  - Religion: Christian  lev                                    1.0000
## Test 7: Age: Below median - Age: Above median lev                                               1.0000
## Test 12: Gender: Woman - Gender: Man lev                                                        1.0000
## Test 12: Education: More than High School - Education: High School or Less lev                  1.0000
## Test 12: Fail AC - Pass AC lev                                                                  1.0000
## Test 7: Education: More than High School - Education: High School or Less lev                   1.0000
## Test 7: Gender: Woman - Gender: Man lev                                                         1.0000
## Test 8: Gender: Woman - Gender: Man lev                                                         1.0000
## Test 8: Non-Married  - Married  lev                                                             1.0000
## Test 12: Age: Below median - Age: Above median lev                                              1.0000
## Test 7: Hrs/day on social media: Below median - Hrs/day on social media: Above median lev       1.0000
## Test 8: Education: More than High School - Education: High School or Less lev                   1.0000
## Test 12: Misinfo Acc Score Pre Below median - Misinfo Acc Score Pre Above median perc           1.0000
## Test 12: Religiosity: Does not attend  - Religiosity: Attends  perc                             1.0000
## Test 8: Hrs/day on social media: Below median - Hrs/day on social media: Above median perc      1.0000
## Test 7: Religion: Non-Christian  - Religion: Christian  perc                                    1.0000
## Test 12: Location: Non Rural  - Location: Mostly Rural  perc                                    1.0000
## Test 12: Non-Married  - Married  perc                                                           1.0000
## Test 7: Fail AC - Pass AC perc                                                                  1.0000
## Test 8: Religion: Non-Christian  - Religion: Christian  perc                                    1.0000
## Test 7: Location: Non Rural  - Location: Mostly Rural  perc                                     1.0000
## Test 12: Base Acc Score Pre Below median - Base Acc Score Pre Above median perc                 1.0000
## Test 7: Hrs/day on social media: Below median - Hrs/day on social media: Above median perc      1.0000
## Test 7: Gender: Woman - Gender: Man perc                                                        1.0000
## Test 12: Religion: Non-Christian  - Religion: Christian  perc                                   1.0000
## Test 12: Gender: Woman - Gender: Man perc                                                       1.0000
## Test 8: Base Acc Score Pre Below median - Base Acc Score Pre Above median perc                  1.0000
## Test 8: Non-Married  - Married  perc                                                            1.0000
## Test 8: Fail AC - Pass AC perc                                                                  1.0000
## Test 12: Education: More than High School - Education: High School or Less perc                 1.0000
## Test 8: Base Rate Pre Below median - Base Rate Pre Above median perc                            1.0000
## Test 12: Fail AC - Pass AC perc                                                                 1.0000
## Test 7: Education: More than High School - Education: High School or Less perc                  1.0000
## Test 12: Age: Below median - Age: Above median perc                                             1.0000
## Test 8: Education: More than High School - Education: High School or Less perc                  1.0000
## Test 7: Base Acc Score Pre Below median - Base Acc Score Pre Above median perc                  1.0000
## Test 8: Age: Below median - Age: Above median perc                                              1.0000
## Test 8: Prop. of content shared below 40%  - Prop. of content shared above 40%  perc            1.0000
## Test 7: Age: Below median - Age: Above median perc                                              1.0000
## Test 8: Misinfo Acc Score Pre Below median - Misinfo Acc Score Pre Above median perc            1.0000
## Test 8: Gender: Woman - Gender: Man perc                                                        1.0000
##                                                                                             p_val_RW
## Test 7: Base Rate Pre Below median - Base Rate Pre Above median lev                           0.0050
## Test 7: Misinfo Rate Pre Below median - Misinfo Rate Pre Above median lev                     0.0050
## Test 12: Misinfo Rate Pre Below median - Misinfo Rate Pre Above median lev                    0.0699
## Test 12: Misinfo Rate Pre Below median - Misinfo Rate Pre Above median perc                   0.1029
## Test 7: Religiosity: Does not attend  - Religiosity: Attends  lev                             0.2797
## Test 7: Religiosity: Does not attend  - Religiosity: Attends  perc                            0.2867
## Test 7: Misinfo Acc Score Pre Below median - Misinfo Acc Score Pre Above median lev           0.4855
## Test 7: Base Rate Pre Below median - Base Rate Pre Above median perc                          0.6374
## Test 7: Misinfo Rate Pre Below median - Misinfo Rate Pre Above median perc                    0.6713
## Test 7: Prop. of content shared below 40%  - Prop. of content shared above 40%  lev           0.6943
## Test 8: Base Rate Pre Below median - Base Rate Pre Above median lev                           0.7143
## Test 12: Base Rate Pre Below median - Base Rate Pre Above median lev                          0.8212
## Test 12: Base Rate Pre Below median - Base Rate Pre Above median perc                         0.9111
## Test 8: Religiosity: Does not attend  - Religiosity: Attends  lev                             0.9760
## Test 8: Religiosity: Does not attend  - Religiosity: Attends  perc                            0.9800
## Test 8: Location: Non Rural  - Location: Mostly Rural  lev                                    0.9880
## Test 8: Location: Non Rural  - Location: Mostly Rural  perc                                   0.9910
## Test 8: Base Acc Score Pre Below median - Base Acc Score Pre Above median lev                 0.9920
## Test 12: Hrs/day on social media: Below median - Hrs/day on social media: Above median lev    0.9940
## Test 12: Hrs/day on social media: Below median - Hrs/day on social media: Above median perc   0.9940
## Test 7: Misinfo Acc Score Pre Below median - Misinfo Acc Score Pre Above median perc          0.9950
## Test 7: Prop. of content shared below 40%  - Prop. of content shared above 40%  perc          0.9960
## Test 7: Non-Married  - Married  perc                                                          0.9970
## Test 12: Prop. of content shared below 40%  - Prop. of content shared above 40%  perc         0.9970
## Test 8: Misinfo Rate Pre Below median - Misinfo Rate Pre Above median perc                    0.9970
## Test 12: Misinfo Acc Score Pre Below median - Misinfo Acc Score Pre Above median lev          0.9990
## Test 8: Hrs/day on social media: Below median - Hrs/day on social media: Above median lev     0.9990
## Test 12: Prop. of content shared below 40%  - Prop. of content shared above 40%  lev          0.9990
## Test 8: Misinfo Acc Score Pre Below median - Misinfo Acc Score Pre Above median lev           0.9990
## Test 7: Religion: Non-Christian  - Religion: Christian  lev                                   1.0000
## Test 12: Religiosity: Does not attend  - Religiosity: Attends  lev                            1.0000
## Test 7: Fail AC - Pass AC lev                                                                 1.0000
## Test 7: Base Acc Score Pre Below median - Base Acc Score Pre Above median lev                 1.0000
## Test 8: Prop. of content shared below 40%  - Prop. of content shared above 40%  lev           1.0000
## Test 7: Non-Married  - Married  lev                                                           1.0000
## Test 12: Location: Non Rural  - Location: Mostly Rural  lev                                   1.0000
## Test 12: Non-Married  - Married  lev                                                          1.0000
## Test 8: Misinfo Rate Pre Below median - Misinfo Rate Pre Above median lev                     1.0000
## Test 8: Fail AC - Pass AC lev                                                                 1.0000
## Test 8: Religion: Non-Christian  - Religion: Christian  lev                                   1.0000
## Test 7: Location: Non Rural  - Location: Mostly Rural  lev                                    1.0000
## Test 8: Age: Below median - Age: Above median lev                                             1.0000
## Test 12: Base Acc Score Pre Below median - Base Acc Score Pre Above median lev                1.0000
## Test 12: Religion: Non-Christian  - Religion: Christian  lev                                  1.0000
## Test 7: Age: Below median - Age: Above median lev                                             1.0000
## Test 12: Gender: Woman - Gender: Man lev                                                      1.0000
## Test 12: Education: More than High School - Education: High School or Less lev                1.0000
## Test 12: Fail AC - Pass AC lev                                                                1.0000
## Test 7: Education: More than High School - Education: High School or Less lev                 1.0000
## Test 7: Gender: Woman - Gender: Man lev                                                       1.0000
## Test 8: Gender: Woman - Gender: Man lev                                                       1.0000
## Test 8: Non-Married  - Married  lev                                                           1.0000
## Test 12: Age: Below median - Age: Above median lev                                            1.0000
## Test 7: Hrs/day on social media: Below median - Hrs/day on social media: Above median lev     1.0000
## Test 8: Education: More than High School - Education: High School or Less lev                 1.0000
## Test 12: Misinfo Acc Score Pre Below median - Misinfo Acc Score Pre Above median perc         1.0000
## Test 12: Religiosity: Does not attend  - Religiosity: Attends  perc                           1.0000
## Test 8: Hrs/day on social media: Below median - Hrs/day on social media: Above median perc    1.0000
## Test 7: Religion: Non-Christian  - Religion: Christian  perc                                  1.0000
## Test 12: Location: Non Rural  - Location: Mostly Rural  perc                                  1.0000
## Test 12: Non-Married  - Married  perc                                                         1.0000
## Test 7: Fail AC - Pass AC perc                                                                1.0000
## Test 8: Religion: Non-Christian  - Religion: Christian  perc                                  1.0000
## Test 7: Location: Non Rural  - Location: Mostly Rural  perc                                   1.0000
## Test 12: Base Acc Score Pre Below median - Base Acc Score Pre Above median perc               1.0000
## Test 7: Hrs/day on social media: Below median - Hrs/day on social media: Above median perc    1.0000
## Test 7: Gender: Woman - Gender: Man perc                                                      1.0000
## Test 12: Religion: Non-Christian  - Religion: Christian  perc                                 1.0000
## Test 12: Gender: Woman - Gender: Man perc                                                     1.0000
## Test 8: Base Acc Score Pre Below median - Base Acc Score Pre Above median perc                1.0000
## Test 8: Non-Married  - Married  perc                                                          1.0000
## Test 8: Fail AC - Pass AC perc                                                                1.0000
## Test 12: Education: More than High School - Education: High School or Less perc               1.0000
## Test 8: Base Rate Pre Below median - Base Rate Pre Above median perc                          1.0000
## Test 12: Fail AC - Pass AC perc                                                               1.0000
## Test 7: Education: More than High School - Education: High School or Less perc                1.0000
## Test 12: Age: Below median - Age: Above median perc                                           1.0000
## Test 8: Education: More than High School - Education: High School or Less perc                1.0000
## Test 7: Base Acc Score Pre Below median - Base Acc Score Pre Above median perc                1.0000
## Test 8: Age: Below median - Age: Above median perc                                            1.0000
## Test 8: Prop. of content shared below 40%  - Prop. of content shared above 40%  perc          1.0000
## Test 7: Age: Below median - Age: Above median perc                                            1.0000
## Test 8: Misinfo Acc Score Pre Below median - Misinfo Acc Score Pre Above median perc          1.0000
## Test 8: Gender: Woman - Gender: Man perc                                                      1.0000

final_subgroup_results[grepl("Gender",rownames(final_subgroup_results)),]

##                                            obs  baseline       estimate
## Test 12: Gender: Woman - Gender: Man lev    NA        NA -0.00903276190
## Test 7: Gender: Woman - Gender: Man lev     NA        NA -0.00489847958
## Test 8: Gender: Woman - Gender: Man lev     NA        NA  0.00413428233
## Test 7: Gender: Woman - Gender: Man perc    NA        NA -0.01801181877
## Test 12: Gender: Woman - Gender: Man perc   NA        NA -0.01795297860
## Test 8: Gender: Woman - Gender: Man perc    NA        NA -0.00005884017
## Test 7: Gender: Man                       5679 0.6173789 -0.08579403300
## Test 8: Gender: Man                       5679 0.6173789 -0.08642861224
## Test 7: Gender: Woman                     3005 0.5876344 -0.09069251258
## Test 8: Gender: Woman                     3005 0.5876344 -0.08229432991
## Test 12: Gender: Woman                    3005 0.5053401 -0.00839818267
## Test 12: Gender: Man                      5679 0.5309503  0.00063457923
##                                              std.err      CI_low      CI_upp
## Test 12: Gender: Woman - Gender: Man lev  0.02641300 -0.06080223  0.04273671
## Test 7: Gender: Woman - Gender: Man lev   0.02631424 -0.05647439  0.04667743
## Test 8: Gender: Woman - Gender: Man lev   0.02687787 -0.04854634  0.05681490
## Test 7: Gender: Woman - Gender: Man perc  0.04724332 -0.11060873  0.07458509
## Test 12: Gender: Woman - Gender: Man perc 0.05130025 -0.11850146  0.08259551
## Test 8: Gender: Woman - Gender: Man perc  0.04828296 -0.09469343  0.09457575
## Test 7: Gender: Man                       0.01546580        -Inf -0.06043012
## Test 8: Gender: Man                       0.01575423        -Inf -0.06059167
## Test 7: Gender: Woman                     0.02128963        -Inf -0.05577752
## Test 8: Gender: Woman                     0.02177668        -Inf -0.04658057
## Test 12: Gender: Woman                    0.02127389 -0.05009500  0.02649099
## Test 12: Gender: Man                      0.01565465 -0.03004853  0.02630821
##                                                      p_val     p_val_holm
## Test 12: Gender: Woman - Gender: Man lev  0.73236465003352 1.000000000000
## Test 7: Gender: Woman - Gender: Man lev   0.85232464639313 1.000000000000
## Test 8: Gender: Woman - Gender: Man lev   0.87775375930282 1.000000000000
## Test 7: Gender: Woman - Gender: Man perc  0.70301299066059 1.000000000000
## Test 12: Gender: Woman - Gender: Man perc 0.72636953753838 1.000000000000
## Test 8: Gender: Woman - Gender: Man perc  0.99902765585119 1.000000000000
## Test 7: Gender: Man                       0.00000003108233 0.000002051434
## Test 8: Gender: Man                       0.00000004399492 0.000002815675
## Test 7: Gender: Woman                     0.00002096820616 0.001006473896
## Test 8: Gender: Woman                     0.00016005561189 0.006402224475
## Test 12: Gender: Woman                    0.69304002707181 1.000000000000
## Test 12: Gender: Man                      0.96766795284772 1.000000000000
##                                              p_val_RW
## Test 12: Gender: Woman - Gender: Man lev  1.000000000
## Test 7: Gender: Woman - Gender: Man lev   1.000000000
## Test 8: Gender: Woman - Gender: Man lev   1.000000000
## Test 7: Gender: Woman - Gender: Man perc  1.000000000
## Test 12: Gender: Woman - Gender: Man perc 1.000000000
## Test 8: Gender: Woman - Gender: Man perc  1.000000000
## Test 7: Gender: Man                       0.000999001
## Test 8: Gender: Man                       0.000999001
## Test 7: Gender: Woman                     0.001998002
## Test 8: Gender: Woman                     0.007992008
## Test 12: Gender: Woman                    1.000000000
## Test 12: Gender: Man                      1.000000000

Subgroup Analysis on base_rate_post for Inoculation against Misinformation

Kristine Koutout

2024-04-11