Loading R library

The purpose of this document is to clean the raw FB Misinformation Chatfuel data. The raw Chatfuel dataset was last pulled on 02/06/2024 and processed by the anonymization script. The anonymized version of dataset contains the variable analytic_id as the unique identifier for each respondent. We started with 159,484 observations and filtered respondents based on the following criteria:

  • Eligibility of respondents based the two versions of Phase 1 and three versions of Phase 2. Ineligible respondents were recorded as a missing value in the NewVersion variable and were dropped from the dataset. Respondents in the technical pilots were also dropped from the dataset.

  • Eligibility of respondents based on country. Respondents who were not from Nigeria, Kenya, Ghana, and South Africa were dropped.

  • We distinguished between attempted_repeaters and actual_repeaters. attempted_repeaters are respondents who tried to repeat the chatbot but were turned away automatically. actual repeaters are respondents who have a stamp in more than one treatment arm and they were dropped from the dataset. The actual repeaters were present at the beggining of Phase 1 before we implemented a chatbot patch that turned away respondents who tried to repeat the chatbot.

  • Abnormal recorded response in treatment arm across Phase 1 and Phase 2: We noticed two people in Phase 1 who had long baseline treatment. They started the chatbot during Phase 1 but only reached randomization phase during Phase 2 and were consequently re-coded to Phase 2. one person who is in Phase 2 of the chatbot but has a video intervention. This person was dropped from the dataset.

  • Abnormal recorded timestamps. We noticed about 82 respondents (<0.10 %) having a negative time duration for the amount of time completing the treatment intervention, the amount of time completing the quiz. Upon inspection, it is likely due to a bug in Chatfuel.

Loading the anonymized dataset for cleaning

Data

The raw data for was downloaded from Chatfuel and undergone an anonymization process before being stored in Sherlock. The anynomized data was then downloaded from Sherlock and stored in the data folder. The cleaning process involved dropping observations who are not in Phase 1 or Phase 2 and dropping actual repeaters. We identified actual repeaters by looking at whether they have a timestamp in more than one treatment arms and dropped them. Attempted repeaters are not actual repeaters, and these respondents were turned away from the chatbot when they tried to enter the chatbot for the second time after completing the first time. Since they did not repeat the chatbot, we don’t drop them from our dataset. We also checked to make sure all observations are unique by comparing the number of unique id with the number of observations in the dataset.

  • The anonymized dataset is stored in Github directory ~fb_misinfo_interventions/data/chatfuel
# file path `~fb_misinfo_interventions/data/chatfuel`
data <- fread("./data/chatfuel/fbmisinfo_anon.csv.gz")

Cleaning phases

  • Looking at respondents across different phases and generating generating a phase identifier (phase_coded)

  • Respondents who were in Version ExperimentLaunchV1 or ExperimentLaunchV2 as Phase 1

  • Respondents who were in Version Phase2, Phase2_V2 or Phase2_V3 as Phase 2

  • Blank values are individuals entered the chatbot but not eligible. These people clicked on on the Messenger pop up so their time was recorded. These participants were dropped from the dataset.

total_count <- data %>% 
  count(NewVersion) %>% 
  summarise(total = sum(n)) %>%
  pull(total)

data %>% 
  count(NewVersion) %>%
  rbind(data.frame(NewVersion = "Total", n = total_count)) %>%
  kable("html", col.names=(c("Phase", "Count"))) %>% 
  kableExtra::kable_styling(bootstrap_options = c("striped", "hover")) %>% scroll_box(width = "100%") 
Phase Count
67684
ExperimentLaunchV1 9906
ExperimentLaunchV2 30577
Final_Technical_Pilot 181
Misinfo_Pilot 290
Phase2 8058
Phase2_V2 27276
Phase2_V3 15512
Total 159484
# generating coded phase 1 and 2 variables and dropping respondents not in Phase 1 or Phase 2
data$phase_coded <-data$NewVersion 
data$phase_coded[data$phase_coded=="ExperimentLaunchV1" | data$phase_coded=="ExperimentLaunchV2"] = "Phase 1"
data$phase_coded[data$phase_coded=="Phase2" | data$phase_coded=="Phase2_V2" | data$phase_coded=="Phase2_V3"] = "Phase 2"

total_count <- data %>% 
  count(phase_coded) %>% 
  summarise(total = sum(n)) %>%
  pull(total)

data %>% 
  count(phase_coded) %>%
  rbind(data.frame(phase_coded = "Total", n = total_count)) %>%
  kable("html", col.names=(c("Phase", "Count"))) %>% 
  kableExtra::kable_styling(bootstrap_options = c("striped", "hover")) %>% scroll_box(width = "100%") 
Phase Count
67684
Final_Technical_Pilot 181
Misinfo_Pilot 290
Phase 1 40483
Phase 2 50846
Total 159484
a= as.character(length(unique(data$`analytic_id`)))


if (nrow(data) != length(unique(data$analytic_id))) {
  stop("The number of rows does not match the number of unique analytic_id values.")
} else {
  # Continue 
  print("Unique observations check passed.")
}
## [1] "Unique observations check passed."
  • The count of unique analytic_id is 159484. The number of unique analytic_id is equal to the number of total observation, so there is no repetition of analytic_id. Each observation is unique

Dropping respondents not in Phase 1 or Phase 2

data <- data %>% 
  filter(phase_coded == "Phase 1" | phase_coded == "Phase 2")

b <- as.character(length(unique(data$`analytic_id`)))
total_count <- data %>% 
  count(phase_coded) %>% 
  summarise(total = sum(n)) %>%
  pull(total)

data %>% 
  count(phase_coded) %>%
  rbind(data.frame(phase_coded = "Total", n = total_count)) %>%
  kable("html", col.names=(c("Phase", "Count"))) %>% 
  kableExtra::kable_styling(bootstrap_options = c("striped", "hover")) %>% scroll_box(width = "100%") 
Phase Count
Phase 1 40483
Phase 2 50846
Total 91329
  • The number of observations before dropping is 159484, and after dropping the number of observations is 91329

Identifying attempted and actual repeaters

#check for entries with multiple treatment arms
data_repeaters <- data %>%
    select(analytic_id, phase_coded,
           repeater,
           arm, 
           InterArms_start_time, InterArmsVideo_end_time, # video start and end time
           InterArmsLongBase_start_time, InterArmsLongBase_end_time, #long baseline start,end time
           InterArmsBase_start_time, InterArmsBase_end_time, #short baseline start,end time
           InterArmsSMS_start_time, InterArmsSMS_end_time, #sms start,end time
           InterArmsGame_start_time, interArmsGame_end_time, # game start and end time
           MisinfoQuiz_start_time) %>%
    mutate(has_longbase_time = if_else(!is.na(InterArmsLongBase_start_time) | !is.na(InterArmsLongBase_end_time), TRUE, FALSE),
           has_video_time = if_else(!is.na(InterArms_start_time) | !is.na(InterArmsVideo_end_time), TRUE, FALSE),
           has_game_time = if_else(!is.na(InterArmsGame_start_time) | !is.na(interArmsGame_end_time), TRUE, FALSE),
           has_shortbase_time = if_else(!is.na(InterArmsBase_start_time) | !is.na(InterArmsBase_end_time), TRUE, FALSE),
           has_sms_time = if_else(!is.na(InterArmsSMS_start_time) | !is.na(InterArmsSMS_end_time), TRUE, FALSE),
           repeated = has_longbase_time + has_shortbase_time + has_sms_time + has_video_time +has_game_time > 1) %>%
    select(analytic_id, arm, repeater, repeated, phase_coded, MisinfoQuiz_start_time)

data_repeaters = data_repeaters |> 
    mutate(
      actual_repeater = repeated,
      attempted_repeater = case_when(
        repeater == 1 & repeated == 0 ~ TRUE,
        TRUE ~ FALSE
      ),
      repeater = actual_repeater | attempted_repeater
    )

# Count reprters and attempted repeaters by phase
data_repeaters |> group_by(phase_coded) |> 
  summarize(
    `Actual Repeaters` = sum(actual_repeater),
    `Attempted Repeaters` = sum(attempted_repeater)
  ) |>
  mutate(
    `Total` = `Actual Repeaters` + `Attempted Repeaters`
  ) |>
  kable("html", col.names = c("Phase", "Actual Repeaters", "Attempted Repeaters", "Total")) |>
  kableExtra::kable_styling(bootstrap_options = c("striped", "hover")) |>
  scroll_box(width = "100%")
Phase Actual Repeaters Attempted Repeaters Total
Phase 1 3432 9645 13077
Phase 2 28 9973 10001
  • Repeaters are classified as attempted repeaters or actual repeaters.

  • Attempted repeaters are respondents trying to repeat the chatbot but were turned away automatically. The chatbot variable repeater identify if a person attempted repeat and based on their personal and chatfuel identification, we can determine if that person had completed the chatbot before.

  • Actual repeaters are respondents who actually repeated the chatbot.These actual repeaters are identified by having a timestamp in at least two treatment arms.

Excluding repeaters

data <- left_join(
  data,
  data_repeaters %>% select(analytic_id, actual_repeater, attempted_repeater),
  by = "analytic_id",
  relationship = "one-to-one",
  unmatched = "error"
  ) %>%
    filter(!actual_repeater)
data |> group_by(phase_coded) |> 
  summarize(
    `Actual Repeaters` = sum(actual_repeater),
    `Attempted Repeaters` = sum(attempted_repeater)
  ) |>
  mutate(
    `Total` = `Actual Repeaters` + `Attempted Repeaters`
  ) |>
  kable("html", col.names = c("Phase", "Actual Repeaters", "Attempted Repeaters", "Total")) |>
  kableExtra::kable_styling(bootstrap_options = c("striped", "hover")) |>
  scroll_box(width = "100%")
Phase Actual Repeaters Attempted Repeaters Total
Phase 1 0 9645 9645
Phase 2 0 9973 9973
c = as.character(length(unique(data$`analytic_id`)))
  • Before dropping the actual repeaters, we had 91329 observations. After dropping, we have 87869 observations.

Cleaning treatment arms variable

  • Checking the raw treatment arm variable, we see the following possible entries: baseline, game, longbaseline, sms, video

  • There is one entry of video in Phase 2, so we’ll treat it as a missing value

  • The coded treatment arm variable is arm_coded

  • Baseline 1 : If arm == “baseline” and phase_coded == “Phase 1”

  • SMS: If arm == “sms”

  • Video: If arm == “video” and phase_coded == “Phase 1”

  • Game : If arm == “game” and phase_coded == “Phase 1”

  • Short Baseline 2 : If arm == “baseline” and phase_coded == “Phase 2”

  • Long Baseline: If arm == “longbaseline” and phase_coded == “Phase 2”

# checking possible treatment responses 
data %>% 
  count(arm) %>%
  kable("html", col.names=(c("Treatment Arm", "Count"))) %>% 
  kableExtra::kable_styling(bootstrap_options = c("striped", "hover")) %>% scroll_box(width = "100%") 
Treatment Arm Count
17064
baseline 13728
game 7498
longbaseline 14163
sms 27862
video 7554
data %>% group_by(phase_coded) %>%
  count(arm)%>%
  kable("html", col.names=(c("Phase", "Treatment Arm", "Count"))) %>% 
  kableExtra::kable_styling(bootstrap_options = c("striped", "hover")) %>% scroll_box(width = "100%") 
Phase Treatment Arm Count
Phase 1 6954
Phase 1 baseline 7474
Phase 1 game 7498
Phase 1 longbaseline 2
Phase 1 sms 7570
Phase 1 video 7553
Phase 2 10110
Phase 2 baseline 6254
Phase 2 longbaseline 14161
Phase 2 sms 20292
Phase 2 video 1
  • We noticed one individual in Phase 2 having arm == “video” so we’ll drop this one observation and two individuals in Phase 1 having arm == “longbaseline” so we’ll drop these two observations
data <- data %>% filter(!(arm == "video" & phase_coded == "Phase 2"))
data <- data %>% filter(!(arm == "longbaseline" & phase_coded == "Phase 1"))
c1 <- as.character(length(unique(data$`analytic_id`)))
  • Before dropping we had 87869 observations. After dropping, we have 87866 observations.

Recoding treatment arm variable

# Recoding treatment arm variable 
data <- data %>%
  mutate(
    arm_coded = case_when(
      arm == "baseline" & phase_coded == "Phase 1" ~ "Original Baseline",
      arm %in% c("SMS", "sms") ~ "SMS",
      arm == "video" & phase_coded == "Phase 1" ~ "Video",
      arm == "game" & phase_coded == "Phase 1" ~ "Game",
      arm == "baseline" & phase_coded == "Phase 2" ~ "Original Baseline",
      arm == "longbaseline" & phase_coded == "Phase 2" ~ "Long Baseline",
      TRUE ~ "None"
    )
  )

Checking if Phase 1 respondents may have Phase 2 attributes other than treatment arm

data %>% group_by(phase_coded) %>%
  count(arm_coded)
## # A tibble: 9 × 3
## # Groups:   phase_coded [2]
##   phase_coded arm_coded             n
##   <chr>       <chr>             <int>
## 1 Phase 1     Game               7498
## 2 Phase 1     None               6954
## 3 Phase 1     Original Baseline  7474
## 4 Phase 1     SMS                7570
## 5 Phase 1     Video              7553
## 6 Phase 2     Long Baseline     14161
## 7 Phase 2     None              10110
## 8 Phase 2     Original Baseline  6254
## 9 Phase 2     SMS               20292
data %>% group_by(phase_coded) %>%
  count(user_payment_condition)
## # A tibble: 5 × 3
## # Groups:   phase_coded [2]
##   phase_coded user_payment_condition     n
##   <chr>       <chr>                  <int>
## 1 Phase 1     ""                     37049
## 2 Phase 2     ""                     10110
## 3 Phase 2     "high"                  9352
## 4 Phase 2     "low"                  21576
## 5 Phase 2     "medium"                9779
data %>% group_by(phase_coded) %>%
  count(attention_check_passed)
## # A tibble: 4 × 3
## # Groups:   phase_coded [2]
##   phase_coded attention_check_passed     n
##   <chr>       <lgl>                  <int>
## 1 Phase 1     NA                     37049
## 2 Phase 2     FALSE                   5335
## 3 Phase 2     TRUE                   23773
## 4 Phase 2     NA                     21709
  • We further checked for instances where individuals who had Phase-1 treatment have recorded at the Phase 2 only variables user_payment_condition and attention_check_passed. We found that there are no such instances.

Participants in Treatment Arm across Phase 1 and Phase 2

data %>%
  group_by(phase_coded, arm_coded) %>%
  summarise(n = n()) |>
  group_by(phase_coded) |>
  mutate(Total = sum(n)) |>
  pivot_wider(names_from = arm_coded, values_from = n, values_fill = 0) |>
  select(
  `Phase` = phase_coded,
  `Original Baseline`, `Game`, `SMS`, `Video`, `Long Baseline`, `Total`) |>
  kable("html") %>% 
kableExtra::kable_styling(bootstrap_options = c("striped", "hover")) %>% scroll_box(width = "100%")
Phase Original Baseline Game SMS Video Long Baseline Total
Phase 1 7474 7498 7570 7553 0 37049
Phase 2 6254 0 20292 0 14161 50817

Cleaning user country

  • Variable user_country indicates where the respondents come from

  • The coded version of user_country is country_coded

  • Responses are messy with some random links and emojis which are treated as missing values

  • Answers who include an explicit name of the country or small typos are renamed as one of the four countries

  • Participants who are not from either Kenya, South Africa, Nigeria, or Ghana are dropped from the dataset

Checking possible country answers

data$user_country <- data$user_country
data %>%
  count(user_country)
##         user_country    n
##   1:                 8935
##   2:        ,nigeria    1
##   3: .\nSouth Africa    1
##   4:  . south Africa    1
##   5:              ..    1
##  ---                     
## 216:  you are stupid    1
## 217:           𝒢𝒽𝒶𝓃𝒶    1
## 218:               😊    1
## 219:            😡😡😡😡    1
## 220:               🤔    1

Cleaning and renaming country responses

# rewriting responses

data = data |> mutate(
  country_coded = case_when(
    tolower(user_country) == "nigeria" ~ "Nigeria",
    tolower(user_country) == "kenya" ~ "Kenya",
    tolower(user_country) == "south africa" ~ "South Africa",
    tolower(user_country) == "ghana" ~ "Ghana",
    TRUE ~ "Other"
  )
)

data %>%
  group_by(phase_coded, country_coded) %>%
  summarise(n = n()) |>
  group_by(phase_coded) |>
  mutate(Total = sum(n)) |>
  pivot_wider(names_from = country_coded, values_from = n, values_fill = 0) |>
  select(
  `Phase` = phase_coded, `Ghana`, `Kenya`, `Nigeria`, `South Africa`, `Other`, `Total`) |>
  kable("html") %>% 
kableExtra::kable_styling(bootstrap_options = c("striped", "hover")) %>% scroll_box(width = "100%")
Phase Ghana Kenya Nigeria South Africa Other Total
Phase 1 4420 4292 14196 9716 4425 37049
Phase 2 4671 6120 20470 14109 5447 50817

Dropping participants who are not from either Kenya, South Africa, Nigeria, or Ghana

  • Respondents who are dropped passed the consent checkpoint but did not come from either four of our countries of interest

  • ineligible_country is a variable that indicates whether the respondent is from a country other than Kenya, South Africa, Nigeria, or Ghana

data = data |> mutate(
  is_ineligible_country = country_coded == "Other"
)

# Calculate the number of unique participants before and after dropping ineligible countries
c2_before = as.character(length(unique(data$`analytic_id`)))

n_other = (data$is_ineligible_country) |> sum()

data = data |> filter(!is_ineligible_country)

c2 = as.character(length(unique(data$`analytic_id`)))
  • Dropped 9872participants who are not from either Kenya, South Africa, Nigeria, or Ghana.

  • Before dropping, we had 87866 observations. After dropping we have 77994 observations.

Checking payment amount for each country

  • The actual payment amount for participation was only recorded for Phase 2 participants by variable user_payment_amount

  • In Version 1 of Phase 1, participants were paid 12.5GHS in Ghana, 150Ksh in Kenya, 20R in South Africa, and 800N in Nigeria while in Version 2 of Phase 1, participants were paid 12.5GHS in Ghana, 150Ksh in Kenya, 10R in South Africa, and 400N in Nigeria

  • The variable payment_amount_coded is created to contain the payment amount in both Phase 1 and Phase 2

# manually entered payment amount for each country in Phase 1 and Phase 2 

data$payment_amount_1[data$NewVersion == "ExperimentLaunchV1" & data$country_coded=="Ghana"] <- "₵12.5"
data$payment_amount_1[data$NewVersion == "ExperimentLaunchV1" & data$country_coded=="Kenya"] <- "150Ksh" 
data$payment_amount_1[data$NewVersion == "ExperimentLaunchV1" & data$country_coded=="South Africa"] <- "R20"
data$payment_amount_1[data$NewVersion == "ExperimentLaunchV1" & data$country_coded=="Nigeria"] <- "₦800"

data$payment_amount_1[data$NewVersion == "ExperimentLaunchV2" & data$country_coded=="Ghana"] <- "₵12.5"
data$payment_amount_1[data$NewVersion == "ExperimentLaunchV2" & data$country_coded=="Kenya"] <- "150Ksh"
data$payment_amount_1[data$NewVersion == "ExperimentLaunchV2" & data$country_coded=="South Africa"] <- "R10"
data$payment_amount_1[data$NewVersion == "ExperimentLaunchV2" & data$country_coded=="Nigeria"] <- "₦400"

data <- data %>%
  mutate(payment_amount_coded = coalesce(payment_amount_1, user_payment_amount))

Checking manipulative / reliable order

In the quiz, for each post, every user first saw the question about their intention to share. Then, they were asked two questions regarding their assessment of the manipulativeness and reliability of these posts. The order in which these questions were asked was randomized at the user level when they started the quiz. Users who never participated in the quiz will therefore have a missing value

data %>%
  group_by(phase_coded, manipulative_reliable_order) %>%
  summarise(n = n()) |>
  group_by(phase_coded) |>
  mutate(Total = sum(n)) |>
  pivot_wider(names_from = manipulative_reliable_order, values_from = n, values_fill = 0) |>
  select(
  `Phase` = phase_coded, `Manipulative First` = `1`, `Reliable First` = `2` , `Missing` = `NA`, `Total`) |>
  kable("html") %>% 
kableExtra::kable_styling(bootstrap_options = c("striped", "hover")) %>% scroll_box(width = "100%")
Phase Manipulative First Reliable First Missing Total
Phase 1 10416 10244 11964 32624
Phase 2 15457 15563 14350 45370

Checking time variables

  • MisinfoChat_start_time : time stamp for starting the chatbot

  • InterArms_start_time and InterArmsVideo_end_time: time stamp for starting and ending the video intervention

  • InterArmsGame_start_time and interArmsGame_end_time: time stamp for starting and ending the game intervention

  • InterArmsSMS_start_time and InterArmsSMS_end_time: time stamp for starting and ending the SMS intervention

  • InterArmsBase_start_time and InterArmsBase_end_time: time stamp for starting and ending the baseline intervention

  • InterArmsLongBase_start_time and InterArmsLongBase_end_time: time stamp for starting and ending the long baseline intervention

  • MisinfoQuiz_start_time and misinfoQuiz_end_time: time stamp for starting and ending the Misinfo quiz

  • baseline_duration: time taken to complete the baseline

  • video_duration: time taken to complete the video intervention

  • game_duration: time taken to complete the game intervention

  • sms_duration: time taken to complete the SMS intervention

  • longbase_duration: time taken to complete the long baseline

  • intervention_duration: time taken to complete the intervention

  • quiz_duration: time taken to complete the quiz

# calculating duration 
data <- data %>%
  mutate(
    baseline_duration = InterArmsBase_end_time - InterArmsBase_start_time,
    video_duration = InterArmsVideo_end_time - InterArms_start_time,
    game_duration = interArmsGame_end_time - InterArmsGame_start_time,
    sms_duration = InterArmsSMS_end_time - InterArmsSMS_start_time,
    longbase_duration = InterArmsLongBase_end_time - InterArmsLongBase_start_time,
    misinfo_quiz_duration = misinfoQuiz_end_time - MisinfoQuiz_start_time,
    total_duration = misinfoQuiz_end_time - MisinfoChat_start_time
    ) %>%
  mutate(intervention_duration = case_when(
    arm_coded == "Video" ~ video_duration,
    arm_coded == "Game" ~ game_duration,
    arm_coded == "SMS" ~ sms_duration,
    arm_coded == "Original Baseline" ~ baseline_duration,
    arm_coded == "Long Baseline" ~ longbase_duration,
    TRUE ~ NA
  )
  )
data <- data |> mutate(
  is_abnormal_time = ((intervention_duration < 0) & !is.na(intervention_duration) ) | ((misinfo_quiz_duration < 0)) & !is.na(misinfo_quiz_duration)
) 

# removing missing values from duration (for those who did not finish the intervention)
is_abnormal_time = as.character(sum(data$is_abnormal_time, na.rm = TRUE))  
  • Checking for abnormal time (negative time values, outliers, etc. ). Noticed 82 time values. Dropping these observations from the dataset.
data <- data |> filter(!is_abnormal_time)

After dropping, we have 77912 observations.

Cleaning and creating dropoff variable

  • started_coded_num: 1 if the participant started the chatbot, 0 otherwise

  • consent_coded_num: 1 if the participant consented to the quiz, 0 otherwise

  • started_arm_coded_num: 1 if the participant started any of the interventions, 0 otherwise

  • completed_arm_coded_num: 1 if the participant completed any of the interventions, 0 otherwise

  • quiz_started_coded_num: 1 if the participant started the quiz, 0 otherwise

  • quiz_consent_coded_num: 1 if the participant consented to the quiz, 0 otherwise

  • quiz_completed_coded_num: 1 if the participant completed the quiz, 0 otherwise

  • completed_all_coded_num: 1 if the participant completed all, 0 otherwise

data <- data %>%
  mutate(started_coded_num = ifelse(!is.na(MisinfoChat_start_time), 1, 0),
         started_arm_coded_num = ifelse(!is.na(InterArms_start_time) | !is.na(InterArmsSMS_start_time) | !is.na(InterArmsGame_start_time) | !is.na(InterArmsBase_start_time) | !is.na(InterArmsLongBase_start_time), 1, 0),
         completed_arm_coded_num = ifelse(!is.na(intervention_duration), 1, 0),
         quiz_started_coded_num = ifelse(!is.na(MisinfoQuiz_start_time), 1, 0),
         quiz_consent_coded_num = case_when(
          quiz_consent == "I acknowledge" ~ 1,
          TRUE ~ 0),
         quiz_completed_coded_num = ifelse(!is.na(misinfoQuiz_end_time), 1, 0),
         completed_all_coded_num = ifelse(!is.na(completed), 1, 0))

Creating dropoff variable

  • stage provides the location where participants dropped off from the chatbot
data <- data |>
  rowwise() |>
  mutate(
    stage_reached_num = sum(
      started_coded_num,
      started_arm_coded_num,
      completed_arm_coded_num,
      quiz_consent_coded_num,
      quiz_started_coded_num,
      quiz_completed_coded_num,
      completed_all_coded_num),
    stage_reached_char = case_when(
      stage_reached_num == 7 ~ "7-Completed all",
      stage_reached_num == 6 ~ "6-Completed quiz",
      stage_reached_num == 5 ~ "5-Consented to quiz",
      stage_reached_num == 4 ~ "4-Reached quiz section",
      stage_reached_num == 3 ~ "3-Completed intervention",
      stage_reached_num == 2 ~ "2-Started intervention",
      stage_reached_num == 1 ~ "1-Started chatbot",
      TRUE ~ "Did not start"
  ))


data |> group_by(phase_coded, stage_reached_num, stage_reached_char) |>
  summarize(N = n()) |> 
  arrange(phase_coded, desc(stage_reached_num)) |>
  group_by(phase_coded) |>
  mutate(Total = cumsum(N)) |>
  arrange(phase_coded, stage_reached_num) |>
  mutate(
    `Reached here (% Total)` = round(Total / Total[1] * 100,2),
    `Reached here (% Previous)` = round(Total / lag(Total) * 100,2),
    `Finished here (% Total)` = round(N / Total[1] * 100,2),
    `Finished here (% Previous)` = round(N / lag(Total) * 100,2)
  ) |>
  select(
    `Phase` = phase_coded,
    `Stage Reached` = stage_reached_char,
    `Reached here (n)` = Total,
    `Reached here (% Total)`,
    `Reached here (% Previous)`,
    `Finished here` = N,
    `Finished here (% Total)`,
    `Finished here (% Previous)`
  ) |>
  kable("html") |>
  kableExtra::kable_styling(bootstrap_options = c("striped", "hover")) |>
  scroll_box(width = "100%")
Phase Stage Reached Reached here (n) Reached here (% Total) Reached here (% Previous) Finished here Finished here (% Total) Finished here (% Previous)
Phase 1 1-Started chatbot 32560 100.00 NA 2540 7.80 NA
Phase 1 2-Started intervention 30020 92.20 92.20 9113 27.99 27.99
Phase 1 3-Completed intervention 20907 64.21 69.64 3 0.01 0.01
Phase 1 4-Reached quiz section 20904 64.20 99.99 310 0.95 1.48
Phase 1 5-Consented to quiz 20594 63.25 98.52 3610 11.09 17.27
Phase 1 6-Completed quiz 16984 52.16 82.47 314 0.96 1.52
Phase 1 7-Completed all 16670 51.20 98.15 16670 51.20 98.15
Phase 2 1-Started chatbot 45352 100.00 NA 5071 11.18 NA
Phase 2 2-Started intervention 40281 88.82 88.82 8893 19.61 19.61
Phase 2 4-Reached quiz section 31388 69.21 77.92 387 0.85 0.96
Phase 2 5-Consented to quiz 31001 68.36 98.77 3840 8.47 12.23
Phase 2 6-Completed quiz 27161 59.89 87.61 317 0.70 1.02
Phase 2 7-Completed all 26844 59.19 98.83 26844 59.19 98.83

Checking quiz completers and treatment groups to see if NA entrants

c4 = as.character(length(unique(data$`analytic_id`)))
abnormal_quiz_completers <- data %>%
  filter(quiz_completed_coded_num == 1) %>%
  select(arm_coded, analytic_id, phase_coded, quiz_completed_coded_num, intervention_duration) %>%
  filter(is.na(arm_coded))

is_abnormal_quiz_completers <- as.character(nrow(abnormal_quiz_completers))

data <- data %>%
  filter(!analytic_id %in% abnormal_quiz_completers$analytic_id)

d <- as.character(length(unique(data$`analytic_id`)))
  • Noticed 0 participants who completed the quiz but did not have an recorded entry for treatment assignment. They did not receive any intervention, so we dropped them from the our dataset.

  • Before dropping, we had 77912 number of observations. After droppping, we have 77912 number of observations

Cleaning post-level variables and outcomes

  • Three variables: sharing, manipulative, and reliable scores are recorded for each post. We shall reshape them individually. While merging we shall impose a one-to-one relationship to ensure that we do not lose any data.
# Reshape post ids
data_long <- data %>% 
  select(starts_with("post_id"), analytic_id, phase_coded, arm_coded, attention_check_passed, quiz_completed_coded_num) %>%
  select(-ends_with("_followup")) %>%
  pivot_longer(
    cols=starts_with("post_id"),
    names_to="post_number",
    values_to="post_id",
    names_prefix="post_id_"
    )

# Reshape manipulative scores
manipulative_score <- data %>% 
  select(starts_with("post_id"), contains(c("_manipulative")), analytic_id) %>%
  select(-ends_with(c("_followup", "_reliable_order"))) %>%
  pivot_longer(
    cols = contains("_manipulative"),
    names_to = "post_number",
    values_to = "manipulative_score",
    names_pattern = "post_(.*)_manipulative"
  ) %>%
  select(post_number, analytic_id, manipulative_score)

# Reshape reliable scores
reliable_score <- data %>% 
  select(starts_with("post_id"), contains(c("_reliable")), analytic_id) %>%
  select(-ends_with(c("_followup", "_reliable_order"))) %>%
  pivot_longer(
    cols = contains("_reliable"),
    names_to = "post_number",
    values_to = "reliable_score",
    names_pattern = "post_(.*)_reliable"
  ) %>%
  select(post_number, analytic_id, reliable_score)

# Reshape share scores
share_score <- data %>%
  select(starts_with("post_id"), ends_with("_share"), analytic_id) %>%
  pivot_longer(
    cols = contains("_share"),
    names_to = "post_number",
    values_to = "share_score",
    names_pattern = "post_(.*)_share"
  ) %>%
  select(post_number, analytic_id, share_score)


# Merging reshaped variables

data_long = data_long |>
  left_join(manipulative_score,
    by = c("analytic_id", "post_number"),
    relationship = "one-to-one",
    unmatched = "error"
  ) |>
  left_join(reliable_score,
    by = c("analytic_id", "post_number"),
    relationship = "one-to-one",
    unmatched = "error"
  ) |>
  left_join(share_score,
    by = c("analytic_id", "post_number"),
    relationship = "one-to-one",
    unmatched = "error"
  )

# converting to numeric responses
    # for manipulative and reliable scores: 1 = not at all, 5 = very
    # for share score: 1 = yes, 0 = no


data_long = data_long |> 
  mutate(
    manipulative_score = parse_number(manipulative_score),
    reliable_score = parse_number(reliable_score),
    share_score = case_when(
      tolower(share_score) == "yes" ~ 1,
      tolower(share_score) == "no" ~ 0,
      TRUE ~ NA_real_
    )
  ) 

Checking abnormal reliable and manipulative scores

  • abnormal_manipulativescore or abnormal_reliablescore responses for manipulative and reliable scores are not in {1,2,3,4,5}.
data_long <- data_long %>%
  mutate(
    abnormal_manipulativescore = ifelse(!(manipulative_score %in% 1:5) & !is.na(manipulative_score), 1, 0),
    abnormal_reliablescore = ifelse(!(reliable_score %in% 1:5) & !is.na(reliable_score), 1, 0)
  ) |>
  mutate(
    manipulative_score = ifelse(abnormal_manipulativescore == 1, NA_real_, manipulative_score) |> as.integer(),
    reliable_score = ifelse(abnormal_reliablescore == 1, NA_real_, reliable_score) |> as.integer()
  )
  • We identified 82 participant-answers with abnormal scores in the manipulative and reliable scores. We will exclude these scores from the analysis by recoding them as missing values.

Checking attention check pass

  • Checking attention check pass across two phases. Phase 1 has notably lower pass rates
# attention check pass variable

data_attention_2 = data |> filter(
  phase_coded == "Phase 2") |>
  mutate(
    attention_check_passed = case_when(
      attention_check_passed == "TRUE" ~ 1,
      attention_check_passed == "FALSE" ~ 0,
      TRUE ~ NA_real_
    )
  ) |>
  select(analytic_id, attention_check_passed)

data_attantion_1 = data_long |>
  filter(phase_coded == "Phase 1") |>
  filter(post_id %in% c("attention_check_1", "attention_check_2")) |>
  mutate(
    attention_check_passed = case_when(
      post_id == "attention_check_1" & manipulative_score == 4 & reliable_score == 4 & share_score == 1 ~ 1,
      post_id == "attention_check_2" & manipulative_score == 2 & reliable_score == 2 & share_score == 0 ~ 1,
      post_id == "attention_check_1" & !(manipulative_score == 4 & reliable_score == 4 & share_score == 1) ~ 0, 
      post_id == "attention_check_2" & !(manipulative_score == 2 & reliable_score == 2 & share_score == 0) ~ 0,
      TRUE ~ NA_real_
    )
  ) |>
  select(analytic_id, attention_check_passed)

data_attention <- bind_rows(data_attention_2, data_attantion_1)


data_attention |> left_join(
    data |> select(analytic_id, phase_coded),
    by = "analytic_id",
    relationship = "one-to-one"
  ) |>
  group_by(phase_coded) |>
  summarize(
    `Attention Check Passed` = sum(attention_check_passed, na.rm = TRUE),
    `Attention Check Non-missing` = sum(!is.na(attention_check_passed)),
    `Pass Rate (%)` = round(`Attention Check Passed` / `Attention Check Non-missing` * 100, 2)
  ) |>
  rename(`Phase` = phase_coded) |>
  kable("html") |>
  kableExtra::kable_styling(bootstrap_options = c("striped", "hover")) |>
  scroll_box(width = "100%")
Phase Attention Check Passed Attention Check Non-missing Pass Rate (%)
Phase 1 495 18569 2.67
Phase 2 23751 29080 81.67

Inspecting missing values

  • We are cautious of missing values in the quiz among quiz completers. We will now evaluate the issue of missingness

  • We will start by dropping the attention check posts from the long dataset as they were coded differently in Phase 2 (and we analyzed them above); we will also drop those that did not complete the quiz (202560 out of 467472 user-posts`).

# Creating post_type for each post
data_long <- data_long |>
  mutate(post_type = case_when(
    str_detect(post_id, "non_misinfo") ~ "non_misinfo",
    str_detect(post_id, "discrediting") ~ "discrediting",
    str_detect(post_id, "false") ~ "false_dichotomy",
    str_detect(post_id, "attention_check") ~ "attention_check",
    str_detect(post_id, "emotional") ~ "emotional",
    TRUE ~ "NA"
  ))

data_long = data_long |> filter(!post_type == "attention_check") |> filter(quiz_completed_coded_num == 1)

Checking missing values in post id

data_long |> group_by(post_id) |> summarize(n = n()) |> arrange(desc(n)) |> kable("html") |> kableExtra::kable_styling(bootstrap_options = c("striped", "hover")) |> scroll_box(width = "100%")
post_id n
emotional_28 22136
non_misinfo_10 22130
false_dichotomy_21 22060
non_misinfo_6 22050
discrediting_17 22036
discrediting_14 21997
non_misinfo_8 21974
false_dichotomy_23 21973
non_misinfo_2 21911
emotional_27 21894
714
emotional_29 2
emotional_24 1
non_misinfo_9 1
abnormal_posts = data_long |>
  filter(is.na(post_id) | post_id == "" | post_id == "emotional_29" | post_id == "emotional_24" | post_id == "non_misinfo_9") |> select(analytic_id) |> distinct() |>
  mutate(abnormal_posts = 1)

data_long = data_long |> filter(!analytic_id %in% abnormal_posts$analytic_id)

data = data |> left_join(
  abnormal_posts,
  by = "analytic_id"
  ) |>
  mutate(abnormal_posts = ifelse(is.na(abnormal_posts), 0, abnormal_posts))  

We shall drop all users who have at least one missing value in the post_id variable. We shall also drop those who saw emotional_29, emotional_24 and non_misinfo_9 as these posts were used in the pilot but not in the main study, a total of 123 users.

missing_treatment_arm <- data_long |>
  filter(arm_coded == "None") |>
  select(analytic_id) |>
  distinct() |>
  mutate(missing_treatment_arm = 1)

data_long = data_long |> filter(!analytic_id %in% missing_treatment_arm$analytic_id)

data = data |> left_join(
  missing_treatment_arm,
  by = "analytic_id"
  ) |>
  mutate(missing_treatment_arm = ifelse(is.na(missing_treatment_arm), 0, missing_treatment_arm))  

Finally, we shall drop all users who have missing treatment arm values, a total of 5 users.

Missing values per phase

In Phase 1 we initially did not enforce that the users select from one of the given options. This results in the relatively higher number of missing values in Phase 1.

By users

data_long |> 
  group_by(phase_coded, analytic_id) |>
  summarize(
    `User has any missing values` = any(is.na(share_score)) | any(is.na(reliable_score)) | any(is.na(manipulative_score)),
    `User has all missing values` = all(is.na(share_score)) & all(is.na(reliable_score)) & all(is.na(manipulative_score)),
    `User has no missing values` = !any(is.na(share_score)) & !any(is.na(reliable_score)) & !any(is.na(manipulative_score))
    ) |>
  group_by(phase_coded) |>
  summarize(
    `Total Users` = n(),
    `Users with any missing values` = sum(`User has any missing values`),
    `Users with all missing values` = sum(`User has all missing values`),
    `Users with no missing values` = sum(`User has no missing values`)
  ) |>
  kable("html", caption = "Users with missing values, per phase") |>
  kableExtra::kable_styling(bootstrap_options = c("striped", "hover")) |>
  scroll_box(width = "100%")
Users with missing values, per phase
phase_coded Total Users Users with any missing values Users with all missing values Users with no missing values
Phase 1 16859 1037 0 15822
Phase 2 27165 46 0 27119

By user-posts

data_long |>
  group_by(phase_coded) |>
  summarize(
    `Total` = n(),
    `Missing Share Score` = sum(is.na(share_score)),
    `Missing Reliable Score` = sum(is.na(reliable_score)),
    `Missing Manipulative Score` = sum(is.na(manipulative_score)),
    `Missing All Scores` = sum(is.na(share_score) & is.na(reliable_score) & is.na(manipulative_score)),
    `All Scores Non-missing` = sum(!is.na(share_score) & !is.na(reliable_score) & !is.na(manipulative_score)),
  ) |>
  kable("html", caption = "Missing values per phase") |>
  kableExtra::kable_styling(bootstrap_options = c("striped", "hover")) |> 
  scroll_box(width = "100%")
Missing values per phase
phase_coded Total Missing Share Score Missing Reliable Score Missing Manipulative Score Missing All Scores All Scores Non-missing
Phase 1 84295 481 610 546 37 82920
Phase 2 135825 35 32 37 22 135770

Missing values per post order

We now look into whether the missing values are concentrated in a particular post order.

data_long |>
  group_by(post_number) |>
  summarize(
    `Total` = n(),
    `Missing Share Score` = sum(is.na(share_score)),
    `Missing Reliable Score` = sum(is.na(reliable_score)),
    `Missing Manipulative Score` = sum(is.na(manipulative_score)),
    `Missing All Scores` = sum(is.na(share_score) & is.na(reliable_score) & is.na(manipulative_score)),
    `All Scores Non-missing` = sum(!is.na(share_score) & !is.na(reliable_score) & !is.na(manipulative_score)),
  ) |>
  kable("html", caption = "Missing values per post order") |>
  kableExtra::kable_styling(bootstrap_options = c("striped", "hover")) |>
  scroll_box(width = "100%")
Missing values per post order
post_number Total Missing Share Score Missing Reliable Score Missing Manipulative Score Missing All Scores All Scores Non-missing
1 36616 91 118 99 5 36348
2 36725 71 89 88 8 36522
3 36717 77 93 84 8 36505
4 36605 65 97 90 9 36397
5 36652 91 107 107 14 36413
6 36805 121 138 115 15 36505

Missing values per post id

  • The missing values are equally distributed across the post ids.
data_long |>
  group_by(post_id) |>
  summarize(
    `Total` = n(),
    `Missing Share Score` = sum(is.na(share_score)),
    `Missing Reliable Score` = sum(is.na(reliable_score)),
    `Missing Manipulative Score` = sum(is.na(manipulative_score)),
    `Missing All Scores` = sum(is.na(share_score) & is.na(reliable_score) & is.na(manipulative_score)),
    `All Scores Non-missing` = sum(!is.na(share_score) & !is.na(reliable_score) & !is.na(manipulative_score)),
  ) |>
  kable("html", caption = "Missing values per post id") |>
  kableExtra::kable_styling(bootstrap_options = c("striped", "hover")) |>
  scroll_box(width = "100%")
Missing values per post id
post_id Total Missing Share Score Missing Reliable Score Missing Manipulative Score Missing All Scores All Scores Non-missing
discrediting_14 21994 50 67 63 6 21854
discrediting_17 22030 59 59 58 10 21889
emotional_27 21893 56 70 57 7 21747
emotional_28 22131 50 68 68 6 21981
false_dichotomy_21 22056 44 68 63 6 21911
false_dichotomy_23 21968 51 63 70 7 21817
non_misinfo_10 22126 37 63 54 5 22002
non_misinfo_2 21908 43 53 48 0 21773
non_misinfo_6 22043 70 59 46 3 21893
non_misinfo_8 21971 56 72 56 9 21823

Missing values per phase and treatment arm

  • The missing values are equally distributed across the treatment arms within each phase.
data_long |>
  group_by(phase_coded, arm_coded) |>
  summarize(
    `Total` = n(),
    `Missing Share Score` = sum(is.na(share_score)),
    `Missing Reliable Score` = sum(is.na(reliable_score)),
    `Missing Manipulative Score` = sum(is.na(manipulative_score)),
    `Missing All Scores` = sum(is.na(share_score) & is.na(reliable_score) & is.na(manipulative_score)),
    `All Scores Non-missing` = sum(!is.na(share_score) & !is.na(reliable_score) & !is.na(manipulative_score)),
  ) |>
  kable("html", caption = "Missing values per phase and treatment arm") |>
  kableExtra::kable_styling(bootstrap_options = c("striped", "hover")) |>
  scroll_box(width = "100%")
Missing values per phase and treatment arm
phase_coded arm_coded Total Missing Share Score Missing Reliable Score Missing Manipulative Score Missing All Scores All Scores Non-missing
Phase 1 Game 17430 103 145 135 6 17118
Phase 1 Original Baseline 24490 128 161 166 11 24115
Phase 1 SMS 21645 122 167 129 9 21282
Phase 1 Video 20730 128 137 116 11 20405
Phase 2 Long Baseline 45880 8 7 10 6 45868
Phase 2 Original Baseline 21355 5 5 6 4 21347
Phase 2 SMS 68590 22 20 21 12 68555

Calculating mean scores for each participant

Sharing discernment is defined as the difference in proportion between non-misinformation posts and misinformation posts shared. For example, if user shared 2 out of 2 non-misinformation posts (i.e. \(1\)) and 1 out of 3 misinformation posts (i.e. \(\frac{1}{3}\)), discernment takes a value of \(\frac{2}{3}\).

Reliability and Manipulation discernment are constructed in such a way that misinformation and non-misinformation are assigned an equal weight. Since each user encounters 3 misinformation posts and 2 non-misinformation posts, non-misinformation is reweighted by \(\frac{3}{2}\)

As shown above there is a number of missing values in the user-post scores but they are primarily concentrated in Phase 1 (due to us initially not enforcing that the users select from one of the given options). For nor now, we shall constuct the scores by ignoring missing values however we shall keep track if any score was missing.

Note that under a restrictive sample definition, where we only consider the users with no missing values, we would loose 1083 users out of 44024 users who completed the quiz. Vast majority of these users are from Phase 1.


\[ Sharing Discernemnt = \frac{\textit{# Non-Misinformation posts Shared}}{3} - \frac{\textit{# Misinformation posts Shared}}{2} \]

\[ \text{Reliability Discernment:} = \begin{cases} \textit{Reliability score} \times \frac{3}{2}, & \text{if non-misinformation} \\ \textit{Reliability score} \times -1, & \text{if misinformation} \\ 0, & \text{otherwise} \end{cases} \]


\[ \text{Manipulation Discernment:} = \begin{cases} \textit{Manipulation score}\times \frac{3}{2}, & \text{if non-misinformation} \\ \textit{Manipulation score} \times -1, & \text{if misinformation} \\ 0, & \text{otherwise} \end{cases} \]

outcomes_misinfo = data_long |>
  filter(post_type %in% c("discrediting", "false_dichotomy", "emotional")) |>
  group_by(analytic_id) |>
  summarize(
    mean_share_misinfo = mean(share_score, na.rm=TRUE),
    mean_reliable_misinfo = mean(reliable_score, na.rm=TRUE),
    mean_manipulative_misinfo = mean(manipulative_score, na.rm=TRUE),
    any_missing_share_misinfo = any(is.na(share_score)),
    any_missing_reliable_misinfo = any(is.na(reliable_score)),
    any_missing_manipulative_misinfo = any(is.na(manipulative_score))
  )

outcomes_nonmisinfo = data_long |>
  filter(post_type == "non_misinfo") |>
  group_by(analytic_id) |>
  summarize(
    mean_share_nonmisinfo = mean(share_score, na.rm=TRUE),
    mean_reliable_nonmisinfo = mean(reliable_score, na.rm=TRUE),
    mean_manipulative_nonmisinfo = mean(manipulative_score, na.rm=TRUE),
    any_missing_share_nonmisinfo = any(is.na(share_score)),
    any_missing_reliable_nonmisinfo = any(is.na(reliable_score)),
    any_missing_manipulative_nonmisinfo = any(is.na(manipulative_score))
  )

outcomes = outcomes_misinfo |>
  left_join(outcomes_nonmisinfo,
  by = "analytic_id",
  relationship = "one-to-one",
  unmatched = "error"
  )

outcomes = outcomes |>
  mutate(
    sharing_discernment = mean_share_nonmisinfo - mean_share_misinfo,
    reliability_discernment = mean_reliable_nonmisinfo - mean_reliable_misinfo,
    manipulation_discernment = mean_manipulative_nonmisinfo - mean_manipulative_misinfo
  )

Generating long dataset

  • Long dataset is exported as misinfo_clean_long.csv
data_long_final <- data_long %>%
  select(analytic_id, post_id, post_number, post_type, share_score, reliable_score, manipulative_score, phase_coded, quiz_completed_coded_num, arm_coded)

# labeling   
library(Hmisc)


label(data_long_final$analytic_id) <- "Analytic ID"
label(data_long_final$post_id) <- "Post ID"
label(data_long_final$post_number) <- "Post Order Number"
label(data_long_final$post_type) <- "Post Type"
label(data_long_final$share_score) <- "Share Score"
label(data_long_final$reliable_score) <- "Reliable Score"
label(data_long_final$manipulative_score) <- "Manipulative Score"
label(data_long_final$phase_coded) <- "Phase"
label(data_long_final$quiz_completed_coded_num) <- "Quiz Completion"
label(data_long_final$arm_coded) <- "Treatment arm"

var_names <- names(data_long_final)

# Extracting labels
var_labels <- sapply(data_long_final, label)

# Combine into a data frame
label_info <- data.frame(Variable = var_names, Label = var_labels, stringsAsFactors = FALSE)

write.csv(label_info, "./data/chatfuel/long_data_variable.csv", row.names = FALSE)

fwrite(data_long_final, "./data/chatfuel/misinfo_clean_long.csv.gz")

saveRDS(data_long_final, "./data/chatfuel/misinfo_clean_long.rds")

Generating wide dataset

  • Wide dataset is exported as misinfo_clean_wide.csv
data_wide = data |>
  select(
    analytic_id, phase_coded, country_coded, score, payment_amount_coded, arm_coded,
    MisinfoChat_start_time, MisinfoChat_end_time, total_duration, intervention_duration, 
    misinfo_quiz_duration, misinfoQuiz_end_time, MisinfoQuiz_start_time,
    missing_treatment_arm, abnormal_posts, stage_reached_char, consent_coded_num, manipulative_reliable_order
  ) |> left_join(
    outcomes,
    by = "analytic_id",
    relationship = "one-to-one"
  ) |> left_join(
    data_attention,
    by = "analytic_id",
    relationship = "one-to-one"
  )
   
#labeling 

label(data_wide$analytic_id) <- "Analytic ID"
label(data_wide$phase_coded) <- "Phase"
label(data_wide$country_coded) <- "Country"
label(data_wide$stage_reached_char) <- "Stage Reached"
label(data_wide$payment_amount_coded) <- "Payment Amount"
label(data_wide$arm_coded) <- "Treatment Arm"
label(data_wide$intervention_duration) <- "Intervention Duration"
label(data_wide$total_duration) <- "Total Duration"
label(data_wide$misinfo_quiz_duration) <- "Misinfo Quiz Duration"
label(data_wide$misinfoQuiz_end_time) <- "Misinfo Quiz End Time"
label(data_wide$MisinfoQuiz_start_time) <- "Misinfo Quiz Start Time"
label(data_wide$consent_coded_num) <- "Consent: 1 for I consent, start now, 0 for No let's stop here"
label(data_wide$attention_check_passed) <- "Attention Check Passed; TRUE = PASS; FALSE = FAIL"
label(data_wide$manipulative_reliable_order) <- "Order of Manipulative and Reliable Questions"


# Save data to csv.gz
fwrite(data_wide, "./data/chatfuel/misinfo_clean_wide.csv.gz")

# Save data to .rds
saveRDS(data_wide, "./data/chatfuel/misinfo_clean_wide.rds")