Project Overview

Problem

COVID-19 vaccine hesitancy has been recognized as a problem across nations. A resistance to getting vaccinated is emerging as a major hurdle, especially in the developing world, where vaccine access issues are still being gradually resolved. Persistent pools of unvaccinated people around the world could present a greater risk for the emergence of new variants of concern. Addressing people’s vaccine hesitancy is hence crucial to curb the spread of COVID-19, and to consequently avert hospitalizations and death.

Objectives

We intend to understand why people are hesitant about getting the COVID-19 vaccine. Hesitancy could not only occur within the unvaccinated population but also in a subset of people who already got vaccinated. Therefore, phase 1 of the project has the following objectives:

  1. Understand why people are hesitant to get the COVID-19 vaccines
  2. Understand ways to best elicit vaccine impediments from respondents
  3. Pinpoint what treatments will help people get vaccinated

Approach

We intend to use chatbot as a medium (on Facebook) to conduct conversations with people to understand how we can best achieve the above three objectives. We have run six pilots as of March 12, 2022, – 2 in the United States using Qualtrics on Lucid, and 4 in South Africa on Facebook. The eventual goal will be running this using multiple chatbots that enable the conversation to flow more naturally than in a survey format. We hypothesize that respondents are more likely to respond to our sensitive questions around vaccine hesitancy if the questions are asked more casually in an open stress-free setting. Therefore, in all version of the pilots, we make our tone as causal as possible (using emojis, GIFs, emphatic prompts) and include delays in the appearance of questions (and empathetic responses after each question) to make the conversation more authentic.

Analysis script used is linked here. Relevant GitHub issue is linked here.

This analysis is based on 962 unvaccinated respondents who completed the current pilot survey waves 4a, 4b, and 5.

We have split the analyses into 3 sections (preceded by a set of key takeaways):

  • Key takeaways
  • Free text response distributions
  • Demographic associations
  • Preferred treatments

Key takeaways

  • Free text response distributions
    • Motivation impediments
      • Vaccinated people can die is a frequent free text response. A variation can be included in the next iteration.
      • 24% free text responses are redundant.
    • Ability impediments
      • 19% free text responses are redundant
      • 10% are misforked (should be in motivation fork).
    • Treatments
      • Job needs vaccination proof is a frequent free text response and can be included in the next iteration.
      • 7% free text responses are redundant here.
  • Demographic associations
    • Most correlations are weak and not statistically significant.
    • That said, among statistically significant correlations for motivation impediments:
      • Bad side effects concerns are lower for white respondents.
      • Vaccinated people can die is more concerning to lower income respondents.
      • Freedom to choose is more a reason for higher income respondents.
    • That said, among statistically significant correlations for ability impediments:
      • Distance to vax site concerns are lower for females, and higher for more educated respondents.
      • Travel cost concerns are correlated with being politically conservative.
      • Being unable to get off work is correlated with being white.
  • Preferred treatments:
    • Nothing would help is the most common treatment response for all frequent motivation free text impediments.
    • Among the second most favorable treatments:
      • More transparency helps respondents concerned about bad side effects, not trusting pharma, personal health reasons, freedom to choose, as well as the ability impediment of vaccination site being too far away.
      • Family support is important for people thinking vaccinated people can die from it.

Free text response distributions

Notes:

  1. Free text response categories that already exist in our chatbot are highlighted in blue.
  2. Redundant responses and simple yes/no responses are excluded from these charts.

Motivation impediments

Question: Are there other reasons why you might not want the vaccine?

Proportion of redundant responses: 24%

is_redundant percent
FALSE 0.757874
TRUE 0.242126


Ability impediments

Question: Are there other reasons it might be hard to get the vaccine?

Proportion of redundant responses: 19%

is_redundant percent
FALSE 0.8058252
TRUE 0.1941748

Proportion of misforked responses: 10%

is_misforked percent
FALSE 0.9029126
TRUE 0.0970874


Treatments

Question: What would best help you to get vaccinated?

Proportion of redundant responses: 7.3%

is_redundant percent
FALSE 0.9266667
TRUE 0.0733333

Demographic associations

The correlation plots here aim to understand what demographic variables and motivation/ability impediments mentioned in free text are directly correlated with each other. We want to filter out the demographic variables that are highly correlated with other demographic variables and the demographic variables that are not related to the motivation/ability impediment to minimize the number of questions respondents have to answer.

Since the correlation matrixes provided by ggcorrplot() shows the correlation coefficients between continuous variables, we mapped binary and ordinal variables to continuous variables.

Details on demographic variable encoding:

  • female: 1 if female, 0 if male
  • income: 0 if the participant is unemployed, 1 if income < R5,000, 2 if income in R5,000 – R9,999, …, 6 if income > R100,000
  • education: 1 if the participant’s education < high school, 2 if education is high school, …, 6 if education is a graduate degree
  • religiosity: 1 if the participant is not very religious, 2 if somewhat religious, 3 if very religious
  • politics: 1 if the participant is conservative, 2 if moderate, 3 if liberal
  • location: 1 if the participant lives in rural, 2 if suburban, 3 if urban,
  • white: 1 if the participant is a white or caucasian, 0 if not

Correlations that are not statistically significant are crossed out.

For top 5 motivation impediments in free text

Bad side effects:

Vaccinated people can die:

Don’t trust pharma:

Personal health reasons:

Freedom to choose:

For top 3 ability impediments in free text

Too far away:

Travel costs:

Getting off work:

Preferred treatments

For top 5 motivation impediments in free text

Bad side effects:

Vaccinated people can die:

Don’t trust pharma:

Personal health reasons:

Freedom to choose:

For top 3 ability impediments in free text

Too far away:

Travel costs:

Getting off work: