Treatment Analysis

Takeaway

By Version

By Treatment

Funnel & Engagement Analysis

Funnel Analysis

Funnel Dropoff Summary

Funnel Dropoff by Fork Path Analysis

Variation of Cost-per-Action Analysis

Engagement Analysis

Variation in enjoyment level

Variation in comfort level

Covariate Analysis

Goals for this section:

  • Understanding the variation in impediments to vaccination faced by our respondents
  • Investigate if these impediments have strong associations with demographic characteristics

Vaccination status

We start off with a plot demonstrating the distribution of vaccinated and unvaccinated participants in this pilot:

Impediments

Overview

We asked participants about whether they have the motivation to get the COVID-19 vaccine and whether they have the ability to get the vaccine for both vaccinated and unvaccinated people. We then fork them into 8 different segments based on the vaccination status, motivation to get the vaccine, and ability to get the vaccine. We obtain the distribution below:

Pilot 4
## Error: `n()` must only be used inside dplyr verbs.
Pilot 4B
## Error: `n()` must only be used inside dplyr verbs.

Takeaways

Analysis for each impediment

Let’s investigate each impediment (motivation & ability) in detail, and see the distributions of the reasons why they have such impediments.

  • Main Level: demonstrates the distribution of the reasons that chosen from the options we provided to them to explain why participants have motivation/ability impediments
  • Sub-level: analyze the explanations why participants choose the specific reason in the Main level

Takeaways

Motivational (Main Level)

We asked: What’s the main reason you don’t want to be vaccinated?

Provided options:

  • there’s no clear benefit (benefit)
  • it’s too risky (risk)
  • against my beliefs (belief)

The distribution of the answers demonstrates below:

## $x
## [1] "Motivational Impediment"
## 
## $y
## [1] "count"
## 
## attr(,"class")
## [1] "labels"
Motivational (Sub-Level)
Benefit

We asked: is there a main reason why you think there isn’t a benefit?

Provided options:

  • covid not dangerous
  • unlikely to get sick
  • had covid already
  • i can recover
  • other reason(s)

The distribution of the answers demonstrates below:

Risk

We asked: is there a main reason why you think there is risk?

Provided options:

  • don’t trust pharma
  • vaccines don’t work
  • bad side effects
  • needles/pain
  • other reason(s)

The distribution of the answers demonstrates below:

Belief

We asked: is there a main reason why against your belief?

Provided options:

  • don’t trust gov
  • religious reasons
  • freedom to choose
  • other reason(s)

The distribution of the answers demonstrates below:

Ability (Main Level)

We asked: What’s the main difficulty of getting vaccinated?

Provided options:

  • no vax available (availability)
  • lack of time (time)
  • lack of money (money)

The distribution of the answers demonstrates below:

Ability (Sub-Level)
Availability

We asked: is there a main reason why there isn’t availability?

Provided options:

  • too far away
  • no vaccines left
  • other reason(s)

Time

We asked: is there a main reason why there isn’t time?

Provided options:

  • no time to research
  • work
  • family
  • keep forgetting
  • other reason(s)

Money

We asked: is there a main reason why there isn’t money?

Provided options:

  • travel costs
  • no insurance
  • no cost

Demographics

We mapped binary and ordinal demographic variables to continuous variables (with value 0, 1, 2,…).

How we did the mapping:

  • ability: 1 if the participant has the ability to get vax, 0 if not
  • female: 1 if female, 0 if male
  • country: 1 if live in South Africa, 0 if not
  • income: 0 if the participant is unemployed, 1 if income < R5,000, 2 if income in R5,000 – R9,999, …, 6 if income > R100,000
  • education: 1 if the participant’s education < high school, 2 if education is high school, …, 6 if education is a graduate degree
  • religiosity: 1 if the participant is not very religious, 2 if somewhat religious, 3 if very religious
  • politics: 1 if the participant is conservative, 2 if moderate, 3 if liberal
  • location: 1 if the participant lives in rural, 2 if suburban, 3 if urban,
  • white: 1 if the participant is a white or caucasian, 0 if not

General Analysis

Free text elaboration Distribution

Density Plot (Overall)

Density Plot (By Vaccination Group)

Statistics (Overall)

Statistics (By Vaccination Group)

##    version   vax_status    N Missing  Mean     SD Min   Q1 Median   Q3 Max
## 1  Pilot 4   Vaccinated   92     268 47.22  45.74   2 15.5   33.5 62.5 274
## 2  Pilot 4 Unvaccinated  154     494 53.31 101.93   1 11.0   20.5 51.0 799
## 3 Pilot 4B   Vaccinated  910    3890 41.73  44.81   2 12.0   28.0 55.0 333
## 4 Pilot 4B Unvaccinated 1566    6306 35.96  47.81   1  9.0   20.0 43.0 430

Regression Analysis (Overall)

## 
## Call:
## lm(formula = nchar ~ version, data = free_text_combined)
## 
## Residuals:
##    Min     1Q Median     3Q    Max 
## -50.03 -29.08 -17.08  10.92 747.97 
## 
## Coefficients:
##                 Estimate Std. Error t value Pr(>|t|)    
## (Intercept)       51.028      3.281  15.552  < 2e-16 ***
## versionPilot 4B  -12.949      3.440  -3.764 0.000171 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 51.46 on 2720 degrees of freedom
##   (10958 observations deleted due to missingness)
## Multiple R-squared:  0.005182,   Adjusted R-squared:  0.004816 
## F-statistic: 14.17 on 1 and 2720 DF,  p-value: 0.0001708

Regresssion Analysis (By Vacccintaion Group)

## 
## Call:
## lm(formula = nchar ~ version + vax_status + version * vax_status, 
##     data = free_text_combined)
## 
## Residuals:
##    Min     1Q Median     3Q    Max 
## -52.31 -28.96 -15.96  10.04 745.69 
## 
## Coefficients:
##                                      Estimate Std. Error t value Pr(>|t|)    
## (Intercept)                            53.305      4.142  12.868  < 2e-16 ***
## versionPilot 4B                       -17.344      4.341  -3.995 6.63e-05 ***
## vax_statusVaccinated                   -6.088      6.774  -0.899   0.3689    
## versionPilot 4B:vax_statusVaccinated   11.852      7.104   1.668   0.0954 .  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 51.4 on 2718 degrees of freedom
##   (10958 observations deleted due to missingness)
## Multiple R-squared:  0.008117,   Adjusted R-squared:  0.007023 
## F-statistic: 7.415 on 3 and 2718 DF,  p-value: 6.05e-05

Free text analyses comparison on best treatment

Density Plot (Overall)

Density Plot (By Vaccination Group)

Statistics (Overall)

Statistics (By Vaccination Group)

##    version   vax_status   N Missing  Mean    SD Min Q1 Median Q3 Max
## 1  Pilot 4   Vaccinated  14       1 45.36 53.76   3 10   25.5 56 186
## 2  Pilot 4 Unvaccinated   9      18 38.89 36.60   4  6   24.0 64 103
## 3 Pilot 4B   Vaccinated 117      33 42.90 41.26   2 12   26.0 67 186
## 4 Pilot 4B Unvaccinated 148      98 28.51 37.55   1  9   17.0 35 264

Regression Analysis (Overall)

## 
## Call:
## lm(formula = nchar ~ version, data = free_text_combined)
## 
## Residuals:
##    Min     1Q Median     3Q    Max 
## -39.83 -25.86 -14.86  12.42 229.14 
## 
## Coefficients:
##                 Estimate Std. Error t value Pr(>|t|)    
## (Intercept)       42.826      8.424   5.084  6.7e-07 ***
## versionPilot 4B   -7.962      8.782  -0.907    0.365    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 40.4 on 286 degrees of freedom
##   (150 observations deleted due to missingness)
## Multiple R-squared:  0.002866,   Adjusted R-squared:  -0.0006208 
## F-statistic: 0.8219 on 1 and 286 DF,  p-value: 0.3654

Regresssion Analysis (By Vacccintaion Group)

## 
## Call:
## lm(formula = nchar ~ version + vax_status + version * vax_status, 
##     data = free_text_combined)
## 
## Residuals:
##    Min     1Q Median     3Q    Max 
## -42.36 -23.51 -13.13  11.49 235.49 
## 
## Coefficients:
##                                      Estimate Std. Error t value Pr(>|t|)   
## (Intercept)                            38.889     13.314   2.921  0.00377 **
## versionPilot 4B                       -10.375     13.713  -0.757  0.44990   
## vax_statusVaccinated                    6.468     17.065   0.379  0.70494   
## versionPilot 4B:vax_statusVaccinated    7.916     17.766   0.446  0.65625   
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 39.94 on 284 degrees of freedom
##   (150 observations deleted due to missingness)
## Multiple R-squared:  0.03223,    Adjusted R-squared:  0.02201 
## F-statistic: 3.153 on 3 and 284 DF,  p-value: 0.02531

Free text analyses comparison on Impediment

Density Plot (Overall)

Density Plot (By Vaccination Group)

Statistics (Overall)

Statistics (By Vaccination Group)

##    version   vax_status   N Missing  Mean     SD Min Q1 Median  Q3 Max
## 1  Pilot 4   Vaccinated  14     136 42.21  26.19   2 30   35.5  59  96
## 2  Pilot 4 Unvaccinated  58     212 99.29 150.47   1 27   51.0 104 799
## 3 Pilot 4B   Vaccinated 142    1508 50.27  44.02   2 23   37.5  65 264
## 4 Pilot 4B Unvaccinated 530    2176 58.83  57.58   1 21   40.5  76 430

Regression Analysis (Overall)

## 
## Call:
## lm(formula = nchar ~ version, data = free_text_combined)
## 
## Residuals:
##    Min     1Q Median     3Q    Max 
## -87.19 -38.02 -18.61  15.18 710.81 
## 
## Coefficients:
##                 Estimate Std. Error t value Pr(>|t|)    
## (Intercept)       88.194      7.944  11.102  < 2e-16 ***
## versionPilot 4B  -31.172      8.359  -3.729 0.000207 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 67.41 on 742 degrees of freedom
##   (4032 observations deleted due to missingness)
## Multiple R-squared:  0.0184, Adjusted R-squared:  0.01708 
## F-statistic: 13.91 on 1 and 742 DF,  p-value: 0.0002067

Regresssion Analysis (By Vacccintaion Group)

## 
## Call:
## lm(formula = nchar ~ version + vax_status + version * vax_status, 
##     data = free_text_combined)
## 
## Residuals:
##    Min     1Q Median     3Q    Max 
## -98.29 -37.83 -18.05  15.73 699.71 
## 
## Coefficients:
##                                      Estimate Std. Error t value Pr(>|t|)    
## (Intercept)                            99.293      8.804  11.279  < 2e-16 ***
## versionPilot 4B                       -40.461      9.273  -4.363 1.46e-05 ***
## vax_statusVaccinated                  -57.079     19.965  -2.859  0.00437 ** 
## versionPilot 4B:vax_statusVaccinated   48.514     20.946   2.316  0.02082 *  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 67.05 on 740 degrees of freedom
##   (4032 observations deleted due to missingness)
## Multiple R-squared:  0.03149,    Adjusted R-squared:  0.02756 
## F-statistic:  8.02 on 3 and 740 DF,  p-value: 2.897e-05

Ads Analysis

Demographics and Ads

Age

Age and Impression count

Age and Conversation Started count

Gender

Gender and Impression count

Gender and Conversation Started count

Region

Region and Impression count

Region and Conversation Started count

Ad Performance Summary

Side-by-Side Chart on Key Metrics

Extreme VS Moderate

Compare by Image

Unnecessary vs Unsafe vs Inaccessible

Survey vs Control vs Airtime