1 Research questions

RQ1: How does FDA approval influence app choice for tracking metabolic health, compared to other app attributes like privacy, invasiveness, health benefits, and burden?

RQ2: Are individuals with stronger (vs. weaker) FDA legitimacy perceptions more likely to choose an FDA-approved app, holding other app attributes constant?

2 Conjoint experiment set up

Trulli

2.1 Conjoint attributes & design

FDA Approval:

  • Yes: Has been regulated and approved by the FDA (Food & Drug Administration).
  • No: Has not been assessed by the FDA (Food & Drug Administration)

Health Benefit:

  • High: Regular users see a large benefit in metabolic health.
  • Low: Regular users see a small benefit in metabolic health.

Privacy:

  • High: Very unlikely to sell data to third-party companies without additional consent.
  • Low: Likely to sell data to third-party companies without additional consent.

Invasiveness:

  • High: Requires wearing a continuous glucose monitor (CGM), with a microneedle in the arm.
  • Low: Does not require wearing a continuous glucose monitor (CGM), with a microneedle in the arm.

Burden:

  • High: Requires manually tracking meals in a food diary.
  • Low: Requires taking photos of meals to automatically track foods

Uniform attribute random sampling

  • 16 trials; randomly sampled 2 of the 32 possible combinations
  • Random attribute ordering per participant (1-5 per participant but not per choice)
  • Forced choice (logistic outcome)

2.2 Legitimacy items

Impartiality:

  • How biased do you think the FDA is when regulating digital health apps? (1-9)
  • How much do you think the FDA considers perspectives from all parts of society when regulating digital health apps? (1-9)

Benevolence:

  • How much do you think the FDA acts to protect the health and well-being of digital health app users when regulating health apps? (1-9)
  • How much do you think the FDA wants to protect the health and well-being of digital health app users when regulating health apps? (1-9)

Appropriate process:

  • How transparent do you think the FDA is in its decision-making process on regulating digital health apps? (1-9)
  • How appropriate do you think FDA’s decision making process is for regulating digital health apps? (1-9)
dbu <- read.csv("~/Documents/digitalbiom.csv", stringsAsFactors = FALSE) %>%
  select("Q_leg_benev_all_1",
         "Q_leg_benev_unbias_1",
         "Q_leg_process_approp_1",
         "Q_leg_process_transp_1",
         "Q_leg_protect1_1",
      "Q_leg_protect2_1", "ResponseId")  %>%
  slice(-c(1, 2))  # This removes the first two rows

dbu$benev1 = as.numeric(dbu$Q_leg_benev_all_1)
dbu$benev2 = as.numeric(dbu$Q_leg_benev_unbias_1)
dbu$process1 = as.numeric(dbu$Q_leg_process_approp_1)
dbu$process2 = as.numeric(dbu$Q_leg_process_transp_1)
dbu$protect1 = as.numeric(dbu$Q_leg_protect1_1)
dbu$protect2 = as.numeric(dbu$Q_leg_protect2_1)

###reverse code
dbu$benev2_r <- 9 - dbu$benev2

dbu$legitimacy= (dbu$benev1 + dbu$benev2_r + dbu$process1 + dbu$process2 + dbu$protect1  +dbu$protect2)/6

items <- dbu %>% dplyr::select(benev1, benev2_r,  process1,  process2, protect1,  protect2)

alpha <- psych::alpha(items)

# Create a kable table from the alpha results
#kable(alpha_table)
# Get summary statistics for legitimacy
legitimacy_summary <- summary(dbu$legitimacy)

# Create a kable table from the alpha results and summary statistics
alpha_table <- data.frame(
  Measure = c("Cronbach's Alpha", "Number of Items", "Min Legitimacy", "1st Qu.", 
              "Median", "Mean", "3rd Qu", "Max"),
  Value = c(round(alpha$total$raw_alpha, digits=2), 
            "6", 
            round(legitimacy_summary[1], 2), 
            round(legitimacy_summary[2], 2), 
            round(legitimacy_summary[3], 2), 
            round(legitimacy_summary[4], 2), 
            round(legitimacy_summary[5], 2), 
            round(legitimacy_summary[6], 2))
)

kable(alpha_table)
Measure Value
Cronbach’s Alpha 0.92
Number of Items 6
Min Legitimacy 0.67
1st Qu. 4.42
Median 5.67
Mean 5.62
3rd Qu 7
Max 9 
hist(dbu$legitimacy, main = "Legitimacy Histogram", xlab = "Legitimacy", col = "lightblue", border = "black")
# Calculate the median
median_value <- median(dbu$legitimacy, na.rm=T)
# Add a dashed line at the median
abline(v = median_value, col = "blue", lwd = 2, lty = 2)  # lty = 2 creates a dashed line

3 Results

3.1 RQ1

RQ1 How does FDA approval influence app choice for tracking metabolic health, compared to other app attributes like privacy, invasiveness, health benefits, and burden?

sample size: n=32 participants, 512 trials total

  • Holding other attirbutes constant, on average, respondents were more likely to select an app that is FDA approved compared to one that is not FDA approved by 16.11% points (large CI of3.89% to 28.3%).

  • On average, a randomly selected digital app in this experimental design would earn 16.11 points (large CI of3.89% to 28.3%) more of the total vote share if it was FDA-approved.

df <- read.csv("~/Documents/Conjoint/final_combined_df.csv", stringsAsFactors = FALSE)

df$choice_n <- as.numeric(ifelse(df$choice == "yes", 1, 0))  # Convert to binary

df$FDA_Approval <- factor(df$FDA_Approval, levels = c("FDA not approved", "FDA approved"))
df$Burden <- factor(df$Burden, levels = c("low burden", "high burden"))
df$Privacy <- factor(df$Privacy, levels = c("no privacy", "yes privacy"))
df$Invasive <- factor(df$Invasive, levels = c("CGM not required", "CGM required"))
df$Benefit <- factor(df$Benefit, levels = c("small benefit", "large benefit"))

model1 <- amce(df, choice_n ~ FDA_Approval + Burden + Privacy + Invasive + Benefit,id = ~ ResponseId)

#model1 %>% as_tibble() 
plot(model1 ) +
  guides(color = "none") +
#  theme_nice() +
  labs(title = "AMCEs")

model_tibble <- model1 %>% as_tibble() %>%
  filter(!is.na(lower))

# Display the resulting tibble
kable(model_tibble)
outcome statistic feature level estimate std.error z p lower upper
choice_n amce FDA_Approval FDA approved 0.1610600 0.0623499 2.583165 0.0097898 0.0388565 0.2832636
choice_n amce Burden high burden -0.0568934 0.0491801 -1.156837 0.2473388 -0.1532847 0.0394979
choice_n amce Privacy yes privacy 0.3922690 0.0694849 5.645382 0.0000000 0.2560810 0.5284569
choice_n amce Invasive CGM required -0.2166866 0.0751536 -2.883248 0.0039360 -0.3639850 -0.0693882
choice_n amce Benefit large benefit 0.1496457 0.0511700 2.924480 0.0034503 0.0493543 0.2499372

3.2 RQ2

sample size: n=32 participants, 512 trials total

RQ2: Are individuals with stronger (vs. weaker ) FDA legitimacy perceptions more likely to choose an FDA-approved app, holding other app attributes constant?

  • Participants with stronger FDA legitimacy perceptions are estimated to be 24% more likely to choose an FDA-approved app, with a confidence interval ranging from 0.00 to 0.48, holding all other attributes constant.

  • Participants with weaker FDA legitimacy perceptions have an estimate of 8% likelihood of choosing and FDA-approved app, with a confidence interval of -0.04 to 0.20. This interval includes zero, holding all other attributes constant.

**Higher/lower is defined as median split.

##### interaction

df_new = merge(df, dbu, by= 'ResponseId')

# Q1 <- quantile(df_new$legitimacy, 0.25, na.rm = TRUE)
# Q3 <- quantile(df_new$legitimacy, 0.75, na.rm = TRUE)
# 
# # Create the split variable based on interquartiles
# df_new$legitimacy_split <- cut(df_new$legitimacy,
#                                 breaks = c(-Inf, Q1, Q3, Inf),
#                                 labels = c("Low", "Medium", "High"),
#                                 right = FALSE)

# Convert to factor
#df_new$legitimacy_split <- as.factor(df_new$legitimacy_split)
median_legitimacy <- median(df_new$legitimacy, na.rm = TRUE)
# # Create the median split variable
df_new$legitimacy_split <- ifelse(df_new$legitimacy > median_legitimacy, "High", "Low")
# # Convert to factor
df_new$legitimacy_split <- as.factor(df_new$legitimacy_split)

model_int1 <- cregg::cj(df_new, choice_n ~ FDA_Approval + Burden
               + Privacy + Invasive + Benefit,id = ~ ResponseId,
               estimate = "amce",
               by = ~ legitimacy_split)

model_int_tibble <- model_int1 %>% as_tibble() %>%
  filter(!is.na(lower))

#kable(model_int_tibble)

# Display the plot
plot_int <- plot(model_int1) +
  facet_wrap(~ BY) +  # Create a separate panel for each legitimacy level
  scale_color_manual(values = c("blue", "red", "yellow",
                               "black", "green"), name = "Legitimacy Level") +
  labs(title = "Interaction Effect by Legitimacy Level",
       x = "Attributes",
      y = "AMCE Estimate") +
  theme_minimal() +
  theme(legend.position = "bottom")

 estimate_high_fda <- model_int_tibble %>%
   filter(feature == "FDA_Approval") %>%
   filter(level == 'FDA approved') %>%
   filter(legitimacy_split == "High")
# print(estimate_high_fda$estimate)
 
 estimate_low_fda <- model_int_tibble %>%
   filter(feature == "FDA_Approval") %>%
   filter(level == 'FDA approved') %>%
   filter(legitimacy_split == "Low")

# Extract estimates and confidence intervals
high_fda <- c(
  estimate = estimate_high_fda$estimate,
  lower_ci = estimate_high_fda$lower,
  upper_ci = estimate_high_fda$upper
)

low_fda <- c(
  estimate = estimate_low_fda$estimate,
  lower_ci = estimate_low_fda$lower,
  upper_ci = estimate_low_fda$upper
)

# Create a data frame for the kable table
results_table <- data.frame(
  Legitimacy = c("High FDA Legitimacy (median + above)", "Low FDA Legitimacy (below median)"),
  Estimate = c(high_fda['estimate'], low_fda['estimate']),
  Lower_CI = c(high_fda['lower_ci'], low_fda['lower_ci']),
  Upper_CI = c(high_fda['upper_ci'], low_fda['upper_ci'])
)

# Print the kable table
kable(results_table, digits = 2, caption = "Estimates and Confidence Intervals for FDA Approval by Legitimacy")
Estimates and Confidence Intervals for FDA Approval by Legitimacy
Legitimacy Estimate Lower_CI Upper_CI
High FDA Legitimacy (median + above) 0.24 0.00 0.48
Low FDA Legitimacy (below median) 0.08 -0.04 0.20 
print(plot_int)

kable(model_int_tibble)
BY outcome statistic feature level estimate std.error z p lower upper legitimacy_split
High choice_n amce FDA_Approval FDA approved 0.2418917 0.1231938 1.9635062 0.0495874 0.0004364 0.4833470 High
High choice_n amce Burden high burden -0.0259779 0.0697921 -0.3722179 0.7097306 -0.1627679 0.1108121 High
High choice_n amce Privacy yes privacy 0.5142546 0.1201851 4.2788541 0.0000188 0.2786961 0.7498131 High
High choice_n amce Invasive CGM required -0.1324961 0.0982719 -1.3482599 0.1775748 -0.3251055 0.0601133 High
High choice_n amce Benefit large benefit 0.0618352 0.0783813 0.7889029 0.4301688 -0.0917892 0.2154597 High
Low choice_n amce FDA_Approval FDA approved 0.0836409 0.0612282 1.3660519 0.1719227 -0.0363642 0.2036459 Low
Low choice_n amce Burden high burden -0.0432917 0.0673382 -0.6428995 0.5202893 -0.1752722 0.0886888 Low
Low choice_n amce Privacy yes privacy 0.3504647 0.0740964 4.7298498 0.0000022 0.2052385 0.4956908 Low
Low choice_n amce Invasive CGM required -0.2834993 0.1113972 -2.5449415 0.0109296 -0.5018338 -0.0651648 Low
Low choice_n amce Benefit large benefit 0.2285415 0.0675143 3.3850808 0.0007116 0.0962158 0.3608672 Low

4 Power Analysis

Based on these preliminary findings:

  • H1.To detect an AMCE of 0.16, with a power of 0.95, approx. 495 participants are needed.

  • H2.To detect an AMCE of 0.08 (smallest effect), with a power of 0.95, approx. 2017 participants are needed.

library(cjpowR)

rq1pwr = cjpowr_amce(amce = 0.16, power = 0.95, levels = 2)
#round(rq1pwr$n, digits =0)  #495

rq2pwr = cjpowr_amce(amce = 0.08, power = 0.95, levels = 2)
#round(rq2pwr$n, digits =0)  #2017