01 - survey analysis: both pilot
Rui
2025-04-15
Data Attrition
## [1] "Original dataset has 599 rows"## Condition Rows_Remaining Rows_Discarded
## 1 2. Do not consent 585 14
## 2 3. Tech_screener failed 538 47
## 3 4. Not in the US 538 0
## 4 5. Not Female 520 18
## 5 6. Use cream less than 3 times a week 444 76
## 6 7. Fail attention check 442 2
## 7 TOTAL 442 157## [1] "Percentage of records kept: 73.79 %"## [1] "Percentage of records discarded: 26.21 %"## [1] "Final filtered dataset (df_cp) dimensions: 442 rows x 84 columns"Those who didn’t do shopping task - thus do not procced to claim question
Survey - product choice
## Column Non_NA_Count Percent_Complete
## age...22 age...22 366 100.00
## use_frequency use_frequency 366 100.00
## edu edu 366 100.00
## income income 366 100.00
## employ employ 366 100.00
## purchase_frequency purchase_frequency 366 100.00
## purchase_amount purchase_amount 366 100.00
## importance importance 366 100.00
## participantId participantId 366 100.00
## race...51 race...51 201 54.92
## Mec_benchmark_1 Mec_benchmark_1 70 19.13
## Mec_study_effet_b_1 Mec_study_effet_b_1 38 10.38
## Mec_clinical_study_b_1 Mec_clinical_study_b_1 37 10.11
## disclaimer_1_b_1 disclaimer_1_b_1 32 8.74
## Mec_study_effet_a_1 Mec_study_effet_a_1 31 8.47
## disclaimer_3_b_1 disclaimer_3_b_1 31 8.47
## Mec_clinical_study_a_1 Mec_clinical_study_a_1 30 8.20
## disclaimer_2_b_1 disclaimer_2_b_1 29 7.92
## disclaimer_1_a_1 disclaimer_1_a_1 23 6.28
## disclaimer_3_a_1 disclaimer_3_a_1 23 6.28
## disclaimer_2_a_1 disclaimer_2_a_1 22 6.01The proportion that choose product 1 when product 1 is treated
## [1] "Proportions of Value = 1 with 95% Confidence Intervals:"## Column Proportion Count Total SE CI_Lower CI_Upper
## 1 Mec_benchmark_1 0.5000000 35 70 0.05976143 0.3828676 0.6171324
## 2 Mec_study_effet_b_1 0.5789474 22 38 0.08009325 0.4219646 0.7359301
## 3 Mec_clinical_study_b_1 0.7837838 29 37 0.06767705 0.6511368 0.9164308
## 4 disclaimer_1_b_1 0.7812500 25 32 0.07307925 0.6380147 0.9244853
## 5 disclaimer_2_b_1 0.8620690 25 29 0.06403288 0.7365645 0.9875734
## 6 disclaimer_3_b_1 0.8064516 25 31 0.07095828 0.6673734 0.9455298
## The proportion that choose product 2 when product 2 is treated
## [1] "Proportions of Value = 2 with 95% Confidence Intervals:"## Column Proportion Count Total SE CI_Lower CI_Upper
## 1 Mec_benchmark_1 0.5000000 35 70 0.05976143 0.3828676 0.6171324
## 2 Mec_study_effet_a_1 0.5483871 17 31 0.08938115 0.3732000 0.7235742
## 3 Mec_clinical_study_a_1 0.6666667 20 30 0.08606630 0.4979767 0.8353566
## 4 disclaimer_1_a_1 0.6956522 16 23 0.09594388 0.5076022 0.8837022
## 5 disclaimer_2_a_1 0.6363636 14 22 0.10255929 0.4353474 0.8373798
## 6 disclaimer_3_a_1 0.7391304 17 23 0.09156054 0.5596718 0.9185891
Combine both: the proportion that choose the treated product
## Column Proportion SE CI_Lower CI_Upper Count Total Label
## 1 Benchmark 0.5000000 0.05976143 0.3828676 0.6171324 35 70 Benchmark
## 2 Mec_study_effet 0.5652174 0.05967869 0.4482472 0.6821876 39 69 Study
## 3 Mec_clinical_study 0.7313433 0.05415295 0.6252035 0.8374831 49 67 Clinical Study
## 4 disclaimer_clinical 0.7454545 0.05873702 0.6303300 0.8605791 41 55 Clinical Disclaimer
## 5 disclaimer_degree 0.7647059 0.05939743 0.6482869 0.8811248 39 51 Degree Disclaimer
## 6 disclaimer_consumer 0.7777778 0.05657501 0.6668908 0.8886648 42 54 Consumer Disclaimer
Check if demographic is balanced: Yes, Balanced between control and treatment group
## Variable Test_Type Test_Statistic P_Value Is_Balanced
## t age...22 t-test -0.4406931 0.6596969 Balanced
## t1 edu t-test -0.3906553 0.6962814 Balanced
## t2 race...51 t-test -0.7362689 0.4624459 Balanced
## t3 income t-test 1.0037299 0.3161880 Balanced
## t4 employ t-test -0.3067283 0.7592256 Balanced
## t5 purchase_frequency t-test 1.0779437 0.2817729 Balanced
## t6 purchase_amount t-test 0.3599635 0.7190833 Balanced
## t7 importance t-test -0.5438597 0.5868710 Balanced##
## Overall Balance Assessment:
## Total variables tested: 8
## Balanced variables: 8
## Imbalanced variables: 0
## Percentage balanced: 100 %Claim credibility
## [1] "Mean values with 95% confidence intervals:"## Column Mean SD SE CI_Lower CI_Upper n
## 1 claim_benchmark 4.729508 1.412490 0.07383201 4.584797 4.874219 366
## 2 Original claim 4.854839 1.226040 0.15570718 4.549653 5.160025 62
## 3 Consumer perception 4.666667 1.297542 0.16751198 4.338343 4.994990 60
## 4 High price 4.783333 1.249972 0.16137066 4.467047 5.099620 60
## 5 Low price 4.903226 1.003431 0.12743592 4.653451 5.153000 62
## 6 Smaller participants 4.633333 1.275143 0.16462020 4.310678 4.955989 60
## 7 Big efficacy number 5.274194 1.381011 0.17538860 4.930432 5.617955 62
##
##
## ### Testing if the distributions of all claim ratings are different #####
## Shapiro-Wilk normality test for each claim variable:## # A tibble: 7 × 4
## Claim variable statistic p
## <fct> <chr> <dbl> <dbl>
## 1 claim_benchmark Rating 0.935 1.33e-11
## 2 Original claim Rating 0.919 5.48e- 4
## 3 Consumer perception Rating 0.938 4.42e- 3
## 4 High price Rating 0.928 1.67e- 3
## 5 Low price Rating 0.894 5.96e- 5
## 6 Smaller participants Rating 0.935 3.41e- 3
## 7 Big efficacy number Rating 0.904 1.42e- 4##
## Not all distributions are normal. Using Kruskal-Wallis test (non-parametric alternative to ANOVA).
##
## Kruskal-Wallis rank sum test
##
## data: Rating by Claim
## Kruskal-Wallis chi-squared = 10.736, df = 6, p-value = 0.09689##
##
## ### Pairwise comparisons between all claim variables #####
## Performing pairwise comparisons using appropriate tests:## Comparing 'Claim Benchmark' vs 'Original Claim' using Mann-Whitney U test: p-value = 0.5834
## Comparing 'Claim Benchmark' vs 'Consumer Perception' using Mann-Whitney U test: p-value = 0.6228
## Comparing 'Claim Benchmark' vs 'High Price' using Mann-Whitney U test: p-value = 0.9198
## Comparing 'Claim Benchmark' vs 'Low Price' using Mann-Whitney U test: p-value = 0.5597
## Comparing 'Claim Benchmark' vs 'Smaller Participants' using Mann-Whitney U test: p-value = 0.5108
## Comparing 'Claim Benchmark' vs 'Big Efficacy Number' using Mann-Whitney U test: p-value = 0.0048
## Comparing 'Original Claim' vs 'Consumer Perception' using Mann-Whitney U test: p-value = 0.3375
## Comparing 'Original Claim' vs 'High Price' using Mann-Whitney U test: p-value = 0.6803
## Comparing 'Original Claim' vs 'Low Price' using Mann-Whitney U test: p-value = 0.8864
## Comparing 'Original Claim' vs 'Smaller Participants' using Mann-Whitney U test: p-value = 0.2962
## Comparing 'Original Claim' vs 'Big Efficacy Number' using Mann-Whitney U test: p-value = 0.0595
## Comparing 'Consumer Perception' vs 'High Price' using Mann-Whitney U test: p-value = 0.6363
## Comparing 'Consumer Perception' vs 'Low Price' using Mann-Whitney U test: p-value = 0.3572
## Comparing 'Consumer Perception' vs 'Smaller Participants' using Mann-Whitney U test: p-value = 0.8755
## Comparing 'Consumer Perception' vs 'Big Efficacy Number' using Mann-Whitney U test: p-value = 0.0103
## Comparing 'High Price' vs 'Low Price' using Mann-Whitney U test: p-value = 0.6669
## Comparing 'High Price' vs 'Smaller Participants' using Mann-Whitney U test: p-value = 0.5311
## Comparing 'High Price' vs 'Big Efficacy Number' using Mann-Whitney U test: p-value = 0.0284
## Comparing 'Low Price' vs 'Smaller Participants' using Mann-Whitney U test: p-value = 0.2602
## Comparing 'Low Price' vs 'Big Efficacy Number' using Mann-Whitney U test: p-value = 0.0463
## Comparing 'Smaller Participants' vs 'Big Efficacy Number' using Mann-Whitney U test: p-value = 0.0063
##
##
## ### Summary of claim variables with significantly different distributions ##### Found 10 significant differences out of 42 comparisons (23.8%)## Significant differences found between:
## - Big Efficacy Number and Claim Benchmark (p = 0.0048)
## - Big Efficacy Number and Consumer Perception (p = 0.0103)
## - Big Efficacy Number and High Price (p = 0.0284)
## - Big Efficacy Number and Low Price (p = 0.0463)
## - Big Efficacy Number and Smaller Participants (p = 0.0063)
## - Claim Benchmark and Big Efficacy Number (p = 0.0048)
## - Consumer Perception and Big Efficacy Number (p = 0.0103)
## - High Price and Big Efficacy Number (p = 0.0284)
## - Low Price and Big Efficacy Number (p = 0.0463)
## - Smaller Participants and Big Efficacy Number (p = 0.0063)Claim credibility - subtract benchmark
## [1] "Mean values of benchmark-adjusted ratings with 95% confidence intervals:"## Column Mean SD SE CI_Lower CI_Upper n
## 1 Original claim_adj -0.11290323 0.8703653 0.1105365 -0.32955476 0.1037483 62
## 2 Consumer perception_adj 0.43333333 1.1697699 0.1510166 0.13734071 0.7293260 60
## 3 High price_adj 0.25000000 1.2020463 0.1551835 -0.05415967 0.5541597 60
## 4 Low price_adj 0.08064516 1.2051581 0.1530552 -0.21934310 0.3806334 62
## 5 Smaller participants_adj -0.45000000 0.9283665 0.1198516 -0.68490914 -0.2150909 60
## 6 Big efficacy number_adj 0.54838710 0.9864229 0.1252758 0.30284646 0.7939277 62
##
##
## ### Testing if the distributions of all benchmark-adjusted claim ratings are different #####
## Shapiro-Wilk normality test for each adjusted claim variable:## # A tibble: 6 × 4
## Claim variable statistic p
## <fct> <chr> <dbl> <dbl>
## 1 Original claim_adj Adjusted_Rating 0.869 0.00000868
## 2 Consumer perception_adj Adjusted_Rating 0.924 0.00108
## 3 High price_adj Adjusted_Rating 0.925 0.00128
## 4 Low price_adj Adjusted_Rating 0.901 0.000116
## 5 Smaller participants_adj Adjusted_Rating 0.884 0.0000351
## 6 Big efficacy number_adj Adjusted_Rating 0.900 0.000104##
## Not all adjusted distributions are normal. Using Kruskal-Wallis test (non-parametric alternative to ANOVA).
##
## Kruskal-Wallis rank sum test
##
## data: Adjusted_Rating by Claim
## Kruskal-Wallis chi-squared = 34.981, df = 5, p-value = 1.518e-06
##
##
## Kruskal-Wallis test is significant. Performing Dunn's test with Bonferroni correction:
## # A tibble: 15 × 9
## .y. group1 group2 n1 n2 statistic p p.adj p.adj.signif
## * <chr> <chr> <chr> <int> <int> <dbl> <dbl> <dbl> <chr>
## 1 Adjusted_Rating Original claim_adj Consumer perception_adj 62 60 2.80 5.18e-3 7.77e-2 ns
## 2 Adjusted_Rating Original claim_adj High price_adj 62 60 2.11 3.49e-2 5.24e-1 ns
## 3 Adjusted_Rating Original claim_adj Low price_adj 62 62 1.58 1.13e-1 1 e+0 ns
## 4 Adjusted_Rating Original claim_adj Smaller participants_adj 62 60 -1.57 1.16e-1 1 e+0 ns
## 5 Adjusted_Rating Original claim_adj Big efficacy number_adj 62 62 3.50 4.64e-4 6.96e-3 **
## 6 Adjusted_Rating Consumer perception_adj High price_adj 60 60 -0.681 4.96e-1 1 e+0 ns
## 7 Adjusted_Rating Consumer perception_adj Low price_adj 60 62 -1.23 2.21e-1 1 e+0 ns
## 8 Adjusted_Rating Consumer perception_adj Smaller participants_adj 60 60 -4.33 1.47e-5 2.21e-4 ***
## 9 Adjusted_Rating Consumer perception_adj Big efficacy number_adj 60 62 0.676 4.99e-1 1 e+0 ns
## 10 Adjusted_Rating High price_adj Low price_adj 60 62 -0.539 5.90e-1 1 e+0 ns
## 11 Adjusted_Rating High price_adj Smaller participants_adj 60 60 -3.65 2.60e-4 3.90e-3 **
## 12 Adjusted_Rating High price_adj Big efficacy number_adj 60 62 1.36 1.73e-1 1 e+0 ns
## 13 Adjusted_Rating Low price_adj Smaller participants_adj 62 60 -3.14 1.67e-3 2.51e-2 *
## 14 Adjusted_Rating Low price_adj Big efficacy number_adj 62 62 1.92 5.52e-2 8.28e-1 ns
## 15 Adjusted_Rating Smaller participants_adj Big efficacy number_adj 60 62 5.04 4.55e-7 6.83e-6 ****Consensus and magnitude confusion
## [1] "Mean values with 95% confidence intervals for confidence degree variables:"## Column Mean SD SE CI_Lower CI_Upper n
## 1 Con-degree-high 2.901099 1.0358752 0.07678418 2.750602 3.051596 182
## 2 Con-degree-low 2.603261 0.8621933 0.06356173 2.478680 2.727842 184
Product choose reason (for all participants)
1 - I have used this product before and liked it. 2 - The packaging/design attracted me. 3 - It was a product I already intended to buy. 4 - The product claims appealed to me. 5 - The price seemed reasonable. 6 - Other (please specify)
Product choose reason (for selected participants)
1 - I have used this product before and liked it. 2 - The packaging/design attracted me. 3 - It was a product I already intended to buy. 4 - The product claims appealed to me. 5 - The price seemed reasonable. 6 - Other (please specify)
Other reasons are mostly: Brand, good reviews, ingredients
Did they search for info online?
