Influencer Ads Summary Analysis
Zelin (James) Li
2024-04-01
Data
Preprocessing
Result 1: Original Study
- original study - contrary to our hypothesis, finds influencer ads are less persuasive
Result 2: Replicate with Larger Sample
- study you just ran - replicates this finding with larger sample, looks at individual diff moderation etc
Moderation Analysis (Only for Aggregate Outcome)
Aggregate
## OLS estimation, Dep. Var.: Aggregate
## Observations: 4,920
## Fixed-effects: ResponseId: 984, Brand: 10
## Standard-errors: Clustered (ResponseId)
## Estimate Std. Error t value Pr(>|t|)
## Treatment 0.024765 0.114192 0.216871 0.828354
## Treatment:TikTokYes 0.097155 0.053589 1.812959 0.070143 .
## Treatment:InstagramYes -0.076421 0.047144 -1.621018 0.105334
## Treatment:SnapchatYes 0.043856 0.050198 0.873668 0.382513
## Treatment:YouTubeYes -0.068990 0.046790 -1.474445 0.140682
## Treatment:buys_off_socialYes 0.071320 0.043422 1.642464 0.100814
## Treatment:social_usageHigh -0.079431 0.049561 -1.602683 0.109326
## Treatment:youngYes -0.007356 0.067987 -0.108203 0.913857
## Treatment:age -0.000198 0.001677 -0.118204 0.905931
## ... 8 variables were removed because of collinearity (TikTokYes, InstagramYes and 6 others [full set in $collin.var])
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## RMSE: 0.547325 Adj. R2: 0.593548
## Within R2: 0.004689Feelings
## OLS estimation, Dep. Var.: Feelings
## Observations: 4,920
## Fixed-effects: ResponseId: 984, Brand: 10
## Standard-errors: Clustered (ResponseId)
## Estimate Std. Error t value Pr(>|t|)
## Treatment 0.129174 0.122940 1.050712 0.293649
## Treatment:TikTokYes 0.093783 0.059107 1.586676 0.112908
## Treatment:InstagramYes -0.105515 0.050325 -2.096685 0.036277 *
## Treatment:SnapchatYes 0.032464 0.054900 0.591339 0.554429
## Treatment:YouTubeYes -0.080173 0.051189 -1.566217 0.117620
## Treatment:buys_off_socialYes 0.095205 0.046825 2.033206 0.042300 *
## Treatment:social_usageHigh -0.109547 0.056034 -1.955025 0.050863 .
## Treatment:youngYes -0.059381 0.079027 -0.751402 0.452590
## Treatment:age -0.001614 0.001822 -0.885425 0.376144
## ... 8 variables were removed because of collinearity (TikTokYes, InstagramYes and 6 others [full set in $collin.var])
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## RMSE: 0.597264 Adj. R2: 0.511577
## Within R2: 0.00559Trustworthy
## OLS estimation, Dep. Var.: Trustworthy
## Observations: 4,920
## Fixed-effects: ResponseId: 984, Brand: 10
## Standard-errors: Clustered (ResponseId)
## Estimate Std. Error t value Pr(>|t|)
## Treatment 0.044018 0.125871 0.349707 0.726633
## Treatment:TikTokYes 0.101130 0.060245 1.678637 0.093541 .
## Treatment:InstagramYes -0.063705 0.053351 -1.194085 0.232733
## Treatment:SnapchatYes 0.070215 0.060246 1.165464 0.244114
## Treatment:YouTubeYes -0.073798 0.052570 -1.403790 0.160697
## Treatment:buys_off_socialYes 0.069323 0.047545 1.458049 0.145146
## Treatment:social_usageHigh -0.070994 0.054503 -1.302570 0.193027
## Treatment:youngYes -0.118686 0.080043 -1.482785 0.138452
## Treatment:age -0.000730 0.001864 -0.391523 0.695495
## ... 8 variables were removed because of collinearity (TikTokYes, InstagramYes and 6 others [full set in $collin.var])
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## RMSE: 0.598454 Adj. R2: 0.496587
## Within R2: 0.004808PurchaseIntention
## OLS estimation, Dep. Var.: Purchase
## Observations: 4,920
## Fixed-effects: ResponseId: 984, Brand: 10
## Standard-errors: Clustered (ResponseId)
## Estimate Std. Error t value Pr(>|t|)
## Treatment -0.098898 0.163177 -0.606077 0.544603
## Treatment:TikTokYes 0.096551 0.071993 1.341117 0.180192
## Treatment:InstagramYes -0.060043 0.063889 -0.939810 0.347546
## Treatment:SnapchatYes 0.028889 0.070858 0.407695 0.683586
## Treatment:YouTubeYes -0.052998 0.064906 -0.816540 0.414389
## Treatment:buys_off_socialYes 0.049431 0.060238 0.820595 0.412076
## Treatment:social_usageHigh -0.057752 0.067517 -0.855365 0.392558
## Treatment:youngYes 0.155998 0.092398 1.688331 0.091665 .
## Treatment:age 0.001749 0.002339 0.747814 0.454751
## ... 8 variables were removed because of collinearity (TikTokYes, InstagramYes and 6 others [full set in $collin.var])
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## RMSE: 0.753257 Adj. R2: 0.586943
## Within R2: 0.002434PCA Analysis on Moderators
## Importance of components:
## PC1 PC2 PC3 PC4 PC5 PC6 PC7
## Standard deviation 1.8135 1.0081 0.9241 0.87381 0.8114 0.7600 0.70668
## Proportion of Variance 0.4111 0.1270 0.1067 0.09544 0.0823 0.0722 0.06242
## Cumulative Proportion 0.4111 0.5381 0.6449 0.74032 0.8226 0.8948 0.95724
## PC8
## Standard deviation 0.58489
## Proportion of Variance 0.04276
## Cumulative Proportion 1.00000## OLS estimation, Dep. Var.: Aggregate
## Observations: 4,920
## Fixed-effects: ResponseId: 984, Brand: 10
## Standard-errors: Clustered (ResponseId)
## Estimate Std. Error t value Pr(>|t|)
## Treatment -0.043009 0.019235 -2.235949 0.025579 *
## Treatment:PC1 0.005296 0.010508 0.503982 0.614387
## Treatment:PC2 -0.014620 0.018819 -0.776901 0.437404
## Treatment:PC3 0.042292 0.020657 2.047312 0.040893 *
## Treatment:PC4 -0.010881 0.022430 -0.485114 0.627704
## Treatment:PC5 0.043072 0.024865 1.732263 0.083540 .
## Treatment:PC6 -0.057985 0.026718 -2.170227 0.030229 *
## Treatment:PC7 -0.010752 0.027385 -0.392630 0.694678
## Treatment:PC8 -0.012252 0.033010 -0.371158 0.710600
## ... 8 variables were removed because of collinearity (PC1, PC2 and 6 others [full set in $collin.var])
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## RMSE: 0.547325 Adj. R2: 0.593548
## Within R2: 0.004689Result 3: Mediation Question Treatment Effect
- additional qs study - shows potentially relevant dimensions on which influencer v traditional ads differ - finds no diff in persuasiveness for influencer v traditional (prob bc asking other qs messed up those outcomes) so can’t do normal mediation analysis
Mechanism Question Treatment Effect
Result 4: Mechanism Analysis
- instead we do item-level mediation analysis where for each of the 20 ads, we calculate persuasiveness (using agg outcome pooling across data from 1 and 2 above) and then do meta-regression predicting persuasiveness using influencer dummy (which should find sig neg effect), and then do that again also including avg value of different additional qs from 3 to see which ones reduce influencer effect size (do you see what i mean?)
##
## Multivariate Meta-Analysis Model (k = 10; method: REML)
##
## Variance Components:
##
## estim sqrt nlvls fixed factor
## sigma^2 0.0238 0.1544 10 no term
##
## Test for Heterogeneity:
## Q(df = 9) = 82.7118, p-val < .0001
##
## Model Results:
##
## estimate se zval pval ci.lb ci.ub
## -0.0501 0.0517 -0.9692 0.3324 -0.1514 0.0512
##
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1Belief
##
## Multivariate Meta-Analysis Model (k = 10; method: REML)
##
## Variance Components:
##
## estim sqrt nlvls fixed factor
## sigma^2 0.0142 0.1190 10 no term
##
## Test for Residual Heterogeneity:
## QE(df = 8) = 47.2366, p-val < .0001
##
## Test of Moderators (coefficient 2):
## QM(df = 1) = 6.1000, p-val = 0.0135
##
## Model Results:
##
## estimate se zval pval ci.lb ci.ub
## intrcpt -0.0960 0.0453 -2.1201 0.0340 -0.1848 -0.0073 *
## fit_out_belief$estimate 0.4934 0.1998 2.4698 0.0135 0.1018 0.8849 *
##
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1Connectedness
##
## Multivariate Meta-Analysis Model (k = 10; method: REML)
##
## Variance Components:
##
## estim sqrt nlvls fixed factor
## sigma^2 0.0078 0.0885 10 no term
##
## Test for Residual Heterogeneity:
## QE(df = 8) = 29.6266, p-val = 0.0002
##
## Test of Moderators (coefficient 2):
## QM(df = 1) = 14.3944, p-val = 0.0001
##
## Model Results:
##
## estimate se zval pval ci.lb
## intrcpt -0.0558 0.0327 -1.7031 0.0885 -0.1200
## fit_out_connectedness$estimate 0.8418 0.2219 3.7940 0.0001 0.4070
## ci.ub
## intrcpt 0.0084 .
## fit_out_connectedness$estimate 1.2767 ***
##
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1Informative
##
## Multivariate Meta-Analysis Model (k = 10; method: REML)
##
## Variance Components:
##
## estim sqrt nlvls fixed factor
## sigma^2 0.0115 0.1075 10 no term
##
## Test for Residual Heterogeneity:
## QE(df = 8) = 40.4665, p-val < .0001
##
## Test of Moderators (coefficient 2):
## QM(df = 1) = 8.6729, p-val = 0.0032
##
## Model Results:
##
## estimate se zval pval ci.lb
## intrcpt -0.0792 0.0393 -2.0178 0.0436 -0.1561
## fit_out_informative$estimate 0.4043 0.1373 2.9450 0.0032 0.1352
## ci.ub
## intrcpt -0.0023 *
## fit_out_informative$estimate 0.6733 **
##
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1Properties
##
## Multivariate Meta-Analysis Model (k = 10; method: REML)
##
## Variance Components:
##
## estim sqrt nlvls fixed factor
## sigma^2 0.0079 0.0891 10 no term
##
## Test for Residual Heterogeneity:
## QE(df = 8) = 30.3853, p-val = 0.0002
##
## Test of Moderators (coefficient 2):
## QM(df = 1) = 14.2394, p-val = 0.0002
##
## Model Results:
##
## estimate se zval pval ci.lb
## intrcpt -0.0913 0.0347 -2.6355 0.0084 -0.1593
## fit_out_properties$estimate 0.6132 0.1625 3.7735 0.0002 0.2947
## ci.ub
## intrcpt -0.0234 **
## fit_out_properties$estimate 0.9316 ***
##
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1Quality
##
## Multivariate Meta-Analysis Model (k = 10; method: REML)
##
## Variance Components:
##
## estim sqrt nlvls fixed factor
## sigma^2 0.0168 0.1296 10 no term
##
## Test for Residual Heterogeneity:
## QE(df = 8) = 55.3581, p-val < .0001
##
## Test of Moderators (coefficient 2):
## QM(df = 1) = 4.2456, p-val = 0.0394
##
## Model Results:
##
## estimate se zval pval ci.lb ci.ub
## intrcpt 0.0427 0.0632 0.6757 0.4992 -0.0811 0.1665
## fit_out_quality$estimate 0.7147 0.3469 2.0605 0.0394 0.0349 1.3946 *
##
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1All
##
## Multivariate Meta-Analysis Model (k = 10; method: REML)
##
## Variance Components:
##
## estim sqrt nlvls fixed factor
## sigma^2 0.0036 0.0601 10 no term
##
## Test for Residual Heterogeneity:
## QE(df = 4) = 9.0117, p-val = 0.0608
##
## Test of Moderators (coefficients 2:6):
## QM(df = 5) = 32.9435, p-val < .0001
##
## Model Results:
##
## estimate se zval pval ci.lb
## intrcpt -0.0069 0.0571 -0.1207 0.9040 -0.1189
## fit_out_quality$estimate 0.1545 0.3754 0.4117 0.6806 -0.5812
## fit_out_properties$estimate 1.3802 0.5258 2.6250 0.0087 0.3497
## fit_out_informative$estimate -0.8733 0.4993 -1.7493 0.0802 -1.8519
## fit_out_connectedness$estimate 1.4342 0.5784 2.4796 0.0132 0.3006
## fit_out_belief$estimate -0.6831 0.4304 -1.5872 0.1125 -1.5266
## ci.ub
## intrcpt 0.1051
## fit_out_quality$estimate 0.8903
## fit_out_properties$estimate 2.4108 **
## fit_out_informative$estimate 0.1052 .
## fit_out_connectedness$estimate 2.5678 *
## fit_out_belief$estimate 0.1604
##
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1Result 5: Analogus to Swiping on TikTok
1-4 shows that influencer ads are less persuasive and gives insight into why. finally, we test another potential advantage of influencer content - do people prefer to watch it? let people skip videos (analgous to swiping on tiktok) and mix ads in with good non-ad content. find that people do watch influencer ads for slightlly longer than traditional ads, but difference is small and likely not very economically meaningful. also, no diff in persusiveness of traditional v influencer ads here
Time Spent
## OLS estimation, Dep. Var.: Time
## Observations: 3,675
## Fixed-effects: ResponseId: 735, Brand: 10
## Standard-errors: Clustered (ResponseId)
## Estimate Std. Error t value Pr(>|t|)
## Treatment 3.21099 1.38411 2.31989 0.02062 *
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## RMSE: 40.3 Adj. R2: 0.166665
## Within R2: 0.001234Purchase Intention and Feelings
Note There seems to be some errors in the data. So I filtered out participants who had more than 5 rows in the panel data. I did not filter based on whether they recognize the brand was shown in the survey or not.
