Connect with me on Open Science Framework | Contact me via LinkedIn

Depending on your machine it might be necessary to use right-click -> open in new browser window.

R analysis script presenting the results for exercise 5 in consumer behavior. These are the questions regarding Biswas et al. (2014).

1 Loading packages that we will use

Beware: R is a context-sensitive language. Thus, ‘data’ will be interpreted not in the same way as ‘Data’ will.

In R most functionality is provided by additional packages.
Most of the packages are well-documented, See: https://cran.r-project.org/

The code chunk below first evaluates if the package pacman is already installed to your machine. If yes, the corresponding package will be loaded. If not, R will install the package.

Alternatively, you can do this manually first by executing install.packages(“pacman”) and then library(pacman).

The second line then loads the package pacman

The third line uses the function p_load() from the pacman package to install (if necessary) and load all packages that we provide as arguments (e.g., pwr, which provides functions for power analysis).

if(!"pacman" %in% rownames(installed.packages())) install.packages("pacman")

library(pacman)

pacman::p_load(tidyverse, dplyr, pwr, tidyselect, compute.es)

Here is the R session info which gives you information on my machine, all loaded packages and their version:

sessionInfo()

## R version 3.6.3 (2020-02-29)
## Platform: x86_64-w64-mingw32/x64 (64-bit)
## Running under: Windows 10 x64 (build 18363)
## 
## Matrix products: default
## 
## locale:
## [1] LC_COLLATE=German_Germany.1252  LC_CTYPE=German_Germany.1252   
## [3] LC_MONETARY=German_Germany.1252 LC_NUMERIC=C                   
## [5] LC_TIME=German_Germany.1252    
## 
## attached base packages:
## [1] stats     graphics  grDevices utils     datasets  methods   base     
## 
## other attached packages:
##  [1] compute.es_0.2-4 tidyselect_1.0.0 pwr_1.3-0        forcats_0.5.0   
##  [5] stringr_1.4.0    dplyr_0.8.5      purrr_0.3.3      readr_1.3.1     
##  [9] tidyr_1.0.2      tibble_2.1.3     ggplot2_3.3.0    tidyverse_1.3.0 
## [13] pacman_0.5.1    
## 
## loaded via a namespace (and not attached):
##  [1] xfun_0.12        haven_2.2.0      lattice_0.20-38  colorspace_1.4-1
##  [5] vctrs_0.2.4      generics_0.0.2   htmltools_0.4.0  yaml_2.2.1      
##  [9] rlang_0.4.5      pillar_1.4.3     glue_1.3.2       withr_2.1.2     
## [13] DBI_1.1.0        dbplyr_1.4.2     modelr_0.1.6     readxl_1.3.1    
## [17] lifecycle_0.2.0  munsell_0.5.0    gtable_0.3.0     cellranger_1.1.0
## [21] rvest_0.3.5      codetools_0.2-16 evaluate_0.14    knitr_1.28      
## [25] fansi_0.4.1      broom_0.5.5      Rcpp_1.0.3       scales_1.1.0    
## [29] backports_1.1.5  jsonlite_1.6.1   fs_1.3.2         hms_0.5.3       
## [33] digest_0.6.25    stringi_1.4.3    grid_3.6.3       cli_2.0.2       
## [37] tools_3.6.3      magrittr_1.5     crayon_1.3.4     pkgconfig_2.0.3 
## [41] xml2_1.2.5       reprex_0.3.0     lubridate_1.7.4  assertthat_0.2.1
## [45] rmarkdown_2.1    httr_1.4.1       rstudioapi_0.11  R6_2.4.1        
## [49] nlme_3.1-145     compiler_3.6.3

2 Finding an answer to question #5

How many subjects have to be recruited in order to replicate the overall effect of Study 1b (Assumptions: $\alpha$=5%, Power=0.8, Chi²-test, effect size=sample)?

We see that the problem is very similar to what we have done in Exercise 2 (Kim, Spence, & Marshall, 2018), as in the article a $\chi^{2}$-test was applied to analyze counts.

2.1 Calculate effect size Cohen’s $\omega$

Information the article provides:

“Fifty-one restaurant patrons ($M_{age}$ = 38 years, 26% female) participated in the experiment in exchange for a free drink and complimentary chocolates.”
“Consumers chose the chocolate sampled sequentially last to a greater extent when the sensory cues were dissimilar (vs. similar) and chose the first chocolate to a greater extent when the sensory cues were similar (vs. dissimilar) ($\chi^{2}$ = 7.07, p < .01).”

From this we are able to extract:

sample size n=51
the $\chi^{2}$ value which is 7.07

Unfortunately, the authors did not report the degrees of freedom (df), which is a suboptimal style. However, based on the information provided, we understand that their analysis is based on a 2x2 contingency table:

	First chocolate chosen	Last chocolate chosen
Similar cues	a	b
Dissimilar cues	c	d

Furthermore, we know from lectures in statistics that the df in a $\chi^{2}$-test equal: (number of rows -1) * (number of columns -1). Thus, df=(2-1)*(2-1)=1. The degrees of freedom are 1.

This is all that we need to first calculate the observed effect size Cohen’s $\omega$ (Cohen, 1988). Remember from exercise: Cohen’s $\omega$=$\sqrt{\frac{chi^{2}}{n}}$. Therefore, we simply need to solve $\sqrt{\frac{7.07}{51}}$

We achieve this by using the next code chunk. Within this code chunk we use the sqrt() function (place the cursor in the function and press ‘F1’ to see help), which calculates the square root of a number. We feed this function with the values of $\chi^{2}(1)$=7.07, and the total sample size n=51. We assign the results to an object that we call ‘Cohen_omega’.

In the next line we call the object and simultaneously round the results to 4 digits. This is obtained by using the round() function.

Cohen_omega <- sqrt(7.07/51)

round(Cohen_omega, digits = 4)

## [1] 0.3723

We can see that the calculated effect size 0.3723 perfectly mirrors the one we have seen in the exercise slides. Common classifications for Cohen’s $\omega$ are: [0.1 | 0.3[ - small effect, [0.3 | 0.5[ - medium effect, and [0.5 | +$\infty$] - large effect.

2.2 Conduct A priori power analysis to search for minimum n

In the question we are asked for a statistical power of 80% (a commonly used threshold in social sciences).

To estimate a minimum sample size n for the replication we apply the pwr.chisq.test() function from the pwr package. This function needs us to provide 5 arguments to fill its parameters. These are (see help by pressing ‘F1’):

w = Effect size Cohen’s $\omega$
N = Total number of observations
df= degrees of freedom
sig.level = Significance level (Type I error probability $\alpha$)
power = Power of test (1 minus Type II error probability)

The function, furthermore, assumes us to set one of these arguments to NULL. By doing so, we tell the function to use the remaining 4 parameters to search for the value of the fifth. In our case, we are searching for ‘N’, therefore, we set ‘N=NULL’.

We assign the results of our power analysis to a new object named ‘results’. Then we call for its content.

results <- pwr.chisq.test(w=Cohen_omega, df=1 ,N=NULL, sig.level=0.05, power = 0.8) 

results

 Chi squared power calculation 

          w = 0.3723271
          N = 56.61837
         df = 1
  sig.level = 0.05
      power = 0.8

NOTE: N is the number of observations

We can see that the calculated sample size 57 perfectly mirrors the one we have seen in G*Power 3 (Faul, Erdfelder, Lang, & Buchner, 2007). Keep in mind: if an original study comes with a medium effect size, then a replication with a power of 80% often needs a comparable sample size as compared to the original study. However, studies reporting only small effects usually need much more participants for a successful replication.

2.3 Visualize results

In a last step we can visualize the relationship between the expected statistical power and different sample sizes.

For this purpose we apply the plot() function with the ‘results’ object and a catchy label for the x-axes as arguments.

plot(results, xlab="sample size")

From this plot we can alternatively extract the same information for sample size planning.

3 Additional reading (not relevant for the exam)

In the above-discussed example, we were planning a replication of the original findings. In doing so, we have made certain assumptions:

We assume that the observed effect size Cohen’s $\omega$ = 0.3723 is the true effect size to find. Strictly speaking, this is wrong. Our calculated value is only a sample estimate of the true effect size in the underlying population. Under the assumption of knowing the true effect size $\omega$ = 0.3723 we calculated that we need at least n=57 participants for a replication of the original findings with a power of 80%. This means that we will have approximately the chance of 80% to find the effect as to be significant at $\alpha$=0.05. Because, the true effect to find might, however, be smaller, this calculation maybe presents an overly optimistic approach. Therefore, other researchers prefer to focus on the confidence intervals for the estimate of effect sizes measures (Thompson, 2002).

Apart from Cohen’s $\omega$, there are plenty of other effect sizes ‘out in the market’. They all can be converted into each other (Borenstein, Hedges, Higgins, & Rothstein, 2009; Fern & Monroe, 1996; Lenhard & Lenhard, 2017; Volker, 2006). In our case (Chi² with df=1), for example, Cohen’s $\omega$ is equivalent to the common language effect size r (Lenhard & Lenhard, 2017).

One R package that is facilitating the calculation of confidence levels for effect sizes is the compute.es package. This package supports the use of r as an effect size measure.

We can apply the chies() function to obtain confidence intervals for our observed effect size. We use the function twice. The first time we assign the results to an object named ‘CI_effect’. The second time we don not assign the results to an object, which is equivalent to printing the results to the screen.

In the second call we feed the function with 5 arguments:

chi.sq = our $\chi^{2}$ value which is 7.07
n = our sample size of 51
level = the level of confidence for the confidence intervals to be calculated
dig = the number of digits to be saved
verbose = TRUE, in order to print detailed results

CI_effect <- chies(chi.sq = 7.07, n =51, level = 95, dig = 3, verbose = FALSE)

chies(chi.sq = 7.07, n =51, level = 95, dig = 3, verbose = TRUE)

## Mean Differences ES: 
##  
##  d [ 95 %CI] = 0.794 [ 0.182 , 1.407 ] 
##   var(d) = 0.093 
##   p-value(d) = 0.012 
##   U3(d) = 78.653 % 
##   CLES(d) = 71.286 % 
##   Cliff's Delta = 0.426 
##  
##  g [ 95 %CI] = 0.782 [ 0.179 , 1.385 ] 
##   var(g) = 0.09 
##   p-value(g) = 0.012 
##   U3(g) = 78.296 % 
##   CLES(g) = 70.991 % 
##  
##  Correlation ES: 
##  
##  r [ 95 %CI] = 0.372 [ 0.101 , 0.592 ] 
##   var(r) = 0.015 
##   p-value(r) = 0.009 
##  
##  z [ 95 %CI] = 0.391 [ 0.101 , 0.681 ] 
##   var(z) = 0.021 
##   p-value(z) = 0.009 
##  
##  Odds Ratio ES: 
##  
##  OR [ 95 %CI] = 4.225 [ 1.391 , 12.83 ] 
##   p-value(OR) = 0.012 
##  
##  Log OR [ 95 %CI] = 1.441 [ 0.33 , 2.552 ] 
##   var(lOR) = 0.306 
##   p-value(Log OR) = 0.012 
##  
##  Other: 
##  
##  NNT = 3.556 
##  Total N = 51

Along the output of the chies()-function we see for the relevant effect size r that we are additionally provided with a test for significance for the effect size itself. This is an alternative to the classical $\chi^{2}$ test. If we reach at a significant result here, this means that the effect size obtained is significantly greater than 0. Put differently, the effect is assumed to exist in the underlying population. Recently researchers in the context effect domain are starting to report tests on effect sizes, especially, when their articles span over multiple studies (Evangelidis, Levav, & Simonson, 2018).

We now recognize that the effect size in the underlying population is likely to be located somewhere in the interval ranging from 0.101 to 0.592. Put differently, in the worst case we have to plan for a replication of an effect that is, indeed, only of the size of $\omega$ = 0.101.

We can briefly calculate how this will change our assumptions about the minimum sample size. We, again, use the pwr.chisq.test() function (see above). This time, we hand in ‘CI_effect$l.r’ as the argument for the effect size parameter w. Calling ‘CI_effect$l.r’ simply returns the value of the lower limit of the 95% confidence interval of the effect size r (=Cohen’s $\omega$ in our case).

in R objects such as ‘CI_effect’ can compromise of different elements. We can assess these elements by using the ‘$’ notation. In the code chunk below, the element ‘l.r’ simply stores the value lower limit of the 95% confidence interval of the effect size r.

results_worst_case <- pwr.chisq.test(w=CI_effect$l.r, df=1 ,N=NULL, sig.level=0.05, power = 0.8) 

results_worst_case

 Chi squared power calculation 

          w = 0.101
          N = 769.4207
         df = 1
  sig.level = 0.05
      power = 0.8

NOTE: N is the number of observations

We can see that the more conservative sample size is 770, which makes a replication of the original findings almost impossible. This is a general problem in consumer research. Most universities (especially the smaller European schools) do not have the necessary budget to replicate findings published in the top-tier journals which, however, decreases trust in the findings’ usefulness.

List of References

Biswas, D., Labrecque, L. I., Lehmann, D. R., & Markos, E. (2014). Making choices while smelling, tasting, and listening: The role of sensory (dis)similarity when sequentially sampling products. Journal of Marketing, 78(1), 112–126. doi: 10.1509/jm.12.0325

Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009). Converting among effect sizes. In M. Borenstein (Ed.), Introduction to meta-analysis (pp. 45–49). doi: 10.1002/9780470743386.ch7

Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Lawrence Erlbaum.

Evangelidis, I., Levav, J., & Simonson, I. (2018). The asymmetric impact of context on advantaged versus disadvantaged options. Journal of Marketing Research, 55(2), 239–253. doi: 10.1509/jmr.14.0483

Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A. (2007). G* power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39(2), 175–191. doi: 10.3758/BF03193146

Fern, E. F., & Monroe, K. B. (1996). Effect-size estimates: Issues and problems in interpretation. Journal of Consumer Research, 23(2), 89–105. doi: 10.2307/2489707

Kim, J., Spence, M. T., & Marshall, R. (2018). The color of choice: The influence of presenting product information in color on the compromise effect. Journal of Retailing, 94(2), 167–185. doi: 10.1016/j.jretai.2018.01.002

Lenhard, W., & Lenhard, A. (2017). Computation of effect sizes. doi: 10.13140/RG.2.2.17823.92329

Thompson, B. (2002). What future quantitative social science research could look like: Confidence intervals for effect sizes. Educational Researcher, 31(3), 25–32. doi: 10.3102/0013189X031003025

Volker, M. A. (2006). Reporting effect size estimates in school psychology research. Psychology in the Schools, 43(6), 653–672. doi: 10.1002/pits.20176

Consumer Behavior - Solutions Exercise 5

Jun.-Prof. Dr. Marcel Lichters

2020-05-04