Consumer Behavior - Solutions Exercise 2
Jun.-Prof. Dr. Marcel Lichters
2020-05-04
Connect with me on Open Science Framework | Contact me via LinkedIn
Depending on your machine it might be necessary to use right-click -> open in new browser window.
R analysis script presenting the results for exercise 2 in consumer behavior. These are the questions regarding Kim, Spence, & Marshall (2018).
1 A very brief intro to R and loading packages that we will use
Beware: R is a context-sensitive language. Thus, ‘data’ will be interpreted not in the same way as ‘Data’ will.
In R most functionality is provided by additional packages.
Most of the packages are well-documented, See: https://cran.r-project.org/
The code chunk below first evaluates if the package pacman is already installed to your machine. If yes, the corresponding package will be loaded. If not, R will install the package.
Alternatively, you can do this manually first by executing install.packages(“pacman”) and then library(pacman).
The second line then loads the package pacman
The third line uses the function p_load() from the pacman package to install (if necessary) and load all packages that we provide as arguments (e.g., pwr, which provides functions for power analysis).
if(!"pacman" %in% rownames(installed.packages())) install.packages("pacman")
library(pacman)
pacman::p_load(tidyverse, dplyr, pwr, tidyselect, compute.es)R is a functional language, meaning we could define a function that takes some arguments as input parameters and delivers a result as an output.
We can see which arguments a function understands by pressing ‘F1’ while setting the cursor into the function’s name (i.e., library() above).
To execute a line call, just set the cursor somewhere in the corresponding code line and press ‘Ctr’+‘Enter’ (on a Windows machine), or use the grafical user interface of RStudio.
Here is the R session info which gives you information on my machine, all loaded packages and their version:
sessionInfo()## R version 3.6.3 (2020-02-29)
## Platform: x86_64-w64-mingw32/x64 (64-bit)
## Running under: Windows 10 x64 (build 18363)
##
## Matrix products: default
##
## locale:
## [1] LC_COLLATE=German_Germany.1252 LC_CTYPE=German_Germany.1252
## [3] LC_MONETARY=German_Germany.1252 LC_NUMERIC=C
## [5] LC_TIME=German_Germany.1252
##
## attached base packages:
## [1] stats graphics grDevices utils datasets methods base
##
## other attached packages:
## [1] compute.es_0.2-4 tidyselect_1.0.0 pwr_1.3-0 forcats_0.5.0
## [5] stringr_1.4.0 dplyr_0.8.5 purrr_0.3.3 readr_1.3.1
## [9] tidyr_1.0.2 tibble_2.1.3 ggplot2_3.3.0 tidyverse_1.3.0
## [13] pacman_0.5.1
##
## loaded via a namespace (and not attached):
## [1] xfun_0.12 haven_2.2.0 lattice_0.20-38 colorspace_1.4-1
## [5] vctrs_0.2.4 generics_0.0.2 htmltools_0.4.0 yaml_2.2.1
## [9] rlang_0.4.5 pillar_1.4.3 glue_1.3.2 withr_2.1.2
## [13] DBI_1.1.0 dbplyr_1.4.2 modelr_0.1.6 readxl_1.3.1
## [17] lifecycle_0.2.0 munsell_0.5.0 gtable_0.3.0 cellranger_1.1.0
## [21] rvest_0.3.5 codetools_0.2-16 evaluate_0.14 knitr_1.28
## [25] fansi_0.4.1 broom_0.5.5 Rcpp_1.0.3 scales_1.1.0
## [29] backports_1.1.5 jsonlite_1.6.1 fs_1.3.2 hms_0.5.3
## [33] digest_0.6.25 stringi_1.4.3 grid_3.6.3 cli_2.0.2
## [37] tools_3.6.3 magrittr_1.5 crayon_1.3.4 pkgconfig_2.0.3
## [41] xml2_1.2.5 reprex_0.3.0 lubridate_1.7.4 assertthat_0.2.1
## [45] rmarkdown_2.1 httr_1.4.1 rstudioapi_0.11 R6_2.4.1
## [49] nlme_3.1-145 compiler_3.6.32 Finding an answer to question #6
For your master thesis it could be an idea to replicate the findings on the compromise effect obtained in Study 1A for computers in the Color condition. Based on a power analysis (\(\alpha\)=5%, \(\beta\)=20%), how many subjects should you recruit for such a study?
2.1 Calculate effect size Cohen’s \(\omega\)
Information the article provides: “However, the compromise effect is significantly positive under the color condition (computer B’s share under the ABC vs. BCD condition=77.1% (27/35) vs. 29.0% (9/31); compromise effect=48.1% (\(\chi^{2}(1)\)=15.35, p<.001)).”
From this we are able to extract:
- sample size n=66 (35+31)
- the \(\chi^{2}\) value which is 15.35
- the degrees of freedom (df=1)
This is all that we need to first calculate the observed effect size Cohen’s \(\omega\) (Cohen, 1988). Remember from exercise: Cohen’s \(\omega\)=\(\sqrt{\frac{chi^{2}}{n}}\). Therefore, we simply need to solve \(\sqrt{\frac{15.35}{66}}\)
We achieve this by using the next code chunk. Within this code chunk we use the sqrt() function (place the cursor in the function and press ‘F1’ to see help), which calculates the square root of a number. We feed this function with the values of \(\chi^{2}(1)\)=15.35, and the total sample size n=66. We assign the results to an object that we call ‘Cohen_omega’.
In the next line we call the object and simultaneously round the results to 4 digits. This is obtained by using the round() function.
Cohen_omega <- sqrt(15.35/66)
round(Cohen_omega, digits = 4)## [1] 0.4823We can see that the calculated effect size 0.4823 perfectly mirrors the one we have seen in the exercise slides. Common classifications for Cohen’s \(\omega\) are: [0.1 | 0.3[ - small effect, [0.3 | 0.5[ - medium effect, and [0.5 | +\(\infty\)] - large effect.
2.2 Conduct A priori power analysis to search for minimum n
In the question we are asked for \(\beta\)=20%, which corresponds to a statistical power of 80% (a commonly used threshold in social sciences).
To estimate a minimum sample size n for the replication we apply the pwr.chisq.test() function from the pwr package. This function needs us to provide 5 arguments to fill its parameters. These are (see help by pressing ‘F1’):
- w = Effect size Cohen’s \(\omega\)
- N = Total number of observations
- df= degrees of freedom
- sig.level = Significance level (Type I error probability \(\alpha\))
- power = Power of test (1 minus Type II error probability)
The function, furthermore, assumes us to set one of these arguments to NULL. By doing so, we tell the function to use the remaining 4 parameters to search for the value of the fifth. In our case, we are searching for ‘N’, therefore, we set ‘N=NULL’.
We assign the results of our power analysis to a new object named ‘results’. Then we call for its content.
results <- pwr.chisq.test(w=Cohen_omega, df=1 ,N=NULL, sig.level=0.05, power = 0.8)
results Chi squared power calculation
w = 0.4822611
N = 33.74754
df = 1
sig.level = 0.05
power = 0.8NOTE: N is the number of observations
We can see that the calculated sample size 33.7475436 perfectly mirrors the one we have seen in G*Power 3 (Faul, Erdfelder, Lang, & Buchner, 2007). Keep in mind: if an original study comes with a relatively large effect size, then a replication with a power of 80% often needs fewer participants than the original study. However, studies reporting only small effects usually need much more participants for a successful replication.
2.3 Visualize results
In a last step we can visualize the relationship between the expected statistical power and different sample sizes.
For this purpose we apply the plot() function with the ‘results’ object and a catchy label for the x-axes as arguments.
plot(results, xlab="sample size")
From this plot we can alternatively extract the same information for sample size planning.
3 Additional reading (not relevant for the exam)
In the above-discussed example, we were planning a replication of the original findings from Kim et al. (2018). In doing so, we have made certain assumptions:
We assume that the observed effect size Cohen’s \(\omega\) = 0.4823 is the true effect size to find. Strictly speaking, this is wrong. Our calculated value is only a sample estimate of the true effect size in the underlying population. Under the assumption of knowing the true effect size \(\omega\) = 0.4823 we calculated that we need at least n=34 participants for a replication of the original findings with a power of 80%. This means that we will have approximately the chance of 80% to find the effect as to be significant at \(\alpha\)=0.05. Because, the true effect to find might, however, be smaller, this calculation maybe presents an overly optimistic approach. Therefore, other researchers prefer to focus on the confidence intervals for the estimate of effect sizes measures (Thompson, 2002).
Apart from Cohen’s \(\omega\), there are plenty of other effect sizes ‘out in the market’. They all can be converted into each other (Borenstein, Hedges, Higgins, & Rothstein, 2009; Fern & Monroe, 1996; Lenhard & Lenhard, 2017; Volker, 2006). In our case (Chi² with df=1), for example, Cohen’s \(\omega\) is equivalent to the common language effect size r (Lenhard & Lenhard, 2017).
One R package that is facilitating the calculation of confidence levels for effect sizes is the compute.es package. This package supports the use of r as an effect size measure.
We can apply the chies() function to obtain confidence intervals for our observed effect size. We use the function twice. The first time we assign the results to an object named ‘CI_effect’. The second time we don not assign the results to an object, which is equivalent to printing the results to the screen.
In the second call we feed the function with 5 arguments:
- chi.sq = our \(\chi^{2}\) value which is 15.35
- n = our sample size of 66
- level = the level of confidence for the confidence intervals to be calculated
- dig = the number of digits to be saved
- verbose = TRUE, in order to print detailed results
CI_effect <- chies(chi.sq = 15.35, n =66, level = 95, dig = 3, verbose = FALSE)
chies(chi.sq = 15.35, n =66, level = 95, dig = 3, verbose = TRUE)## Mean Differences ES:
##
## d [ 95 %CI] = 1.093 [ 0.527 , 1.658 ]
## var(d) = 0.08
## p-value(d) = 0
## U3(d) = 86.273 %
## CLES(d) = 78.013 %
## Cliff's Delta = 0.56
##
## g [ 95 %CI] = 1.08 [ 0.521 , 1.639 ]
## var(g) = 0.078
## p-value(g) = 0
## U3(g) = 85.988 %
## CLES(g) = 77.743 %
##
## Correlation ES:
##
## r [ 95 %CI] = 0.482 [ 0.268 , 0.651 ]
## var(r) = 0.009
## p-value(r) = 0
##
## z [ 95 %CI] = 0.526 [ 0.274 , 0.778 ]
## var(z) = 0.016
## p-value(z) = 0
##
## Odds Ratio ES:
##
## OR [ 95 %CI] = 7.256 [ 2.601 , 20.245 ]
## p-value(OR) = 0
##
## Log OR [ 95 %CI] = 1.982 [ 0.956 , 3.008 ]
## var(lOR) = 0.264
## p-value(Log OR) = 0
##
## Other:
##
## NNT = 2.506
## Total N = 66Along the output of the chies()-function we see for the relevant effect size r that we are additionally provided with a test for significance for the effect size itself. This is an alternative to the classical \(\chi^{2}\) test. If we reach at a significant result here, this means that the effect size obtained is significantly greater than 0. Put differently, the effect is assumed to exist in the underlying population. Recently researchers in the context effect domain are starting to report tests on effect sizes, especially, when their articles span over multiple studies (Evangelidis, Levav, & Simonson, 2018).
We now recognize that the effect size in the underlying population is likely to be located somewhere in the interval ranging from 0.268 to 0.651. Put differently, in the worst case we have to plan for a replication of an effect that is, indeed, only of the size of \(\omega\) = 0.268.
We can briefly calculate how this will change our assumptions about the minimum sample size. We, again, use the pwr.chisq.test() function (see above). This time, we hand in ‘CI_effect$l.r’ as the argument for the effect size parameter w. Calling ‘CI_effect$l.r’ simply returns the value of the lower limit of the 95% confidence interval of the effect size r (=Cohen’s \(\omega\) in our case).
in R objects such as ‘CI_effect’ can compromise of different elements. We can assess these elements by using the ‘$’ notation. In the code chunk below, the element ‘l.r’ simply stores the value lower limit of the 95% confidence interval of the effect size r.
results_worst_case <- pwr.chisq.test(w=CI_effect$l.r, df=1 ,N=NULL, sig.level=0.05, power = 0.8)
results_worst_case Chi squared power calculation
w = 0.268
N = 109.2791
df = 1
sig.level = 0.05
power = 0.8NOTE: N is the number of observations
We can see that the more conservative sample size is 109.2790782, which makes a replication of the original findings harder.
List of References
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009). Converting among effect sizes. In M. Borenstein (Ed.), Introduction to meta-analysis (pp. 45–49). doi: 10.1002/9780470743386.ch7
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Lawrence Erlbaum.
Evangelidis, I., Levav, J., & Simonson, I. (2018). The asymmetric impact of context on advantaged versus disadvantaged options. Journal of Marketing Research, 55(2), 239–253. doi: 10.1509/jmr.14.0483
Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A. (2007). G* power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39(2), 175–191. doi: 10.3758/BF03193146
Fern, E. F., & Monroe, K. B. (1996). Effect-size estimates: Issues and problems in interpretation. Journal of Consumer Research, 23(2), 89–105. doi: 10.2307/2489707
Kim, J., Spence, M. T., & Marshall, R. (2018). The color of choice: The influence of presenting product information in color on the compromise effect. Journal of Retailing, 94(2), 167–185. doi: 10.1016/j.jretai.2018.01.002
Lenhard, W., & Lenhard, A. (2017). Computation of effect sizes. doi: 10.13140/RG.2.2.17823.92329
Thompson, B. (2002). What future quantitative social science research could look like: Confidence intervals for effect sizes. Educational Researcher, 31(3), 25–32. doi: 10.3102/0013189X031003025
Volker, M. A. (2006). Reporting effect size estimates in school psychology research. Psychology in the Schools, 43(6), 653–672. doi: 10.1002/pits.20176