Reproducibility Report: Group B Choice 2
For this exercise, please try to reproduce the results from Experiment 1 of the associated paper (Ko, Sadler & Galinsky, 2015). The PDF of the paper is included in the same folder as this Rmd file.
Methods summary:
A sense of power has often been tied to how we perceive each other’s voice. Social hierarchy is embedded into the structure of society and provides a metric by which others relate to one another. In 1956, the Brunswik Lens Model was introduced to examine how vocal cues might influence hierarchy. In “The Sound of Power: Conveying and Detecting Hierarchical Rank Through Voice,” Ko and colleagues investigated how manipulation of hierarchal rank within a situation might impact vocal acoustic cues. Using the Brunswik Model, six acoustic metrics were utilized (pitch mean & variability, loudness mean & variability, and resonance mean & variability) to isolate a potential contribution between individuals of different hierarchal rank. In the first experiment, Ko, Sadler & Galinsky examined the vocal acoustic cues of individuals before and after being assigned a hierarchal rank in a sample of 161 subjects (80 male). Each of the six hierarchy acoustic cues were analyzed with a 2 (high vs. low rank condition) x 2 (male vs. female) analysis of covariance, controlling for the baseline of the respective acoustic cue.
Target outcomes:
Below is the specific result you will attempt to reproduce (quoted directly from the results section of Experiment 1):
The impact of hierarchical rank on speakers’ acoustic cues. Each of the six hierarchy-based (i.e., postmanipulation) acoustic variables was submitted to a 2 (condition: high rank, low rank) × 2 (speaker’s sex: female, male) between-subjects analysis of covariance, controlling for the corresponding baseline acoustic variable. Table 4 presents the adjusted means by condition. Condition had a significant effect on pitch, pitch variability, and loudness variability. Speakers’ voices in the high-rank condition had higher pitch, F(1, 156) = 4.48, p < .05; were more variable in loudness, F(1, 156) = 4.66, p < .05; and were more monotone (i.e., less variable in pitch), F(1, 156) = 4.73, p < .05, compared with speakers’ voices in the low-rank condition (all other Fs < 1; see the Supplemental Material for additional analyses of covariance involving pitch and loudness). (from Ko et al., 2015, p. 6; emphasis added)
The adjusted means for these analyses are reported in Table 4 (Table4_AdjustedMeans.png, included in the same folder as this Rmd file).
Step 1: Load packages
library(tidyverse) # for data munging
library(knitr) # for kable table formating
library(haven) # import and export 'SPSS', 'Stata' and 'SAS' Files
library(readxl) # import excel files
library(rstatix)
#optional packages:
library(psych)
library(car) # for ANCOVA
# library(compute.es) # for ANCOVA
library(lsmeans) # for ANCOVAStep 2: Load data
# Just Experiment 1
d <-read_csv("data/S1_voice_level_Final.csv")
# DT::datatable(d)Step 3: Tidy data
Not sure what to tidy up here.
Step 4: Run analysis
Pre-processing
d <- d%>%
mutate(power_level = ifelse(plev == -1, "low","high"))
d$power_level <- factor(d$power_level, levels = c("high","low"))Descriptive statistics
In the paper, the adjusted means by condition are reported (see Table 4, or Table4_AdjustedMeans.png, included in the same folder as this Rmd file). Reproduce these values below:
Each mean was adjusted by its corresponding baseline acoustic cue. pitch_smean_adjusted <- get_emmeans(emmeans_test(d,pitch_smean ~ power_level, covariate = pitch_rmean, p.adjust.method = "bonferroni"))
pitch_svar_adjusted <- get_emmeans(emmeans_test(d,pitch_svar ~ power_level, covariate = pitch_rvar, p.adjust.method = "bonferroni"))
intense_smean_adjusted <- get_emmeans(emmeans_test(d,intense_smean ~ power_level, covariate = intense_rmean, p.adjust.method = "bonferroni"))
intense_svar_adjusted <- get_emmeans(emmeans_test(d,intense_svar ~ power_level, covariate = intense_rvar, p.adjust.method = "bonferroni"))
form_smean_adjusted <- get_emmeans(emmeans_test(d,form_smean ~ power_level, covariate = form_rmean, p.adjust.method = "bonferroni"))
form_svar_adjusted <- get_emmeans(emmeans_test(d,form_svar ~ power_level, covariate = form_rvar, p.adjust.method = "bonferroni"))
tabl <- t(Reduce(cbind,list(pitch_smean_adjusted[,3],pitch_svar_adjusted[,3],
intense_smean_adjusted[,3],intense_svar_adjusted[,3],
form_smean_adjusted[,3],form_svar_adjusted[,3])))
colnames(tabl) <- c("high_rank_condition","slow_rank_condition")
rownames(tabl) <- c("pitch_smean","pitch_svar",
"intense_smean","intense_svar",
"form_smean","form_svar")
round(tabl,2)## high_rank_condition slow_rank_condition
## pitch_smean 158.71 155.39
## pitch_svar 1417.62 1662.21
## intense_smean 59.32 58.71
## intense_svar 197.13 182.86
## form_smean 1129.05 1129.51
## form_svar 42143.43 43710.00
Figure 1. Results in the original study.
Inferential statistics
The impact of hierarchical rank on speakers’ acoustic cues. Each of the six hierarchy-based (i.e., postmanipulation) acoustic variables was submitted to a 2 (condition: high rank, low rank) × 2 (speaker’s sex: female, male) between-subjects analysis of covariance, controlling for the corresponding baseline acoustic variable. […] Condition had a significant effect on pitch, pitch variability, and loudness variability. Speakers’ voices in the high-rank condition had higher pitch, F(1, 156) = 4.48, p < .05; were more variable in loudness, F(1, 156) = 4.66, p < .05; and were more monotone (i.e., less variable in pitch), F(1, 156) = 4.73, p < .05, compared with speakers’ voices in the low-rank condition (all other Fs < 1; see the Supplemental Material for additional analyses of covariance involving pitch and loudness).
the three significant findings
# reproduce the above results here
#higher pitch
aov1 <- aov(pitch_smean ~ power_level * sex + pitch_rmean,d)
Anova(aov1)## Anova Table (Type II tests)
##
## Response: pitch_smean
## Sum Sq Df F value Pr(>F)
## power_level 379 1 4.4744 0.03599 *
## sex 460 1 5.4218 0.02117 *
## pitch_rmean 43334 1 511.2208 < 2e-16 ***
## power_level:sex 222 1 2.6156 0.10783
## Residuals 13223 156
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1lsmeans(aov1,pairwise ~ power_level,adjust="tukey")$contrasts## contrast estimate SE df t.ratio p.value
## high - low 3.09 1.46 156 2.117 0.0358
##
## Results are averaged over the levels of: sex#more variable in loudness
aov2 <- aov(intense_svar ~ power_level * sex + intense_rvar,d)
Anova(aov2)## Anova Table (Type II tests)
##
## Response: intense_svar
## Sum Sq Df F value Pr(>F)
## power_level 7052 1 4.6687 0.03224 *
## sex 29860 1 19.7693 1.651e-05 ***
## intense_rvar 184927 1 122.4323 < 2.2e-16 ***
## power_level:sex 40 1 0.0262 0.87160
## Residuals 235630 156
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1lsmeans(aov2,pairwise ~ power_level,adjust="tukey")$contrasts## contrast estimate SE df t.ratio p.value
## high - low 13.3 6.14 156 2.159 0.0324
##
## Results are averaged over the levels of: sex# more monotone
aov3 <- aov(pitch_svar ~ power_level * sex + pitch_rvar,d)
Anova(aov3)## Anova Table (Type II tests)
##
## Response: pitch_svar
## Sum Sq Df F value Pr(>F)
## power_level 2006828 1 4.7340 0.03108 *
## sex 26142260 1 61.6676 6.133e-13 ***
## pitch_rvar 27760551 1 65.4850 1.548e-13 ***
## power_level:sex 37857 1 0.0893 0.76546
## Residuals 66131879 156
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1lsmeans(aov3,pairwise ~ power_level,adjust="tukey")$contrasts## contrast estimate SE df t.ratio p.value
## high - low -223 103 156 -2.174 0.0312
##
## Results are averaged over the levels of: sexother non-significant results
aov4 <- aov(intense_smean ~ power_level * sex + intense_rmean,d)
Anova(aov4)## Anova Table (Type II tests)
##
## Response: intense_smean
## Sum Sq Df F value Pr(>F)
## power_level 17.69 1 2.1833 0.1415291
## sex 104.18 1 12.8542 0.0004499 ***
## intense_rmean 1195.54 1 147.5165 < 2.2e-16 ***
## power_level:sex 11.69 1 1.4424 0.2315772
## Residuals 1264.30 156
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1aov5 <- aov(form_smean ~ power_level * sex + form_rmean,d)
Anova(aov5)## Anova Table (Type II tests)
##
## Response: form_smean
## Sum Sq Df F value Pr(>F)
## power_level 13 1 0.0018 0.9662798
## sex 80559 1 11.5306 0.0008684 ***
## form_rmean 89319 1 12.7845 0.0004656 ***
## power_level:sex 170 1 0.0243 0.8763623
## Residuals 1089894 156
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1aov6 <- aov(form_svar ~ power_level * sex + form_rvar,d)
Anova(aov6)## Anova Table (Type II tests)
##
## Response: form_svar
## Sum Sq Df F value Pr(>F)
## power_level 8.8735e+07 1 0.4764 0.4911
## sex 4.2843e+07 1 0.2300 0.6322
## form_rvar 4.9801e+08 1 2.6739 0.1040
## power_level:sex 1.5279e+07 1 0.0820 0.7749
## Residuals 2.9055e+10 156Step 5: Reflection
Were you able to reproduce the results you attempted to reproduce? If not, what part(s) were you unable to reproduce?
More or less so.
For reasons that I don’t understand, the adjusted means did not match the exact numbers in the paper, but were nonetheless very similar. This difference in digits did not alter my conclusions.
For the ANCOVAs, the significant F values matched those reported in the paper. Although the authors reported that all other F values were smaller than 1, I found that the effect of power level on mean loudness has F = 2.18 > 1. However, this did not alter the conclusions.
How difficult was it to reproduce your results?
I would rate 5 out of 10. It took me some time to work out what is the adjusted mean in Table 4. The six ANCOVAs were quite intuitive.
What aspects made it difficult? What aspects made it easy?
What made it easy: I really appreciate the codebook; it has made variable-searching much easier. However, I would really appreciate that the variable names themselves could be more intuitive.
What made it difficult: It was a bit confusing to me why the authors reported adjusted mean in Table 4.