One of my research interests entails the language use multilingual speaking communities use to communicate through their intergenerational teaching. I use ethnographic methods to collect my data, however, I find that the proposed study for replication brings a new perspective I can include in my research. The participants I conduct my research with are multilingual and speak a number of combinations that include Zapotec (an Indigenous language native to Mexico), Spanish, and English. One of my research interests includes the language preference my participants use among peers their age and adults who are experienced members in the culture-specific learning setting. I was intrigued by the findings of this article because the researchers examine how implicit messages vary in the degree to which the communicator’s language is concrete rather than abstract. Hence, research has demonstrated that subtle linguistic variations can have a substantial effect on what is conveyed about a target (Porter, Rheinschmidt-Same, & Richeson, 2016). As I design my dissertation, this study has helped me realize the importance of potential favorable or unfavorable LIB (defined in the next section) within the intragroup setting of my study.
The procedures of the study required participants to complete all tasks on a computer as they were asked to read a passage and then respond to questions. The beginning of the passage was the same across all conditions: “Imagine that someone is communicating with you about a man named Peter. Peter is American, has an interest in politics, and voted for Barack Obama.” This information was intended to subtly imply that Peter (the target) was a Democrat. The stimuli are introduced in the second part of the passage as participants were given the communicator’s description of Peter’s helpful and rude behaviors. In the favorable linguistic intergroup bias (favorable-LIB) condition, which is the use of abstract language to describe in-group members’ desirable behaviors and concrete language to describe their undesirable behaviors, Peter’s helping behavior was described abstractly (such as “Peter is someone who stands up for the interests of others”). In the unfavorable LIB condition, Peter’s helping behavior is described more concretely (such as “Peter helped another person, even if it did not benefit him”) and his rude behavior was described more abstractly (such as “Peter is cold.”).
Participants were asked to assess the likelihood that the communicator was a Republican or Democrat after reading the passage. Participants assessed their ratings on a 7-point scale, anchored by 1, definitely a Republican, and 7, definitely a Democrat. Next, in order to check the effectiveness of the LIB manipulation, the authors asked participants to estimate the percentage of future situation in which Peter was likely to be helpful, along with the percentage of future situations in which he was likely to be rude. To conclude, participants completed a demographic questionnaire that asked about their gender, political-party affiliation, and the degree to which they endorsed liberal and conservative beliefs, on a 7-point scale ranging from 1, strongly disagree, to 7, strongly agree. I can foresee finding a sample size to the original study (n = 88) that is similar to be challenging. Additionally, given that the complete materials were not provided in the article, the length of the entire procedure is unknown and may be a challenge as well.
Link to the repository: https://github.com/mmesinas/porter2016
Original link to paper: http://journals.sagepub.com.stanford.idm.oclc.org/doi/pdf/10.1177/0956797615612202
An original effect size was not mentioned nor a power analysis for samples to achieve 80%, 90%, 95% power to detect that effect size. Considerations of feasibility for selecting planned sample size was discussed in the study, please see the next section for more details.
From past research in this area, the authors conservatively estimated the sample size needed to find an effect. They concluded that they needed to sample 90 participants and stopped collecting data once that number was reached. However, two participants did not complete the dependent-variable measure and the final sample size was 88. Participants were recruited from Amazon Mechanical Turk (MTurk.com).
“Participants completed all tasks on a computer. They were asked to read a passage and then respond to questions. The beginning of the passage was the same for all participants: “Imagine that someone is communicating with you about a man named Peter. Peter is American, has an interest in politics, and voted for Barack Obama.” This information was intended to subtly imply that Peter (the target) was a Democrat. In the second part of the passage, participants were provided with the communicator’s description of Peter’s helpful and rude behaviors (for the complete text of these descriptions, see Table S1 in the Supplemental Material available online). In the favorable-LIB condition, Peter’s helping behavior was described abstractly (e.g., “[Peter] is someone who stands up for the interests of others”), and his rude behavior was described concretely (e.g., “Peter said something rude to another person recently”). In the unfavorable-LIB condition, Peter’s helping behavior was described concretely (e.g., “Peter helped another person, even when it did not benefit him”), and his rude behavior was described abstractly (e.g., “[Peter] is cold and unfriendly”).”
The Supplemental Materials can be found here: http://journals.sagepub.com.stanford.idm.oclc.org/doi/suppl/10.1177/0956797615612202/suppl_file/suppl-material.pdf
“After reading the passage, participants were asked to assess the likelihood that the communicator was either a Democrat or a Republican. Ratings were made on a 7-point scale, anchored by 1, definitely a Republican, and 7, definitely a Democrat. As a check of the effectiveness of the LIB manipulation, we asked participants to estimate the percentage of future situations in which Peter was likely to be helpful and the percentage of future situations in which he was likely to be rude (Semin & de Poot, 1997). Finally, participants completed a demographic questionnaire that asked their gender, their political-party affiliation, and the degree to which they endorsed liberal and conservative beliefs (on 7-point scales ranging from 1, strongly disagree, to 7, strongly agree).”
The survey can be accesed via this link: https://gse.qualtrics.com/jfe/form/SV_0Ca9vv5SVM4U8sd
The analysis plan will follow the original study’s plan. I will conduct the same manipulation check that was conducted in study 1a through the calculation of raw mean ratings of Peter’s (the target) future likelihood of being rude and helpful. Further, the mean rated likelihood that the communicator and target shared group membership for each condition (favorable- and unfavorable-linguistic intergroup bias).
I will calculate t-statistics, effect sizes, and p-values for these analyses. A t-test will be used because I want to compare the means of two independent groups.
Explicitly describe known differences in sample, setting, procedure, and analysis plan from original study. The goal, of course, is to minimize those differences, but differences will inevitably occur. Also, note whether such differences are anticipated to make a difference based on claims in the original article or subsequent published research on the conditions for obtaining the effect. To be determined.
You can comment this section out prior to final report with data collection. To be determined.
Sample size, demographics, data exclusions based on rules spelled out in analysis plan To be determined.
Any differences from what was described as the original plan, or “none”. To be determined.
I will import the data for study 1a to familiarize myself with the organization of the dataset. I will make sure to understand how each variable was labeled and inputted, especially for the LIB and ULIB conditions.
“The primary dependent measure was participants’ inferences regarding the communicator’s political affiliation. As predicted, participants in the favorable-LIB condition were significantly more likely to believe that the communicator was a Democrat, and thus shared a party affiliation with the target, than were participants in the unfavorable-LIB condition, t(86) = 2.89, p = .005, d = 0.62 (Fig. 1). This difference was not moderated by participants’ self-reported political- party affiliation or ideological endorsement (ps > .18). Our findings suggested initial support for our hypothesis that individuals can infer a communicator’s social identity from his or her language, regardless of their own social identity.”
I ran the following code to test my pilot data:
library(tidyverse)
pilot1 <- read_csv(“/Users/melissamesinas/Desktop/PSYCH 251/data/Porteretal2016-pilot.csv”)
pilot1 <- pilot1[-c(1:2), ] %>% # drops rows 1 and 2, which had meta-data select(Condition, Q4) %>% drop_na() # pilot1\(Q4[is.na(pilot1\)Q4)]
pilot1\(Q4[pilot1\)Q4 == “7 - Definitely Democrat”] <- “7” pilot1\(Condition <- as.factor(pilot1\)Condition) pilot1\(Q4 <- as.numeric(pilot1\)Q4)
t.test(Q4 ~ Condition, data = pilot1)
ggplot(pilot1, aes(Condition, Q4)) + geom_boxplot()
The results from the pilot data are t(2.8824) = -3.2733, p = 0.04942.
Any follow-up analyses desired (not required).
Open the discussion section with a paragraph summarizing the primary result from the confirmatory analysis and the assessment of whether it replicated, partially replicated, or failed to replicate the original result.
Add open-ended commentary (if any) reflecting (a) insights from follow-up exploratory analysis, (b) assessment of the meaning of the replication (or not) - e.g., for a failure to replicate, are the differences between original and present study ones that definitely, plausibly, or are unlikely to have been moderators of the result, and (c) discussion of any objections or challenges raised by the current and original authors about the replication attempt. None of these need to be long.