Replication of Study 3 by Porter et al. (2016, Psychological Science)
Hannah Mieczkowski
December 13, 2018
Introduction
One of my main research interests is the way in which someone’s language can affect perceptions of them. I am choosing to replicate Study 3 from “Inferring Identity From Language: Linguistic Intergroup Bias Informs Social Categorization” because it falls into my interest area and I might reference their hypothesis in my future work. Additionally, their focus on political party affiliation is relevant for the current political moment. The results of this study suggest favorable linguistic integroup bias (LIB), or “us[ing] abstract language to describe in-group members’ desirable behaviors and concrete language to describe their undesirable behaviors,” increased the likelihood that the participant believed the target shared the same group membership. Most people do not think about whether or not they should describe someone abstractly or concretely when talking about them, but it appears that this distinction has a notable effect. Replicating this finding would provide additional evidence that this phenomenon exists and that differences in language choice can have an effect on perception.
The stimuli in this experiment were passages about a fictional man named Peter that included indications about his political party affiliation as well as his helpful and rude behaviors. This study had four conditions: favorable/unfavorable LIB Democratic, and favorable/unfavorable LIB Republican. In the favorable LIB conditions, helpful behavior was written in abstract language and rude behavior in concrete language. The unfavorable conditions had helpful behavior in concrete language and rude behavior in abstract language. Party affiliation was indicated by past voting behavior. After reading the passages, participants answered questions about the Peter’s likely political group membership, how likely they would be friends with him and information about their own party affiliations and ideology. Participant information was used in the analysis, which was a 2 (LIB condition) x 2 (Peter’s affiliation) x 3 (participant affiliation) ANOVA. Manipulation checks on the effectiveness of the LIB treatment will be conducted as well.
The participants in this replication study will be able to complete this on Amazon Mechanical Turk, which was also the venue for the original study. When conducting this experiment, I will be careful to properly set up the study on both Qualtrics and MTurk so that participants do not run into any issues that inhibit their completion of the study. Other challenges might include cleaning the data from Qualtrics, which has been difficult for me in the past. Hopefully my new knowledge of how to tidy the data will help!
Repository: https://github.com/hnmiecz/replication_paper Original Paper: https://github.com/hnmiecz/replication_paper/blob/master/original_paper/porter2016.pdf Registration: https://osf.io/fd9pz/
Methods
Power Analysis
The original authors note that the effect sizes they are using in Study 3 are taken from Study 1a. The effect size for study 3 was d = 0.51. Using G*Power, I conducted an a priori power analysis to obtain 80% power with alpha = .05.
Planned Sample
From the power analysis, I concluded that a sample size of 123 would be appropriate. In order to avoid the extra MTurk fee, I plan to collect data from 126 participants (smallest multiple of 9 that is above 123).
Materials
The following paragraphs are taken directly from the supplemental materials provided by Porter et al. (2016) and will be used in this study in the same form:
“Study 3 Communicator Statements
Favorable LIB Condition The communicator states the following about Peter: On one occasion, there is a person in a wheelchair who needs assistance getting up a ramp. Peter reaches for the handles of the wheelchair. Peter is helpful. On the other hand, on another occasion, while at work, Peter talks to one of his colleagues. While the colleague is still talking, Peter yawns loudly, turns and begins walking away. Peter walks back to his office.
Unfavorable LIB Condition The communicator states the following about Peter: On one occasion, there is a person in a wheelchair who needs assistance getting up a ramp. Peter reaches for the handles of the wheelchair. Peter pushes the wheelchair up the ramp. On the other hand, on another occasion, while at work, Peter talks to one of his colleagues. While the colleague is still talking, Peter yawns loudly, turns and begins walking away. Peter is rude."
Procedure
The following procedure is described in Porter et al. (2016) and will be followed precisely:
“As in Study 1a, participants were asked to read a passage and then respond to questions. In the Republican-target condition, the passage indicated that Peter had voted for John McCain; in the Democratic-target condition, Peter had voted for Barack Obama. In the second part of the passage, participants were again provided with an unknown communicator’s description of Peter’s helpful and rude behaviors. Following Wigboldus et al. (2000), we included a description of one discrete episode, expressed in the present tense, for each type of behavior (for the full descriptions, see Table S1 in the Supplemental Material available online). For example, the description of helpful behavior in the favorable-LIB condition was written in abstract language and read as follows: “On one occasion, there is a person in a wheelchair who needs assistance getting up a ramp. Peter reaches for the handles of the wheelchair. Peter is helpful.” In the unfavorable-LIB condition, helpful behavior was described concretely: “On one occasion, there is a person in a wheelchair who needs assistance getting up a ramp. Peter reaches for the handles of the wheelchair. Peter pushes the wheelchair up the ramp.” After reading the passage, participants indicated the likely group membership of the communicator on an 8-point scale anchored by 1, definitely a Democrat, and 8, definitely a Republican. They then rated the likelihood that they would be friends with the communicator, using a 5-point scale ranging from 1, it is not at all likely, to 5, it is extremely likely. Finally, participants completed the manipulation-check items and a demographic questionnaire on which they reported their political-party affiliation and political ideology.”
The manipulation check and political-party affiliation questions were described as follows: “We asked participants to estimate the percentage of future situations in which Peter was likely to be helpful and the percentage of future situations in which he was likely to be rude (Semin & de Poot, 1997). Finally, participants completed a demographic questionnaire that asked their gender, their political-party affiliation, and the degree to which they endorsed liberal and conservative beliefs (on 7-point scales ranging from 1, strongly disagree, to 7, strongly agree).”
The survey can be found here: https://stanforduniversity.qualtrics.com/jfe/form/SV_6LGMhD4QCpxHBVX
Analysis Plan
First, like in the original paper, a LIB manipulation check will be conducted through an ANOVA. Participants in the favorable-LIB condition should believe that Peter will be more helpful in the future and participants in the unfavorable-LIB condition should indicate Peter will be rude in the future. If this is not the case, then the treatment will have failed. However, if it is shown to be successful, a factorial ANOVA (as completed in the original paper) will be conducted in order to ascertain any differences in dependent variable outcomes between conditions. The dependent variable is social category inference.
The key analysis of interest is an ANOVA, testing for the aforementioned main effect of LIB condition on social category inference. In order for the original main effect of LIB condition to replicate, I will look for a p-value of less than .01.
Differences from Original Study
Since the data for this study was presumably collected in 2013 (when the first draft of the manuscript was received), differences in assumptions about political partisanship may occur. As a result of alleged “echo chambers” in the current political moment, social category ratings may not replicate.
Additionally, I highlighted the word “communicator” in the question asking about social category inference to make it more salient.
Methods Addendum (Post Data Collection)
Actual Sample
Sample size: 126 Demographics: -Gender: 39.7% Female, 60.3% Male -Political Party Affiliation: 54.8% Democrat, 20.6% Republican, 22.2% Independent, 2.4% Other Data exclusions based on rules spelled out in analysis plan: No proposed exclusions
Differences from pre-data collection methods plan
None
Results
Data preparation
Load Relevant Libraries and Functions
tidyverse, dplyr, ggplot, etc.
Import data
Data from qualtrics will be in CSV format. In Excel I will remove unnecessary rows at the top of the file and rename the columns to make the variables of interest clear.
Data exclusion / filtering Exclude participants with “other” or “none” as political affiliation.
Prepare data for analysis - create columns etc. Since Qualtrics will provide data in a wide format, it will be necessary to change it into a long format. Each row will be a participant’s ratings for the questions they were asked. Since this is partially a between-subjects design, there will be four “groups” of columns. One group will be for the favorable-LIB Democrat conditon questions, another for the favorable-LIB Republican condition questions, etc. I will create new columns indicating which condition the participant is in (coded 1 to 4). In order to replicate some of the graphs, I will also need to make other columns indicating whether the participant was in a favorable or unfavorable LIB condition (LIB_condition), as well as whether they were in a Democrat or Republican condition (POL_condition). Then I will gather the data using dpylr so that question names specific to conditions are in one column and related responses are in an adjacent column.
###Data Preparation
####Load Relevant Libraries and Functions
#tidyverse, dplyr, ggplot, etc.
library(tidyverse)## ── Attaching packages ────────## ✔ ggplot2 3.1.0 ✔ purrr 0.2.5
## ✔ tibble 1.4.2 ✔ dplyr 0.7.7
## ✔ tidyr 0.8.2 ✔ stringr 1.3.1
## ✔ readr 1.1.1 ✔ forcats 0.3.0## ── Conflicts ─────────────────
## ✖ dplyr::filter() masks stats::filter()
## ✖ dplyr::lag() masks stats::lag()library(dplyr)
library(ggplot2)
library(sjstats)## Warning in checkMatrixPackageVersion(): Package version inconsistency detected.
## TMB was built with Matrix version 1.2.15
## Current Matrix version is 1.2.14
## Please re-install 'TMB' from source using install.packages('TMB', type = 'source') or ask CRAN for a binary version of 'TMB' matching CRAN's 'Matrix' package#formula for error bars
sem <- function(x) {sd(x, na.rm=TRUE) / sqrt(sum(!is.na((x))))}
ci <- function(x) {sem(x) * 1.96} # reasonable approximation
####Import data
#data from qualtrics will be in CSV format
porter3 = read.csv("/Users/Hannah/Documents/stanford/2018 autumn/psych251_full.csv")
porter3## subid comm_pol1 friendship_likelihood1 peter_helpful1 peter_rude1
## 1 1 NA NA NA NA
## 2 2 NA NA NA NA
## 3 3 NA NA NA NA
## 4 4 NA NA NA NA
## 5 5 5 3 50 50
## 6 6 NA NA NA NA
## 7 7 NA NA NA NA
## 8 8 NA NA NA NA
## 9 9 NA NA NA NA
## 10 10 6 3 60 45
## 11 11 NA NA NA NA
## 12 12 NA NA NA NA
## 13 13 NA NA NA NA
## 14 14 4 1 35 60
## 15 15 NA NA NA NA
## 16 16 NA NA NA NA
## 17 17 NA NA NA NA
## 18 18 NA NA NA NA
## 19 19 5 2 60 40
## 20 20 NA NA NA NA
## 21 21 4 1 50 50
## 22 22 8 4 50 50
## 23 23 4 1 95 1
## 24 24 NA NA NA NA
## 25 25 NA NA NA NA
## 26 26 1 4 80 20
## 27 27 NA NA NA NA
## 28 28 NA NA NA NA
## 29 29 NA NA NA NA
## 30 30 3 3 70 30
## 31 31 NA NA NA NA
## 32 32 4 3 30 80
## 33 33 NA NA NA NA
## 34 34 NA NA NA NA
## 35 35 2 2 80 20
## 36 36 NA NA NA NA
## 37 37 NA NA NA NA
## 38 38 NA NA NA NA
## 39 39 NA NA NA NA
## 40 40 NA NA NA NA
## 41 41 NA NA NA NA
## 42 42 NA NA NA NA
## 43 43 6 4 4 4
## 44 44 NA NA NA NA
## 45 45 NA NA NA NA
## 46 46 NA NA NA NA
## 47 47 NA NA NA NA
## 48 48 NA NA NA NA
## 49 49 3 4 75 50
## 50 50 NA NA NA NA
## 51 51 NA NA NA NA
## 52 52 NA NA NA NA
## 53 53 NA NA NA NA
## 54 54 NA NA NA NA
## 55 55 NA NA NA NA
## 56 56 NA NA NA NA
## 57 57 NA NA NA NA
## 58 58 NA NA NA NA
## 59 59 NA NA NA NA
## 60 60 6 3 45 76
## 61 61 NA NA NA NA
## 62 62 3 2 60 25
## 63 63 NA NA NA NA
## 64 64 NA NA NA NA
## 65 65 6 4 80 20
## 66 66 3 2 5 95
## 67 67 NA NA NA NA
## 68 68 NA NA NA NA
## 69 69 NA NA NA NA
## 70 70 NA NA NA NA
## 71 71 1 5 100 0
## 72 72 NA NA NA NA
## 73 73 NA NA NA NA
## 74 74 NA NA NA NA
## 75 75 NA NA NA NA
## 76 76 NA NA NA NA
## 77 77 NA NA NA NA
## 78 78 7 5 70 30
## 79 79 NA NA NA NA
## 80 80 NA NA NA NA
## 81 81 NA NA NA NA
## 82 82 NA NA NA NA
## 83 83 NA NA NA NA
## 84 84 NA NA NA NA
## 85 85 5 2 50 50
## 86 86 NA NA NA NA
## 87 87 NA NA NA NA
## 88 88 NA NA NA NA
## 89 89 NA NA NA NA
## 90 90 8 1 70 15
## 91 91 5 2 80 20
## 92 92 NA NA NA NA
## 93 93 NA NA NA NA
## 94 94 NA NA NA NA
## 95 95 NA NA NA NA
## 96 96 NA NA NA NA
## 97 97 NA NA NA NA
## 98 98 1 1 35 45
## 99 99 NA NA NA NA
## 100 100 NA NA NA NA
## 101 101 3 1 25 75
## 102 102 NA NA NA NA
## 103 103 4 2 10 1
## 104 104 NA NA NA NA
## 105 105 6 2 70 60
## 106 106 NA NA NA NA
## 107 107 NA NA NA NA
## 108 108 1 4 60 10
## 109 109 4 2 50 50
## 110 110 NA NA NA NA
## 111 111 NA NA NA NA
## 112 112 NA NA NA NA
## 113 113 NA NA NA NA
## 114 114 NA NA NA NA
## 115 115 NA NA NA NA
## 116 116 4 2 80 30
## 117 117 6 3 50 50
## 118 118 NA NA NA NA
## 119 119 NA NA NA NA
## 120 120 NA NA NA NA
## 121 121 6 4 50 40
## 122 122 NA NA NA NA
## 123 123 NA NA NA NA
## 124 124 NA NA NA NA
## 125 125 NA NA NA NA
## 126 126 3 3 60 20
## comm_pol2 friendship_likelihood2 peter_helpful2 peter_rude2 comm_pol3
## 1 NA NA NA NA NA
## 2 NA NA NA NA 7
## 3 NA NA NA NA NA
## 4 NA NA NA NA 6
## 5 NA NA NA NA NA
## 6 NA NA NA NA NA
## 7 5 3 50 50 NA
## 8 NA NA NA NA 4
## 9 2 2 75 25 NA
## 10 NA NA NA NA NA
## 11 3 1 50 50 NA
## 12 NA NA NA NA 5
## 13 NA NA NA NA 3
## 14 NA NA NA NA NA
## 15 NA NA NA NA NA
## 16 NA NA NA NA 2
## 17 7 5 50 25 NA
## 18 NA NA NA NA 5
## 19 NA NA NA NA NA
## 20 NA NA NA NA NA
## 21 NA NA NA NA NA
## 22 NA NA NA NA NA
## 23 NA NA NA NA NA
## 24 NA NA NA NA NA
## 25 NA NA NA NA NA
## 26 NA NA NA NA NA
## 27 NA NA NA NA NA
## 28 6 1 50 50 NA
## 29 NA NA NA NA NA
## 30 NA NA NA NA NA
## 31 7 4 75 82 NA
## 32 NA NA NA NA NA
## 33 NA NA NA NA 3
## 34 NA NA NA NA 1
## 35 NA NA NA NA NA
## 36 NA NA NA NA 5
## 37 NA NA NA NA 8
## 38 NA NA NA NA NA
## 39 NA NA NA NA NA
## 40 NA NA NA NA NA
## 41 NA NA NA NA NA
## 42 5 3 40 60 NA
## 43 NA NA NA NA NA
## 44 NA NA NA NA 7
## 45 NA NA NA NA 5
## 46 7 4 75 80 NA
## 47 NA NA NA NA NA
## 48 6 3 50 60 NA
## 49 NA NA NA NA NA
## 50 NA NA NA NA NA
## 51 NA NA NA NA 8
## 52 NA NA NA NA 3
## 53 5 4 35 56 NA
## 54 7 2 60 50 NA
## 55 6 3 50 50 NA
## 56 NA NA NA NA NA
## 57 5 3 50 50 NA
## 58 NA NA NA NA 2
## 59 NA NA NA NA NA
## 60 NA NA NA NA NA
## 61 6 2 50 50 NA
## 62 NA NA NA NA NA
## 63 4 3 50 50 NA
## 64 NA NA NA NA NA
## 65 NA NA NA NA NA
## 66 NA NA NA NA NA
## 67 NA NA NA NA NA
## 68 NA NA NA NA 3
## 69 NA NA NA NA NA
## 70 NA NA NA NA 8
## 71 NA NA NA NA NA
## 72 NA NA NA NA 8
## 73 NA NA NA NA 5
## 74 2 2 70 30 NA
## 75 3 4 40 60 NA
## 76 NA NA NA NA NA
## 77 NA NA NA NA NA
## 78 NA NA NA NA NA
## 79 8 1 95 5 NA
## 80 2 3 90 10 NA
## 81 NA NA NA NA NA
## 82 NA NA NA NA NA
## 83 2 3 50 10 NA
## 84 NA NA NA NA 6
## 85 NA NA NA NA NA
## 86 NA NA NA NA 1
## 87 5 1 80 20 NA
## 88 3 2 55 45 NA
## 89 NA NA NA NA NA
## 90 NA NA NA NA NA
## 91 NA NA NA NA NA
## 92 NA NA NA NA 5
## 93 4 2 15 10 NA
## 94 1 1 25 75 NA
## 95 NA NA NA NA NA
## 96 NA NA NA NA 2
## 97 NA NA NA NA NA
## 98 NA NA NA NA NA
## 99 NA NA NA NA 8
## 100 7 3 50 50 NA
## 101 NA NA NA NA NA
## 102 NA NA NA NA 5
## 103 NA NA NA NA NA
## 104 NA NA NA NA NA
## 105 NA NA NA NA NA
## 106 3 1 70 60 NA
## 107 7 1 10 90 NA
## 108 NA NA NA NA NA
## 109 NA NA NA NA NA
## 110 NA NA NA NA NA
## 111 NA NA NA NA NA
## 112 NA NA NA NA 3
## 113 NA NA NA NA NA
## 114 NA NA NA NA 8
## 115 4 2 50 50 NA
## 116 NA NA NA NA NA
## 117 NA NA NA NA NA
## 118 NA NA NA NA 4
## 119 4 1 75 20 NA
## 120 NA NA NA NA NA
## 121 NA NA NA NA NA
## 122 5 1 82 11 NA
## 123 NA NA NA NA NA
## 124 3 1 90 10 NA
## 125 NA NA NA NA NA
## 126 NA NA NA NA NA
## friendship_likelihood3 peter_helpful3 peter_rude3 comm_pol4
## 1 NA NA NA 6
## 2 4 80 20 NA
## 3 NA NA NA 5
## 4 2 60 20 NA
## 5 NA NA NA NA
## 6 NA NA NA 8
## 7 NA NA NA NA
## 8 3 75 25 NA
## 9 NA NA NA NA
## 10 NA NA NA NA
## 11 NA NA NA NA
## 12 2 50 50 NA
## 13 3 100 25 NA
## 14 NA NA NA NA
## 15 NA NA NA 7
## 16 1 50 50 NA
## 17 NA NA NA NA
## 18 3 50 50 NA
## 19 NA NA NA NA
## 20 NA NA NA 6
## 21 NA NA NA NA
## 22 NA NA NA NA
## 23 NA NA NA NA
## 24 NA NA NA 4
## 25 NA NA NA 8
## 26 NA NA NA NA
## 27 NA NA NA 5
## 28 NA NA NA NA
## 29 NA NA NA 8
## 30 NA NA NA NA
## 31 NA NA NA NA
## 32 NA NA NA NA
## 33 3 80 20 NA
## 34 4 50 30 NA
## 35 NA NA NA NA
## 36 3 60 40 NA
## 37 5 99 98 NA
## 38 NA NA NA 5
## 39 NA NA NA 1
## 40 NA NA NA 6
## 41 NA NA NA 4
## 42 NA NA NA NA
## 43 NA NA NA NA
## 44 2 40 60 NA
## 45 2 60 40 NA
## 46 NA NA NA NA
## 47 NA NA NA 5
## 48 NA NA NA NA
## 49 NA NA NA NA
## 50 NA NA NA 4
## 51 4 85 80 NA
## 52 3 40 50 NA
## 53 NA NA NA NA
## 54 NA NA NA NA
## 55 NA NA NA NA
## 56 NA NA NA 4
## 57 NA NA NA NA
## 58 5 25 75 NA
## 59 NA NA NA 6
## 60 NA NA NA NA
## 61 NA NA NA NA
## 62 NA NA NA NA
## 63 NA NA NA NA
## 64 NA NA NA 4
## 65 NA NA NA NA
## 66 NA NA NA NA
## 67 NA NA NA 1
## 68 3 77 80 NA
## 69 NA NA NA 6
## 70 2 60 40 NA
## 71 NA NA NA NA
## 72 1 20 80 NA
## 73 3 50 50 NA
## 74 NA NA NA NA
## 75 NA NA NA NA
## 76 NA NA NA 3
## 77 NA NA NA 8
## 78 NA NA NA NA
## 79 NA NA NA NA
## 80 NA NA NA NA
## 81 NA NA NA 4
## 82 NA NA NA 5
## 83 NA NA NA NA
## 84 5 1 2 NA
## 85 NA NA NA NA
## 86 1 50 50 NA
## 87 NA NA NA NA
## 88 NA NA NA NA
## 89 NA NA NA 5
## 90 NA NA NA NA
## 91 NA NA NA NA
## 92 2 60 10 NA
## 93 NA NA NA NA
## 94 NA NA NA NA
## 95 NA NA NA 7
## 96 3 20 40 NA
## 97 NA NA NA 7
## 98 NA NA NA NA
## 99 1 20 80 NA
## 100 NA NA NA NA
## 101 NA NA NA NA
## 102 2 25 50 NA
## 103 NA NA NA NA
## 104 NA NA NA 5
## 105 NA NA NA NA
## 106 NA NA NA NA
## 107 NA NA NA NA
## 108 NA NA NA NA
## 109 NA NA NA NA
## 110 NA NA NA 8
## 111 NA NA NA 6
## 112 1 15 65 NA
## 113 NA NA NA 6
## 114 2 30 70 NA
## 115 NA NA NA NA
## 116 NA NA NA NA
## 117 NA NA NA NA
## 118 3 90 10 NA
## 119 NA NA NA NA
## 120 NA NA NA 5
## 121 NA NA NA NA
## 122 NA NA NA NA
## 123 NA NA NA 8
## 124 NA NA NA NA
## 125 NA NA NA 4
## 126 NA NA NA NA
## friendship_likelihood4 peter_helpful4 peter_rude4 gender
## 1 1 50 50 2
## 2 NA NA NA 1
## 3 3 82 5 1
## 4 NA NA NA 1
## 5 NA NA NA 2
## 6 1 5 95 2
## 7 NA NA NA 1
## 8 NA NA NA 2
## 9 NA NA NA 2
## 10 NA NA NA 1
## 11 NA NA NA 1
## 12 NA NA NA 1
## 13 NA NA NA 2
## 14 NA NA NA 1
## 15 1 25 75 1
## 16 NA NA NA 2
## 17 NA NA NA 2
## 18 NA NA NA 1
## 19 NA NA NA 1
## 20 3 60 40 2
## 21 NA NA NA 2
## 22 NA NA NA 2
## 23 NA NA NA 1
## 24 2 50 50 2
## 25 4 70 30 2
## 26 NA NA NA 1
## 27 2 65 35 2
## 28 NA NA NA 1
## 29 3 50 50 1
## 30 NA NA NA 2
## 31 NA NA NA 2
## 32 NA NA NA 1
## 33 NA NA NA 1
## 34 NA NA NA 1
## 35 NA NA NA 2
## 36 NA NA NA 2
## 37 NA NA NA 2
## 38 1 50 50 2
## 39 1 50 50 1
## 40 3 80 15 1
## 41 2 60 40 1
## 42 NA NA NA 2
## 43 NA NA NA 1
## 44 NA NA NA 1
## 45 NA NA NA 1
## 46 NA NA NA 1
## 47 3 30 50 1
## 48 NA NA NA 2
## 49 NA NA NA 1
## 50 3 50 50 1
## 51 NA NA NA 2
## 52 NA NA NA 1
## 53 NA NA NA 1
## 54 NA NA NA 1
## 55 NA NA NA 1
## 56 2 50 50 1
## 57 NA NA NA 1
## 58 NA NA NA 1
## 59 1 50 50 1
## 60 NA NA NA 1
## 61 NA NA NA 2
## 62 NA NA NA 1
## 63 NA NA NA 1
## 64 3 40 25 1
## 65 NA NA NA 1
## 66 NA NA NA 2
## 67 2 60 40 2
## 68 NA NA NA 2
## 69 2 50 50 2
## 70 NA NA NA 1
## 71 NA NA NA 2
## 72 NA NA NA 1
## 73 NA NA NA 1
## 74 NA NA NA 1
## 75 NA NA NA 1
## 76 2 65 35 1
## 77 5 50 50 1
## 78 NA NA NA 2
## 79 NA NA NA 1
## 80 NA NA NA 1
## 81 2 50 50 1
## 82 2 25 66 1
## 83 NA NA NA 1
## 84 NA NA NA 1
## 85 NA NA NA 2
## 86 NA NA NA 1
## 87 NA NA NA 2
## 88 NA NA NA 1
## 89 3 60 40 2
## 90 NA NA NA 2
## 91 NA NA NA 2
## 92 NA NA NA 1
## 93 NA NA NA 1
## 94 NA NA NA 1
## 95 2 20 80 1
## 96 NA NA NA 1
## 97 4 60 40 1
## 98 NA NA NA 2
## 99 NA NA NA 1
## 100 NA NA NA 2
## 101 NA NA NA 2
## 102 NA NA NA 2
## 103 NA NA NA 1
## 104 3 40 70 2
## 105 NA NA NA 2
## 106 NA NA NA 2
## 107 NA NA NA 1
## 108 NA NA NA 1
## 109 NA NA NA 2
## 110 1 50 65 1
## 111 2 75 40 1
## 112 NA NA NA 1
## 113 2 50 25 1
## 114 NA NA NA 2
## 115 NA NA NA 2
## 116 NA NA NA 2
## 117 NA NA NA 2
## 118 NA NA NA 2
## 119 NA NA NA 1
## 120 1 45 55 1
## 121 NA NA NA 1
## 122 NA NA NA 2
## 123 2 50 50 1
## 124 NA NA NA 1
## 125 2 60 20 2
## 126 NA NA NA 1
## participant_pol liberal_beliefs conservative_beliefs
## 1 1 6 1
## 2 3 2 7
## 3 1 5 3
## 4 1 5 4
## 5 2 4 4
## 6 1 7 1
## 7 1 6 3
## 8 3 3 5
## 9 1 7 1
## 10 3 2 5
## 11 2 4 4
## 12 1 6 1
## 13 2 4 4
## 14 3 2 6
## 15 1 7 2
## 16 3 1 6
## 17 1 6 7
## 18 2 4 4
## 19 1 7 1
## 20 4 5 1
## 21 1 5 3
## 22 3 6 5
## 23 2 5 2
## 24 1 7 1
## 25 2 4 7
## 26 1 5 3
## 27 3 2 6
## 28 1 6 2
## 29 1 7 1
## 30 1 7 1
## 31 3 6 7
## 32 1 6 4
## 33 2 4 4
## 34 1 6 2
## 35 1 6 1
## 36 2 6 2
## 37 3 6 6
## 38 1 7 1
## 39 3 5 7
## 40 2 6 1
## 41 1 7 1
## 42 3 1 7
## 43 1 6 6
## 44 1 6 2
## 45 1 7 1
## 46 1 6 5
## 47 1 5 4
## 48 1 5 3
## 49 1 5 2
## 50 1 6 3
## 51 3 7 6
## 52 1 6 5
## 53 1 5 3
## 54 1 7 2
## 55 2 5 2
## 56 1 5 2
## 57 2 4 4
## 58 1 6 2
## 59 2 4 4
## 60 1 5 3
## 61 1 6 2
## 62 1 6 2
## 63 2 5 5
## 64 3 3 5
## 65 3 6 3
## 66 1 6 2
## 67 2 4 4
## 68 1 7 1
## 69 1 7 1
## 70 3 2 6
## 71 3 5 6
## 72 1 7 1
## 73 1 5 3
## 74 2 5 3
## 75 2 6 5
## 76 3 2 6
## 77 3 1 7
## 78 3 6 5
## 79 1 7 1
## 80 2 6 2
## 81 3 3 5
## 82 1 6 2
## 83 1 6 2
## 84 2 6 6
## 85 3 3 6
## 86 2 4 4
## 87 1 7 1
## 88 1 5 2
## 89 1 5 5
## 90 1 7 2
## 91 4 4 1
## 92 1 6 2
## 93 1 6 1
## 94 3 4 6
## 95 1 5 5
## 96 1 6 1
## 97 2 4 4
## 98 1 4 2
## 99 1 7 2
## 100 3 3 6
## 101 2 4 4
## 102 1 7 1
## 103 1 3 4
## 104 1 4 4
## 105 2 6 2
## 106 1 5 2
## 107 1 5 4
## 108 1 7 1
## 109 1 6 2
## 110 2 3 3
## 111 2 2 6
## 112 1 7 1
## 113 1 6 2
## 114 1 7 1
## 115 2 4 4
## 116 2 5 5
## 117 3 5 3
## 118 1 5 4
## 119 1 6 1
## 120 4 6 1
## 121 3 6 6
## 122 2 3 3
## 123 2 5 5
## 124 3 1 7
## 125 1 6 3
## 126 1 7 1#### Data exclusion / filtering
list(unique(porter3$participant_pol))## [[1]]
## [1] 1 3 2 4list(unique(porter3$gender))## [[1]]
## [1] 2 1porter3$participant_pol = recode(porter3$participant_pol, "1" = "Democrat", "2" = "Independent", "3" = "Republican", "4" = "Other")
porter3$gender = recode(porter3$gender, "1" ="Male", "2" ="Female")
substrRight <- function(x, n){
substr(x, nchar(x)-n+1, nchar(x))
}
cut_last_number = function(x) {
substr(x, 0, nchar(x)-1)
}
#### Prepare data for analysis - create columns etc.
porter3_long = porter3 %>%
gather("question", "response",
-c(subid, participant_pol, gender,
liberal_beliefs, conservative_beliefs)) %>%
filter(!is.na(response)) %>%
mutate(condition = substrRight(question, 1)) %>%
mutate(question = cut_last_number(question)) %>%
mutate(condition = factor(
condition, levels = 1:4,
labels = c("Favorable LIB ~~~ Democrat",
"Unfavorable LIB ~~~ Democrat",
"Unfavorable LIB ~~~ Republican",
"Favorable LIB ~~~ Republican"))) %>%
separate(condition, into=c("LIB_condition", "POL_condition"),
sep=" ~~~ ") %>%
spread(question, response)
porter3_long## subid gender participant_pol liberal_beliefs conservative_beliefs
## 1 1 Female Democrat 6 1
## 2 2 Male Republican 2 7
## 3 3 Male Democrat 5 3
## 4 4 Male Democrat 5 4
## 5 5 Female Independent 4 4
## 6 6 Female Democrat 7 1
## 7 7 Male Democrat 6 3
## 8 8 Female Republican 3 5
## 9 9 Female Democrat 7 1
## 10 10 Male Republican 2 5
## 11 11 Male Independent 4 4
## 12 12 Male Democrat 6 1
## 13 13 Female Independent 4 4
## 14 14 Male Republican 2 6
## 15 15 Male Democrat 7 2
## 16 16 Female Republican 1 6
## 17 17 Female Democrat 6 7
## 18 18 Male Independent 4 4
## 19 19 Male Democrat 7 1
## 20 20 Female Other 5 1
## 21 21 Female Democrat 5 3
## 22 22 Female Republican 6 5
## 23 23 Male Independent 5 2
## 24 24 Female Democrat 7 1
## 25 25 Female Independent 4 7
## 26 26 Male Democrat 5 3
## 27 27 Female Republican 2 6
## 28 28 Male Democrat 6 2
## 29 29 Male Democrat 7 1
## 30 30 Female Democrat 7 1
## 31 31 Female Republican 6 7
## 32 32 Male Democrat 6 4
## 33 33 Male Independent 4 4
## 34 34 Male Democrat 6 2
## 35 35 Female Democrat 6 1
## 36 36 Female Independent 6 2
## 37 37 Female Republican 6 6
## 38 38 Female Democrat 7 1
## 39 39 Male Republican 5 7
## 40 40 Male Independent 6 1
## 41 41 Male Democrat 7 1
## 42 42 Female Republican 1 7
## 43 43 Male Democrat 6 6
## 44 44 Male Democrat 6 2
## 45 45 Male Democrat 7 1
## 46 46 Male Democrat 6 5
## 47 47 Male Democrat 5 4
## 48 48 Female Democrat 5 3
## 49 49 Male Democrat 5 2
## 50 50 Male Democrat 6 3
## 51 51 Female Republican 7 6
## 52 52 Male Democrat 6 5
## 53 53 Male Democrat 5 3
## 54 54 Male Democrat 7 2
## 55 55 Male Independent 5 2
## 56 56 Male Democrat 5 2
## 57 57 Male Independent 4 4
## 58 58 Male Democrat 6 2
## 59 59 Male Independent 4 4
## 60 60 Male Democrat 5 3
## 61 61 Female Democrat 6 2
## 62 62 Male Democrat 6 2
## 63 63 Male Independent 5 5
## 64 64 Male Republican 3 5
## 65 65 Male Republican 6 3
## 66 66 Female Democrat 6 2
## 67 67 Female Independent 4 4
## 68 68 Female Democrat 7 1
## 69 69 Female Democrat 7 1
## 70 70 Male Republican 2 6
## 71 71 Female Republican 5 6
## 72 72 Male Democrat 7 1
## 73 73 Male Democrat 5 3
## 74 74 Male Independent 5 3
## 75 75 Male Independent 6 5
## 76 76 Male Republican 2 6
## 77 77 Male Republican 1 7
## 78 78 Female Republican 6 5
## 79 79 Male Democrat 7 1
## 80 80 Male Independent 6 2
## 81 81 Male Republican 3 5
## 82 82 Male Democrat 6 2
## 83 83 Male Democrat 6 2
## 84 84 Male Independent 6 6
## 85 85 Female Republican 3 6
## 86 86 Male Independent 4 4
## 87 87 Female Democrat 7 1
## 88 88 Male Democrat 5 2
## 89 89 Female Democrat 5 5
## 90 90 Female Democrat 7 2
## 91 91 Female Other 4 1
## 92 92 Male Democrat 6 2
## 93 93 Male Democrat 6 1
## 94 94 Male Republican 4 6
## 95 95 Male Democrat 5 5
## 96 96 Male Democrat 6 1
## 97 97 Male Independent 4 4
## 98 98 Female Democrat 4 2
## 99 99 Male Democrat 7 2
## 100 100 Female Republican 3 6
## 101 101 Female Independent 4 4
## 102 102 Female Democrat 7 1
## 103 103 Male Democrat 3 4
## 104 104 Female Democrat 4 4
## 105 105 Female Independent 6 2
## 106 106 Female Democrat 5 2
## 107 107 Male Democrat 5 4
## 108 108 Male Democrat 7 1
## 109 109 Female Democrat 6 2
## 110 110 Male Independent 3 3
## 111 111 Male Independent 2 6
## 112 112 Male Democrat 7 1
## 113 113 Male Democrat 6 2
## 114 114 Female Democrat 7 1
## 115 115 Female Independent 4 4
## 116 116 Female Independent 5 5
## 117 117 Female Republican 5 3
## 118 118 Female Democrat 5 4
## 119 119 Male Democrat 6 1
## 120 120 Male Other 6 1
## 121 121 Male Republican 6 6
## 122 122 Female Independent 3 3
## 123 123 Male Independent 5 5
## 124 124 Male Republican 1 7
## 125 125 Female Democrat 6 3
## 126 126 Male Democrat 7 1
## LIB_condition POL_condition comm_pol friendship_likelihood
## 1 Favorable LIB Republican 6 1
## 2 Unfavorable LIB Republican 7 4
## 3 Favorable LIB Republican 5 3
## 4 Unfavorable LIB Republican 6 2
## 5 Favorable LIB Democrat 5 3
## 6 Favorable LIB Republican 8 1
## 7 Unfavorable LIB Democrat 5 3
## 8 Unfavorable LIB Republican 4 3
## 9 Unfavorable LIB Democrat 2 2
## 10 Favorable LIB Democrat 6 3
## 11 Unfavorable LIB Democrat 3 1
## 12 Unfavorable LIB Republican 5 2
## 13 Unfavorable LIB Republican 3 3
## 14 Favorable LIB Democrat 4 1
## 15 Favorable LIB Republican 7 1
## 16 Unfavorable LIB Republican 2 1
## 17 Unfavorable LIB Democrat 7 5
## 18 Unfavorable LIB Republican 5 3
## 19 Favorable LIB Democrat 5 2
## 20 Favorable LIB Republican 6 3
## 21 Favorable LIB Democrat 4 1
## 22 Favorable LIB Democrat 8 4
## 23 Favorable LIB Democrat 4 1
## 24 Favorable LIB Republican 4 2
## 25 Favorable LIB Republican 8 4
## 26 Favorable LIB Democrat 1 4
## 27 Favorable LIB Republican 5 2
## 28 Unfavorable LIB Democrat 6 1
## 29 Favorable LIB Republican 8 3
## 30 Favorable LIB Democrat 3 3
## 31 Unfavorable LIB Democrat 7 4
## 32 Favorable LIB Democrat 4 3
## 33 Unfavorable LIB Republican 3 3
## 34 Unfavorable LIB Republican 1 4
## 35 Favorable LIB Democrat 2 2
## 36 Unfavorable LIB Republican 5 3
## 37 Unfavorable LIB Republican 8 5
## 38 Favorable LIB Republican 5 1
## 39 Favorable LIB Republican 1 1
## 40 Favorable LIB Republican 6 3
## 41 Favorable LIB Republican 4 2
## 42 Unfavorable LIB Democrat 5 3
## 43 Favorable LIB Democrat 6 4
## 44 Unfavorable LIB Republican 7 2
## 45 Unfavorable LIB Republican 5 2
## 46 Unfavorable LIB Democrat 7 4
## 47 Favorable LIB Republican 5 3
## 48 Unfavorable LIB Democrat 6 3
## 49 Favorable LIB Democrat 3 4
## 50 Favorable LIB Republican 4 3
## 51 Unfavorable LIB Republican 8 4
## 52 Unfavorable LIB Republican 3 3
## 53 Unfavorable LIB Democrat 5 4
## 54 Unfavorable LIB Democrat 7 2
## 55 Unfavorable LIB Democrat 6 3
## 56 Favorable LIB Republican 4 2
## 57 Unfavorable LIB Democrat 5 3
## 58 Unfavorable LIB Republican 2 5
## 59 Favorable LIB Republican 6 1
## 60 Favorable LIB Democrat 6 3
## 61 Unfavorable LIB Democrat 6 2
## 62 Favorable LIB Democrat 3 2
## 63 Unfavorable LIB Democrat 4 3
## 64 Favorable LIB Republican 4 3
## 65 Favorable LIB Democrat 6 4
## 66 Favorable LIB Democrat 3 2
## 67 Favorable LIB Republican 1 2
## 68 Unfavorable LIB Republican 3 3
## 69 Favorable LIB Republican 6 2
## 70 Unfavorable LIB Republican 8 2
## 71 Favorable LIB Democrat 1 5
## 72 Unfavorable LIB Republican 8 1
## 73 Unfavorable LIB Republican 5 3
## 74 Unfavorable LIB Democrat 2 2
## 75 Unfavorable LIB Democrat 3 4
## 76 Favorable LIB Republican 3 2
## 77 Favorable LIB Republican 8 5
## 78 Favorable LIB Democrat 7 5
## 79 Unfavorable LIB Democrat 8 1
## 80 Unfavorable LIB Democrat 2 3
## 81 Favorable LIB Republican 4 2
## 82 Favorable LIB Republican 5 2
## 83 Unfavorable LIB Democrat 2 3
## 84 Unfavorable LIB Republican 6 5
## 85 Favorable LIB Democrat 5 2
## 86 Unfavorable LIB Republican 1 1
## 87 Unfavorable LIB Democrat 5 1
## 88 Unfavorable LIB Democrat 3 2
## 89 Favorable LIB Republican 5 3
## 90 Favorable LIB Democrat 8 1
## 91 Favorable LIB Democrat 5 2
## 92 Unfavorable LIB Republican 5 2
## 93 Unfavorable LIB Democrat 4 2
## 94 Unfavorable LIB Democrat 1 1
## 95 Favorable LIB Republican 7 2
## 96 Unfavorable LIB Republican 2 3
## 97 Favorable LIB Republican 7 4
## 98 Favorable LIB Democrat 1 1
## 99 Unfavorable LIB Republican 8 1
## 100 Unfavorable LIB Democrat 7 3
## 101 Favorable LIB Democrat 3 1
## 102 Unfavorable LIB Republican 5 2
## 103 Favorable LIB Democrat 4 2
## 104 Favorable LIB Republican 5 3
## 105 Favorable LIB Democrat 6 2
## 106 Unfavorable LIB Democrat 3 1
## 107 Unfavorable LIB Democrat 7 1
## 108 Favorable LIB Democrat 1 4
## 109 Favorable LIB Democrat 4 2
## 110 Favorable LIB Republican 8 1
## 111 Favorable LIB Republican 6 2
## 112 Unfavorable LIB Republican 3 1
## 113 Favorable LIB Republican 6 2
## 114 Unfavorable LIB Republican 8 2
## 115 Unfavorable LIB Democrat 4 2
## 116 Favorable LIB Democrat 4 2
## 117 Favorable LIB Democrat 6 3
## 118 Unfavorable LIB Republican 4 3
## 119 Unfavorable LIB Democrat 4 1
## 120 Favorable LIB Republican 5 1
## 121 Favorable LIB Democrat 6 4
## 122 Unfavorable LIB Democrat 5 1
## 123 Favorable LIB Republican 8 2
## 124 Unfavorable LIB Democrat 3 1
## 125 Favorable LIB Republican 4 2
## 126 Favorable LIB Democrat 3 3
## peter_helpful peter_rude
## 1 50 50
## 2 80 20
## 3 82 5
## 4 60 20
## 5 50 50
## 6 5 95
## 7 50 50
## 8 75 25
## 9 75 25
## 10 60 45
## 11 50 50
## 12 50 50
## 13 100 25
## 14 35 60
## 15 25 75
## 16 50 50
## 17 50 25
## 18 50 50
## 19 60 40
## 20 60 40
## 21 50 50
## 22 50 50
## 23 95 1
## 24 50 50
## 25 70 30
## 26 80 20
## 27 65 35
## 28 50 50
## 29 50 50
## 30 70 30
## 31 75 82
## 32 30 80
## 33 80 20
## 34 50 30
## 35 80 20
## 36 60 40
## 37 99 98
## 38 50 50
## 39 50 50
## 40 80 15
## 41 60 40
## 42 40 60
## 43 4 4
## 44 40 60
## 45 60 40
## 46 75 80
## 47 30 50
## 48 50 60
## 49 75 50
## 50 50 50
## 51 85 80
## 52 40 50
## 53 35 56
## 54 60 50
## 55 50 50
## 56 50 50
## 57 50 50
## 58 25 75
## 59 50 50
## 60 45 76
## 61 50 50
## 62 60 25
## 63 50 50
## 64 40 25
## 65 80 20
## 66 5 95
## 67 60 40
## 68 77 80
## 69 50 50
## 70 60 40
## 71 100 0
## 72 20 80
## 73 50 50
## 74 70 30
## 75 40 60
## 76 65 35
## 77 50 50
## 78 70 30
## 79 95 5
## 80 90 10
## 81 50 50
## 82 25 66
## 83 50 10
## 84 1 2
## 85 50 50
## 86 50 50
## 87 80 20
## 88 55 45
## 89 60 40
## 90 70 15
## 91 80 20
## 92 60 10
## 93 15 10
## 94 25 75
## 95 20 80
## 96 20 40
## 97 60 40
## 98 35 45
## 99 20 80
## 100 50 50
## 101 25 75
## 102 25 50
## 103 10 1
## 104 40 70
## 105 70 60
## 106 70 60
## 107 10 90
## 108 60 10
## 109 50 50
## 110 50 65
## 111 75 40
## 112 15 65
## 113 50 25
## 114 30 70
## 115 50 50
## 116 80 30
## 117 50 50
## 118 90 10
## 119 75 20
## 120 45 55
## 121 50 40
## 122 82 11
## 123 50 50
## 124 90 10
## 125 60 20
## 126 60 20#Proportions of participants reporting different genders, political party affiliation
prop.table(table(porter3_long$gender))##
## Female Male
## 0.3968254 0.6031746prop.table(table(porter3_long$participant_pol))##
## Democrat Independent Other Republican
## 0.54761905 0.22222222 0.02380952 0.20634921Manipulation Check Analysis
Two main analyses will be conducted. The first is the LIB manipulation check. I will utilize an ANOVA to see if there are significant differences between the conditions in terms of how helpful or rude the participants expect Peter to be in the future. Tukey HSD tests will be conducted post-hoc if significant differences are found.
manip_check_rude.aov = lm(peter_rude ~ LIB_condition*POL_condition, porter3_long)
anova(manip_check_rude.aov)## Analysis of Variance Table
##
## Response: peter_rude
## Df Sum Sq Mean Sq F value Pr(>F)
## LIB_condition 1 225 224.51 0.4325 0.5120
## POL_condition 1 1242 1241.71 2.3921 0.1245
## LIB_condition:POL_condition 1 215 214.72 0.4137 0.5213
## Residuals 122 63329 519.09anova_stats(manip_check_rude.aov)## term df sumsq meansq statistic p.value
## 1 LIB_condition 1 224.509 224.509 0.433 0.512
## 2 POL_condition 1 1241.715 1241.715 2.392 0.125
## 3 LIB_condition:POL_condition 1 214.725 214.725 0.414 0.521
## 4 Residuals 122 63329.051 519.091 NA NA
## etasq partial.etasq omegasq partial.omegasq cohens.f power
## 1 0.003 0.004 -0.004 -0.005 0.060 0.101
## 2 0.019 0.019 0.011 0.011 0.140 0.340
## 3 0.003 0.003 -0.005 -0.005 0.058 0.099
## 4 NA NA NA NA NA NAmanip_check_helpful.aov = lm(peter_helpful ~ LIB_condition*POL_condition, porter3_long)
anova(manip_check_helpful.aov)## Analysis of Variance Table
##
## Response: peter_helpful
## Df Sum Sq Mean Sq F value Pr(>F)
## LIB_condition 1 60 59.62 0.1231 0.7263
## POL_condition 1 688 687.84 1.4200 0.2357
## LIB_condition:POL_condition 1 7 6.61 0.0136 0.9072
## Residuals 122 59096 484.40anova_stats(manip_check_helpful.aov)## term df sumsq meansq statistic p.value
## 1 LIB_condition 1 59.616 59.616 0.123 0.726
## 2 POL_condition 1 687.841 687.841 1.420 0.236
## 3 LIB_condition:POL_condition 1 6.607 6.607 0.014 0.907
## 4 Residuals 122 59096.293 484.396 NA NA
## etasq partial.etasq omegasq partial.omegasq cohens.f power
## 1 0.001 0.001 -0.007 -0.007 0.032 0.064
## 2 0.011 0.012 0.003 0.003 0.108 0.222
## 3 0.000 0.000 -0.008 -0.008 0.011 0.052
## 4 NA NA NA NA NA NAUnlike in Porter et al., LIB condition did not have a significant effect on the likelihood of Peter being helpful or rude in the future.
Confirmatory Analysis
Similar ANOVAs will be conducted for the social category inference DV, this time taking into account the participant’s political affiliation. Tukey HSD tests will be conducted post-hoc if significant differences are found.
#test for main effect of LIB condition on social category inference
social_category_hyp.aov = lm(comm_pol ~ LIB_condition*POL_condition*participant_pol, porter3_long)
anova(social_category_hyp.aov)## Analysis of Variance Table
##
## Response: comm_pol
## Df Sum Sq Mean Sq F value
## LIB_condition 1 0.53 0.5336 0.1435
## POL_condition 1 14.51 14.5119 3.9020
## participant_pol 3 5.52 1.8385 0.4943
## LIB_condition:POL_condition 1 8.33 8.3294 2.2397
## LIB_condition:participant_pol 2 21.21 10.6044 2.8514
## POL_condition:participant_pol 3 4.58 1.5283 0.4109
## LIB_condition:POL_condition:participant_pol 2 30.82 15.4108 4.1437
## Residuals 112 416.53 3.7191
## Pr(>F)
## LIB_condition 0.70556
## POL_condition 0.05069 .
## participant_pol 0.68693
## LIB_condition:POL_condition 0.13732
## LIB_condition:participant_pol 0.06197 .
## POL_condition:participant_pol 0.74545
## LIB_condition:POL_condition:participant_pol 0.01836 *
## Residuals
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1anova_stats(social_category_hyp.aov)## term df sumsq meansq statistic
## 1 LIB_condition 1 0.534 0.534 0.143
## 2 POL_condition 1 14.512 14.512 3.902
## 3 participant_pol 3 5.516 1.839 0.494
## 4 LIB_condition:POL_condition 1 8.329 8.329 2.240
## 5 LIB_condition:participant_pol 2 21.209 10.604 2.851
## 6 POL_condition:participant_pol 3 4.585 1.528 0.411
## 7 LIB_condition:POL_condition:participant_pol 2 30.822 15.411 4.144
## 8 Residuals 112 416.534 3.719 NA
## p.value etasq partial.etasq omegasq partial.omegasq cohens.f power
## 1 0.706 0.001 0.001 -0.006 -0.007 0.036 0.067
## 2 0.051 0.029 0.034 0.021 0.023 0.187 0.506
## 3 0.687 0.011 0.013 -0.011 -0.012 0.115 0.152
## 4 0.137 0.017 0.020 0.009 0.010 0.141 0.322
## 5 0.062 0.042 0.048 0.027 0.029 0.226 0.561
## 6 0.745 0.009 0.011 -0.013 -0.014 0.105 0.133
## 7 0.018 0.061 0.069 0.046 0.048 0.272 0.734
## 8 NA NA NA NA NA NA NA#prepare data for plotting
porter3_long_plot = porter3_long %>%
group_by(LIB_condition, POL_condition) %>%
summarise(mean_comm_pol = mean(comm_pol),
ci = ci(comm_pol))
porter3_long_plot## # A tibble: 4 x 4
## # Groups: LIB_condition [?]
## LIB_condition POL_condition mean_comm_pol ci
## <chr> <chr> <dbl> <dbl>
## 1 Favorable LIB Democrat 4.28 0.672
## 2 Favorable LIB Republican 5.41 0.614
## 3 Unfavorable LIB Democrat 4.65 0.671
## 4 Unfavorable LIB Republican 4.83 0.826#plot data
ggplot(porter3_long_plot, aes(x=POL_condition, y=mean_comm_pol, fill=LIB_condition)) +
geom_bar(position="dodge", stat="identity") +
geom_errorbar(aes(ymin = mean_comm_pol-ci, ymax = mean_comm_pol+ci), width = .2, position = position_dodge(.9)) +
xlab("Target's Political Affiliation") +
ylab(expression(atop("Likelihood that Communicator and", paste("Target Share Same Political Affiliation")))) +
labs(fill="LIB Condition") +
scale_fill_brewer(palette="Set2")
Porter et al., Figure 3
Exploratory analyses
Porter et al. (2016) also conducted an analysis to test the interaction of Peter’s political affiliation and participant’s political affiliation on the likelihood that the participant would be friends with the target. A replication would be a significant interaction effect with a p-value of <.01. Additionally, since the manipulation check of LIB condition on Peter’s likelihood of rudeness and helpfulness failed, I removed the participants that indicated there would be a 50% likelihood of Peter being rude and a 50% likelihood of Peter being helpful.
#test for interaction effect
friendship_likelihood_hyp.aov = lm(friendship_likelihood ~ LIB_condition*POL_condition*participant_pol, porter3_long)
anova(friendship_likelihood_hyp.aov)## Analysis of Variance Table
##
## Response: friendship_likelihood
## Df Sum Sq Mean Sq F value
## LIB_condition 1 0.115 0.1154 0.0899
## POL_condition 1 0.066 0.0658 0.0513
## participant_pol 3 8.552 2.8508 2.2197
## LIB_condition:POL_condition 1 3.914 3.9142 3.0477
## LIB_condition:participant_pol 2 1.992 0.9961 0.7756
## POL_condition:participant_pol 3 2.399 0.7996 0.6226
## LIB_condition:POL_condition:participant_pol 2 2.492 1.2460 0.9702
## Residuals 112 143.842 1.2843
## Pr(>F)
## LIB_condition 0.76488
## POL_condition 0.82129
## participant_pol 0.08977 .
## LIB_condition:POL_condition 0.08359 .
## LIB_condition:participant_pol 0.46290
## POL_condition:participant_pol 0.60186
## LIB_condition:POL_condition:participant_pol 0.38218
## Residuals
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1#prepare data for plotting
porter3_long_plot2 = porter3_long %>%
select(participant_pol, POL_condition, friendship_likelihood) %>%
filter(participant_pol == "Democrat"| participant_pol == "Republican"|participant_pol =="Independent") %>%
group_by(POL_condition, participant_pol) %>%
summarise(mean_friendship = mean(friendship_likelihood),
ci = ci(friendship_likelihood))
porter3_long_plot2## # A tibble: 6 x 4
## # Groups: POL_condition [?]
## POL_condition participant_pol mean_friendship ci
## <chr> <chr> <dbl> <dbl>
## 1 Democrat Democrat 2.38 0.388
## 2 Democrat Independent 2.21 0.511
## 3 Democrat Republican 3.07 0.725
## 4 Republican Democrat 2.26 0.304
## 5 Republican Independent 2.64 0.637
## 6 Republican Republican 2.83 0.794#plot data
ggplot(porter3_long_plot2, aes(x=participant_pol, y=mean_friendship, fill=POL_condition)) +
geom_bar(position="dodge", stat="identity") +
geom_errorbar(aes(ymin = mean_friendship-ci, ymax = mean_friendship+ci), width = .2, position = position_dodge(.9)) +
xlab("Participant's Political Affiliation") +
ylab(expression(atop("Likelihood of Becoming Friends", paste("with the Communicator")))) +
labs(fill="Target's Political Affiliation") +
scale_fill_brewer(palette="Set2")
#test if removing 50/50 alters manipulation check results
porter3_filter = porter3_long %>% filter(peter_helpful == 50 & peter_rude == 50)
porter3_filter## subid gender participant_pol liberal_beliefs conservative_beliefs
## 1 1 Female Democrat 6 1
## 2 5 Female Independent 4 4
## 3 7 Male Democrat 6 3
## 4 11 Male Independent 4 4
## 5 12 Male Democrat 6 1
## 6 16 Female Republican 1 6
## 7 18 Male Independent 4 4
## 8 21 Female Democrat 5 3
## 9 22 Female Republican 6 5
## 10 24 Female Democrat 7 1
## 11 28 Male Democrat 6 2
## 12 29 Male Democrat 7 1
## 13 38 Female Democrat 7 1
## 14 39 Male Republican 5 7
## 15 50 Male Democrat 6 3
## 16 55 Male Independent 5 2
## 17 56 Male Democrat 5 2
## 18 57 Male Independent 4 4
## 19 59 Male Independent 4 4
## 20 61 Female Democrat 6 2
## 21 63 Male Independent 5 5
## 22 69 Female Democrat 7 1
## 23 73 Male Democrat 5 3
## 24 77 Male Republican 1 7
## 25 81 Male Republican 3 5
## 26 85 Female Republican 3 6
## 27 86 Male Independent 4 4
## 28 100 Female Republican 3 6
## 29 109 Female Democrat 6 2
## 30 115 Female Independent 4 4
## 31 117 Female Republican 5 3
## 32 123 Male Independent 5 5
## LIB_condition POL_condition comm_pol friendship_likelihood
## 1 Favorable LIB Republican 6 1
## 2 Favorable LIB Democrat 5 3
## 3 Unfavorable LIB Democrat 5 3
## 4 Unfavorable LIB Democrat 3 1
## 5 Unfavorable LIB Republican 5 2
## 6 Unfavorable LIB Republican 2 1
## 7 Unfavorable LIB Republican 5 3
## 8 Favorable LIB Democrat 4 1
## 9 Favorable LIB Democrat 8 4
## 10 Favorable LIB Republican 4 2
## 11 Unfavorable LIB Democrat 6 1
## 12 Favorable LIB Republican 8 3
## 13 Favorable LIB Republican 5 1
## 14 Favorable LIB Republican 1 1
## 15 Favorable LIB Republican 4 3
## 16 Unfavorable LIB Democrat 6 3
## 17 Favorable LIB Republican 4 2
## 18 Unfavorable LIB Democrat 5 3
## 19 Favorable LIB Republican 6 1
## 20 Unfavorable LIB Democrat 6 2
## 21 Unfavorable LIB Democrat 4 3
## 22 Favorable LIB Republican 6 2
## 23 Unfavorable LIB Republican 5 3
## 24 Favorable LIB Republican 8 5
## 25 Favorable LIB Republican 4 2
## 26 Favorable LIB Democrat 5 2
## 27 Unfavorable LIB Republican 1 1
## 28 Unfavorable LIB Democrat 7 3
## 29 Favorable LIB Democrat 4 2
## 30 Unfavorable LIB Democrat 4 2
## 31 Favorable LIB Democrat 6 3
## 32 Favorable LIB Republican 8 2
## peter_helpful peter_rude
## 1 50 50
## 2 50 50
## 3 50 50
## 4 50 50
## 5 50 50
## 6 50 50
## 7 50 50
## 8 50 50
## 9 50 50
## 10 50 50
## 11 50 50
## 12 50 50
## 13 50 50
## 14 50 50
## 15 50 50
## 16 50 50
## 17 50 50
## 18 50 50
## 19 50 50
## 20 50 50
## 21 50 50
## 22 50 50
## 23 50 50
## 24 50 50
## 25 50 50
## 26 50 50
## 27 50 50
## 28 50 50
## 29 50 50
## 30 50 50
## 31 50 50
## 32 50 50manip_check_rude_filter.aov = lm(peter_rude ~ LIB_condition*POL_condition, porter3_filter)
anova(manip_check_rude_filter.aov)## Warning in anova.lm(manip_check_rude_filter.aov): ANOVA F-tests on an
## essentially perfect fit are unreliable## Analysis of Variance Table
##
## Response: peter_rude
## Df Sum Sq Mean Sq F value Pr(>F)
## LIB_condition 1 3.5340e-28 3.5341e-28 0.7424 0.3962
## POL_condition 1 2.2390e-28 2.2394e-28 0.4704 0.4984
## LIB_condition:POL_condition 1 1.8000e-28 1.7995e-28 0.3780 0.5436
## Residuals 28 1.3329e-26 4.7602e-28manip_check_helpful_filter.aov = lm(peter_helpful ~ LIB_condition*POL_condition, porter3_filter)
anova(manip_check_helpful_filter.aov)## Warning in anova.lm(manip_check_helpful_filter.aov): ANOVA F-tests on an
## essentially perfect fit are unreliable## Analysis of Variance Table
##
## Response: peter_helpful
## Df Sum Sq Mean Sq F value Pr(>F)
## LIB_condition 1 3.5340e-28 3.5341e-28 0.7424 0.3962
## POL_condition 1 2.2390e-28 2.2394e-28 0.4704 0.4984
## LIB_condition:POL_condition 1 1.8000e-28 1.7995e-28 0.3780 0.5436
## Residuals 28 1.3329e-26 4.7602e-28The interaction effect did not replicate, and removing the 50/50 participants did not not alter the results of the manipulation check substantially. The effect of LIB condition of helpfulness/rudeness did not replicate.
Discussion
Summary of Replication Attempt
This replication attempt failed to replicate the results of Porter et al. (2016) that were in question. Namely, the manipulation check of LIB condition on Peter’s likelihood to be helpful OR rude was not significant, the main effect of LIB condition on social category inference was not significant, and the interaction effect of Peter’s political affiliation and the participant’s political affiliation on friendship likelihood was not significant. The failed replication of LIB condition is the most notable result, as this effect was present in all four of the studies in the original paper.
Commentary
As discussed in Mike’s blog (http://babieslearninglanguage.blogspot.com/2018/12/how-to-run-study-that-doesnt-replicate.html), there are several aspects of this study that may contribute to a failed replication. For instance, both social category inference and frienship likelihood are single-item DVs; the study is between subjects, and the manipulation is induced by a single paragraph. Past work has shown that these factors are associated with lower likelihoods of replication.
Despite this failed replication, I think there still might be an effect of LIB on some outcome variables. Other work has shown that very subtle changes in language do have an effect on people’s perceptions (Pennebaker, 2011), so I would not be surprised if LIB does influence variables such as social category inference. However, in order to truly understand these effects, a better study design would be necessary. Perhaps a within-subjects setup could be used, as well as multiple-item DVs.