Replication of Study 3 by Kwan, Dai & Wyer,JR (2017, Journal of Consumer Research)
Design Overview
- One factor was manipulated. The space around the message.
- Three measures: response time, favorability and recall
- Between-subject.
- Yes, each measure was repeated 10 times for the 10 stimuli.
- No, it is not possible to do a within-subject design. Participants’ responses may be influenced by previous conditions, leading to confounding results.
- Yes, we used between-subject design, so participants were unaware of what the manipulation is. Also, we randomized the order of 10 quotes for each participant, so their responses to the first quote is not likely to influence their responses to subsequent questions.
- One potential confound could be the attrition bias, participants may have more difficulty recalling the information in the more space condition, thus they feel frustrated and drop out of the study more. This creates a bias in the remaining data.
Introduction
The main objective of this replication project is to thoroughly assess the findings of “Contextual Influences on Message Persuasion: The Effect of Empty Space” by Kwan, Dai, and Wyer from 2017. The original finding was that message recipients infer a weaker opinion when a message is surrounded by empty space, making them less likely to accept its implications. We aim to focus on replicating Study 3 from the original article, where the authors found that people evaluated the statements less favorably and paid less attention to a message when it was surrounded by substantial empty space. By replicating the study’s methods and analyzing the data in a comparable setting, we intend to evaluate the reliability and applicability of the original conclusions. This project will enhance our understanding of the contextual effect and strengthen the credibility of the influence of empty space on persuasion in the field.
Methods
Power Analysis
To preface, the following estimate is based on default inputs in GPower. The original paper did not report sufficient parameters to accurately determine an effect size. Based on the f-test we conducted in GPower, we found that for an effect size of 0.25, an estimated 102 participants would be needed to achieve a power of 0.8.
Planned Sample
Ninety-four US residents were recruited on Mechanical Turk for a monetary incentive. There were no preselection rules to our knowledge. In the paper there were 36 male participants. The amount of the monetary incentive was not disclosed but in an independent study mentioned in the paper the particpants were payed $.20.
Materials
We will create ten statements drawn from social media, covering topics such as romance, happiness, and personal values. These statements will vary in length from 5 to 9 words and will be presented uniformly in terms of font type, size, text positioning, line spacing, paragraphing, and background graphics.
The quote we selected is as follows: Keep calm and carry on. Men never remember but women never forget. The best mirror is an old friend. Happiness shared is happiness doubled. Love is shown more in deeds than in words. Every day is a chance to grow. Take each day one step at a time. A heart in love has no limits. Life is too precious to waste on regrets. Make time for what makes you smile.
In the Limited Space Condition, quotes will be shown in a box sized between 420 × 315 pixels and 660 × 165 pixels, with no extra space around the border. In the Empty Space Condition, quotes will be displayed in a box ranging from 960 × 720 pixels to 960 × 240 pixels, surrounded by significant empty space.
Procedure
Participants first completed a survey called “Quotes-of-the-Year,” during which they evaluated 10 statements sourced from social media platforms like Twitter and Facebook. These statements varied in length from 3 to 11 words and addressed various topics, including ro- mance (e.g., “try to reason about love and you will lose your reason”), happiness (e.g., “life is too short for tears”), and personal values (e.g., “follow your heart”). All statements were pre- sented in the same font type, size, text positioning, line spacing, paragraphing, and background graphics.
In the limited space condition, each quote was displayed in a box measuring between 420 × 315 pixels and 660 × 165 pixels, with no extra space around the border. In contrast, the empty space condition featured a box size ranging from 960 × 720 pixels to 960 × 240 pixels, surrounded by significant empty space.
Participants reviewed all 10 quotes in both conditions. After reading each quote, they indicated how much they liked it and how important they thought it was, using a scale from 1 (not at all) to 7 (very much). Their responses were averaged to create a single measure of message persuasiveness ( = .88). Additionally, the time spent evaluating each quote was recorded, with the total time serving as an indicator of message deliberation. Finally, participants reported their age and gender.
Analysis Plan
We plan to run F-tests to compare favorability, response time between the empty space condition and the limited space condition.
Differences from Original Study
As the original appendix only provides three pairs of examples from the ten pairs of materials used in the original study, we plan to recreate the other seven pairs of materials using quotes gathered from social media. Based on the findings of the original article, if empty spaces cause people to rate the quotes less favorably, the same effect should also exist if the content of the materials changes. Thus, we argue that this difference should not influence our replication of the original findings.
Methods Addendum (Post Data Collection)
You can comment this section out prior to final report with data collection.
Actual Sample
Sample size, demographics, data exclusions based on rules spelled out in analysis plan
Differences from pre-data collection methods plan
Any differences from what was described as the original plan, or “none”.
Results
Data preparation
Data preparation would include: a) Data import and initial inspection
data <- read.csv(“/Users/a123/Downloads/replication_pilot 3/Data_3.csv”, header = TRUE)
str(data) summary(data) #inspect head(data) #preview
#Assigning conditions
empty_space <- data\(Response.Time[data\)Condition == “empty_space”] limited_space <- data\(Response.Time[data\)Condition == “limited_space”]
my_data <- na.omit(data) # Removes rows with any NA values ### Convert space_condition to factor if not already my_data\(space_condition <- as.factor(my_data\)space_condition) ### Checking for duplicates using (dulicated) to ensure no rows are repeated
Removing participants with incomplete responses
my_data <- filter(my_data, complete_cases == TRUE)
Check the structure after data preparation
str(data) summary(data)
Load Relevant Libraries and Functions
library(“dplyr”) library (“lavaan”) #### Import data my_data <- read.csv(file.choose()) #### Data exclusion / filtering sample_n(my_data, 10) favor.ftest <- var.test(empty_space ~ limited_space, data = data) time.ftest <- var.test(empty_time ~ limited_time, data = my_data) favor.ftest favor.ftest\(estimate favor.ftest\)p.value time.ftest time.ftest\(estimate time.ftest\)p.value
Justification of F test
The main finding from the original study suggests that participants evaluated statements less favorably in the “empty space” condition compared to the “limited space” condition. This indicates that the core hypothesis is centered around a mean difference in favorability scores between these two conditions.
An ANOVA F-test is designed to compare means between groups, which is precisely what’s needed here. It will provide a single F-statistic and p-value to confirm whether the difference in evaluation scores between the “empty space” and “limited space” conditions is statistically significant.
The F-test is both powerful and well-suited for comparing two or more groups, especially with continuous data like evaluation scores. It is more straightforward than alternatives (e.g., t-tests) when confirming findings in studies with multiple conditions.
Further, the original study used an F-test or ANOVA to analyze this comparison, then using the same approach is consistent with replication goals, further justifying its use.
```
Confirmatory analysis
#F test
f_test_result <- var.test(empty_space, limited_space)
#Print the result of F test print(f_test_result)
#Anova
anova_result <- aov(Response.Time ~ Condition, data = data) summary(anova_result)
Discussion
Summary of Replication Attempt
Open the discussion section with a paragraph summarizing the primary result from the confirmatory analysis and the assessment of whether it replicated, partially replicated, or failed to replicate the original result.
Commentary
Add open-ended commentary (if any) reflecting (a) insights from follow-up exploratory analysis, (b) assessment of the meaning of the replication (or not) - e.g., for a failure to replicate, are the differences between original and present study ones that definitely, plausibly, or are unlikely to have been moderators of the result, and (c) discussion of any objections or challenges raised by the current and original authors about the replication attempt. None of these need to be long.