Replication of Study 3 by Kwan, Dai & Wyer,JR (2017, Journal of Consumer Research)
Introduction
The main objective of this replication project is to thoroughly assess the findings of “Contextual Influences on Message Persuasion: The Effect of Empty Space” by Kwan, Dai, and Wyer from 2017. The original finding was that message recipients infer a weaker opinion when a message is surrounded by empty space, making them less likely to accept its implications. We aim to focus on replicating Study 3 from the original article, where the authors found that people evaluated the statements less favorably and paid less attention to a message when it was surrounded by substantial empty space. By replicating the study’s methods and analyzing the data in a comparable setting, we intend to evaluate the reliability and applicability of the original conclusions. This project will enhance our understanding of the contextual effect and strengthen the credibility of the influence of empty space on persuasion in the field.
Methods
Power Analysis
Original effect size, power analysis for samples to achieve 80%, 90%, 95% power to detect that effect size. Considerations of feasibility for selecting planned sample size.
Planned Sample
Ninety-four US residents were recruited on Mechanical Turk for a monetary incentive. There were no preselection rules to our knowledge. In the paper there were 36 male participants. The amount of the monetary incentive was not disclosed but in an independent study mentioned in the paper the particpants were payed $.20.
Materials
We will create ten statements drawn from social media, covering topics such as romance, happiness, and personal values. These statements will vary in length from 5 to 9 words and will be presented uniformly in terms of font type, size, text positioning, line spacing, paragraphing, and background graphics.
The quote we selected is as follows: Keep calm and carry on. Men never remember but women never forget. The best mirror is an old friend. Happiness shared is happiness doubled. Love is shown more in deeds than in words. Every day is a chance to grow. Take each day one step at a time. A heart in love has no limits. Life is too precious to waste on regrets. Make time for what makes you smile.
In the Limited Space Condition, quotes will be shown in a box sized between 420 × 315 pixels and 660 × 165 pixels, with no extra space around the border. In the Empty Space Condition, quotes will be displayed in a box ranging from 960 × 720 pixels to 960 × 240 pixels, surrounded by significant empty space.
Procedure
Participants first completed a survey called “Quotes-of-the-Year,” during which they evaluated 10 statements sourced from social media platforms like Twitter and Facebook. These statements varied in length from 3 to 11 words and addressed various topics, including ro- mance (e.g., “try to reason about love and you will lose your reason”), happiness (e.g., “life is too short for tears”), and personal values (e.g., “follow your heart”). All statements were pre- sented in the same font type, size, text positioning, line spacing, paragraphing, and background graphics.
In the limited space condition, each quote was displayed in a box measuring between 420 × 315 pixels and 660 × 165 pixels, with no extra space around the border. In contrast, the empty space condition featured a box size ranging from 960 × 720 pixels to 960 × 240 pixels, surrounded by significant empty space.
Participants reviewed all 10 quotes in both conditions. After reading each quote, they indicated how much they liked it and how important they thought it was, using a scale from 1 (not at all) to 7 (very much). Their responses were averaged to create a single measure of message persuasiveness ( = .88). Additionally, the time spent evaluating each quote was recorded, with the total time serving as an indicator of message deliberation. Finally, participants reported their age and gender before being given the first few words of each quote as a cue and asked to recall it as accurately as possible.
In the end of the study, we include an attention check question “The color test you are about to take part in is very simple, when asked for your favorite color you must select ‘Green’. This is an attention check.Based on the text you read above, what color have you been asked to enter? (Red/Blue/Green/Orange/Brown) We exclude participants choosing other options besides ‘Green.’
Analysis Plan
We plan to run F-tests to compare favorability, response time, and ability to recall between the empty space condition and the limited space condition. Additionally, we plan to build an SPSS model (model 4) to explore whether message recall mediates the effect of empty space on message persuasion, using a bootstrap sample of 5,000.
Differences from Original Study
As the original appendix only provides three pairs of examples from the ten pairs of materials used in the original study, we plan to recreate the other seven pairs of materials using quotes gathered from social media. Based on the findings of the original article, if empty spaces cause people to rate the quotes less favorably and recall them less, the same effect should also exist if the content of the materials changes. Thus, we argue that this difference should not influence our replication of the original findings.
Methods Addendum (Post Data Collection)
You can comment this section out prior to final report with data collection.
Actual Sample
Sample size, demographics, data exclusions based on rules spelled out in analysis plan
Differences from pre-data collection methods plan
Any differences from what was described as the original plan, or “none”.
Results
Data preparation
Data preparation would include: a) Data import and initial inspection my_data <- read.csv(file.choose()) str(my_data) summary(my_data) #inspect head(my_data) #preview
Data Cleaning my_data <- na.omit(my_data) # Removes rows with any NA values # Convert space_condition to factor if not already my_data\(space_condition <- as.factor(my_data\)space_condition) #Checking for duplicates using (dulicated) to ensure no rows are repeated
Defining exclusion criteria #Removing participants with incomplete responses my_data <- filter(my_data, complete_cases == TRUE) #Removing participants who failed the attention check my_data <- my_data %>% filter(attention_check == “Green”)
Final Preparation # Check the structure after data preparation str(my_data) summary(my_data)
Load Relevant Libraries and Functions
library(“dplyr”) library (“lavaan”) #### Import data my_data <- read.csv(file.choose()) #### Data exclusion / filtering sample_n(my_data, 10) favor.ftest <- var.test(empty_favor ~ limited_favor, data = my_data) time.ftest <- var.test(empty_time ~ limited_time, data = my_data) favor.ftest favor.ftest\(estimate favor.ftest\)p.value time.ftest time.ftest\(estimate time.ftest\)p.value #### Prepare data for analysis - create columns etc. # The numbers of correct recall
Specify the quote to match
specific_quote1 <- “Keep calm and carry on.” specific_quote2 <- “Men never remember but women never forget.” specific_quote3 <- “The best mirror is an old friend.” specific_quote4 <- “Happiness shared is happiness doubled.” specific_quote5 <- “Love is shown more in deeds than in words.” specific_quote6 <- “Every day is a chance to grow.” specific_quote7 <- “Take each day one step at a time.” specific_quote8 <- “A heart in love has no limits.” specific_quote9 <- “Life is too precious to waste on regrets.” specific_quote10 <- “Make time for what makes you smile.”
Create a new column to indicate if there is an exact match
my_data\(match1 <- ifelse(my_data\)quote1 == specific_quote1, TRUE, FALSE) my_data\(match2 <- ifelse(my_data\)quote2 == specific_quote2, TRUE, FALSE) my_data\(match3 <- ifelse(my_data\)quote3 == specific_quote3, TRUE, FALSE) my_data\(match4 <- ifelse(my_data\)quote4 == specific_quote4, TRUE, FALSE) my_data\(match5 <- ifelse(my_data\)quote5 == specific_quote5, TRUE, FALSE) my_data\(match6 <- ifelse(my_data\)quote6 == specific_quote6, TRUE, FALSE) my_data\(match7 <- ifelse(my_data\)quote7 == specific_quote7, TRUE, FALSE) my_data\(match8 <- ifelse(my_data\)quote8 == specific_quote8, TRUE, FALSE) my_data\(match9 <- ifelse(my_data\)quote9 == specific_quote9, TRUE, FALSE) my_data\(match10 <- ifelse(my_data\)quote10 == specific_quote10, TRUE, FALSE)
Create a new column to indicate the numbers of exact match for a participant
my_data\(recall <- rowSums(my_data\)match1, my_data\(match2, my_data\)match3, my_data\(match4,my_data\)match5,my_data\(match6,my_data\)match7,my_data\(match8,my_data\)match9,my_data$match10)
Compare the numbers of recall for two conditions, and run another F test
Build SPSS Model 4 to run mediation analysis
View the updated data frame
print(data)
#Justification of F test #The main finding from the original study suggests that participants evaluated statements less favorably in the “empty space” condition compared to the “limited space” condition. This indicates that the core hypothesis is centered around a mean difference in favorability scores between these two conditions.
#An ANOVA F-test is designed to compare means between groups, which is precisely what’s needed here. It will provide a single F-statistic and p-value to confirm whether the difference in evaluation scores between the “empty space” and “limited space” conditions is statistically significant.
#The F-test is both powerful and well-suited for comparing two or more groups, especially with continuous data like evaluation scores. It is more straightforward than alternatives (e.g., t-tests) when confirming findings in studies with multiple conditions.
#Further, the original study used an F-test or ANOVA to analyze this comparison, then using the same approach is consistent with replication goals, further justifying its use.
```
Confirmatory analysis
Define the CFA model
#“favor”, “time”, and “recall” are latent factors with observed variables. # Replace favor_var1, favor_var2, etc., with the actual names of observed variables cfa_model <- ’ # Define latent variables favor =~ favor_var1 + favor_var2 + favor_var3 time =~ time_var1 + time_var2 + time_var3 recall =~ recall_var1 + recall_var2 + recall_var3 ’
Fit the model to the data
cfa_fit <- cfa(cfa_model, data = my_data)
Summarize the CFA model results
summary(cfa_fit, fit.measures = TRUE, standardized = TRUE)
View modification indices (to suggest model improvements)
modificationIndices(cfa_fit, sort = TRUE)
Side-by-side graph with original graph is ideal here
Exploratory analyses
Any follow-up analyses desired (not required).
Discussion
Summary of Replication Attempt
Open the discussion section with a paragraph summarizing the primary result from the confirmatory analysis and the assessment of whether it replicated, partially replicated, or failed to replicate the original result.
Commentary
Add open-ended commentary (if any) reflecting (a) insights from follow-up exploratory analysis, (b) assessment of the meaning of the replication (or not) - e.g., for a failure to replicate, are the differences between original and present study ones that definitely, plausibly, or are unlikely to have been moderators of the result, and (c) discussion of any objections or challenges raised by the current and original authors about the replication attempt. None of these need to be long.