Replication of Study 3 by Kwan, Dai & Wyer,JR (2017, Journal of Consumer Research)

Author

Arushi Srivastava, Shirley Agustin, Elaine Young (arsrivastava@ucsd.edu )

Published

Invalid Date

Design Overview

  1. One factor was manipulated. The space around the message.
  2. Three measures: response time, favorability and recall
  3. Between-subject.
  4. Yes, each measure was repeated 10 times for the 10 stimuli.
  5. No, it is not possible to do a within-subject design. Participants’ responses may be influenced by previous conditions, leading to confounding results.
  6. Yes, we used between-subject design, so participants were unaware of what the manipulation is. Also, we randomized the order of 10 quotes for each participant, so their responses to the first quote is not likely to influence their responses to subsequent questions.
  7. One potential confound could be the attrition bias, participants may have more difficulty recalling the information in the more space condition, thus they feel frustrated and drop out of the study more. This creates a bias in the remaining data.

Introduction

The main objective of this replication project is to thoroughly assess the findings of “Contextual Influences on Message Persuasion: The Effect of Empty Space” by Kwan, Dai, and Wyer from 2017. The original finding was that message recipients infer a weaker opinion when a message is surrounded by empty space, making them less likely to accept its implications. We aim to focus on replicating Study 3 from the original article, where the authors found that people evaluated the statements less favorably and paid less attention to a message when it was surrounded by substantial empty space. By replicating the study’s methods and analyzing the data in a comparable setting, we intend to evaluate the reliability and applicability of the original conclusions. This project will enhance our understanding of the contextual effect and strengthen the credibility of the influence of empty space on persuasion in the field.

GitHub Repository

Original Paper

Methods

Power Analysis

Original effect size, power analysis for samples to achieve 80%, 90%, 95% power to detect that effect size. Considerations of feasibility for selecting planned sample size.

Planned Sample

Ninety-four US residents were recruited on Mechanical Turk for a monetary incentive. There were no preselection rules to our knowledge. In the paper there were 36 male participants. The amount of the monetary incentive was not disclosed but in an independent study mentioned in the paper the particpants were payed $.20.

Materials

We will create ten statements drawn from social media, covering topics such as romance, happiness, and personal values. These statements will vary in length from 5 to 9 words and will be presented uniformly in terms of font type, size, text positioning, line spacing, paragraphing, and background graphics.

The quote we selected is as follows: Keep calm and carry on. Men never remember but women never forget. The best mirror is an old friend. Happiness shared is happiness doubled. Love is shown more in deeds than in words. Every day is a chance to grow. Take each day one step at a time. A heart in love has no limits. Life is too precious to waste on regrets. Make time for what makes you smile.

In the Limited Space Condition, quotes will be shown in a box sized between 420 × 315 pixels and 660 × 165 pixels, with no extra space around the border. In the Empty Space Condition, quotes will be displayed in a box ranging from 960 × 720 pixels to 960 × 240 pixels, surrounded by significant empty space.

Procedure

Participants first completed a survey called “Quotes-of-the-Year,” during which they evaluated 10 statements sourced from social media platforms like Twitter and Facebook. These statements varied in length from 3 to 11 words and addressed various topics, including ro- mance (e.g., “try to reason about love and you will lose your reason”), happiness (e.g., “life is too short for tears”), and personal values (e.g., “follow your heart”). All statements were pre- sented in the same font type, size, text positioning, line spacing, paragraphing, and background graphics.

In the limited space condition, each quote was displayed in a box measuring between 420 × 315 pixels and 660 × 165 pixels, with no extra space around the border. In contrast, the empty space condition featured a box size ranging from 960 × 720 pixels to 960 × 240 pixels, surrounded by significant empty space.

Participants reviewed all 10 quotes in both conditions. After reading each quote, they indicated how much they liked it and how important they thought it was, using a scale from 1 (not at all) to 7 (very much). Their responses were averaged to create a single measure of message persuasiveness ( = .88). Additionally, the time spent evaluating each quote was recorded, with the total time serving as an indicator of message deliberation. Finally, participants reported their age and gender before being given the first few words of each quote as a cue and asked to recall it as accurately as possible.

In the end of the study, we include an attention check question “The color test you are about to take part in is very simple, when asked for your favorite color you must select ‘Green’. This is an attention check.Based on the text you read above, what color have you been asked to enter? (Red/Blue/Green/Orange/Brown) We exclude participants choosing other options besides ‘Green.’

Analysis Plan

We plan to run F-tests to compare favorability, response time, and ability to recall between the empty space condition and the limited space condition. Additionally, we plan to build an SPSS model (model 4) to explore whether message recall mediates the effect of empty space on message persuasion, using a bootstrap sample of 5,000.

Differences from Original Study

As the original appendix only provides three pairs of examples from the ten pairs of materials used in the original study, we plan to recreate the other seven pairs of materials using quotes gathered from social media. Based on the findings of the original article, if empty spaces cause people to rate the quotes less favorably and recall them less, the same effect should also exist if the content of the materials changes. Thus, we argue that this difference should not influence our replication of the original findings.

Methods Addendum (Post Data Collection)

You can comment this section out prior to final report with data collection.

Actual Sample

Sample size, demographics, data exclusions based on rules spelled out in analysis plan

Differences from pre-data collection methods plan

Any differences from what was described as the original plan, or “none”.

Results

Data preparation

Data preparation would include: a) Data import and initial inspection

data <- read.csv(“/Users/a91981/Downloads/replication_pilot/Data2.csv”, header = TRUE)

str(data) summary(data) #inspect head(data) #preview

read.csv(“/Users/a91981/Downloads/replication_pilot/Data2.csv”, row.names = NULL, check.names = FALSE)

#Assigning conditions

empty_space <- data\(Response.Time[data\)Condition == “empty_space”] limited_space <- data\(Response.Time[data\)Condition == “limited_space”]

  1. Data Cleaning my_data <- na.omit(my_data) # Removes rows with any NA values ### Convert space_condition to factor if not already my_data\(space_condition <- as.factor(my_data\)space_condition) ### Checking for duplicates using (dulicated) to ensure no rows are repeated

  2. Defining exclusion criteria ### Removing participants with incomplete responses my_data <- filter(my_data, complete_cases == TRUE) ### Removing participants who failed the attention check my_data <- my_data %>% filter(attention_check == “Green”)

  3. Final Preparation ### Check the structure after data preparation str(data) summary(data)

Load Relevant Libraries and Functions

library(“dplyr”) library (“lavaan”) #### Import data my_data <- read.csv(file.choose()) #### Data exclusion / filtering sample_n(my_data, 10) favor.ftest <- var.test(empty_space ~ limited_space, data = data) time.ftest <- var.test(empty_time ~ limited_time, data = my_data) favor.ftest favor.ftest\(estimate favor.ftest\)p.value time.ftest time.ftest\(estimate time.ftest\)p.value

Prepare data for analysis - create columns etc.

The numbers of correct recall

Specify the quote to match

specific_quote1 <- “Keep calm and carry on.” specific_quote2 <- “Men never remember but women never forget.” specific_quote3 <- “The best mirror is an old friend.” specific_quote4 <- “Happiness shared is happiness doubled.” specific_quote5 <- “Love is shown more in deeds than in words.” specific_quote6 <- “Every day is a chance to grow.” specific_quote7 <- “Take each day one step at a time.” specific_quote8 <- “A heart in love has no limits.” specific_quote9 <- “Life is too precious to waste on regrets.” specific_quote10 <- “Make time for what makes you smile.”

Create a new column to indicate if there is an exact match

my_data\(match1 <- ifelse(my_data\)quote1 == specific_quote1, TRUE, FALSE) my_data\(match2 <- ifelse(my_data\)quote2 == specific_quote2, TRUE, FALSE) my_data\(match3 <- ifelse(my_data\)quote3 == specific_quote3, TRUE, FALSE) my_data\(match4 <- ifelse(my_data\)quote4 == specific_quote4, TRUE, FALSE) my_data\(match5 <- ifelse(my_data\)quote5 == specific_quote5, TRUE, FALSE) my_data\(match6 <- ifelse(my_data\)quote6 == specific_quote6, TRUE, FALSE) my_data\(match7 <- ifelse(my_data\)quote7 == specific_quote7, TRUE, FALSE) my_data\(match8 <- ifelse(my_data\)quote8 == specific_quote8, TRUE, FALSE) my_data\(match9 <- ifelse(my_data\)quote9 == specific_quote9, TRUE, FALSE) my_data\(match10 <- ifelse(my_data\)quote10 == specific_quote10, TRUE, FALSE)

Create a new column to indicate the numbers of exact match for a participant

my_data\(recall <- rowSums(my_data\)match1, my_data\(match2, my_data\)match3, my_data\(match4,my_data\)match5,my_data\(match6,my_data\)match7,my_data\(match8,my_data\)match9,my_data$match10)

Compare the numbers of recall for two conditions, and run another F test

Build SPSS Model 4 to run mediation analysis

View the updated data frame

print(data)

Justification of F test

The main finding from the original study suggests that participants evaluated statements less favorably in the “empty space” condition compared to the “limited space” condition. This indicates that the core hypothesis is centered around a mean difference in favorability scores between these two conditions.

An ANOVA F-test is designed to compare means between groups, which is precisely what’s needed here. It will provide a single F-statistic and p-value to confirm whether the difference in evaluation scores between the “empty space” and “limited space” conditions is statistically significant.

The F-test is both powerful and well-suited for comparing two or more groups, especially with continuous data like evaluation scores. It is more straightforward than alternatives (e.g., t-tests) when confirming findings in studies with multiple conditions.

Further, the original study used an F-test or ANOVA to analyze this comparison, then using the same approach is consistent with replication goals, further justifying its use.

```

Confirmatory analysis

#F test

f_test_result <- var.test(empty_space, limited_space) print(f_test_result)

#T Test

Perform independent t-test

t_test_result <- t.test(empty_space, limited_space)