Replication of Study 3 by Kwan, Dai & Wyer,JR (2017, Journal of Consumer Research)

Author

Shirley Agustin (shagustin@ucsd.edu), Arushi Srivastava (arsrivastava@ucsd.edu), Elaine Young

Published

November 23, 2024

Introduction

This replication project is based on the third study that was run in the “Contextual Influences on Message Persuasion: The Effect of Empty Space” by Kwan, Dai, and Wyer,JR (2017, Journal of Consumer Research). In this study it was found that participants did not evaluate the statements with empty space highly. However, participants were able to recall more statements in the empty space rather than the ones in the limited space. This project will further our understanding of empty space in the field of advertising while also getting a better understanding of visual cues and verbal messages.

GitHub Repository

Original Paper

Experimental Paradigm

Methods

Power Analysis

To preface, the following estimate is based on default inputs in G Power. The original paper did not report enough parameters to determine an effect size accurately. Based on the f-test that we ran on G Power, we found that for an effect size of 0.25 we would need an estimate of 102 participants to achieve 0.8 power.

Design Overview

The factor that was manipulated in this study was the empty spaces surrounding the statements presented to the participants. These two conditions are labeled as limited space condition and empty space condition.

The measure that was taken in this study was a survey given to participants to understand favorability of the statements in different conditions. This measure was not repeated and the study used within-participants.

The experiment could not use between-participants as particpants need to be exposed to both conditions for the experiment to run.

Steps were not taken to reduce demand characteristics. The potential confounds in the study would be the amount of words in the statements selected and the themes of the statements. They may be more likely to recall quotes with a smaller amount of words and some themes may resonate with participants more which may cause participants to recall certain quotes.

Planned Sample

The original study had ninety four US residents that were recruited on Mechanical Turk for a monetary incentive. The amount of the monetary incentive was not disclosed by the authors. Although in an independent study mentioned in the general discussion of the paper it was disclosed that participants were payed $.20. The only known demographics in this sample was the mention of 36 male participants in the study.

There were no preselection rules mentioned in the study.

Materials

“We will create ten statements drawn from social media, covering topics such as romance, happiness, and personal values. These statements will vary in length from 3 to 11 words and will be presented uniformly in terms of font type, size, text positioning, line spacing, paragraphing, and background graphics”.

“In the Limited Space Condition, quotes will be shown in a box sized between 420 × 315 pixels and 660 × 165 pixels, with no extra space around the border. In the Empty Space Condition, quotes will be displayed in a box ranging from 960 × 720 pixels to 960 × 240 pixels, surrounded by significant empty space”.

The materials above will be precisely the same.

The quotes that will be presented to the participants are the following:

  1. Keep calm and carry on.
  2. Men never remember but women never forget.
  3. The best mirror is an old friend.
  4. Happiness shared is happiness doubled.
  5. Love is shown more in deeds than in words.
  6. Every day is a chance to grow.
  7. Take each day one step at a time.
  8. A heart in love has no limits.
  9. Life is too precious to waste on regrets.
  10. Make time for what makes you smile.

Procedure

“Participants first completed a survey called ‘Quotes-of-the-Year,’ during which they evaluated 10 statements sourced from social media platforms like Twitter and Facebook. These statements varied in length from 3 to 11 words and addressed various topics, including ro- mance (e.g., ‘try to reason about love and you will lose your reason’), happiness (e.g., ‘life is too short for tears’), and personal values (e.g., ‘follow your heart’). All statements were pre- sented in the same font type, size, text positioning, line spacing, paragraphing, and background graphics”.

“In the limited space condition, each quote was displayed in a box measuring between 420 × 315 pixels and 660 × 165 pixels, with no extra space around the border. In contrast, the empty space condition featured a box size ranging from 960 × 720 pixels to 960 × 240 pixels, surrounded by significant empty space”.

“Participants reviewed all 10 quotes in both conditions. After reading each quote, they indicated how much they liked it and how important they thought it was, using a scale from 1 (not at all) to 7 (very much). Their responses were averaged to create a single measure of message persuasiveness ( = .88). Additionally, the time spent evaluating each quote was recorded, with the total time serving as an indicator of message deliberation”.

“Finally, participants reported their age and gender before being given the first few words of each quote as a cue and asked to recall it as accurately as possible.”

Analysis Plan

We plan to run F-tests to compare favorability, and response time between the empty space condition and the limited space condition.

Differences from Original Study

Our group is going to select new messages to present to the participants. In the original appendix of the study there is only three pairs of examples provided. The group will select messages from social media. The findings in the original study claim that empty spaces will cause the messages to be rated less favorably. Therefore, the content of the messages should not play a big factor in recall rate and favorability.

Methods Addendum (Post Data Collection)

You can comment this section out prior to final report with data collection.

Actual Sample

Sample size, demographics, data exclusions based on rules spelled out in analysis plan

Differences from pre-data collection methods plan

Any differences from what was described as the original plan, or “none”.

Results

Data preparation

  1. Data import and initial inspection
data <-  read.csv("/Users/shirley/Downloads/PSYCH201A/replication_pilot 2/Data_3.csv", header = TRUE)

str(data)
'data.frame':   156 obs. of  9 variables:
 $ P1           : chr  "P1" "P1" "P1" "P1" ...
 $ Trial.type   : chr  "preload" "html-keyboard-response" "instructions" "html-keyboard-response" ...
 $ Trial.index  : int  0 1 2 3 4 5 6 7 8 9 ...
 $ Time.elapsed : int  145 6029 10035 16304 27215 32778 39633 43125 48979 54924 ...
 $ Condition    : chr  "limited_space" "limited_space" "limited_space" "limited_space" ...
 $ Response.Time: int  NA 5884 4004 6268 10907 5561 6851 3490 5850 5942 ...
 $ Response.2   : chr  "" "Alt" "" "ArrowDown" ...
 $ Trial.ID     : chr  "" "" "" "" ...
 $ Ques.order   : chr  "" "" "" "" ...
summary(data) #inspect
      P1             Trial.type         Trial.index    Time.elapsed   
 Length:156         Length:156         Min.   : 0.0   Min.   :   145  
 Class :character   Class :character   1st Qu.: 6.0   1st Qu.: 43905  
 Mode  :character   Mode  :character   Median :12.5   Median :100148  
                                       Mean   :12.5   Mean   :143438  
                                       3rd Qu.:19.0   3rd Qu.:191862  
                                       Max.   :25.0   Max.   :490934  
                                                                      
  Condition         Response.Time     Response.2          Trial.ID        
 Length:156         Min.   :  1641   Length:156         Length:156        
 Class :character   1st Qu.:  4797   Class :character   Class :character  
 Mode  :character   Median :  7616   Mode  :character   Mode  :character  
                    Mean   : 12351                                        
                    3rd Qu.: 12732                                        
                    Max.   :252943                                        
                    NA's   :6                                             
  Ques.order       
 Length:156        
 Class :character  
 Mode  :character  
                   
                   
                   
                   
head(data) #preview
  P1             Trial.type Trial.index Time.elapsed     Condition
1 P1                preload           0          145 limited_space
2 P1 html-keyboard-response           1         6029 limited_space
3 P1           instructions           2        10035 limited_space
4 P1 html-keyboard-response           3        16304 limited_space
5 P1          survey-likert           4        27215 limited_space
6 P1          survey-likert           5        32778 limited_space
  Response.Time      Response.2        Trial.ID Ques.order
1            NA                                           
2          5884             Alt                           
3          4004                                           
4          6268       ArrowDown                           
5         10907 {"Q0":1,"Q1":3} likert_question      [0,1]
6          5561 {"Q0":1,"Q1":2} likert_question      [0,1]
read.csv("/Users/shirley/Downloads/PSYCH201A/replication_pilot 2/Data_3.csv", row.names = NULL, check.names = FALSE)
    P1             Trial type Trial index Time elapsed     Condition
1   P1                preload           0          145 limited_space
2   P1 html-keyboard-response           1         6029 limited_space
3   P1           instructions           2        10035 limited_space
4   P1 html-keyboard-response           3        16304 limited_space
5   P1          survey-likert           4        27215 limited_space
6   P1          survey-likert           5        32778 limited_space
7   P1          survey-likert           6        39633 limited_space
8   P1          survey-likert           7        43125 limited_space
9   P1          survey-likert           8        48979 limited_space
10  P1          survey-likert           9        54924 limited_space
11  P1          survey-likert          10        59134 limited_space
12  P1          survey-likert          11        78365 limited_space
13  P1          survey-likert          12        83775 limited_space
14  P1          survey-likert          13        91845 limited_space
15  P1            survey-text          14        95433 limited_space
16  P1    survey-multi-choice          15        97076 limited_space
17  P1            survey-text          16       119289 limited_space
18  P1            survey-text          17       125953 limited_space
19  P1            survey-text          18       132728 limited_space
20  P1            survey-text          19       136608 limited_space
21  P1            survey-text          20       143139 limited_space
22  P1            survey-text          21       157852 limited_space
23  P1            survey-text          22       162891 limited_space
24  P1            survey-text          23       172141 limited_space
25  P1            survey-text          24       180051 limited_space
26  P1            survey-text          25       185765 limited_space
27  P2                preload           0          367   empty_space
28  P2 html-keyboard-response           1         5260   empty_space
29  P2           instructions           2        17985   empty_space
30  P2 html-keyboard-response           3        28927   empty_space
31  P2          survey-likert           4        63157   empty_space
32  P2          survey-likert           5        75894   empty_space
33  P2          survey-likert           6        93182   empty_space
34  P2          survey-likert           7       102575   empty_space
35  P2          survey-likert           8       118386   empty_space
36  P2          survey-likert           9       126028   empty_space
37  P2          survey-likert          10       158077   empty_space
38  P2          survey-likert          11       170448   empty_space
39  P2          survey-likert          12       179493   empty_space
40  P2          survey-likert          13       189483   empty_space
41  P2            survey-text          14       194640   empty_space
42  P2    survey-multi-choice          15       197906   empty_space
43  P2            survey-text          16       209245   empty_space
44  P2            survey-text          17       290074   empty_space
45  P2            survey-text          18       311182   empty_space
46  P2            survey-text          19       319477   empty_space
47  P2            survey-text          20       329970   empty_space
48  P2            survey-text          21       363255   empty_space
49  P2            survey-text          22       374130   empty_space
50  P2            survey-text          23       383333   empty_space
51  P2            survey-text          24       455551   empty_space
52  P2            survey-text          25       475648   empty_space
53  P3                preload           0          282   empty_space
54  P3 html-keyboard-response           1         4616   empty_space
55  P3           instructions           2         7071   empty_space
56  P3 html-keyboard-response           3        17104   empty_space
57  P3          survey-likert           4        25360   empty_space
58  P3          survey-likert           5        30143   empty_space
59  P3          survey-likert           6        39928   empty_space
60  P3          survey-likert           7        46784   empty_space
61  P3          survey-likert           8        56919   empty_space
62  P3          survey-likert           9        67744   empty_space
63  P3          survey-likert          10        91823   empty_space
64  P3          survey-likert          11        98808   empty_space
65  P3          survey-likert          12       104519   empty_space
66  P3          survey-likert          13       357463   empty_space
67  P3            survey-text          14       361639   empty_space
68  P3    survey-multi-choice          15       363407   empty_space
69  P3            survey-text          16       378222   empty_space
70  P3            survey-text          17       385927   empty_space
71  P3            survey-text          18       393263   empty_space
72  P3            survey-text          19       402022   empty_space
73  P3            survey-text          20       408551   empty_space
74  P3            survey-text          21       428383   empty_space
75  P3            survey-text          22       434191   empty_space
76  P3            survey-text          23       441454   empty_space
77  P3            survey-text          24       480286   empty_space
78  P3            survey-text          25       490934   empty_space
79  P4                preload           0          216   empty_space
80  P4 html-keyboard-response           1         4077   empty_space
81  P4           instructions           2        16028   empty_space
82  P4 html-keyboard-response           3        18984   empty_space
83  P4          survey-likert           4        30055   empty_space
84  P4          survey-likert           5        34960   empty_space
85  P4          survey-likert           6        41374   empty_space
86  P4          survey-likert           7        45771   empty_space
87  P4          survey-likert           8        53771   empty_space
88  P4          survey-likert           9        59736   empty_space
89  P4          survey-likert          10        64722   empty_space
90  P4          survey-likert          11        68792   empty_space
91  P4          survey-likert          12        76386   empty_space
92  P4          survey-likert          13        80434   empty_space
93  P4            survey-text          14        84427   empty_space
94  P4    survey-multi-choice          15        86462   empty_space
95  P4            survey-text          16       103062   empty_space
96  P4            survey-text          17       110980   empty_space
97  P4            survey-text          18       116826   empty_space
98  P4            survey-text          19       120747   empty_space
99  P4            survey-text          20       130607   empty_space
100 P4            survey-text          21       135469   empty_space
101 P4            survey-text          22       142425   empty_space
102 P4            survey-text          23       147398   empty_space
103 P4            survey-text          24       151993   empty_space
104 P4            survey-text          25       158287   empty_space
105 P5                preload           0          196   empty_space
106 P5 html-keyboard-response           1         4428   empty_space
107 P5           instructions           2         6184   empty_space
108 P5 html-keyboard-response           3         8454   empty_space
109 P5          survey-likert           4        17513   empty_space
110 P5          survey-likert           5        26422   empty_space
111 P5          survey-likert           6        30769   empty_space
112 P5          survey-likert           7        34927   empty_space
113 P5          survey-likert           8        38841   empty_space
114 P5          survey-likert           9        44165   empty_space
115 P5          survey-likert          10        49010   empty_space
116 P5          survey-likert          11        52236   empty_space
117 P5          survey-likert          12        56605   empty_space
118 P5          survey-likert          13        60795   empty_space
119 P5            survey-text          14        63914   empty_space
120 P5    survey-multi-choice          15        65915   empty_space
121 P5            survey-text          16        83129   empty_space
122 P5            survey-text          17        87211   empty_space
123 P5            survey-text          18        92228   empty_space
124 P5            survey-text          19        94344   empty_space
125 P5            survey-text          20        98955   empty_space
126 P5            survey-text          21       104699   empty_space
127 P5            survey-text          22       110710   empty_space
128 P5            survey-text          23       116861   empty_space
129 P5            survey-text          24       119727   empty_space
130 P5            survey-text          25       123344   empty_space
131 P6                preload           0         2893 limited_space
132 P7 html-keyboard-response           1        15723 limited_space
133 P7           instructions           2        28284 limited_space
134 P7 html-keyboard-response           3        35987 limited_space
135 P7          survey-likert           4        49645 limited_space
136 P7          survey-likert           5        58314 limited_space
137 P7          survey-likert           6       101340 limited_space
138 P7          survey-likert           7       116904 limited_space
139 P7          survey-likert           8       134304 limited_space
140 P7          survey-likert           9       158369 limited_space
141 P7          survey-likert          10       174104 limited_space
142 P7          survey-likert          11       190936 limited_space
143 P7          survey-likert          12       208843 limited_space
144 P7          survey-likert          13       217100 limited_space
145 P7            survey-text          14       225010 limited_space
146 P7    survey-multi-choice          15       227615 limited_space
147 P7            survey-text          16       252362 limited_space
148 P7            survey-text          17       266020 limited_space
149 P7            survey-text          18       281995 limited_space
150 P7            survey-text          19       292058 limited_space
151 P7            survey-text          20       309489 limited_space
152 P7            survey-text          21       333828 limited_space
153 P7            survey-text          22       353243 limited_space
154 P7            survey-text          23       386986 limited_space
155 P7            survey-text          24       404330 limited_space
156 P7            survey-text          25       423304 limited_space
    Response Time                                  Response 2
1              NA                                            
2            5884                                         Alt
3            4004                                            
4            6268                                   ArrowDown
5           10907                             {"Q0":1,"Q1":3}
6            5561                             {"Q0":1,"Q1":2}
7            6851                             {"Q0":3,"Q1":4}
8            3490                             {"Q0":1,"Q1":1}
9            5850                             {"Q0":1,"Q1":1}
10           5942                             {"Q0":2,"Q1":2}
11           4207                             {"Q0":1,"Q1":1}
12          19229                             {"Q0":1,"Q1":2}
13           5407                             {"Q0":1,"Q1":1}
14           8067                             {"Q0":2,"Q1":3}
15           3583                                 {"Q0":"35"}
16           1641                         {"gender":"Female"}
17          22203 {"Q0":"love (i dont remember the rest)\\n"}
18           6661                   {"Q0":"a chance to grow"}
19           6772                  {"Q0":"happiness doubled"}
20           3877                           {"Q0":"carry on"}
21           6529                  {"Q0":"short for regrets"}
22          14710    {"Q0":"through deeds rather than words"}
23           5037                  {"Q0":"the little things"}
24           9247             {"Q0":"but women never forget"}
25           7908                  {"Q0":"one day at a time"}
26           5712                   {"Q0":"is an old friend"}
27             NA                                            
28           4889                                            
29          12723                                            
30          10941                                            
31          34226                             {"Q0":4,"Q1":4}
32          12735                             {"Q0":5,"Q1":6}
33          17284                             {"Q0":2,"Q1":3}
34           9390                             {"Q0":5,"Q1":6}
35          15808                             {"Q0":2,"Q1":3}
36           7639                             {"Q0":6,"Q1":6}
37          32047                             {"Q0":2,"Q1":3}
38          12368                             {"Q0":6,"Q1":6}
39           9041                             {"Q0":6,"Q1":5}
40           9987                             {"Q0":5,"Q1":4}
41           5151                                 {"Q0":"35"}
42           3263                         {"gender":"Female"}
43          11329                 {"Q0":"love has no limits"}
44          80825                       {"Q0":"learning day"}
45          21102                  {"Q0":"happiness doubled"}
46           8290                          {"Q0":"carry on "}
47          10488         {"Q0":"short to live with regrets"}
48          33281               {"Q0":"with deeds not words"}
49          10870         {"Q0":"things that make you smile"}
50           9200                 {"Q0":"women never forget"}
51          72214                 {"Q0":"as a day to teach "}
52          20094           {"Q0":"is seeing an old friend "}
53             NA                                            
54           4334                                       Enter
55           2455                                            
56          10032                                       Shift
57           8255                             {"Q0":0,"Q1":0}
58           4782                             {"Q0":0,"Q1":0}
59           9783                             {"Q0":0,"Q1":3}
60           6855                             {"Q0":0,"Q1":0}
61          10135                             {"Q0":4,"Q1":0}
62          10823                             {"Q0":0,"Q1":4}
63          24079                             {"Q0":0,"Q1":0}
64           6984                             {"Q0":6,"Q1":3}
65           5710                             {"Q0":0,"Q1":0}
66         252943                             {"Q0":6,"Q1":6}
67           4174                                 {"Q0":"52"}
68           1767                           {"gender":"Male"}
69          14811                              {"Q0":"love "}
70           7703                   {"Q0":"a chance to grow"}
71           7335                  {"Q0":"doubled happiness"}
72           8758                           {"Q0":"and rest"}
73           6527                  {"Q0":"short for regrets"}
74          19830        {"Q0":"with action more than words"}
75           5807                            {"Q0":"friends"}
76           7262                 {"Q0":"women never forget"}
77          38831                 {"Q0":"one step at a time"}
78          10647                   {"Q0":"is an old friend"}
79             NA                                            
80           3859                                            
81          11949                                            
82           2954                                            
83          11067                             {"Q0":4,"Q1":5}
84           4902                             {"Q0":5,"Q1":6}
85           6411                             {"Q0":6,"Q1":6}
86           4395                             {"Q0":4,"Q1":5}
87           7997                             {"Q0":6,"Q1":4}
88           5961                             {"Q0":1,"Q1":3}
89           4982                             {"Q0":6,"Q1":6}
90           4067                             {"Q0":4,"Q1":5}
91           7592                             {"Q0":2,"Q1":2}
92           4045                             {"Q0":5,"Q1":6}
93           3988                                 {"Q0":"20"}
94           2032                           {"gender":"Male"}
95          16574               {"Q0":"love knows no limits"}
96           7907                   {"Q0":"a chance to grow"}
97           5837                  {"Q0":"happiness doubled"}
98           3912                           {"Q0":"carry on"}
99           9853                  {"Q0":"short to waste it"}
100          4851                           {"Q0":"by deeds"}
101          6947               {"Q0":"what makes you happy"}
102          4963             {"Q0":"but women never forget"}
103          4584                 {"Q0":"one step at a time"}
104          6281                   {"Q0":"is an old friend"}
105            NA                                            
106          4231                                            
107          1754                                            
108          2269                                            
109          9055                             {"Q0":5,"Q1":6}
110          8906                             {"Q0":0,"Q1":0}
111          4345                             {"Q0":6,"Q1":6}
112          4155                             {"Q0":6,"Q1":6}
113          3910                             {"Q0":6,"Q1":6}
114          5322                             {"Q0":4,"Q1":3}
115          4843                             {"Q0":6,"Q1":5}
116          3225                             {"Q0":3,"Q1":3}
117          4367                             {"Q0":2,"Q1":3}
118          4188                             {"Q0":1,"Q1":1}
119          3116                                 {"Q0":"38"}
120          1998                         {"gender":"Female"}
121         17205                               {"Q0":"oops"}
122          4081                           {"Q0":"precious"}
123          5015                  {"Q0":"happiness doubled"}
124          2113                           {"Q0":"carry on"}
125          4609       {"Q0":"precious to waste on regrets"}
126          5742              {"Q0":"in deeds not in words"}
127          6010                         {"Q0":"self care?"}
128          6150             {"Q0":"but women never forget"}
129          2865                 {"Q0":"one step at a time"}
130          3615                   {"Q0":"is an old friend"}
131            NA                                            
132         12828                                            
133         12560                                            
134          7701                                            
135         13657                             {"Q0":5,"Q1":6}
136          8667                             {"Q0":6,"Q1":5}
137         43022                             {"Q0":5,"Q1":6}
138         15561                             {"Q0":4,"Q1":5}
139         17398                             {"Q0":6,"Q1":5}
140         24063                             {"Q0":4,"Q1":6}
141         15734                             {"Q0":5,"Q1":4}
142         16830                             {"Q0":5,"Q1":5}
143         17905                             {"Q0":5,"Q1":6}
144          8255                             {"Q0":5,"Q1":6}
145          7907                                 {"Q0":"24"}
146          2603                         {"gender":"Female"}
147         24736                  {"Q0":"love has no limit"}
148         13653                   {"Q0":"a chance to grow"}
149         15971                  {"Q0":"happiness doubled"}
150         10058                       {"Q0":"and carry on"}
151         17427       {"Q0":"precious to waste on regrets"}
152         24337        {"Q0":"more in deeds than in words"}
153         19410               {"Q0":"what makes you smile"}
154         33739             {"Q0":"but women never forget"}
155         17340                          {"Q0":"at a time"}
156         18969                   {"Q0":"is an old friend"}
               Trial ID Ques order
1                                 
2                                 
3                                 
4                                 
5       likert_question      [0,1]
6       likert_question      [0,1]
7       likert_question      [0,1]
8       likert_question      [0,1]
9       likert_question      [0,1]
10      likert_question      [0,1]
11      likert_question      [0,1]
12      likert_question      [0,1]
13      likert_question      [0,1]
14      likert_question      [0,1]
15         age_question           
16      gender_question        [0]
17   recall_statement_1           
18   recall_statement_2           
19   recall_statement_3           
20   recall_statement_4           
21   recall_statement_5           
22   recall_statement_6           
23   recall_statement_7           
24   recall_statement_8           
25   recall_statement_9           
26  recall_statement_10           
27                                
28                                
29                                
30                                
31      likert_question      [0,1]
32      likert_question      [0,1]
33      likert_question      [0,1]
34      likert_question      [0,1]
35      likert_question      [0,1]
36      likert_question      [0,1]
37      likert_question      [0,1]
38      likert_question      [0,1]
39      likert_question      [0,1]
40      likert_question      [0,1]
41         age_question           
42      gender_question        [0]
43   recall_statement_1           
44   recall_statement_2           
45   recall_statement_3           
46   recall_statement_4           
47   recall_statement_5           
48   recall_statement_6           
49   recall_statement_7           
50   recall_statement_8           
51   recall_statement_9           
52  recall_statement_10           
53                                
54                                
55                                
56                                
57      likert_question      [0,1]
58      likert_question      [0,1]
59      likert_question      [0,1]
60      likert_question      [0,1]
61      likert_question      [0,1]
62      likert_question      [0,1]
63      likert_question      [0,1]
64      likert_question      [0,1]
65      likert_question      [0,1]
66      likert_question      [0,1]
67         age_question           
68      gender_question        [0]
69   recall_statement_1           
70   recall_statement_2           
71   recall_statement_3           
72   recall_statement_4           
73   recall_statement_5           
74   recall_statement_6           
75   recall_statement_7           
76   recall_statement_8           
77   recall_statement_9           
78  recall_statement_10           
79                                
80                                
81                                
82                                
83      likert_question      [0,1]
84      likert_question      [0,1]
85      likert_question      [0,1]
86      likert_question      [0,1]
87      likert_question      [0,1]
88      likert_question      [0,1]
89      likert_question      [0,1]
90      likert_question      [0,1]
91      likert_question      [0,1]
92      likert_question      [0,1]
93         age_question           
94      gender_question        [0]
95   recall_statement_1           
96   recall_statement_2           
97   recall_statement_3           
98   recall_statement_4           
99   recall_statement_5           
100  recall_statement_6           
101  recall_statement_7           
102  recall_statement_8           
103  recall_statement_9           
104 recall_statement_10           
105                               
106                               
107                               
108                               
109     likert_question      [0,1]
110     likert_question      [0,1]
111     likert_question      [0,1]
112     likert_question      [0,1]
113     likert_question      [0,1]
114     likert_question      [0,1]
115     likert_question      [0,1]
116     likert_question      [0,1]
117     likert_question      [0,1]
118     likert_question      [0,1]
119        age_question           
120     gender_question        [0]
121  recall_statement_1           
122  recall_statement_2           
123  recall_statement_3           
124  recall_statement_4           
125  recall_statement_5           
126  recall_statement_6           
127  recall_statement_7           
128  recall_statement_8           
129  recall_statement_9           
130 recall_statement_10           
131                               
132                               
133                               
134                               
135     likert_question      [0,1]
136     likert_question      [0,1]
137     likert_question      [0,1]
138     likert_question      [0,1]
139     likert_question      [0,1]
140     likert_question      [0,1]
141     likert_question      [0,1]
142     likert_question      [0,1]
143     likert_question      [0,1]
144     likert_question      [0,1]
145        age_question           
146     gender_question        [0]
147  recall_statement_1           
148  recall_statement_2           
149  recall_statement_3           
150  recall_statement_4           
151  recall_statement_5           
152  recall_statement_6           
153  recall_statement_7           
154  recall_statement_8           
155  recall_statement_9           
156 recall_statement_10

#Assigning conditions

empty_space <- data$Response.Time[data$Condition == "empty_space"]
limited_space <- data$Response.Time[data$Condition == "limited_space"]
  1. Data Cleaning
my_data <- na.omit(data)  # Removes rows with any NA values
### Convert space_condition to factor if not already
my_data$Condition <- as.factor(my_data$Condition)
### Checking for duplicates using (duplicated) to ensure no rows are repeated
  1. Defining exclusion criteria ### Removing participants with incomplete responses
#my_data <- filter(my_data, complete_cases == TRUE)
  1. Final Preparation ### Check the structure after data preparation
str(data)
'data.frame':   156 obs. of  9 variables:
 $ P1           : chr  "P1" "P1" "P1" "P1" ...
 $ Trial.type   : chr  "preload" "html-keyboard-response" "instructions" "html-keyboard-response" ...
 $ Trial.index  : int  0 1 2 3 4 5 6 7 8 9 ...
 $ Time.elapsed : int  145 6029 10035 16304 27215 32778 39633 43125 48979 54924 ...
 $ Condition    : chr  "limited_space" "limited_space" "limited_space" "limited_space" ...
 $ Response.Time: int  NA 5884 4004 6268 10907 5561 6851 3490 5850 5942 ...
 $ Response.2   : chr  "" "Alt" "" "ArrowDown" ...
 $ Trial.ID     : chr  "" "" "" "" ...
 $ Ques.order   : chr  "" "" "" "" ...
summary(my_data)
      P1             Trial.type         Trial.index  Time.elapsed   
 Length:150         Length:150         Min.   : 1   Min.   :  4077  
 Class :character   Class :character   1st Qu.: 7   1st Qu.: 49169  
 Mode  :character   Mode  :character   Median :13   Median :103790  
                                       Mean   :13   Mean   :149148  
                                       3rd Qu.:19   3rd Qu.:197090  
                                       Max.   :25   Max.   :490934  
         Condition   Response.Time     Response.2          Trial.ID        
 empty_space  :100   Min.   :  1641   Length:150         Length:150        
 limited_space: 50   1st Qu.:  4797   Class :character   Class :character  
                     Median :  7616   Mode  :character   Mode  :character  
                     Mean   : 12351                                        
                     3rd Qu.: 12732                                        
                     Max.   :252943                                        
  Ques.order       
 Length:150        
 Class :character  
 Mode  :character

Justification of F test

The original study compares the favorability of empty spaces and limited space conditions. The comparison of two samples suggests that an F test should be used to test for variance. ANOVA will also be used in this analysis to determine if there is a significant difference between the two groups in the study.

The original study used an F-test or ANOVA to analyze this comparison, then using the same approach is consistent with replication goals, further justifying its use.

Confirmatory analysis

F test

f_test_result <- var.test(empty_space, limited_space)
print(f_test_result)

    F test to compare two variances

data:  empty_space and limited_space
F = 10.461, num df = 99, denom df = 49, p-value = 4.441e-15
alternative hypothesis: true ratio of variances is not equal to 1
95 percent confidence interval:
  6.287354 16.710132
sample estimates:
ratio of variances 
          10.46093

Perform ANOVA

anova_result <- aov(Response.Time ~ Condition, data = my_data)
summary(anova_result)
             Df    Sum Sq   Mean Sq F value Pr(>F)
Condition     1 4.095e+06   4095008   0.008  0.929
Residuals   148 7.571e+10 511536219

For Pilot B the average time participants took to complete the Pilot B was around 6 minutes.

Exploratory analyses

Any follow-up analyses desired (not required).

Discussion

Summary of Replication Attempt

Open the discussion section with a paragraph summarizing the primary result from the confirmatory analysis and the assessment of whether it replicated, partially replicated, or failed to replicate the original result.

Commentary

Add open-ended commentary (if any) reflecting (a) insights from follow-up exploratory analysis, (b) assessment of the meaning of the replication (or not) - e.g., for a failure to replicate, are the differences between original and present study ones that definitely, plausibly, or are unlikely to have been moderators of the result, and (c) discussion of any objections or challenges raised by the current and original authors about the replication attempt. None of these need to be long.