Justification: By replicating this project, I want to gain experience running experiments with dyadic interaction. I am interested in understanding how social norms form, and this kind of repeated games are a common way to investigate this problem, hence will continue to be useful in my research program.
Stimuli and Procedures: Participants will play a 2-player iterated cooperative decision-making task which I will implement in Empirica. Participants will be informed that they must select a parking spot in a virtual parking lot over the course of several days. Different spots will cost different amounts of a virtual currency (Monetary Units; price remain fixed over days). Participants will be incentivized to minimize cost paid as low cost paid corresponds to higher pay for participating in the experiment. There are two zones in the lot: an orange zone and a purple zone, and two parking spots per zone, and participants will selecte a spot to park before seeing their partner’s selection. Participants will be informed that selecting the same color zone as their partner would give them a “group discount”, but selecting the exact same parking spot will incur a penalty price. After making their decisions, participants were shown the actions that their partner took and the price each participant paid.
Pilot A: Results and preliminary graphs corresponding to the original paper can be found at this github repo under the pilotA_results folder:
Outcome Measure: The success of the replication will be measured based on how well the distribuion of strategies that the partners adopt match the distribution from the original paper. More specifically, we will use the same statistical analyses to compare the results obtained in condition 3 with the results from condition 1 and 2 of the experiment to see if the qualitative differences that they report replicate. The main result we want to replicate is that “participants developed an alternating norm more frequently than in the control (β = 0.10,CrI = [0.01,0.20])”. Additionally, we are also interested in the result that “participants were less likely to converge on stable selection on orange compared to the condition 1 (β=−0.75, CrI = [−0.92,−0.59]) and 2 (β=−0.29, CrI = [−0.46,−0.13]) * they failed to form any norm more frequently compared to the control (β= 0.18, CrI = [0.04, 0.31])”.
We will do the same statistical tests comparing results from other conditions in the original paper with our replication results of condition 3, to see whether these observations still hold.
We will also confirm whether “participants paid less over the course of the game” still holds, which is a key indicator of whether participants converged to systematic norms over time.
Running the power analysis below shows that we should use 52 pairs, hence recruit 104 participants to get a power of 0.8.
library(pwr)
p1 <- 9.5/84 # probability of alternating on purple in condition 3
p2 <- 1/102 # probability of alternating on purple in condition 1
h <- ES.h(p1, p2)
result <- pwr.2p.test(h = h, sig.level = 0.05, alternative = "greater",power=0.8)
result$n[1] 52.0063We plan to run 104 participants.
The experiment will be run online. Participants will be recruited through Prolific, and sent to I used Empirica where they will be paired up with another participant. The code used for the online experiment is available at this link.
This is the procedure as reported in the original paper: > [Participants] were shown the parking lot of their assigned treatment and were asked to write a strategic plan describing how they would ideally play the game. After writing, participants progressed to a treatment-specific waiting room and were paired with the first available partner. They played 12 trials of the game; after each trial, participants were shown their partner’s move and cost paid on the previous trial, and needed to indicate the cost they themselves paid as an attention check. The task took a median time of 12 minutes. Participants were paid a base rate of $12.50/hr; they were incentivized to minimize their overall cost in the game through a performance-based compensation bonus. Participants were informed they would play multiple trials but not told precisely how many, to induce uncertainty in the time horizon.
We follow the same procedure except we pay them a base rate of $8/hr.
We follow the same procedure as in the paper quoted below: > We excluded from analysis participants who failed to select a parking spot in the allotted time and did not finish the game, as well as participants who wrote fewer than 10 characters in a pre-game writing task.
Clarify key analysis of interest here The key analysis of interest is whether the proportion of participants who used the strategy of alternating between the two purple parking spots was greater in condition 3 than in the control (condition 1). We will compare results from condition 1 in the original paper against the condition 3 results we obtain and use a bayesian regression model brm(isAlternatingPurple ~ condition) to see if the credible interval is above 0. We will also use a different metric, the prop.test function to see test whether the proportions of the strategy of alternating purple is different in condition 3 and 1.
Here are the main differences from the original study:
We are only replicating condition 3 of the experiment, hence the results for condition 3 we replicate will be compared with results from condition 1 of the authors’ paper instead of results from our own experiment.
The classification algorithm for the different strategies is not publicly available so I created my own algorithm to classify the strategies. However, there should be minimal differences, as there is little ambiguity in which traces of the interaction should be classified into different strategies.
Participants were paid $8/hour instead of $12.5/hour as in the original study
However, we do not expect these differences to be significant in influencing the outcome of this study.
You can comment this section out prior to final report with data collection.
Sample size, demographics, data exclusions based on rules spelled out in analysis plan
Any differences from what was described as the original plan, or “none”.
Data preparation following the analysis plan.
The analyses as specified in the analysis plan.
Side-by-side graph with original graph is ideal here
Any follow-up analyses desired (not required).
Open the discussion section with a paragraph summarizing the primary result from the confirmatory analysis and the assessment of whether it replicated, partially replicated, or failed to replicate the original result.
Add open-ended commentary (if any) reflecting (a) insights from follow-up exploratory analysis, (b) assessment of the meaning of the replication (or not) - e.g., for a failure to replicate, are the differences between original and present study ones that definitely, plausibly, or are unlikely to have been moderators of the result, and (c) discussion of any objections or challenges raised by the current and original authors about the replication attempt. None of these need to be long.