Reproducibility Report for ‘Reinforcement learning and Bayesian inference provide complementary models for the unique advantage of adolescents in stochastic reversal’ by Eckstein et al. (2022, Developmental Cognitive Neuroscience)
Introduction
In “Reinforcement Learning and Bayesian Inference”, Eckstein et al. investigated the relationship between performance on a volatile reversal-learning task and age, predicting that adolescents would out-perform both younger and older participants. Indeed, this hypothesis aligned with data collected from 291 participants with ages spanning from 8-30 years. The researchers then built several candidate Reinforcement Learning (RL) and Bayesian Inference (BI) models to investigate the cognitive processes potentially producing this pattern. The models which produced simulated behavior most closely aligned with participants’ included a 4-parameter RL model (𝛽: inverse decision temperature, α+: factual learning rate, α- counter-factual learning rate, and 𝑝: persistence) as well as a 4-parameter BI model (𝛽, 𝑝, 𝑝switch: task volatility, and 𝑝reward: reward stochasticity). Analyzing these models’ parameters with respect to age revealed 𝛽 and 𝑝 increased with age. Interestingly however, the winning RL model’s α- parameter, as well as the winning BI model’s 𝑝switch parameter, were lowest mid-adolescents. This result suggests mid-adolescents’ strong performance in the researchers’ volatile reversal-learning task may be related to a difference in the rate they incorporate negative feedback.
Key analysis of interest: Reproduce the 4-parameter RL and BI models and evaluate parameters’ relationships with age.
Justification for choice of study
My research interests include cognitive modeling and decision making; I have had limited opportunities to explore this interest outside of lectures and am excited by the opportunity to do so with this computational reproducibility project. I also feel reproducing the RL and BI models from the original paper would be an excellent way to strengthen my computational skills and gain exposure to developmental cognitive science literature.
Anticipated challenges
To conduct this experiment, an important first step will be to simulate the volatile reversal-learning task (this step has been completed). Next, candidate RL and BI models will be built and used to simulate agents’ behavior. Before fitting the models to the original data, I will engage in parameter recovery, model recovery, and finally, model comparison. Next I will investigate 1) whether my winning models align with those identified in the original paper and 2) the extent to which my winning models exhibit similar behaviors to those in the original paper. For example, after fitting my models to the original data, I will examine the relationship between participants’ recovered parameters and age.
Potential challenges include 1) building the BI model (I am less experienced with BI than RL though am eager to improve!) and 2) reproducing the final piece of the original analysis (section “2.4. Integrating RL and BI—Going beyond specific models”).
Links
Project repository (on Github): https://github.com/vic-moreno/eckstein2022
Original paper (as hosted in your repo): https://github.com/vic-moreno/eckstein2022/blob/main/original_paper/eckstein2022.pdf
Methods
Description of the steps required to reproduce the results
Please describe all the steps necessary to reproduce the key result(s) of this study.
Differences from original study
Explicitly describe known differences in the analysis pipeline between the original paper and yours (e.g., computing environment). The goal, of course, is to minimize those differences, but differences may occur. Also, note whether such differences are anticipated to influence your ability to reproduce the original results.
Project Progress Check 1
Measure of success
Please describe the outcome measure for the success or failure of your reproduction and how this outcome will be computed.
Pipeline progress
Earlier in this report, you described the steps necessary to reproduce the key result(s) of this study. Please describe your progress on each of these steps (e.g., data preprocessing, model fitting, model evaluation).
Results
Data preparation
Data preparation following the analysis plan.
Key analysis
The analyses as specified in the analysis plan.
Side-by-side graph with original graph is ideal here
Exploratory analyses
Any follow-up analyses desired (not required).
Discussion
Summary of Reproduction Attempt
Open the discussion section with a paragraph summarizing the primary result from the key analysis and assess whether you successfully reproduced it, partially reproduced it, or failed to reproduce it.
Commentary
Add open-ended commentary (if any) reflecting (a) insights from follow-up exploratory analysis of the dataset, (b) assessment of the meaning of the successful or unsuccessful reproducibility attempt - e.g., for a failure to reproduce the original findings, are the differences between original and present analyses ones that definitely, plausibly, or are unlikely to have been moderators of the result, and (c) discussion of any objections or challenges raised by the current and original authors about the reproducibility attempt (if you contacted them). None of these need to be long.