Replication of Naive_Theory (2012, Shtulman)

https://github.com/antzavala/shtulman.2012

Anthony Zavala

2024-10-13

Introduction:

I chose to replicate the naive theories because of how it relates to my research interest in misinformation in how people develop their beliefs in ideas from social media such as the far right. This theory is based on the idea of what happens to theories when they are in conflict with previous notions of naive theories. I would like to know more about how people tend to keep their naive theories even after being proved wrong such as what factors affect this the most. I have found a lot of misinformation mostly from the internet and in person and as this study will be replicated as this I think it will be interesting as we will use crowdsource workers from the internet.

I would conduct the procedure by doing an experiment with stimuli presented to participants with MediaLab v1.21 software, which records the speed and accuracy of their judgments the same as the one before. I see myself doing this with common ideas proven wrong or ones that are more surprisingly similar to the one presented in the article. I would like to do this in order to get a better result but also making sure it doesn’t differ too much from the original. A hurdle may be because it’s conducted online but I believe it will still yield a great result due to many naive theories nowadays coming from social media. I see this relating a lot more to how people compare naive theories to scientific ones when seeing them online says it’s going to be conducted in that way.

Methods Power Analysis Original effect size, power analysis for samples to achieve 80%, 90%, 95% power to detect that effect size. Considerations of feasibility for selecting planned sample size.

Planned Sample Planned sample size and/or termination rule, sampling frame, known demographics if any, preselection rules if any.

Materials All materials - can quote directly from original article - just put the text in quotations and note that this was followed precisely. Or, quote directly and just point out exceptions to what was described in the original article. 2

Procedure Can quote directly from original article - just put the text in quotations and note that this was followed precisely. Or, quote directly and just point out exceptions to what was described in the original article.

Analysis Plan Can also quote directly, though it is less often spelled out effectively for an analysis strategy section. The key is to report an analysis strategy that is as close to the original - data cleaning rules, data exclusion rules, covariates, etc. - as possible. Clarify key analysis of interest here You can also pre-specify additional analyses you plan to do. Differences from Original Study Explicitly describe known differences in sample, setting, procedure, and analysis plan from original study. The goal, of course, is to minimize those differences, but differences will in- evitably occur. Also, note whether such differences are anticipated to make a difference based on claims in the original article or subsequent published research on the conditions for obtain- ing the effect.

Methods Addendum (Post Data Collection) You can comment this section out prior to final report with data collection. Actual Sample Sample size, demographics, data exclusions based on rules spelled out in analysis plan Differences from pre-data collection methods plan Any differences from what was described as the original plan, or “none”. Results Data preparation Data preparation following the analysis plan. 3

Confirmatory analysis The analyses as specified in the analysis plan. Side-by-side graph with original graph is ideal here Exploratory analyses Any follow-up analyses desired (not required). Discussion

Summary of Replication Attempt Open the discussion section with a paragraph summarizing the primary result from the con- firmatory analysis and the assessment of whether it replicated, partially replicated, or failed to replicate the original result.

Commentary Add open-ended commentary (if any) reflecting (a) insights from follow-up exploratory anal- ysis, (b) assessment of the meaning of the replication (or not) - e.g., for a failure to replicate, are the differences between original and present study ones that definitely, plausibly, or are unlikely to have been moderators of the result, and (c) discussion of any objections or chal- lenges raised by the current and original authors about the replication attempt. None of these need to be long.