My research interests lie in the realm of Education and the different levers that are involved in the learning sciences. Motivation and growth mindset are very well studied and integral parts of researching how humans learn, how humans view intelligence, and the science behind continuous learning both outside and inside the school environment. For this reason, Haimovitz’s study on children’s implicit theories on intelligence and how they are developed in the home environment is very applicable to my research interests. I believe that understanding how children develop mindsets about their intelligence and how this impacts their motivation to learn.
This study focuses on the distinction between fixed and growth mindset for children in school and the effect that this has on their motivation. The potential that this intelligence mindset has on children is tremendous, and thus it is important to study where and how these mindsets get developed from the home environment — specifically the parents’ own intelligence mindset. There are four different studies that study this link. For the sake of this replication project, I will focus on study 4, which examines “the causal effect of parents’ failure mind-sets on their responses to their children’s hypothetical failure.”
In this study, 132 parents were recruited to take a survey about failure mindsets and tasked to imagine a scenario in which in which their child came home from school with a failing grade on a math quiz. They were then asked what they would do. It was shown that parents that were induced a “failure-is-debilitating mind-set were more likely to react with concerns about their child’s performance and lack of ability.” These results were then used to conclude that parental beliefs about failure are visible and can shape children’s attitudes and beliefs about their own intelligence. This experiment requires a study that is 1) able to induce and differentiate between the two intelligence mindsets for parents, 2) a case where parents can tap into realistic hypothetical failures, and 3) a method to establish causation between parents’ failure mindsets their perspective on their children’s failures. Some challenges include the effectiveness with which we can induce different failure mindsets and accuracy with which the survey questions can accurately portray parental beliefs towards their children.
Link to Repository: git@github.com:psych251/haimovitz_rescue.git
Link to Paper: https://pdfs.semanticscholar.org/7172/b578bfc9a77fbcd34c8e2417b674977f73fc.pdf?te=1&nl=nyt-parenting&emc=edit_ptg_20200108
The prior replication attempt made for the Haimovitz paper from previous years made a great effort towards replicating the effects that were seen on the effect of parents’ failure mindsets on their responses to their children’s hypothetical failures. The replication study followed a similar method route – they both recruited a group of parents from a crowdsourcing platform, requested survey responses, and used the same coding framework for data analysis. The replication study, however, used a different crowdsourcing platform (Prolific) and recruited 120 parents (190 less than the original) to achieve 80% power. The sample included the following demographics:
The replication study also added additional funnel debrief questions to collect more demographic information about the subjects’ children and relabeled some survey questions for clearness. The coding scheme remained relatively the same. Lastly, during analysis, only one rater coded the responses (compared to two raters in the original study).
Original effect size, power analysis for samples to achieve 80%, 90%, 95% power to detect that effect size. Considerations of feasibility for selecting planned sample size.
How much power does your planned sample have for original effect? For an attenuated effect that is half the size of the original?
(If power analysis is not possible or precise, discuss more fully how you determined a sample size that would be sufficient for rescue.)
For my sample, I plan to collect more data using a similar or larger sample size using the same crowdsourcing platfom. I will be hoping to control for the sample’s sex (male or female parents), but will still attempt to achieve 80% power.
All materials - can quote directly from original article - just put the text in quotations and note that this was followed precisely. Or, quote directly and just point out exceptions to what was described in the original article.
From the paper: Two raters, blind to condition, coded the openended responses.
“Participants completed an online survey initially assessing several beliefs, including their perceptions of their child’s competence (assessed with same measure as in Study 1; α = .79). Then we temporarily manipulated failure mind-sets by randomly assigning the parents to complete one of two five-item biased questionnaires, written to foster agreement with either a failure-is-debilitating mind-set (e.g., “Experiencing failure can lead to negative feelings, like shame or sadness, that interfere with learning”) or a failure-isenhancing mind-set (e.g., “Experiencing failure can improve performance in the long run if you learn from it”). All measures used a 6-point rating scale from 1 (strongly disagree) to 6 (strongly agree). One-sample t tests comparing the mean in each priming condition with the scale’s midpoint (3.5) showed that participants’ agreement with the intended mind-set was above the midpoint in both the failure-is-debilitating condition (M = 4.41, SD = 1.07), t(56) = 6.45, p < .001, and the failure-isenhancing condition (M = 5.14, SD = 0.829), t(74) = 17.11, p < .001. We then asked participants to read and vividly imagine a scenario in which their child came home from school with a failing grade on a math quiz, as in Study 2. They then wrote what they would do, think, and feel in response. Finally, participants reported on their failure mind-sets (α = .82), using the same items as in Study 1, as part of a survey that included a few other items.” Can quote directly from original article - just put the text in quotations and note that this was followed precisely. Or, quote directly and just point out exceptions to what was described in the original article.”
The link to the qualtrics survey that I created for this replication study is here: https://stanforduniversity.qualtrics.com/jfe/form/SV_9tVwoAgXkfFqHOK
Quality control measures that I will be adding to the experiment are: changing some of the coding measures to better interpret the open-ended questions, the coders gave a score of 1 each time that a code was present. However, I think that there needs to be more nuance that captures the varying intensity of the different open-ended responses.
To conduct analysis, we first must separate the likert scale questions and the open response questions. The open responses will be analyzed using the coding schema used for both the original and the replication study:
“Two raters, blind to condition, coded the open-ended responses. The first author developed a coding scheme on the basis of an initial reading of the responses and then made clarifying revisions on the basis of feedback from the two raters. The codes were broken down into two main categories of interest: performance-oriented responses and learning-oriented responses. Coders gave a score of 1 each time a code was present.”
Clarify key analysis of interest here I also attempt to see the link between different demographic factors (i.e. age, race, gender) and its effect on the response towards the child’s failure.
I hope to minimize the differences between my replication study and the previous attempts. First, I hope to utilize the same survey questions and coding schemas as done before – with the exception of adding a little bit more nuance to the decision of the coding schema. Other than that and the different parental demographics, I hope to retain the same method and analyses of both prior studies.
You can comment this section out prior to final report with data collection.
Sample size, demographics, data exclusions based on rules spelled out in analysis plan
Any differences from what was described as the original plan, or “none”.
How did people perform on any quality control checks or positive and negative controls?
After conducting the analysis, the goal was to determine whether or not parents that held a failure-is-debilitating mind-set is more likely to react with concerns about their child’s performance and whether or not this concern was related to their child’s ability. This was done with an unpaired two-tailed t-test. The code for this confirmatory analyses of my pilot study is here:
enhancing_coded <- survey_responses %>%
filter(Condition == "Enhancing")
debilitating_coded <- survey_responses %>%
filter(Condition == "Debilitating")
# conduct the t-test
#t_test_result <- t.test(enhancing_coded$code_results, debilitating_code$code_results, alternative = "two.sided", var.equal #= FALSE)Three-panel graph with original, 1st replication, and your replication is ideal here
Any follow-up analyses desired (not required).
Combining across the original paper, 1st replication, and 2nd replication, what is the aggregate effect size?
Open the discussion section with a paragraph summarizing the primary result from the confirmatory analysis and the assessment of whether it replicated, partially replicated, or failed to replicate the original result.
Add open-ended commentary (if any) reflecting (a) insights from follow-up exploratory analysis, (b) assessment of the meaning of the replication (or not) - e.g., for a failure to replicate, are the differences between original and present study ones that definitely, plausibly, or are unlikely to have been moderators of the result, and (c) discussion of any objections or challenges raised by the current and original authors about the replication attempt. None of these need to be long.