As the experiment put the respondents into the manipulation
conditions, we had to do manipulation checks.
1). The first manipulation check (one-sample t-test)
This test is used to assess participants’ agreement with the
manipulated condition (i.e., failure-enhancing condition and
failure-debilitating condition), by comparing the mean in each
priming condition with the scale midpoint (3.5) and confirming it was
above the midpoint. If the mean is above than the midpoint we can assume
that parents agreed with the intended mindset.
Was the parents’ agreement with each of the intended mindsets
above the midpoint?
Original study : YES
One-sample t tests comparing the mean in each priming condition with
the scale’s midpoint (3.5) showed that participants’ agreement with the
intended mind-set was above the midpoint in both the
failure-is-debilitating condition (M = 4.41, SD = 1.07), t(56) = 6.45, p
< .001, and the failure-is-enhancing condition (M = 5.14, SD =
0.829), t(74) = 17.11, p < .001.
1st replication : YES
(The sentence below was written by me based on the interpretation of
the coding in Takada’s Rpub)
Participants’ agreement with the intended mindset was above the
midpoint in both the failure-is-debilitating condition (M = 3.98, SD not
provided), t(58) = 4.03, p < .001, and the failure-is-enhancing
condition (M = 5.19, SD not provided), t(55) = 15.15, p < .001.
Rescue project : YES
Participants’ agreement with the intended mindset was above the midpoint in both the failure-is-debilitating condition (M = 4.48, SD = 1.01), t(71) = 8.22, p < .001, and the failure-is-enhancing condition (M = 5.33, SD = 0.68), t(68) = 22.35, p < .001. This result indicated that we can assume parents agreed with the intended mindset.
The code and the result were listed in below.
# calculate the mean. sd of each condition (1. calculate mean of the items 2. calculate the mean of the condition)
enhancing_data <- subset(d_condition, Condition == "Enhancing")
enhancing_cols <- enhancing_data[, c("Enhancing_1", "Enhancing_2", "Enhancing_3", "Enhancing_4", "Enhancing_5")]
mean_enhancing <- rowMeans(enhancing_cols)
sd_enhancing <- sd(mean_enhancing)
debilitating_data <- subset(d_condition, Condition == "Debilitating")
debilitating_cols <- debilitating_data[, c("Debilitating_1", "Debilitating_2", "Debilitating_3", "Debilitating_4", "Debilitating_5")]
mean_debilitating <- rowMeans(debilitating_cols)
sd_debilitating <- sd(mean_debilitating)
#t-test for each condition
t_test_enhancing <- t.test(mean_enhancing, mu = 3.5, alternative = "greater")
t_test_debilitating <- t.test(mean_debilitating, mu = 3.5, alternative = "greater")
# Output the results
t_test_enhancing
##
## One Sample t-test
##
## data: mean_enhancing
## t = 22.349, df = 68, p-value < 2.2e-16
## alternative hypothesis: true mean is greater than 3.5
## 95 percent confidence interval:
## 5.19654 Inf
## sample estimates:
## mean of x
## 5.333333
t_test_debilitating
##
## One Sample t-test
##
## data: mean_debilitating
## t = 8.2183, df = 71, p-value = 3.291e-12
## alternative hypothesis: true mean is greater than 3.5
## 95 percent confidence interval:
## 4.281708 Inf
## sample estimates:
## mean of x
## 4.480556
# read the sd
sd_enhancing
## [1] 0.6814057
sd_debilitating
## [1] 1.012407
2). The second manipulation check (independent t-test)
This test is used to assess whether the biased-questionnaire
manipulation effectively changed parents’ self-reported failure mindsets
at the end of the survey, by comparing the means of failure mindset
responses with the two conditions. The higher mean of the survey
indicates the enhancing mindset and the lower mean of the survey
indicates the debilitating mindset. If the mean of failure-enhancing
condition is higher and the mean difference between conditions is
significant, we can say that the failure-enhancing condition has more
enhancing mindset.
Did the parents in the failure-is-enhancing condition report
more of a failure-is-enhancing mindset than the parents in the
failure-is-enhancing condition?
Original study : YES
Indeed, the manipulation seemed to shift parents’ mind-sets, t(124) =
2.53, p = 0.013: Parents in the failure-is-enhancing condition reported
more of a failure- is-enhancing mind-set than did parents in the
failure-is- debilitating condition.
1st replication : YES
(The sentence below was written by me based on the interpretation of
the coding in Takada’s Rpub and in her study. she reverse coded
the positively-worded questions (i.e., failure-enhancing mindset), so
the higher mean indicated the higher failure-debilitating
mindset).
The manipulation check yielded significant results, t(111.64) =
4.5965, p = 1.138e-05, indicating that parents in the
failure-is-debilitating condition reported a stronger
failure-debilitating mindset than those in the failure-is-enhancing
condition.
Rescue project : YES
The manipulation significantly shifted parents' mindsets, t(138.88) = 4.8675, p = 3.03e-06. Parents in the failure-is-enhancing condition demonstrated a more pronounced failure-is-enhancing mindset compared to those in the failure-is-debilitating condition, as indicated by the statistically significant difference in means.
The code and the result were listed in below.
# Set the maximum value for the Closing items
max_value <- 6
# 1). Reverse code the specified columns and update them in the dataframe
d_condition$Closing_4 <- max_value + 1 - d_condition$Closing_4
d_condition$Closing_5 <- max_value + 1 - d_condition$Closing_5
d_condition$Closing_6 <- max_value + 1 - d_condition$Closing_6
# Create a new dataframe 'd_reverse' to store the reversed data
d_reverse <- data.frame(d_condition)
# 2). Calculate the mean of all Closing items, including the reversed ones, and store in 'd_reverse'
d_reverse$Closing_mean <- rowMeans(d_reverse[, c("Closing_1", "Closing_2", "Closing_3", "Closing_4", "Closing_5", "Closing_6")], na.rm = TRUE)
# 3). Subset data by condition for t-test
enhancing_closing <- subset(d_reverse, Condition == "Enhancing")
debilitating_closing <- subset(d_reverse, Condition == "Debilitating")
# 4).Perform t-test on Closing means between the two conditions
t_test_result <- t.test(enhancing_closing$Closing_mean, debilitating_closing$Closing_mean, alternative = "two.sided")
# Output the t-test result
t_test_result
##
## Welch Two Sample t-test
##
## data: enhancing_closing$Closing_mean and debilitating_closing$Closing_mean
## t = 4.8675, df = 138.88, p-value = 3.03e-06
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
## 0.441043 1.044464
## sample estimates:
## mean of x mean of y
## 4.603865 3.861111
3). Main analysis (independent t-test)
Main Analysis of Interest:
Were parents who hold a failure-is-debilitating mind-set more
likely to react with concerns about their child’s performance and lack
of ability?
Original Study: YES
Parents who were induced to hold a failure-is-debilitating mind-set
were more likely to react with concerns about their child’s per-
formance and lack of ability, t(131) = 3.246, p < .001, ηp2 = .075,
and less likely to react with support for their child’s learning and
mastery, t(131) = −2.04, p = .043, ηp2 = .031, compared with those who
were induced to hold a failure-is-enhancing mind-set (see Fig. 2).
Parents in both conditions did not report performance-oriented responses
(M = 0.485, SD = 0.693) nearly as often as learning-oriented responses
(M = 2.38, SD = 1.53).
1st replication: Slightly YES
The replication study showed similar trends to this original finding.
The effect was not as large and there was no statistical difference in
the number of performance-oriented responses between the two conditions,
t(113) = -1.9291, p = 0.0562, ηp2 = .031. However, I think my
replication was rather successful, given that my sample size was
smaller.
Rescue project: NO
Parents who adopted a failure-is-debilitating mindset did not show a statistically significant difference in performance-oriented responses to their child's failures (M = 1.043, SD = 1.117) compared to those with a failure-is-enhancing mindset (M = 1.153, SD = 0.929), t(132.35) = -0.63017, p = 0.5297. Similarly, there was no significant difference in learning-oriented responses between parents with a failure-is-debilitating mindset (M = 2.696, SD = 1.264) and those with a failure-is-enhancing mindset (M = 2.861, SD = 1.367), t(138.83) = -0.74679, p = 0.4565. The results indicate that while there is a trend towards a failure-is-enhancing mindset being associated with slightly higher learning-oriented responses, the differences were not statistically significant.
The code, result, and the figure is presented below.
# count each condition
data_coded_response_fie <- data_coded_response %>%
filter(Condition == "Enhancing")
count_fie <- nrow(data_coded_response_fie)
data_coded_response_fid <- data_coded_response %>%
filter(Condition == "Debilitating")
count_fid <- nrow(data_coded_response_fid)
# Output the counts
print(paste("Count for Failure-Is-Enhancing condition:", count_fie))
## [1] "Count for Failure-Is-Enhancing condition: 69"
print(paste("Count for Failure-Is-Debilitating condition:", count_fid))
## [1] "Count for Failure-Is-Debilitating condition: 72"
# T-Test
data_coded_response_fie <- data_coded_response %>%
filter(Condition == "Enhancing")
data_coded_response_fid <- data_coded_response %>%
filter(Condition == "Debilitating")
sd_fie_p <-sd(data_coded_response_fie$sum_performance)
sd_fid_p <-sd(data_coded_response_fid$sum_performance)
sd_fie_l <-sd(data_coded_response_fie$sum_learning)
sd_fid_l <-sd(data_coded_response_fid$sum_learning)
print(sd_fie_p)
## [1] 1.117176
print(sd_fid_p)
## [1] 0.9293299
print(sd_fie_l)
## [1] 1.263799
print(sd_fid_l)
## [1] 1.366661
# perforamnce
t_test_por <- t.test(data_coded_response_fie$sum_performance, data_coded_response_fid$sum_performance, alternative = "two.sided", var.equal = FALSE)
# learning
t_test_lor <- t.test(data_coded_response_fie$sum_learning, data_coded_response_fid$sum_learning, alternative = "two.sided", var.equal = FALSE)
print(t_test_por)
##
## Welch Two Sample t-test
##
## data: data_coded_response_fie$sum_performance and data_coded_response_fid$sum_performance
## t = -0.63017, df = 132.35, p-value = 0.5297
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
## -0.4523835 0.2337844
## sample estimates:
## mean of x mean of y
## 1.043478 1.152778
print(t_test_lor)
##
## Welch Two Sample t-test
##
## data: data_coded_response_fie$sum_learning and data_coded_response_fid$sum_learning
## t = -0.74679, df = 138.83, p-value = 0.4565
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
## -0.6035270 0.2726091
## sample estimates:
## mean of x mean of y
## 2.695652 2.861111
## compute relevant information to generate figure
data_coded_response_combined_fig <- data_coded_response %>%
pivot_longer(sum_performance:sum_learning,
names_to = "response_type",
values_to = "num_response") %>%
group_by(Condition, response_type) %>%
summarise(count = n(),
mean = mean(num_response, na.rm = TRUE),
sd = sd(num_response, na.rm = TRUE),
sem = sd / sqrt(count),)
## `summarise()` has grouped output by 'Condition'. You can override using the
## `.groups` argument.
## generate figure
response_labels <- c("Learning-Oriented", "Performance-Oriented")
response_breaks <- c("sum_performance", "sum_learning")
fill_labels <- c("Failure-Is-Debilitating Condition", "Failure-Is-Enhancing Condition")
p_values <- c(paste("p =", round(t_test_por$p.value, digits = 3)),
paste("p =", round(t_test_lor$p.value, digits = 3)))
y_breaks <- seq(0, 5, by = 1)
fig_coded_response <- ggplot(data_coded_response_combined_fig,
aes(x=response_type, y=mean, group=Condition, fill=Condition)) +
geom_bar(position="dodge", stat="identity") +
geom_errorbar(aes(ymin=mean-sem, ymax=mean+sem),
width=0.1, position=position_dodge(0.9)) +
scale_x_discrete("Parental Responses to Child-Failure Scenario",
limits=response_breaks, labels=response_labels) +
scale_y_continuous("Number of Responses", breaks=y_breaks) +
scale_fill_brewer(labels=fill_labels, palette="Blues") +
annotate("text", x=1:2, y=3.5, label=p_values) +
theme_bw() +
theme(legend.position="right", legend.title=element_blank(),
panel.grid=element_blank(), panel.border=element_blank(), axis.line=element_line())
# Print the figure
fig_coded_response
#### 3-1). Main analysis (considering the control variable)
Were parents who hold a failure-is-debilitating mind-set more
likely to react with concerns about their child’s performance and lack
of ability even after controlling parents’ perception of their
children’s competence in school?
ANCOVA used to compare mean of parents’ responses (i.e.,
learning-oriented, performance-oriented) in each condition (i.e.,
failure-is-debilitating and failure-is-enhancing) while controlling for
the effect of a covariate (i.e., parents’ perceptions on child’s
competence.
Original study:
When we controlled for parents’ perception of their children’s
competence in school, failure-mind-set condition still predicted
performance-oriented, t(131) = 3.249, p = .002, and learning-oriented,
t(1, 131) = −2.02, p = .046, responses to children’s failure.
Rescue project:
<div style=“font-size: larger;”,“margin-left: 30px;”> Even
after controlling for parents’ perception of their children’s competence
in school, the failure-mindset condition still did not
significantly predict performance-oriented responses (t(138) = -0.669, p
= 0.504), nor learning-oriented responses (t(138) = -0.757, p =
0.450). These results indicate that, unlike in Study 2, the
failure-mindset condition—whether debilitating or enhancing—was not a
significant factor in determining how parents responded to their child’s
failure after accounting for their perception of their child’s
competence. </ div>
control <-read.csv("../data/Final Data Coding _EJM_control.csv")
control <- control %>%
mutate(mean_competence = rowMeans(select(., Subject_1:Subject_4), na.rm = TRUE))
# Split the data by condition
control_fie <- control %>%
filter(Condition == "Enhancing")
control_fid <- control %>%
filter(Condition == "Debilitating")
# sum_performance and sum_learning are the dependent variables,
# Condition is the independent variable (factor),
# and mean_competence is the covariate.
# ANCOVA with linear regression for learning
lm_performance <- lm(sum_performance ~ Condition + mean_competence, data = control)
summary(lm_performance)
##
## Call:
## lm(formula = sum_performance ~ Condition + mean_competence, data = control)
##
## Residuals:
## Min 1Q Median 3Q Max
## -1.2399 -1.0385 -0.1097 0.8312 3.9615
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 0.89845 0.38307 2.345 0.0204 *
## ConditionEnhancing -0.11603 0.17335 -0.669 0.5044
## mean_competence 0.05691 0.08133 0.700 0.4852
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 1.027 on 138 degrees of freedom
## Multiple R-squared: 0.006397, Adjusted R-squared: -0.008003
## F-statistic: 0.4442 on 2 and 138 DF, p-value: 0.6422
lm_learning <- lm(sum_learning ~ Condition + mean_competence, data = control)
summary(lm_learning)
##
## Call:
## lm(formula = sum_learning ~ Condition + mean_competence, data = control)
##
## Residuals:
## Min 1Q Median 3Q Max
## -2.8832 -0.8620 0.1309 1.1309 5.0956
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 2.73473 0.49282 5.549 1.42e-07 ***
## ConditionEnhancing -0.16880 0.22302 -0.757 0.450
## mean_competence 0.02828 0.10463 0.270 0.787
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 1.322 on 138 degrees of freedom
## Multiple R-squared: 0.00451, Adjusted R-squared: -0.009917
## F-statistic: 0.3126 on 2 and 138 DF, p-value: 0.7321
4). Demographics of Prolific Sample
1). Parent’s Gender
gender_table <- d_condition %>%
# Recode combined categories
mutate(Gender = case_when(
Gender == "Agender,Genderqueer, gender fluid, or non-binary" ~ "Non-binary",
Gender == "Genderqueer, gender fluid, or non-binary" ~ "Non-binary",
Gender == "Man or Male,Transgender" ~ "Transgender",
Gender == "Man or Male" ~ "Male",
Gender == "Woman or Female" ~ "Female",
TRUE ~ as.character(Gender)
)) %>%
# Convert Education to a factor and reorder levels
mutate(Gender = factor(Gender, levels = c("Female", "Male", "Transgender", "Non-binary"))) %>%
# Count and calculate percentages
count(Gender) %>%
mutate(Percent = n / sum(n) * 100) %>%
mutate(Percent = paste0(round(Percent, 1), "%")) %>%
# Rename for the table
rename(`Gender` = Gender)
# table with kable()
kable(gender_table, col.names = c("Gender", "N", "%"))
| Female |
98 |
69.5% |
| Male |
40 |
28.4% |
| Transgender |
1 |
0.7% |
| Non-binary |
2 |
1.4% |
#figure
ggplot(gender_table, aes(x = `Gender`, y = n, fill = `Gender`)) +
geom_bar(stat = "identity") +
theme_minimal() +
labs(x = "Gender", y = "Percentage", fill = "Gender") +
geom_text(aes(label = Percent), vjust = -0.4, size =3.5) +
scale_fill_brewer(palette = "Greys") +
theme_bw()+
theme(legend.position="right",
legend.title = element_blank(),
panel.grid.major=element_blank(),
panel.grid.minor=element_blank(),
panel.border=element_blank(),
axis.line=element_line())
##### 2).Parents’ Education
# education
education_table <- d_condition %>%
# Recode combined categories
mutate(Education = case_when(
Education == "College degree,Post Graduate degree" ~ "Post Graduate degree",
Education == "HS degree,College degree" ~ "College degree",
TRUE ~ as.character(Education)
)) %>%
# Convert Education to a factor and reorder levels
mutate(Education = factor(Education, levels = c("No degree", "HS degree", "College degree", "Post Graduate degree"))) %>%
# Count and calculate percentages
count(Education) %>%
mutate(Percent = n / sum(n) * 100) %>%
mutate(Percent = paste0(round(Percent, 1), "%")) %>%
# Rename for the table
rename(`Education Level` = Education)
# table with kable()
kable(education_table, col.names = c("Education Level", "N", "%"))
| No degree |
2 |
1.4% |
| HS degree |
49 |
34.8% |
| College degree |
63 |
44.7% |
| Post Graduate degree |
27 |
19.1% |
# plot
ggplot(education_table, aes(x = `Education Level`, y = n, fill = `Education Level`)) +
geom_bar(stat = "identity") +
theme_minimal() +
labs(x = "Education Level", y = "Percentage", fill = "Education Level") +
geom_text(aes(label = Percent), vjust = -0.4, size =3.5) +
scale_fill_brewer(palette = "Greys") +
theme_bw() +
theme(legend.position="right",
legend.title = element_blank(),
panel.grid.major=element_blank(),
panel.grid.minor=element_blank(),
panel.border=element_blank(),
axis.line=element_line())
3).Parents’ Race
# Recode combined categories into separate rows
race_table <- d_condition %>%
mutate(Race = case_when(
Race == "Black or African American,White" ~ "Black or African American;White",
Race == "Hispanic, Latino, or Spanish origin,White" ~ "Hispanic, Latino, or Spanish origin;White",
Race == "American Indian or Alaska Native,White" ~ "American Indian or Alaska Native;White",
Race == "Other"~ "Fill in their own answer",
TRUE ~ as.character(Race)
)) %>%
separate_rows(Race, sep = ";") %>%
mutate(Race = factor(Race, levels = c("White", "Black or African American", "Asian", "Hispanic, Latino, or Spanish origin",
"American Indian or Alaska Native", "Fill in their own answer", "NA"))) %>%
count(Race, name = "Count") %>%
mutate(Percent = Count / sum(Count) * 100) %>%
mutate(Percent = paste0(round(Percent, 1), "%"))
# table with kable()
knitr::kable(race_table, col.names = c("Race", "N", "%"))
| White |
110 |
75.9% |
| Black or African American |
17 |
11.7% |
| Asian |
8 |
5.5% |
| Hispanic, Latino, or Spanish origin |
5 |
3.4% |
| American Indian or Alaska Native |
2 |
1.4% |
| Fill in their own answer |
2 |
1.4% |
| NA |
1 |
0.7% |
# plot
ggplot(race_table, aes(x = Race, y = Count, fill = Race)) +
geom_bar(stat = "identity") +
theme_minimal() +
labs(title = "Race Distribution", x = "Race Category", y = "Count") +
geom_text(aes(label = Percent), vjust = -0.3, size = 2.5) +
scale_fill_brewer(palette = "Greys") +
theme_bw() +
theme(legend.position="right",
legend.title = element_blank(),
panel.grid.major = element_blank(),
panel.grid.minor = element_blank(),
panel.border = element_blank(),
axis.line = element_line()) +
theme(axis.text.x = element_text(angle = 45, hjust = 1))
