##Introduction
The study led by Hernandez and colleagues (2021) is one of the first papers to experimentally test the impact of a strength based framework on perceptions of academic persistence, state self-esteem, and the endorsement of background specific strengths among students of color. The strength-based framework is not a new approach (i.e., community cultural wealth model; Yosso, 2005). Many scholars of color have examined the cultural values that students bring into systems of higher education using rich qualitative and correlational approaches. Recently the applied educational intervention field has used this framework to create adaptive interventions that enable students of color to reach academic persistence and achievement.
For the purposes of promoting open science I am replicating work led by Hernandez and colleagues (2021) because their strength-based experimental paradigm will teach me how to create methods that empower students of color to reflect and consider the strengths and values that they bring with them as they embark in higher education. I also hope to use the data from this replication to teach me how to clean, analyze, and write scientific reports using RStudio. Altogether, the opportunity to replicate work by Hernandez and colleagues (2021) is going to teach me (1) how to develop strength-based experimental paradigms to inform the development of my intervention work (i.e., my program of research), and (2) prepare with the necessary understanding of open-science, new experimental paradigms, using statistical software (i.e., R Studio, GIT, GITHub,GitBash) which are pivotal skills for the job market post-graduation.
###Study 1
##Method
Description of Study 1. In this work they recruited 200 university students through Prolific. Their criteria for recruitment were students who had a household income of no more than $70,000 a year (i.e., the goal was to target lower income samples). Out of the undergraduates they recruited they had 52.3% White students, 14.1% Asian students, 14.1% Latinx students, 7.5% Multiracial students, 1% Middle Eastern/Arab, .5% American Indian/Native American/Alaska Native, .5% Pacific Islander, 3% unknown ethnic background. It appears that for study one the authors recruited a convenience sample.
Exclusions: They excluded one participants who failed the attention check.So, for my present work, I will only exclude students who fail the attention check.
###Procedure
First, participants were assigned to one of three conditions: background specific strengths (BSS), active control condition, and the passive control condition. After random assignment students had to complete a writing activity (except for the passive control condition) followed by reporting endorsement of background-specific strengths, academic persistence, and state self-esteem.
###Materials
Background specific strengths (BSS) manipulation: “You may have heard or come to believe that certain people have advantages and are more likely to do better in school and in life, like maybe people who have more money or a higher socioeconomic status. You may have even heard an assumption that people from lower socioeconomic status backgrounds are disadvantaged. Actually though, a growing body of empirical research indicates that students from lower socioeconomic status backgrounds have acquired a lot of knowledge, skills, and abilities from their lived experiences (e.g., as a result of adversity). Collectively, we can call these “background-specific strengths”. Students from lower socioeconomic status backgrounds have acquired these background-specific strengths from their lived experiences and these unique strengths can serve as assets that can help them in school, that can benefit their schools, and that can benefit society. We acknowledge the multiple strengths of these students and want to learn more about them.”
Students were then asked to reflect and respond to the following two questions for five minutes: “(1)What “background-specific strengths” do you have that you have developed or acquired from your lived experiences (associated with your socioeconomic background)? You can think of these as strengths that are unique to you and people from similar backgrounds. In other words, people from other groups (e.g., from higher socioeconomic status) probably wouldn’t have these unique strengths. (2) How can you use these background-specific strengths to help you in your education, to benefit your school, and/or to benefit society?”
After reflecting on these two questions participants were asked to write and reflect the lessons they had learned to another students through their writing and this is known as the “saying is believing technique.”
Active control condition: Write for five minutes about your schedule.
Passive control condition: No information before self-reporting the primary outcome measures.
##Measures (Directly from supplementary materials)
Endorsement of background-specific strengths
Scale 1 (strongly disagree) to 6 (strongly agree) 1) I understand what background-specific strengths are. 2) I have certain background-specific strengths that can benefit my school. 3) I have certain background-specific strengths that can benefit society. 4) Due to my family’s socioeconomic background, I have no background-specific strengths to offer. (reverse scored) 5) I am an asset to my school. 6) I am an asset to society. 7) Due to my socioeconomic background, I have obtained value. 8) The background-specific strengths that I have learned growing up have a lot of value that can benefit my school. 9) The background-specific strengths that I have learned growing up have a lot of value that can benefit society. 10) There are really no strengths I have learned from my lived experiences and background. (reverse scored)
State self-esteem (adapted from Heatherton & Polivy, 1991)
Scale: 1 (strongly disagree) to 6 (strongly agree) 1) I am satisfied with myself. 2) I trust in my abilities. 3) I am satisfied with my appearance.
Perceived academic difficulty as impossible (Oyserman, Destin, & Novin, 2015)
Scale: 1 (strongly disagree) to 6 (strongly agree) 1) When I feel stuck on a school task, it’s a sign that my effort is better spent elsewhere. 2) Sometimes people work at things that just aren’t meant for them. If a school task feels too difficult, I should move on to something else. 3) Finding a school task really difficult tells me that I can’t complete it successfully. 4) If a school task feels really difficult, it may not be possible for me.
Status uncertainty (Destin, Rheinschmidt-Same, & Richeson, 2017)
Scale: 1 (strongly disagree) to 7 (strongly agree) 1) My beliefs about where I stand in society often conflict with one another. 2) On one day I might have one opinion of my social standing and on another day I might have a different opinion. 3) I spend a lot of time wondering about where I stand in society. 4) Sometimes I feel that I am not really the social status that others think I am. 5) When I think about the kind of person I have been in the past, I’m not sure what it means for my current social standing. 6) I seldom experience conflict between where I’ve been and where I’m going in society. (reverse scored; see Destin, Rheinschmidt-Same, & Richeson, 2017 for inclusion decision) 7) Sometimes I think it’s easier to identify where other people stand in society than to identify where I stand. 8) My beliefs about where I stand in society seem to change frequently. 9) If I were asked to describe my standing in society, my description might end up being different from one day to another day. 10) Even if I wanted to, I don’t think I could tell someone how I view my own social standing. 11) In general, I have a clear sense of where I stand in society. (reverse scored) 12) It is often hard for me to make up my mind about things because I don’t have a clear sense of my status in society.
Campus belonging (Hurtado & Carter, 1997)
Scale: 1 (strongly disagree) to 6 (strongly agree) 1) I see myself as a part of the campus community. 2) I feel that I am a member of the campus community. 3) I feel a sense of belonging to the campus community.
Academic efficacy (Midgely et al., 2000)
Scale: 1 (strongly disagree) to 6 (strongly agree) 1) I’m certain I can master the skills taught in my courses this year. 2) I’m certain I can figure out how to do the most difficult course work. 3) I can do almost all the work in my courses if I don’t give up. 4) Even if the work is hard, I can learn it. 5) I can do even the hardest work in my courses if I try.
Self-concept clarity (Campbell et al., 1996)
Scale: 1 (strongly disagree) to 7 (strongly agree) 1) My beliefs about myself often conflict with one another. 2) On one day I might have one opinion of myself and on another day I might have a different opinion. 3) I spend a lot of time wondering about what kind of person I really am. 4) Sometimes I feel that I am not really the person that I appear to be. 5) When I think about the kind of person I have been in the past, I’m not sure what I was really like. 6) I seldom experience conflict between the different aspects of my personality. (reverse scored) 7) Sometimes I think I know other people better than I know myself. 8) My beliefs about myself seem to change very frequently. 9) If I were asked to describe my personality, my description might end up being different from one day to another day. 10) Even if I wanted to, I don’t think I could tell someone what I’m really like. 11) In general, I have a clear sense of who I am and what I am. (reverse scored) 12) It is often hard for me to make up my mind about things because I don’t really know what I want.
Attention check and other items
If prolific lets me recruit participants who have an income below $70,000 a year I should be able to replicate this study and have no differences from the original study. Although, this study had a mix of different racial groups:52.3% White students, 14.1% Asian students, 14.1% Latinx students, 7.5% Multiracial students, 1% Middle Eastern/Arab, .5% American Indian/Native American/Alaska Native, .5% Pacific Islander, 3% unknown ethnic background. So, in the present work I would need it to be racially diverse.
###Analysis Plan & Confirmatory Analysis Main Inference of the Paper: Reflecting on background specific strengths will create more adaptive recursive psychological cycles (higher state self esteem and the perceived ability to persist in the face of challenges) compared to control.
There are two orthogonal contrasts and the first one is background specific strengths (+1) compared to passive (-.5) and active control (-.5).The second contrast is the residual contrast which directly compares the passive control (+.5) and the active control condition (-.5). So, in the second contrast they are hoping that the residuals from the control conditions do not differ from each other. These contrast will be used to examine all three primary outcome variables.
Primary Outcome Variables: Endorsement of background specific strengths, academic persistence, and state self-esteem.
Background Specific Strengths Using those two contrasts they examined the impact of the intervention on endorsement of background specific strengths using a one-way ANOVA. For this they reported everything including the p-value, effect size, and the confidence interval. Also, they had 99% power to detect a small effect size (B = .32).They found that participants in the background specific strength condition were significantly different than control. In the second contrast, they found that the control conditions did not differ from each other.
Academic Persistence They were underpowered to detect a very small effect size (B = 0.15) but still found that the BSS conditions had higher self-reported academic persistence than control. They also found that the control conditions did not differ on academic persistence.
State Self-Esteem Despite being underpowered to detect a very small effect size (B = .07) they found that the background specific strength condition had higher state self-esteem than control. They also found that the control conditions did not differ from each other.
###Results ##Data Preparation
library(readxl)## Warning: package 'readxl' was built under R version 4.1.3hernandez <- read_excel("C:/Users/Leslie Remache/OneDrive - Stanford/Stanford PhD/Classes/Fall 2022/Statistics/hernandez2021/Henanadez.xlsx")
View(hernandez)#loading libraries for data cleaning.
library(foreign)## Warning: package 'foreign' was built under R version 4.1.3library(tidyr)## Warning: package 'tidyr' was built under R version 4.1.3library(dplyr)## Warning: package 'dplyr' was built under R version 4.1.3##
## Attaching package: 'dplyr'## The following objects are masked from 'package:stats':
##
## filter, lag## The following objects are masked from 'package:base':
##
## intersect, setdiff, setequal, unionlibrary(stringr)## Warning: package 'stringr' was built under R version 4.1.3library(ggplot2)## Warning: package 'ggplot2' was built under R version 4.1.3#take a look at the column names.
colnames(hernandez)## [1] "Duration (in seconds)" "Condition" "BSS Phase 3"
## [4] "BSS1" "BSS2" "BSS3"
## [7] "BSS4" "BSS4R" "BSS5"
## [10] "BSS6" "BSS7" "BSS8"
## [13] "BSS9" "BSS10" "BSS10R"
## [16] "BSSM" "SSE1" "SSE2"
## [19] "SSE3" "SSE Mean" "ADI1"
## [22] "ADI2" "ADI3" "ADI4"
## [25] "ADI Mean" "Att1" "Att2"
## [28] "Att3" "Age" "Gender"
## [31] "RaceEth" "FirstGen" "Sibling"#Clearning up a bit.
#Step 1: lets get rid of the columns and rows that I do not need.
filtered_hernandez = hernandez %>%
filter(is.na(Condition)==FALSE) #excluding participants who do not have a condition number
filtered_hernandez$ID <- seq.int(nrow(filtered_hernandez))###Adding a numbering ID for participants
View(filtered_hernandez)colnames(filtered_hernandez)[colnames(filtered_hernandez) == "Duration (in seconds)"] = "duration" ###how to reanme a column
colnames(filtered_hernandez)[colnames(filtered_hernandez) == "SSE Mean"] = "SSEM" ###how to reanme a column
colnames(filtered_hernandez)[colnames(filtered_hernandez) == "ADI Mean"] = "ADIM" ###how to reanme a column
print(filtered_hernandez)## # A tibble: 6 x 34
## duration Condition `BSS Phase 3` BSS1 BSS2 BSS3 BSS4 BSS4R BSS5 BSS6
## <chr> <dbl> <chr> <chr> <chr> <chr> <chr> <dbl> <chr> <chr>
## 1 65 2 <NA> 3 5 3 2 5 3 4
## 2 115 3 <NA> 3 4 4 1 6 3 4
## 3 119 1 Not sure 1 2 3 3 4 1 2
## 4 179 3 <NA> 2 4 5 1 6 5 5
## 5 94 2 <NA> 3 1 3 <NA> NA <NA> 4
## 6 30 1 <NA> <NA> <NA> <NA> <NA> NA <NA> <NA>
## # ... with 24 more variables: BSS7 <chr>, BSS8 <chr>, BSS9 <chr>, BSS10 <chr>,
## # BSS10R <dbl>, BSSM <dbl>, SSE1 <chr>, SSE2 <chr>, SSE3 <chr>, SSEM <dbl>,
## # ADI1 <chr>, ADI2 <chr>, ADI3 <chr>, ADI4 <chr>, ADIM <dbl>, Att1 <chr>,
## # Att2 <chr>, Att3 <chr>, Age <chr>, Gender <chr>, RaceEth <chr>,
## # FirstGen <chr>, Sibling <chr>, ID <int>#computing the means of different groups.
#BSSM
filtered_hernandez %>%
group_by(Condition) %>%
summarize(AvgScore = mean(BSSM, na.rm=T)) # the na.rm tells R to ignore NA values## # A tibble: 3 x 2
## Condition AvgScore
## <dbl> <dbl>
## 1 1 3.1
## 2 2 3.36
## 3 3 4.4#SSEM
filtered_hernandez %>%
group_by(Condition) %>%
summarize(AvgScore = mean(SSEM, na.rm=T)) # the na.rm tells R to ignore NA values## # A tibble: 3 x 2
## Condition AvgScore
## <dbl> <dbl>
## 1 1 3.33
## 2 2 3
## 3 3 4#ADIM
filtered_hernandez %>%
group_by(Condition) %>%
summarize(AvgScore = mean(ADIM, na.rm=T)) # the na.rm tells R to ignore NA values## # A tibble: 3 x 2
## Condition AvgScore
## <dbl> <dbl>
## 1 1 3.5
## 2 2 3
## 3 3 2.62##Confirmatory Analyses
#Running ANOVAS
#BSSM
# Compute the analysis of variance
res.aov <- aov(BSSM ~ Condition, data = filtered_hernandez)
# Summary of the analysis
summary(res.aov)## Df Sum Sq Mean Sq F value Pr(>F)
## Condition 1 1.3931 1.3931 8.999 0.0577 .
## Residuals 3 0.4644 0.1548
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 1 observation deleted due to missingness#SSEM
# Compute the analysis of variance
res.aov <- aov(SSEM ~ Condition, data = filtered_hernandez)
# Summary of the analysis
summary(res.aov)## Df Sum Sq Mean Sq F value Pr(>F)
## Condition 1 0.5143 0.5143 0.565 0.507
## Residuals 3 2.7302 0.9101
## 1 observation deleted due to missingness# Compute the analysis of variance
res.aov <- aov(ADIM ~ Condition, data = filtered_hernandez)
# Summary of the analysis
summary(res.aov)## Df Sum Sq Mean Sq F value Pr(>F)
## Condition 1 0.5143 0.5143 1.964 0.256
## Residuals 3 0.7857 0.2619
## 1 observation deleted due to missingness##Discussion
##Summary of Replication Attempt Open the discussion section with a paragraph summarizing the primary result from the confirmatory analysis and the assessment of whether it replicated, partially replicated, or failed to replicate the original result.
##Commentary Add open-ended commentary (if any) reflecting (a) insights from follow-up exploratory analysis, (b) assessment of the meaning of the replication (or not) - e.g., for a failure to replicate, are the differences between original and present study ones that definitely, plausibly, or are unlikely to have been moderators of the result, and (c) discussion of any objections or challenges raised by the current and original authors about the replication attempt. None of these need to be long.