CARPS Reproducibility Report
Report Details
articleID <- "16-9-2014_PS" # insert the article ID code here e.g., "10-3-2015_PS"
reportType <- 'final' # specify whether this is the 'pilot' report or 'final' report
pilotNames <- "Manuel Bohn" # insert the pilot's name here e.g., "Tom Hardwicke". If there are multiple cpilots enter both names in a character string e.g., "Tom Hardwicke, Bob Dylan"
copilotNames <- "Tom Hardwicke" # # insert the co-pilot's name here e.g., "Michael Frank". If there are multiple co-pilots enter both names in a character string e.g., "Tom Hardwicke, Bob Dylan"
pilotTTC <- 360 # insert the pilot's estimated time to complete (in minutes, fine to approximate) e.g., 120
copilotTTC <- 60 # insert the co-pilot's estimated time to complete (in minutes, fine to approximate) e.g., 120
pilotStartDate <- as.Date("06/26/18", format = "%m/%d/%y") # insert the pilot's start date in US format e.g., as.Date("01/25/18", format = "%m/%d/%y")
copilotStartDate <- as.Date("10/11/18", format = "%m/%d/%y") # insert the co-pilot's start date in US format e.g., as.Date("01/25/18", format = "%m/%d/%y")
completionDate <- as.Date("10/11/18", format = "%m/%d/%y") # copilot insert the date of final report completion (after any necessary rounds of author assistance) in US format e.g., as.Date("01/25/18", format = "%m/%d/%y")Methods summary:
The authors compared people from different racial backgrounds on their attitudes towards people from different racial groups, including their own. They used implicit measures of relative preference between races. Results suggest that people evaluated their own racial group as the most positive. Across racial groups, people evaluated Whites most positively, followed by Asians, Blacks and Hispanics when evaluating racial groups other than their own.
Target outcomes:
Results
Sample sizes vary among the tests reported because of missing data. For participants of all racial and ethnic groups, the order of implicit racial preferences was the same (Fig. 1). Whites, Asians, Blacks, and Hispanics exhibited the most positive associations for their own racial group. In addition, their implicit evaluations of the remaining racial groups always placed White people first, followed by Asian, Black, and then Hispanic people, all pairwise ts(1467–31,567) > 4.06, all ps < .001, all ds > 0.06, average d = 0.2, except for the comparison between the Black aggregate and Hispanic aggregate variables among Asian participants, t(1468) = 0.86, p = .394. Participants who identified with racial groups other than the four target groups (e.g., American Indians, Pacific Islanders) showed the same ordinal pattern, all ts(3291–3303) > 4.2, all ps < .001, all ds > 0.08, average d = 0.10.
The Supplemental Material provides t and d values for implicit evaluations (Table S1)
Step 1: Load packages and prepare report object
# load packages
library(tidyverse) # for data munging
library(knitr) # for kable table formating
library(haven) # import and export 'SPSS', 'Stata' and 'SAS' Files
library(readxl) # import excel files
library(ReproReports) # custom report functions
library (lsr)# Prepare report object. This will be updated automatically by the reproCheck function each time values are compared
reportObject <- data.frame(dummyRow = TRUE, reportedValue = NA, obtainedValue = NA, valueType = NA, percentageError = NA, comparisonOutcome = NA, eyeballCheck = NA)Step 2: Load data
d <- read.csv("data/Study1Data.txt" ,header=T, sep="\t")Step 3: Tidy data
d.tidy <- d %>%
gather(test,score,-c(1:8))%>%
arrange(SessionID)Step 4: Run analysis
Descriptive statistics
Filtering is done based syntax file shared by the authors. There it says:
filter_$=((AmericanCitizen=1 OR AmericanResidence=1) & TooFastPct<.1).
For all descriptive and inferential statistics, we apply the same filter. In addition, rows with NAs for a given measure are filtered out, too.
Reproducing Table 1.
tab1 <- d.tidy%>%
filter(TooFastPct < .1, AmericanCitizen == 1 | AmericanResidence == 1, test == "WhiteImplicit" | test == "AsianImplicit" | test == "BlackImplicit" | test == "HispImplicit")%>%
na.omit(score)%>%
group_by(Race,test)%>%
summarise(n = length(unique(SessionID)),
M = mean(score),
sd = sd(score))%>%
gather(type,measure,-c(1:2))%>%
mutate(type = paste(test,type,sep="_"))%>%
mutate(order = 1:12)%>%
select(-test,-order)%>%
spread(type,measure)%>%
ungroup()%>%
mutate(
Race = factor(Race, levels = c("White", "Asian","Black","Hispanic","Other"))
)%>%
arrange(Race)%>%
select(1,12, 11, 13, 3, 2, 4, 6, 5, 7, 9, 8, 10)%>%
select_all(~sub("Implicit","",.))Here is Table 1 from the paper:
Table 1
tab1%>%
kable(digits = 2)| Race | White_n | White_M | White_sd | Asian_n | Asian_M | Asian_sd | Black_n | Black_M | Black_sd | Hisp_n | Hisp_M | Hisp_sd |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| White | 31572 | 0.15 | 0.36 | 31600 | -0.01 | 0.30 | 31590 | -0.05 | 0.33 | 31580 | -0.09 | 0.29 |
| Asian | 1463 | 0.06 | 0.34 | 1468 | 0.26 | 0.32 | 1466 | -0.15 | 0.33 | 1466 | -0.16 | 0.29 |
| Black | 3661 | -0.03 | 0.34 | 3665 | -0.08 | 0.31 | 3662 | 0.22 | 0.34 | 3661 | -0.11 | 0.30 |
| Hispanic | 4402 | 0.04 | 0.34 | 4405 | -0.04 | 0.31 | 4404 | -0.10 | 0.34 | 4412 | 0.10 | 0.32 |
| Other | 3198 | 0.08 | 0.35 | 3200 | 0.04 | 0.32 | 3206 | -0.03 | 0.35 | 3198 | -0.08 | 0.30 |
Means and SDs all seem to match, the sample sizes, however, differ from the ones reported in the paper. This is somehow confusing as the matching in all the summary statistics suggests that the filtering is done right. The sample sizes also differ if NAs are included or when any or all of the filters are omitted.
# N for Whites
reportObject <- reproCheck(reportedValue = "31656", obtainedValue = tab1 %>% filter(Race == "White") %>% pull(White_n), valueType = "n")## [1] "MINOR_ERROR for n. The reported value (31656) and the obtained value (31572) differed by 0.27%. Note that the obtained value was rounded to 0 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = "1469", obtainedValue = tab1 %>% filter(Race == "Asian") %>% pull(White_n), valueType = "n")## [1] "MINOR_ERROR for n. The reported value (1469) and the obtained value (1463) differed by 0.41%. Note that the obtained value was rounded to 0 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = "3676", obtainedValue = tab1 %>% filter(Race == "Black") %>% pull(White_n), valueType = "n")## [1] "MINOR_ERROR for n. The reported value (3676) and the obtained value (3661) differed by 0.41%. Note that the obtained value was rounded to 0 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = "4411", obtainedValue = tab1 %>% filter(Race == "Hispanic") %>% pull(White_n), valueType = "n")## [1] "MINOR_ERROR for n. The reported value (4411) and the obtained value (4402) differed by 0.2%. Note that the obtained value was rounded to 0 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = "3307", obtainedValue = tab1 %>% filter(Race == "Other") %>% pull(White_n), valueType = "n")## [1] "MINOR_ERROR for n. The reported value (3307) and the obtained value (3198) differed by 3.3%. Note that the obtained value was rounded to 0 decimal places to match the reported value."# Means for Whites
reportObject <- reproCheck(reportedValue = "0.15", obtainedValue = tab1 %>% filter(Race == "White") %>% pull(White_M), valueType = "mean")## [1] "MATCH for mean. The reported value (0.15) and the obtained value (0.15) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = "0.06", obtainedValue = tab1 %>% filter(Race == "Asian") %>% pull(White_M), valueType = "mean")## [1] "MATCH for mean. The reported value (0.06) and the obtained value (0.06) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = "-0.03", obtainedValue = tab1 %>% filter(Race == "Black") %>% pull(White_M), valueType = "mean")## [1] "MATCH for mean. The reported value (-0.03) and the obtained value (-0.03) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = "0.04", obtainedValue = tab1 %>% filter(Race == "Hispanic") %>% pull(White_M), valueType = "mean")## [1] "MATCH for mean. The reported value (0.04) and the obtained value (0.04) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = "0.08", obtainedValue = tab1 %>% filter(Race == "Other") %>% pull(White_M), valueType = "mean")## [1] "MATCH for mean. The reported value (0.08) and the obtained value (0.08) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."# SD for Whites
reportObject <- reproCheck(reportedValue = "0.36", obtainedValue = tab1 %>% filter(Race == "White") %>% pull(White_sd), valueType = "sd")## [1] "MATCH for sd. The reported value (0.36) and the obtained value (0.36) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = "0.34", obtainedValue = tab1 %>% filter(Race == "Asian") %>% pull(White_sd), valueType = "sd")## [1] "MATCH for sd. The reported value (0.34) and the obtained value (0.34) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = "0.34", obtainedValue = tab1 %>% filter(Race == "Black") %>% pull(White_sd), valueType = "sd")## [1] "MATCH for sd. The reported value (0.34) and the obtained value (0.34) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = "0.34", obtainedValue = tab1 %>% filter(Race == "Hispanic") %>% pull(White_sd), valueType = "sd")## [1] "MATCH for sd. The reported value (0.34) and the obtained value (0.34) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = "0.35", obtainedValue = tab1 %>% filter(Race == "Other") %>% pull(White_sd), valueType = "sd")## [1] "MATCH for sd. The reported value (0.35) and the obtained value (0.35) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."# N for Asians
reportObject <- reproCheck(reportedValue = "31684", obtainedValue = tab1 %>% filter(Race == "White") %>% pull(Asian_n), valueType = "n")## [1] "MINOR_ERROR for n. The reported value (31684) and the obtained value (31600) differed by 0.27%. Note that the obtained value was rounded to 0 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = "1474", obtainedValue = tab1 %>% filter(Race == "Asian") %>% pull(Asian_n), valueType = "n")## [1] "MINOR_ERROR for n. The reported value (1474) and the obtained value (1468) differed by 0.41%. Note that the obtained value was rounded to 0 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = "3680", obtainedValue = tab1 %>% filter(Race == "Black") %>% pull(Asian_n), valueType = "n")## [1] "MINOR_ERROR for n. The reported value (3680) and the obtained value (3665) differed by 0.41%. Note that the obtained value was rounded to 0 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = "4414", obtainedValue = tab1 %>% filter(Race == "Hispanic") %>% pull(Asian_n), valueType = "n")## [1] "MINOR_ERROR for n. The reported value (4414) and the obtained value (4405) differed by 0.2%. Note that the obtained value was rounded to 0 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = "3311", obtainedValue = tab1 %>% filter(Race == "Other") %>% pull(Asian_n), valueType = "n")## [1] "MINOR_ERROR for n. The reported value (3311) and the obtained value (3200) differed by 3.35%. Note that the obtained value was rounded to 0 decimal places to match the reported value."# Means for Asians
reportObject <- reproCheck(reportedValue = "-0.01", obtainedValue = tab1 %>% filter(Race == "White") %>% pull(Asian_M), valueType = "mean")## [1] "MATCH for mean. The reported value (-0.01) and the obtained value (-0.01) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = "0.26", obtainedValue = tab1 %>% filter(Race == "Asian") %>% pull(Asian_M), valueType = "mean")## [1] "MATCH for mean. The reported value (0.26) and the obtained value (0.26) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = "-0.08", obtainedValue = tab1 %>% filter(Race == "Black") %>% pull(Asian_M), valueType = "mean")## [1] "MATCH for mean. The reported value (-0.08) and the obtained value (-0.08) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = "-0.04", obtainedValue = tab1 %>% filter(Race == "Hispanic") %>% pull(Asian_M), valueType = "mean")## [1] "MATCH for mean. The reported value (-0.04) and the obtained value (-0.04) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = "0.04", obtainedValue = tab1 %>% filter(Race == "Other") %>% pull(Asian_M), valueType = "mean")## [1] "MATCH for mean. The reported value (0.04) and the obtained value (0.04) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."# SD for Asians
reportObject <- reproCheck(reportedValue = "0.30", obtainedValue = tab1 %>% filter(Race == "White") %>% pull(Asian_sd), valueType = "sd")## [1] "MATCH for sd. The reported value (0.3) and the obtained value (0.3) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = "0.32", obtainedValue = tab1 %>% filter(Race == "Asian") %>% pull(Asian_sd), valueType = "sd")## [1] "MATCH for sd. The reported value (0.32) and the obtained value (0.32) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = "0.31", obtainedValue = tab1 %>% filter(Race == "Black") %>% pull(Asian_sd), valueType = "sd")## [1] "MATCH for sd. The reported value (0.31) and the obtained value (0.31) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = "0.31", obtainedValue = tab1 %>% filter(Race == "Hispanic") %>% pull(Asian_sd), valueType = "sd")## [1] "MATCH for sd. The reported value (0.31) and the obtained value (0.31) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = "0.32", obtainedValue = tab1 %>% filter(Race == "Other") %>% pull(Asian_sd), valueType = "sd")## [1] "MATCH for sd. The reported value (0.32) and the obtained value (0.32) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."# N for Blacks
reportObject <- reproCheck(reportedValue = "31674", obtainedValue = tab1 %>% filter(Race == "White") %>% pull(Black_n), valueType = "n")## [1] "MINOR_ERROR for n. The reported value (31674) and the obtained value (31590) differed by 0.27%. Note that the obtained value was rounded to 0 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = "1472", obtainedValue = tab1 %>% filter(Race == "Asian") %>% pull(Black_n), valueType = "n")## [1] "MINOR_ERROR for n. The reported value (1472) and the obtained value (1466) differed by 0.41%. Note that the obtained value was rounded to 0 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = "3677", obtainedValue = tab1 %>% filter(Race == "Black") %>% pull(Black_n), valueType = "n")## [1] "MINOR_ERROR for n. The reported value (3677) and the obtained value (3662) differed by 0.41%. Note that the obtained value was rounded to 0 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = "4413", obtainedValue = tab1 %>% filter(Race == "Hispanic") %>% pull(Black_n), valueType = "n")## [1] "MINOR_ERROR for n. The reported value (4413) and the obtained value (4404) differed by 0.2%. Note that the obtained value was rounded to 0 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = "3317", obtainedValue = tab1 %>% filter(Race == "Other") %>% pull(Black_n), valueType = "n")## [1] "MINOR_ERROR for n. The reported value (3317) and the obtained value (3206) differed by 3.35%. Note that the obtained value was rounded to 0 decimal places to match the reported value."# Means for Blacks
reportObject <- reproCheck(reportedValue = "-0.05", obtainedValue = tab1 %>% filter(Race == "White") %>% pull(Black_M), valueType = "mean")## [1] "MATCH for mean. The reported value (-0.05) and the obtained value (-0.05) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = "-0.15", obtainedValue = tab1 %>% filter(Race == "Asian") %>% pull(Black_M), valueType = "mean")## [1] "MATCH for mean. The reported value (-0.15) and the obtained value (-0.15) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = "0.22", obtainedValue = tab1 %>% filter(Race == "Black") %>% pull(Black_M), valueType = "mean")## [1] "MATCH for mean. The reported value (0.22) and the obtained value (0.22) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = "-0.10", obtainedValue = tab1 %>% filter(Race == "Hispanic") %>% pull(Black_M), valueType = "mean")## [1] "MATCH for mean. The reported value (-0.1) and the obtained value (-0.1) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = "-0.03", obtainedValue = tab1 %>% filter(Race == "Other") %>% pull(Black_M), valueType = "mean")## [1] "MATCH for mean. The reported value (-0.03) and the obtained value (-0.03) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."# SD for Blacks
reportObject <- reproCheck(reportedValue = "0.33", obtainedValue = tab1 %>% filter(Race == "White") %>% pull(Black_sd), valueType = "sd")## [1] "MATCH for sd. The reported value (0.33) and the obtained value (0.33) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = "0.32", obtainedValue = tab1 %>% filter(Race == "Asian") %>% pull(Black_sd), valueType = "sd")## [1] "MINOR_ERROR for sd. The reported value (0.32) and the obtained value (0.33) differed by 3.13%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = "0.34", obtainedValue = tab1 %>% filter(Race == "Black") %>% pull(Black_sd), valueType = "sd")## [1] "MATCH for sd. The reported value (0.34) and the obtained value (0.34) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = "0.34", obtainedValue = tab1 %>% filter(Race == "Hispanic") %>% pull(Black_sd), valueType = "sd")## [1] "MATCH for sd. The reported value (0.34) and the obtained value (0.34) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = "0.35", obtainedValue = tab1 %>% filter(Race == "Other") %>% pull(Black_sd), valueType = "sd")## [1] "MATCH for sd. The reported value (0.35) and the obtained value (0.35) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."# N for Hispanics
reportObject <- reproCheck(reportedValue = "31664", obtainedValue = tab1 %>% filter(Race == "White") %>% pull(Hisp_n), valueType = "n")## [1] "MINOR_ERROR for n. The reported value (31664) and the obtained value (31580) differed by 0.27%. Note that the obtained value was rounded to 0 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = "1472", obtainedValue = tab1 %>% filter(Race == "Asian") %>% pull(Hisp_n), valueType = "n")## [1] "MINOR_ERROR for n. The reported value (1472) and the obtained value (1466) differed by 0.41%. Note that the obtained value was rounded to 0 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = "3676", obtainedValue = tab1 %>% filter(Race == "Black") %>% pull(Hisp_n), valueType = "n")## [1] "MINOR_ERROR for n. The reported value (3676) and the obtained value (3661) differed by 0.41%. Note that the obtained value was rounded to 0 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = "4411", obtainedValue = tab1 %>% filter(Race == "Hispanic") %>% pull(Hisp_n), valueType = "n")## [1] "MINOR_ERROR for n. The reported value (4411) and the obtained value (4412) differed by 0.02%. Note that the obtained value was rounded to 0 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = "3307", obtainedValue = tab1 %>% filter(Race == "Other") %>% pull(Hisp_n), valueType = "n")## [1] "MINOR_ERROR for n. The reported value (3307) and the obtained value (3198) differed by 3.3%. Note that the obtained value was rounded to 0 decimal places to match the reported value."# Means for Hispanics
reportObject <- reproCheck(reportedValue = "-0.09", obtainedValue = tab1 %>% filter(Race == "White") %>% pull(Hisp_M), valueType = "mean")## [1] "MATCH for mean. The reported value (-0.09) and the obtained value (-0.09) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = "-0.16", obtainedValue = tab1 %>% filter(Race == "Asian") %>% pull(Hisp_M), valueType = "mean")## [1] "MATCH for mean. The reported value (-0.16) and the obtained value (-0.16) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = "-0.11", obtainedValue = tab1 %>% filter(Race == "Black") %>% pull(Hisp_M), valueType = "mean")## [1] "MATCH for mean. The reported value (-0.11) and the obtained value (-0.11) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = "0.10", obtainedValue = tab1 %>% filter(Race == "Hispanic") %>% pull(Hisp_M), valueType = "mean")## [1] "MATCH for mean. The reported value (0.1) and the obtained value (0.1) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = "-0.08", obtainedValue = tab1 %>% filter(Race == "Other") %>% pull(Hisp_M), valueType = "mean")## [1] "MATCH for mean. The reported value (-0.08) and the obtained value (-0.08) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."# SD for Hispanics
reportObject <- reproCheck(reportedValue = "0.29", obtainedValue = tab1 %>% filter(Race == "White") %>% pull(Hisp_sd), valueType = "sd")## [1] "MATCH for sd. The reported value (0.29) and the obtained value (0.29) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = "0.29", obtainedValue = tab1 %>% filter(Race == "Asian") %>% pull(Hisp_sd), valueType = "sd")## [1] "MATCH for sd. The reported value (0.29) and the obtained value (0.29) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = "0.30", obtainedValue = tab1 %>% filter(Race == "Black") %>% pull(Hisp_sd), valueType = "sd")## [1] "MATCH for sd. The reported value (0.3) and the obtained value (0.3) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = "0.32", obtainedValue = tab1 %>% filter(Race == "Hispanic") %>% pull(Hisp_sd), valueType = "sd")## [1] "MATCH for sd. The reported value (0.32) and the obtained value (0.32) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = "0.30", obtainedValue = tab1 %>% filter(Race == "Other") %>% pull(Hisp_sd), valueType = "sd")## [1] "MATCH for sd. The reported value (0.3) and the obtained value (0.3) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."Inferential statistics
Reproducing top section of Table S1. We run the t-tests separate for each race in order to put them in the order (ranked by race preference within each race) they appear in the corresponding table. Outcomes are later joined into a table.
# order for whites
d.tidy%>%
filter(Race == "White",test == "WhiteImplicit" | test == "AsianImplicit" | test == "BlackImplicit" | test == "HispImplicit")%>%
group_by(test)%>%
na.omit()%>%
summarize(mean = mean(score))%>%
arrange(-mean)## # A tibble: 4 x 2
## test mean
## <chr> <dbl>
## 1 WhiteImplicit 0.146
## 2 AsianImplicit -0.0133
## 3 BlackImplicit -0.0458
## 4 HispImplicit -0.0867# comparisons
white <- d.tidy%>%
filter(Race == "White",TooFastPct < .1, AmericanCitizen == 1 | AmericanResidence == 1,
test == "WhiteImplicit" | test == "AsianImplicit" | test == "BlackImplicit" | test == "HispImplicit")%>%
group_by(SessionID,Race,test)%>%
group_by(Race,test)%>%
summarize(score = list(score))%>%
spread(test,score)%>%
mutate(t_1_2 = t.test(unlist(WhiteImplicit),unlist(AsianImplicit), paired = T)$statistic,
d_1_2 = cohensD(unlist(WhiteImplicit),unlist(AsianImplicit), method = "paired"),
t_2_3 = t.test(unlist(AsianImplicit),unlist(BlackImplicit), paired = T)$statistic,
d_2_3 = cohensD(unlist(AsianImplicit),unlist(BlackImplicit), method = "paired"),
t_3_4 = t.test(unlist(BlackImplicit),unlist(HispImplicit), paired = T)$statistic,
d_3_4 = cohensD(unlist(BlackImplicit),unlist(HispImplicit), method = "paired"))%>%
select(-2,-3,-4,-5)# order for asians
d.tidy%>%
filter(Race == "Asian",test == "WhiteImplicit" | test == "AsianImplicit" | test == "BlackImplicit" | test == "HispImplicit")%>%
group_by(test)%>%
na.omit()%>%
summarize(mean = mean(score))%>%
arrange(-mean)## # A tibble: 4 x 2
## test mean
## <chr> <dbl>
## 1 AsianImplicit 0.252
## 2 WhiteImplicit 0.0553
## 3 BlackImplicit -0.152
## 4 HispImplicit -0.157#comparisons
asian <- d.tidy%>%
filter(Race == "Asian",TooFastPct < .1, AmericanCitizen == 1 | AmericanResidence == 1,
test == "WhiteImplicit" | test == "AsianImplicit" | test == "BlackImplicit" | test == "HispImplicit")%>%
group_by(SessionID,Race,test)%>%
group_by(Race,test)%>%
summarize(score = list(score))%>%
spread(test,score)%>%
mutate(t_1_2 = t.test(unlist(AsianImplicit),unlist(WhiteImplicit), paired = T)$statistic,
d_1_2 = cohensD(unlist(AsianImplicit),unlist(WhiteImplicit), method = "paired"),
t_2_3 = t.test(unlist(WhiteImplicit),unlist(BlackImplicit), paired = T)$statistic,
d_2_3 = cohensD(unlist(WhiteImplicit),unlist(BlackImplicit), method = "paired"),
t_3_4 = t.test(unlist(BlackImplicit),unlist(HispImplicit), paired = T)$statistic,
d_3_4 = cohensD(unlist(BlackImplicit),unlist(HispImplicit), method = "paired"))%>%
select(-2,-3,-4,-5)# order for blacks
d.tidy%>%
filter(Race == "Black",test == "WhiteImplicit" | test == "AsianImplicit" | test == "BlackImplicit" | test == "HispImplicit")%>%
group_by(test)%>%
na.omit()%>%
summarize(mean = mean(score))%>%
arrange(-mean)## # A tibble: 4 x 2
## test mean
## <chr> <dbl>
## 1 BlackImplicit 0.215
## 2 WhiteImplicit -0.0313
## 3 AsianImplicit -0.0777
## 4 HispImplicit -0.108#comparisons
black <- d.tidy%>%
filter(Race == "Black",TooFastPct < .1, AmericanCitizen == 1 | AmericanResidence == 1,
test == "WhiteImplicit" | test == "AsianImplicit" | test == "BlackImplicit" | test == "HispImplicit")%>%
group_by(SessionID,Race,test)%>%
group_by(Race,test)%>%
summarize(score = list(score))%>%
spread(test,score)%>%
mutate(t_1_2 = t.test(unlist(BlackImplicit),unlist(WhiteImplicit), paired = T)$statistic,
d_1_2 = cohensD(unlist(BlackImplicit),unlist(WhiteImplicit), method = "paired"),
t_2_3 = t.test(unlist(WhiteImplicit),unlist(AsianImplicit), paired = T)$statistic,
d_2_3 = cohensD(unlist(WhiteImplicit),unlist(AsianImplicit), method = "paired"),
t_3_4 = t.test(unlist(AsianImplicit),unlist(HispImplicit), paired = T)$statistic,
d_3_4 = cohensD(unlist(AsianImplicit),unlist(HispImplicit), method = "paired"))%>%
select(-2,-3,-4,-5)# order for hispanics
d.tidy%>%
filter(Race == "Hispanic",test == "WhiteImplicit" | test == "AsianImplicit" | test == "BlackImplicit" | test == "HispImplicit")%>%
group_by(test)%>%
na.omit()%>%
summarize(mean = mean(score))%>%
arrange(-mean)## # A tibble: 4 x 2
## test mean
## <chr> <dbl>
## 1 HispImplicit 0.0875
## 2 WhiteImplicit 0.0436
## 3 AsianImplicit -0.0427
## 4 BlackImplicit -0.0895#comparisons
hispanic <- d.tidy%>%
filter(Race == "Hispanic",TooFastPct < .1, AmericanCitizen == 1 | AmericanResidence == 1,
test == "WhiteImplicit" | test == "AsianImplicit" | test == "BlackImplicit" | test == "HispImplicit")%>%
group_by(SessionID,Race,test)%>%
group_by(Race,test)%>%
summarize(score = list(score))%>%
spread(test,score)%>%
mutate(t_1_2 = t.test(unlist(HispImplicit),unlist(WhiteImplicit), paired = T)$statistic,
d_1_2 = cohensD(unlist(HispImplicit),unlist(WhiteImplicit), method = "paired"),
t_2_3 = t.test(unlist(WhiteImplicit),unlist(AsianImplicit), paired = T)$statistic,
d_2_3 = cohensD(unlist(WhiteImplicit),unlist(AsianImplicit), method = "paired"),
t_3_4 = t.test(unlist(AsianImplicit),unlist(BlackImplicit), paired = T)$statistic,
d_3_4 = cohensD(unlist(AsianImplicit),unlist(BlackImplicit), method = "paired"))%>%
select(-2,-3,-4,-5)# order for others
d.tidy%>%
filter(Race == "Other",test == "WhiteImplicit" | test == "AsianImplicit" | test == "BlackImplicit" | test == "HispImplicit")%>%
group_by(test)%>%
na.omit()%>%
summarize(mean = mean(score))%>%
arrange(-mean)## # A tibble: 4 x 2
## test mean
## <chr> <dbl>
## 1 WhiteImplicit 0.0804
## 2 AsianImplicit 0.0364
## 3 BlackImplicit -0.0354
## 4 HispImplicit -0.0813#comparisons
other <- d.tidy%>%
filter(Race == "Other",TooFastPct < .1, AmericanCitizen == 1 | AmericanResidence == 1,
test == "WhiteImplicit" | test == "AsianImplicit" | test == "BlackImplicit" | test == "HispImplicit")%>%
group_by(SessionID,Race,test)%>%
group_by(Race,test)%>%
summarize(score = list(score))%>%
spread(test,score)%>%
mutate(t_1_2 = t.test(unlist(WhiteImplicit),unlist(AsianImplicit), paired = T)$statistic,
d_1_2 = cohensD(unlist(WhiteImplicit),unlist(AsianImplicit), method = "paired"),
t_2_3 = t.test(unlist(AsianImplicit),unlist(BlackImplicit), paired = T)$statistic,
d_2_3 = cohensD(unlist(AsianImplicit),unlist(BlackImplicit), method = "paired"),
t_3_4 = t.test(unlist(BlackImplicit),unlist(HispImplicit), paired = T)$statistic,
d_3_4 = cohensD(unlist(BlackImplicit),unlist(HispImplicit), method = "paired"))%>%
select(-2,-3,-4,-5)# joining data files
tabS1 <- bind_rows(
white,
asian,
black,
hispanic,
other
)Here is Table S1 from the supplementary material:
Table S1
tabS1%>%
kable(digits = 2)| Race | t_1_2 | d_1_2 | t_2_3 | d_2_3 | t_3_4 | d_3_4 |
|---|---|---|---|---|---|---|
| White | 55.84 | 0.31 | 11.02 | 0.06 | 16.21 | 0.09 |
| Asian | 14.91 | 0.39 | 13.87 | 0.36 | 0.85 | 0.02 |
| Black | 27.19 | 0.45 | 4.82 | 0.08 | 4.06 | 0.07 |
| Hispanic | 6.60 | 0.10 | 11.10 | 0.17 | 6.87 | 0.10 |
| Other | 4.24 | 0.07 | 7.43 | 0.13 | 5.59 | 0.10 |
# t for Whites
reportObject <- reproCheck(reportedValue = "55.84", obtainedValue = white %>% pull(t_1_2), valueType = "t")## [1] "MATCH for t. The reported value (55.84) and the obtained value (55.84) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = "11.02", obtainedValue = white %>% pull(t_2_3), valueType = "t")## [1] "MATCH for t. The reported value (11.02) and the obtained value (11.02) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = "16.21", obtainedValue = white %>% pull(t_3_4), valueType = "t")## [1] "MATCH for t. The reported value (16.21) and the obtained value (16.21) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."# d for Whites
reportObject <- reproCheck(reportedValue = ".31", obtainedValue = white %>% pull(d_1_2), valueType = "d")## [1] "MATCH for d. The reported value (0.31) and the obtained value (0.31) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = ".06", obtainedValue = white %>% pull(d_2_3), valueType = "d")## [1] "MATCH for d. The reported value (0.06) and the obtained value (0.06) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = ".09", obtainedValue = white %>% pull(d_3_4), valueType = "d")## [1] "MATCH for d. The reported value (0.09) and the obtained value (0.09) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."# t for Asians
reportObject <- reproCheck(reportedValue = "14.91", obtainedValue = asian %>% pull(t_1_2), valueType = "t")## [1] "MATCH for t. The reported value (14.91) and the obtained value (14.91) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = "13.87", obtainedValue = asian %>% pull(t_2_3), valueType = "t")## [1] "MATCH for t. The reported value (13.87) and the obtained value (13.87) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = ".86", obtainedValue = asian %>% pull(t_3_4), valueType = "t")## [1] "MINOR_ERROR for t. The reported value (0.86) and the obtained value (0.85) differed by 1.16%. Note that the obtained value was rounded to 2 decimal places to match the reported value."# d for Asians
reportObject <- reproCheck(reportedValue = ".39", obtainedValue = asian %>% pull(d_1_2), valueType = "d")## [1] "MATCH for d. The reported value (0.39) and the obtained value (0.39) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = ".36", obtainedValue = asian %>% pull(d_2_3), valueType = "d")## [1] "MATCH for d. The reported value (0.36) and the obtained value (0.36) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = ".02", obtainedValue = asian %>% pull(d_3_4), valueType = "d")## [1] "MATCH for d. The reported value (0.02) and the obtained value (0.02) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."# t for Blacks
reportObject <- reproCheck(reportedValue = "27.20", obtainedValue = black %>% pull(t_1_2), valueType = "t")## [1] "MINOR_ERROR for t. The reported value (27.2) and the obtained value (27.19) differed by 0.04%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = "4.82", obtainedValue = black %>% pull(t_2_3), valueType = "t")## [1] "MATCH for t. The reported value (4.82) and the obtained value (4.82) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = "4.06", obtainedValue = black %>% pull(t_3_4), valueType = "t")## [1] "MATCH for t. The reported value (4.06) and the obtained value (4.06) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."# d for Blacks
reportObject <- reproCheck(reportedValue = ".45", obtainedValue = black %>% pull(d_1_2), valueType = "d")## [1] "MATCH for d. The reported value (0.45) and the obtained value (0.45) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = ".08", obtainedValue = black %>% pull(d_2_3), valueType = "d")## [1] "MATCH for d. The reported value (0.08) and the obtained value (0.08) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = ".07", obtainedValue = black %>% pull(d_3_4), valueType = "d")## [1] "MATCH for d. The reported value (0.07) and the obtained value (0.07) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."# t for Hispanics
reportObject <- reproCheck(reportedValue = "6.60", obtainedValue = hispanic %>% pull(t_1_2), valueType = "t")## [1] "MATCH for t. The reported value (6.6) and the obtained value (6.6) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = "11.10", obtainedValue = hispanic %>% pull(t_2_3), valueType = "t")## [1] "MATCH for t. The reported value (11.1) and the obtained value (11.1) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = "6.87", obtainedValue = hispanic %>% pull(t_3_4), valueType = "t")## [1] "MATCH for t. The reported value (6.87) and the obtained value (6.87) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."# d for Hispanics
reportObject <- reproCheck(reportedValue = ".10", obtainedValue = hispanic %>% pull(d_1_2), valueType = "d")## [1] "MATCH for d. The reported value (0.1) and the obtained value (0.1) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = ".17", obtainedValue = hispanic %>% pull(d_2_3), valueType = "d")## [1] "MATCH for d. The reported value (0.17) and the obtained value (0.17) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = ".10", obtainedValue = hispanic %>% pull(d_3_4), valueType = "d")## [1] "MATCH for d. The reported value (0.1) and the obtained value (0.1) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."# t for Others
reportObject <- reproCheck(reportedValue = "4.24", obtainedValue = other %>% pull(t_1_2), valueType = "t")## [1] "MATCH for t. The reported value (4.24) and the obtained value (4.24) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = "7.43", obtainedValue = other %>% pull(t_2_3), valueType = "t")## [1] "MATCH for t. The reported value (7.43) and the obtained value (7.43) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = "5.59", obtainedValue = other %>% pull(t_3_4), valueType = "t")## [1] "MATCH for t. The reported value (5.59) and the obtained value (5.59) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."# d for Others
reportObject <- reproCheck(reportedValue = ".08", obtainedValue = other %>% pull(d_1_2), valueType = "d", round = FALSE)## [1] "MINOR_ERROR for d. The reported value (0.08) and the obtained value (0.0739600733259669) differed by 7.55%. Note that the obtained value was not rounded."reportObject <- reproCheck(reportedValue = ".13", obtainedValue = other %>% pull(d_2_3), valueType = "d")## [1] "MATCH for d. The reported value (0.13) and the obtained value (0.13) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = ".10", obtainedValue = other %>% pull(d_3_4), valueType = "d")## [1] "MATCH for d. The reported value (0.1) and the obtained value (0.1) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."Step 5: Conclusion
The sample sizes reported in the paper differed from the ones obtained but given the large numbers, the deviations were only minor errors. Overall, all target outcomes could be successfully reproduced.
Author_Assistance = FALSE # was author assistance provided? (if so, enter TRUE)
Insufficient_Information_Errors <- 0 # how many discrete insufficient information issues did you encounter?
# Assess the causal locus (discrete reproducibility issues) of any reproducibility errors. Note that there doesn't necessarily have to be a one-to-one correspondance between discrete reproducibility issues and reproducibility errors. For example, it could be that the original article neglects to mention that a Greenhouse-Geisser correct was applied to ANOVA outcomes. This might result in multiple reproducibility errors, but there is a single causal locus (discrete reproducibility issue).
locus_typo <- NA # how many discrete issues did you encounter that related to typographical errors?
locus_specification <- NA # how many discrete issues did you encounter that related to incomplete, incorrect, or unclear specification of the original analyses?
locus_analysis <- NA # how many discrete issues did you encounter that related to errors in the authors' original analyses?
locus_data <- NA # how many discrete issues did you encounter that related to errors in the data files shared by the authors?
locus_unidentified <- NA # how many discrete issues were there for which you could not identify the cause
# How many of the above issues were resolved through author assistance?
locus_typo_resolved <- NA # how many discrete issues did you encounter that related to typographical errors?
locus_specification_resolved <- NA # how many discrete issues did you encounter that related to incomplete, incorrect, or unclear specification of the original analyses?
locus_analysis_resolved <- NA # how many discrete issues did you encounter that related to errors in the authors' original analyses?
locus_data_resolved <- NA # how many discrete issues did you encounter that related to errors in the data files shared by the authors?
locus_unidentified_resolved <- NA # how many discrete issues were there for which you could not identify the cause
Affects_Conclusion <- FALSE # Do any reproducibility issues encounter appear to affect the conclusions made in the original article? TRUE, FALSE, or NA. This is a subjective judgement, but you should taking into account multiple factors, such as the presence/absence of decision errors, the number of target outcomes that could not be reproduced, the type of outcomes that could or could not be reproduced, the difference in magnitude of effect sizes, and the predictions of the specific hypothesis under scrutiny.reportObject <- reportObject %>%
filter(dummyRow == FALSE) %>% # remove the dummy row
select(-dummyRow) %>% # remove dummy row designation
mutate(articleID = articleID) %>% # add variables to report
select(articleID, everything()) # make articleID first column
# decide on final outcome
if(any(!(reportObject$comparisonOutcome %in% c("MATCH", "MINOR_ERROR"))) | Insufficient_Information_Errors > 0){
finalOutcome <- "Failure without author assistance"
if(Author_Assistance == T){
finalOutcome <- "Failure despite author assistance"
}
}else{
finalOutcome <- "Success without author assistance"
if(Author_Assistance == T){
finalOutcome <- "Success with author assistance"
}
}
# collate report extra details
reportExtras <- data.frame(articleID, pilotNames, copilotNames, pilotTTC, copilotTTC, pilotStartDate, copilotStartDate, completionDate, Author_Assistance, finalOutcome, Insufficient_Information_Errors, locus_typo, locus_specification, locus_analysis, locus_data, locus_unidentified, locus_typo_resolved, locus_specification_resolved, locus_analysis_resolved, locus_data_resolved, locus_unidentified_resolved)
# save report objects
if(reportType == "pilot"){
write_csv(reportObject, "pilotReportDetailed.csv")
write_csv(reportExtras, "pilotReportExtras.csv")
}
if(reportType == "final"){
write_csv(reportObject, "finalReportDetailed.csv")
write_csv(reportExtras, "finalReportExtras.csv")
}Session information
devtools::session_info()## ─ Session info ───────────────────────────────────────────────────────────────
## setting value
## version R version 4.0.0 (2020-04-24)
## os macOS Catalina 10.15.4
## system x86_64, darwin17.0
## ui X11
## language (EN)
## collate en_US.UTF-8
## ctype en_US.UTF-8
## tz Europe/London
## date 2020-05-12
##
## ─ Packages ───────────────────────────────────────────────────────────────────
## package * version date lib
## assertthat 0.2.1 2019-03-21 [1]
## backports 1.1.6 2020-04-05 [1]
## broom 0.5.6 2020-04-20 [1]
## callr 3.4.3 2020-03-28 [1]
## cellranger 1.1.0 2016-07-27 [1]
## cli 2.0.2 2020-02-28 [1]
## colorspace 1.4-1 2019-03-18 [1]
## crayon 1.3.4 2017-09-16 [1]
## DBI 1.1.0 2019-12-15 [1]
## dbplyr 1.4.3 2020-04-19 [1]
## desc 1.2.0 2018-05-01 [1]
## devtools 2.3.0 2020-04-10 [1]
## digest 0.6.25 2020-02-23 [1]
## dplyr * 0.8.5 2020-03-07 [1]
## ellipsis 0.3.0 2019-09-20 [1]
## evaluate 0.14 2019-05-28 [1]
## fansi 0.4.1 2020-01-08 [1]
## forcats * 0.5.0 2020-03-01 [1]
## fs 1.4.1 2020-04-04 [1]
## generics 0.0.2 2018-11-29 [1]
## ggplot2 * 3.3.0 2020-03-05 [1]
## glue 1.4.0 2020-04-03 [1]
## gtable 0.3.0 2019-03-25 [1]
## haven * 2.2.0 2019-11-08 [1]
## highr 0.8 2019-03-20 [1]
## hms 0.5.3 2020-01-08 [1]
## htmltools 0.4.0 2019-10-04 [1]
## httr 1.4.1 2019-08-05 [1]
## jsonlite 1.6.1 2020-02-02 [1]
## knitr * 1.28 2020-02-06 [1]
## lattice 0.20-41 2020-04-02 [1]
## lifecycle 0.2.0 2020-03-06 [1]
## lsr * 0.5 2015-03-02 [1]
## lubridate 1.7.8 2020-04-06 [1]
## magrittr 1.5 2014-11-22 [1]
## memoise 1.1.0 2017-04-21 [1]
## modelr 0.1.7 2020-04-30 [1]
## munsell 0.5.0 2018-06-12 [1]
## nlme 3.1-147 2020-04-13 [1]
## pillar 1.4.4 2020-05-05 [1]
## pkgbuild 1.0.7 2020-04-25 [1]
## pkgconfig 2.0.3 2019-09-22 [1]
## pkgload 1.0.2 2018-10-29 [1]
## prettyunits 1.1.1 2020-01-24 [1]
## processx 3.4.2 2020-02-09 [1]
## ps 1.3.2 2020-02-13 [1]
## purrr * 0.3.4 2020-04-17 [1]
## R6 2.4.1 2019-11-12 [1]
## Rcpp 1.0.4.6 2020-04-09 [1]
## readr * 1.3.1 2018-12-21 [1]
## readxl * 1.3.1 2019-03-13 [1]
## remotes 2.1.1 2020-02-15 [1]
## reprex 0.3.0 2019-05-16 [1]
## ReproReports * 0.1 2020-05-06 [1]
## rlang 0.4.6 2020-05-02 [1]
## rmarkdown 2.1 2020-01-20 [1]
## rprojroot 1.3-2 2018-01-03 [1]
## rstudioapi 0.11 2020-02-07 [1]
## rvest 0.3.5 2019-11-08 [1]
## scales 1.1.0 2019-11-18 [1]
## sessioninfo 1.1.1 2018-11-05 [1]
## stringi 1.4.6 2020-02-17 [1]
## stringr * 1.4.0 2019-02-10 [1]
## testthat 2.3.2 2020-03-02 [1]
## tibble * 3.0.1 2020-04-20 [1]
## tidyr * 1.0.2 2020-01-24 [1]
## tidyselect 1.0.0 2020-01-27 [1]
## tidyverse * 1.3.0 2019-11-21 [1]
## usethis 1.6.1 2020-04-29 [1]
## utf8 1.1.4 2018-05-24 [1]
## vctrs 0.2.4 2020-03-10 [1]
## withr 2.2.0 2020-04-20 [1]
## xfun 0.13 2020-04-13 [1]
## xml2 1.3.2 2020-04-23 [1]
## yaml 2.2.1 2020-02-01 [1]
## source
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## Github (METRICS-CARPS/CARPSreports@3277f85)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
##
## [1] /Library/Frameworks/R.framework/Versions/4.0/Resources/library