CARPS Reproducibility Report
Report Details
articleID <- "2-10-2014_PS" # insert the article ID code here e.g., "10-3-2015_PS"
reportType <- "final" # specify whether this is the 'pilot' report or 'final' report
pilotNames <- "Manuel Bohn" # insert the pilot's name here e.g., "Tom Hardwicke". If there are multiple cpilots enter both names in a character string e.g., "Tom Hardwicke, Bob Dylan"
copilotNames <- "Tom Hardwicke" # # insert the co-pilot's name here e.g., "Michael Frank". If there are multiple co-pilots enter both names in a character string e.g., "Tom Hardwicke, Bob Dylan"
pilotTTC <- 180 # insert the pilot's estimated time to complete (in minutes, fine to approximate) e.g., 120
copilotTTC <- 30 # insert the co-pilot's estimated time to complete (in minutes, fine to approximate) e.g., 120
pilotStartDate <- as.Date("08/27/18", format = "%m/%d/%y") # insert the pilot's start date in US format e.g., as.Date("01/25/18", format = "%m/%d/%y")
copilotStartDate <- as.Date("09/06/18", format = "%m/%d/%y") # insert the co-pilot's start date in US format e.g., as.Date("01/25/18", format = "%m/%d/%y")
completionDate <- as.Date("09/06/18", format = "%m/%d/%y") # copilot insert the date of final report completion (after any necessary rounds of author assistance) in US format e.g., as.Date("01/25/18", format = "%m/%d/%y")Methods summary:
Participants were asked to note down several facts and experiences (e.g. social events, recent conversation, etc.). For each item they were asked how curious they think they will be to find out what they noted down if they were asked again in a couple of month time. Results show that participants underestimated their future curiosity, that is curiosity ratings were higher than predicted when they were asked again before actually revealing what they noted down.
Target outcomes:
Table 1 provides descriptive statistics for each measure for Study 1.
Participants’ Time 1 predictions of their curiosity (M = 3.99, SD = 1.32) were lower than their actual curiosity ratings at Time 2, immediately before reading their responses (M = 4.34, SD = 1.25), t(105) = 2.88, p = .005, d = 0.27. Participants also underestimated how interesting they would find their responses. Predictions of interest at Time 1 (M = 3.54, SD = 1.01) were lower than ratings of actual interest experienced at Time 2 (M = 3.82, SD = 0.89), t(105) = 3.10, p = .003, d = 0.29.
Step 1: Load packages and prepare report object
# load packages
library(tidyverse) # for data munging
library(knitr) # for kable table formating
library(haven) # import and export 'SPSS', 'Stata' and 'SAS' Files
library(readxl) # import excel files
library(ReproReports) # custom report functions
library(lsr) # calculate effect sizes# Prepare report object. This will be updated automatically by the reproCheck function each time values are compared
reportObject <- data.frame(dummyRow = TRUE, reportedValue = NA, obtainedValue = NA, valueType = NA, percentageError = NA, comparisonOutcome = NA, eyeballCheck = NA)Step 2: Load data
raw_data <- read_sav("data/Study1_Data.sav")Step 3: Tidy data and preprocessing
tidy_data <- raw_data %>%
mutate(Interesting_diff = T2_Interesting - T1_Interesting,
Meaningful_diff = T2_Meaningful - T1_Meaningful,
Surprised_diff = T2_Surprised - T1_Surprised)%>%
gather(measure, score, -Gender, -Year_born, - Order, - T2_Finished)%>%
filter(T2_Finished == 1)Step 4: Run analysis
Descriptive statistics
# Filtering relevant conditions and changing data structure to get values reported in table 1
t1_data <- tidy_data %>%
filter(measure == "T2_Interesting" | measure == "T1_Interesting" |measure == "T1_Curious" | measure == "T1_Surprised" | measure == "T1_Meaningful" | measure == "T1_Interest_Composite" | measure == "Interest_composite_diff" | measure == "Curious_diff" | measure == "T2_Curious" | measure == "T2_Surprised" | measure == "T2_Meaningful" | measure == "T2_Interest_Composite"| measure == "Interesting_diff" | measure == "Surprised_diff" | measure == "Meaningful_diff")%>%
mutate(time = ifelse(grepl("T1",measure),"Time 1", ifelse(grepl( "T2",measure), "Time 2", "Underestimate")),
measure = ifelse(time == "Underestimate",measure, substr(measure,4, length(measure))),
measure = ifelse(measure == "Curious_diff", "Curious", measure),
measure = ifelse(measure == "Surprised_diff", "Surprised", measure),
measure = ifelse(measure == "Interesting_diff", "Interesting", measure),
measure = ifelse(measure == "Meaningful_diff", "Meaningful", measure),
measure = ifelse(measure == "Interest_composite_diff", "Interest_Composite", measure))Below we check each descriptive value for study 1 shown in table 1. Here is the table.
Table 1
The manuscript does not specify the way that confidence intervals were computed. Given the large sample size, I assumed they used a normal distribution as the basis. Given that all the values match, this is probably what they did.
# Curiosity
## Time 1
### mean
reportObject <- reproCheck(reportedValue = "3.99", obtainedValue = mean(t1_data%>%filter(time == "Time 1", measure == "Curious")%>%pull(score)), valueType = "mean")## [1] "MATCH for mean. The reported value (3.99) and the obtained value (3.99) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."### sd
reportObject <- reproCheck(reportedValue = "1.32", obtainedValue = sd(t1_data%>%filter(time == "Time 1", measure == "Curious")%>%pull(score)), valueType = "sd")## [1] "MATCH for sd. The reported value (1.32) and the obtained value (1.32) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."### ci
#### lower
reportObject <- reproCheck(reportedValue = "3.74", obtainedValue =
mean(t1_data%>%filter(time == "Time 1", measure == "Curious")%>%pull(score)) -
qnorm(0.975)*sd(t1_data%>%filter(time == "Time 1", measure == "Curious")%>%pull(score)) /
sqrt(length(unique(t1_data$Order))),
valueType = "ci")## [1] "MATCH for ci. The reported value (3.74) and the obtained value (3.74) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."#### upper
reportObject <- reproCheck(reportedValue = "4.24", obtainedValue =
mean(t1_data%>%filter(time == "Time 1", measure == "Curious")%>%pull(score)) +
qnorm(0.975)*sd(t1_data%>%filter(time == "Time 1", measure == "Curious")%>%pull(score)) /
sqrt(length(unique(t1_data$Order))),
valueType = "ci")## [1] "MATCH for ci. The reported value (4.24) and the obtained value (4.24) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."## Time 2
### mean
reportObject <- reproCheck(reportedValue = "4.34", obtainedValue = mean(t1_data%>%filter(time == "Time 2", measure == "Curious")%>%pull(score)), valueType = "mean")## [1] "MATCH for mean. The reported value (4.34) and the obtained value (4.34) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."### sd
reportObject <- reproCheck(reportedValue = "1.32", obtainedValue = sd(t1_data%>%filter(time == "Time 2", measure == "Curious")%>%pull(score)), valueType = "sd")## [1] "MINOR_ERROR for sd. The reported value (1.32) and the obtained value (1.25) differed by 5.3%. Note that the obtained value was rounded to 2 decimal places to match the reported value."### ci
#### lower
reportObject <- reproCheck(reportedValue = "4.10", obtainedValue =
mean(t1_data%>%filter(time == "Time 2", measure == "Curious")%>%pull(score)) -
qnorm(0.975)*sd(t1_data%>%filter(time == "Time 2", measure == "Curious")%>%pull(score)) /
sqrt(length(unique(t1_data$Order))),
valueType = "ci")## [1] "MATCH for ci. The reported value (4.1) and the obtained value (4.1) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."#### upper
reportObject <- reproCheck(reportedValue = "4.58", obtainedValue =
mean(t1_data%>%filter(time == "Time 2", measure == "Curious")%>%pull(score)) +
qnorm(0.975)*sd(t1_data%>%filter(time == "Time 2", measure == "Curious")%>%pull(score)) /
sqrt(length(unique(t1_data$Order))),
valueType = "ci")## [1] "MATCH for ci. The reported value (4.58) and the obtained value (4.58) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."## Diff
### mean
reportObject <- reproCheck(reportedValue = "0.35", obtainedValue = mean(t1_data%>%filter(time == "Underestimate", measure == "Curious")%>%pull(score)), valueType = "mean")## [1] "MATCH for mean. The reported value (0.35) and the obtained value (0.35) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."### ci
#### lower
reportObject <- reproCheck(reportedValue = "0.11", obtainedValue =
mean(t1_data%>%filter(time == "Underestimate", measure == "Curious")%>%pull(score)) -
qnorm(0.975)*sd(t1_data%>%filter(time == "Underestimate", measure == "Curious")%>%pull(score)) /
sqrt(length(unique(t1_data$Order))),
valueType = "ci")## [1] "MATCH for ci. The reported value (0.11) and the obtained value (0.11) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."#### upper
reportObject <- reproCheck(reportedValue = "0.59", obtainedValue =
mean(t1_data%>%filter(time == "Underestimate", measure == "Curious")%>%pull(score)) +
qnorm(0.975)*sd(t1_data%>%filter(time == "Underestimate", measure == "Curious")%>%pull(score)) /
sqrt(length(unique(t1_data$Order))),
valueType = "ci")## [1] "MATCH for ci. The reported value (0.59) and the obtained value (0.59) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."# Interest Composite
## Time 1
### mean
reportObject <- reproCheck(reportedValue = "3.54", obtainedValue = mean(t1_data%>%filter(time == "Time 1", measure == "Interest_Composite")%>%pull(score)), valueType = "mean")## [1] "MATCH for mean. The reported value (3.54) and the obtained value (3.54) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."### sd
reportObject <- reproCheck(reportedValue = "1.01", obtainedValue = sd(t1_data%>%filter(time == "Time 1", measure == "Interest_Composite")%>%pull(score)), valueType = "sd")## [1] "MATCH for sd. The reported value (1.01) and the obtained value (1.01) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."### ci
#### lower
reportObject <- reproCheck(reportedValue = "3.34", obtainedValue =
mean(t1_data%>%filter(time == "Time 1", measure == "Interest_Composite")%>%pull(score)) -
qnorm(0.975)*sd(t1_data%>%filter(time == "Time 1", measure == "Interest_Composite")%>%pull(score)) /
sqrt(length(unique(t1_data$Order))),
valueType = "ci")## [1] "MATCH for ci. The reported value (3.34) and the obtained value (3.34) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."#### upper
reportObject <- reproCheck(reportedValue = "3.73", obtainedValue =
mean(t1_data%>%filter(time == "Time 1", measure == "Interest_Composite")%>%pull(score)) +
qnorm(0.975)*sd(t1_data%>%filter(time == "Time 1", measure == "Interest_Composite")%>%pull(score)) /
sqrt(length(unique(t1_data$Order))),
valueType = "ci")## [1] "MATCH for ci. The reported value (3.73) and the obtained value (3.73) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."## Time 2
### mean
reportObject <- reproCheck(reportedValue = "3.82", obtainedValue = mean(t1_data%>%filter(time == "Time 2", measure == "Interest_Composite")%>%pull(score)), valueType = "mean")## [1] "MATCH for mean. The reported value (3.82) and the obtained value (3.82) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."### sd
reportObject <- reproCheck(reportedValue = "0.89", obtainedValue = sd(t1_data%>%filter(time == "Time 2", measure == "Interest_Composite")%>%pull(score)), valueType = "sd")## [1] "MATCH for sd. The reported value (0.89) and the obtained value (0.89) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."### ci
#### lower
reportObject <- reproCheck(reportedValue = "3.65", obtainedValue =
mean(t1_data%>%filter(time == "Time 2", measure == "Interest_Composite")%>%pull(score)) -
qnorm(0.975)*sd(t1_data%>%filter(time == "Time 2", measure == "Interest_Composite")%>%pull(score)) /
sqrt(length(unique(t1_data$Order))),
valueType = "ci")## [1] "MATCH for ci. The reported value (3.65) and the obtained value (3.65) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."#### upper
reportObject <- reproCheck(reportedValue = "4.00", obtainedValue =
mean(t1_data%>%filter(time == "Time 2", measure == "Interest_Composite")%>%pull(score)) +
qnorm(0.975)*sd(t1_data%>%filter(time == "Time 2", measure == "Interest_Composite")%>%pull(score)) /
sqrt(length(unique(t1_data$Order))),
valueType = "ci")## [1] "MINOR_ERROR for ci. The reported value (4) and the obtained value (3.99) differed by 0.25%. Note that the obtained value was rounded to 2 decimal places to match the reported value."## Diff
### mean
reportObject <- reproCheck(reportedValue = "0.29", obtainedValue = mean(t1_data%>%filter(time == "Underestimate", measure == "Interest_Composite")%>%pull(score)), valueType = "mean")## [1] "MATCH for mean. The reported value (0.29) and the obtained value (0.29) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."### ci
#### lower
reportObject <- reproCheck(reportedValue = "0.10", obtainedValue =
mean(t1_data%>%filter(time == "Underestimate", measure == "Interest_Composite")%>%pull(score)) -
qnorm(0.975)*sd(t1_data%>%filter(time == "Underestimate", measure == "Interest_Composite")%>%pull(score)) /
sqrt(length(unique(t1_data$Order))),
valueType = "ci")## [1] "MATCH for ci. The reported value (0.1) and the obtained value (0.1) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."#### upper
reportObject <- reproCheck(reportedValue = "0.47", obtainedValue =
mean(t1_data%>%filter(time == "Underestimate", measure == "Interest_Composite")%>%pull(score)) +
qnorm(0.975)*sd(t1_data%>%filter(time == "Underestimate", measure == "Interest_Composite")%>%pull(score)) /
sqrt(length(unique(t1_data$Order))),
valueType = "ci")## [1] "MATCH for ci. The reported value (0.47) and the obtained value (0.47) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."# Surprise
## Time 1
### mean
reportObject <- reproCheck(reportedValue = "2.84", obtainedValue = mean(t1_data%>%filter(time == "Time 1", measure == "Surprised")%>%pull(score)), valueType = "mean")## [1] "MATCH for mean. The reported value (2.84) and the obtained value (2.84) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."### ci
#### lower
reportObject <- reproCheck(reportedValue = "2.64", obtainedValue =
mean(t1_data%>%filter(time == "Time 1", measure == "Surprised")%>%pull(score)) -
qnorm(0.975)*sd(t1_data%>%filter(time == "Time 1", measure == "Surprised")%>%pull(score)) /
sqrt(length(unique(t1_data$Order))),
valueType = "ci")## [1] "MATCH for ci. The reported value (2.64) and the obtained value (2.64) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."#### upper
reportObject <- reproCheck(reportedValue = "3.05", obtainedValue =
mean(t1_data%>%filter(time == "Time 1", measure == "Surprised")%>%pull(score)) +
qnorm(0.975)*sd(t1_data%>%filter(time == "Time 1", measure == "Surprised")%>%pull(score)) /
sqrt(length(unique(t1_data$Order))),
valueType = "ci")## [1] "MATCH for ci. The reported value (3.05) and the obtained value (3.05) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."## Time 2
### mean
reportObject <- reproCheck(reportedValue = "3.25", obtainedValue = mean(t1_data%>%filter(time == "Time 2", measure == "Surprised")%>%pull(score)), valueType = "mean")## [1] "MATCH for mean. The reported value (3.25) and the obtained value (3.25) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."### ci
#### lower
reportObject <- reproCheck(reportedValue = "3.06", obtainedValue =
mean(t1_data%>%filter(time == "Time 2", measure == "Surprised")%>%pull(score)) -
qnorm(0.975)*sd(t1_data%>%filter(time == "Time 2", measure == "Surprised")%>%pull(score)) /
sqrt(length(unique(t1_data$Order))),
valueType = "ci")## [1] "MATCH for ci. The reported value (3.06) and the obtained value (3.06) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."#### upper
reportObject <- reproCheck(reportedValue = "3.44", obtainedValue =
mean(t1_data%>%filter(time == "Time 2", measure == "Surprised")%>%pull(score)) +
qnorm(0.975)*sd(t1_data%>%filter(time == "Time 2", measure == "Surprised")%>%pull(score)) /
sqrt(length(unique(t1_data$Order))),
valueType = "ci")## [1] "MATCH for ci. The reported value (3.44) and the obtained value (3.44) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."## Diff
### mean
reportObject <- reproCheck(reportedValue = "0.40", obtainedValue = mean(t1_data%>%filter(time == "Underestimate", measure == "Surprised")%>%pull(score)), valueType = "mean")## [1] "MATCH for mean. The reported value (0.4) and the obtained value (0.4) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."### ci
#### lower
reportObject <- reproCheck(reportedValue = "0.19", obtainedValue =
mean(t1_data%>%filter(time == "Underestimate", measure == "Surprised")%>%pull(score)) -
qnorm(0.975)*sd(t1_data%>%filter(time == "Underestimate", measure == "Surprised")%>%pull(score)) /
sqrt(length(unique(t1_data$Order))),
valueType = "ci")## [1] "MATCH for ci. The reported value (0.19) and the obtained value (0.19) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."#### upper
reportObject <- reproCheck(reportedValue = "0.62", obtainedValue =
mean(t1_data%>%filter(time == "Underestimate", measure == "Surprised")%>%pull(score)) +
qnorm(0.975)*sd(t1_data%>%filter(time == "Underestimate", measure == "Surprised")%>%pull(score)) /
sqrt(length(unique(t1_data$Order))),
valueType = "ci")## [1] "MATCH for ci. The reported value (0.62) and the obtained value (0.62) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."# Meaningfulness
## Time 1
### mean
reportObject <- reproCheck(reportedValue = "3.81", obtainedValue = mean(t1_data%>%filter(time == "Time 1", measure == "Meaningful")%>%pull(score)), valueType = "mean")## [1] "MATCH for mean. The reported value (3.81) and the obtained value (3.81) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."### ci
#### lower
reportObject <- reproCheck(reportedValue = "3.60", obtainedValue =
mean(t1_data%>%filter(time == "Time 1", measure == "Meaningful")%>%pull(score)) -
qnorm(0.975)*sd(t1_data%>%filter(time == "Time 1", measure == "Meaningful")%>%pull(score)) /
sqrt(length(unique(t1_data$Order))),
valueType = "ci")## [1] "MINOR_ERROR for ci. The reported value (3.6) and the obtained value (3.61) differed by 0.28%. Note that the obtained value was rounded to 2 decimal places to match the reported value."#### upper
reportObject <- reproCheck(reportedValue = "4.03", obtainedValue =
mean(t1_data%>%filter(time == "Time 1", measure == "Meaningful")%>%pull(score)) +
qnorm(0.975)*sd(t1_data%>%filter(time == "Time 1", measure == "Meaningful")%>%pull(score)) /
sqrt(length(unique(t1_data$Order))),
valueType = "ci")## [1] "MINOR_ERROR for ci. The reported value (4.03) and the obtained value (4.02) differed by 0.25%. Note that the obtained value was rounded to 2 decimal places to match the reported value."## Time 2
### mean
reportObject <- reproCheck(reportedValue = "4.04", obtainedValue = mean(t1_data%>%filter(time == "Time 2", measure == "Meaningful")%>%pull(score)), valueType = "mean")## [1] "MATCH for mean. The reported value (4.04) and the obtained value (4.04) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."### ci
#### lower
reportObject <- reproCheck(reportedValue = "3.84", obtainedValue =
mean(t1_data%>%filter(time == "Time 2", measure == "Meaningful")%>%pull(score)) -
qnorm(0.975)*sd(t1_data%>%filter(time == "Time 2", measure == "Meaningful")%>%pull(score)) /
sqrt(length(unique(t1_data$Order))),
valueType = "ci")## [1] "MINOR_ERROR for ci. The reported value (3.84) and the obtained value (3.85) differed by 0.26%. Note that the obtained value was rounded to 2 decimal places to match the reported value."#### upper
reportObject <- reproCheck(reportedValue = "4.23", obtainedValue =
mean(t1_data%>%filter(time == "Time 2", measure == "Meaningful")%>%pull(score)) +
qnorm(0.975)*sd(t1_data%>%filter(time == "Time 2", measure == "Meaningful")%>%pull(score)) /
sqrt(length(unique(t1_data$Order))),
valueType = "ci")## [1] "MATCH for ci. The reported value (4.23) and the obtained value (4.23) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."## Diff
### mean
reportObject <- reproCheck(reportedValue = "0.22", obtainedValue = mean(t1_data%>%filter(time == "Underestimate", measure == "Meaningful")%>%pull(score)), valueType = "mean")## [1] "MATCH for mean. The reported value (0.22) and the obtained value (0.22) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."### ci
#### lower
reportObject <- reproCheck(reportedValue = "0.03", obtainedValue =
mean(t1_data%>%filter(time == "Underestimate", measure == "Meaningful")%>%pull(score)) -
qnorm(0.975)*sd(t1_data%>%filter(time == "Underestimate", measure == "Meaningful")%>%pull(score)) /
sqrt(length(unique(t1_data$Order))),
valueType = "ci")## [1] "MATCH for ci. The reported value (0.03) and the obtained value (0.03) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."#### upper
reportObject <- reproCheck(reportedValue = "0.42", obtainedValue =
mean(t1_data%>%filter(time == "Underestimate", measure == "Meaningful")%>%pull(score)) +
qnorm(0.975)*sd(t1_data%>%filter(time == "Underestimate", measure == "Meaningful")%>%pull(score)) /
sqrt(length(unique(t1_data$Order))),
valueType = "ci")## [1] "MINOR_ERROR for ci. The reported value (0.42) and the obtained value (0.41) differed by 2.38%. Note that the obtained value was rounded to 2 decimal places to match the reported value."# Interest
## Time 1
### mean
reportObject <- reproCheck(reportedValue = "3.95", obtainedValue = mean(t1_data%>%filter(time == "Time 1", measure == "Interesting")%>%pull(score)), valueType = "mean")## [1] "MATCH for mean. The reported value (3.95) and the obtained value (3.95) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."### ci
#### lower
reportObject <- reproCheck(reportedValue = "3.73", obtainedValue =
mean(t1_data%>%filter(time == "Time 1", measure == "Interesting")%>%pull(score)) -
qnorm(0.975)*sd(t1_data%>%filter(time == "Time 1", measure == "Interesting")%>%pull(score)) /
sqrt(length(unique(t1_data$Order))),
valueType = "ci")## [1] "MATCH for ci. The reported value (3.73) and the obtained value (3.73) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."#### upper
reportObject <- reproCheck(reportedValue = "4.18", obtainedValue =
mean(t1_data%>%filter(time == "Time 1", measure == "Interesting")%>%pull(score)) +
qnorm(0.975)*sd(t1_data%>%filter(time == "Time 1", measure == "Interesting")%>%pull(score)) /
sqrt(length(unique(t1_data$Order))),
valueType = "ci")## [1] "MINOR_ERROR for ci. The reported value (4.18) and the obtained value (4.17) differed by 0.24%. Note that the obtained value was rounded to 2 decimal places to match the reported value."## Time 2
### mean
reportObject <- reproCheck(reportedValue = "4.19", obtainedValue = mean(t1_data%>%filter(time == "Time 2", measure == "Interesting")%>%pull(score)), valueType = "mean")## [1] "MATCH for mean. The reported value (4.19) and the obtained value (4.19) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."### ci
#### lower
reportObject <- reproCheck(reportedValue = "4.00", obtainedValue =
mean(t1_data%>%filter(time == "Time 2", measure == "Interesting")%>%pull(score)) -
qnorm(0.975)*sd(t1_data%>%filter(time == "Time 2", measure == "Interesting")%>%pull(score)) /
sqrt(length(unique(t1_data$Order))),
valueType = "ci")## [1] "MATCH for ci. The reported value (4) and the obtained value (4) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."#### upper
reportObject <- reproCheck(reportedValue = "4.38", obtainedValue =
mean(t1_data%>%filter(time == "Time 2", measure == "Interesting")%>%pull(score)) +
qnorm(0.975)*sd(t1_data%>%filter(time == "Time 2", measure == "Interesting")%>%pull(score)) /
sqrt(length(unique(t1_data$Order))),
valueType = "ci")## [1] "MATCH for ci. The reported value (4.38) and the obtained value (4.38) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."## Diff
### mean
reportObject <- reproCheck(reportedValue = "0.23", obtainedValue = mean(t1_data%>%filter(time == "Underestimate", measure == "Interesting")%>%pull(score)), valueType = "mean")## [1] "MATCH for mean. The reported value (0.23) and the obtained value (0.23) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."### ci
#### lower
reportObject <- reproCheck(reportedValue = "0.02", obtainedValue =
mean(t1_data%>%filter(time == "Underestimate", measure == "Interesting")%>%pull(score)) -
qnorm(0.975)*sd(t1_data%>%filter(time == "Underestimate", measure == "Interesting")%>%pull(score)) /
sqrt(length(unique(t1_data$Order))),
valueType = "ci")## [1] "MATCH for ci. The reported value (0.02) and the obtained value (0.02) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."#### upper
reportObject <- reproCheck(reportedValue = "0.45", obtainedValue =
mean(t1_data%>%filter(time == "Underestimate", measure == "Interesting")%>%pull(score)) +
qnorm(0.975)*sd(t1_data%>%filter(time == "Underestimate", measure == "Interesting")%>%pull(score)) /
sqrt(length(unique(t1_data$Order))),
valueType = "ci")## [1] "MATCH for ci. The reported value (0.45) and the obtained value (0.45) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."Inferential statistics
Across the nine prompts, participants’ ratings of their curiosity and interest were highly intercorrelated (αcuriosity = .93, αinterest = .90). We therefore present results collapsed across the prompts. Participants’ Time 1 predictions of their curiosity (M = 3.99, SD = 1.32) were lower than their actual curiosity ratings at Time 2, immediately before read- ing their responses (M = 4.34, SD = 1.25), t(105) = 2.88, p = .005, d = 0.27. Participants also underestimated how interesting they would find their responses. Predictions of interest at Time 1 (M = 3.54, SD = 1.01) were lower than ratings of actual interest experienced at Time 2 (M = 3.82, SD = 0.89), t(105) = 3.10, p = .003, d = 0.29.
# Curiosity
t_cur <- t.test(t1_data%>%filter(time == "Time 2", measure == "Curious")%>%pull(score),t1_data%>%filter(time == "Time 1", measure == "Curious")%>%pull(score), paired = T)
d_cur <-cohensD(t1_data%>%filter(time == "Time 2", measure == "Curious")%>%pull(score),t1_data%>%filter(time == "Time 1", measure == "Curious")%>%pull(score))
# Interest composite score
t_int_comp <- t.test(t1_data%>%filter(time == "Time 2", measure == "Interest_Composite")%>%pull(score),t1_data%>%filter(time == "Time 1", measure == "Interest_Composite")%>%pull(score), paired = T)
d_int_comp <-cohensD(t1_data%>%filter(time == "Time 2", measure == "Interest_Composite")%>%pull(score),t1_data%>%filter(time == "Time 1", measure == "Interest_Composite")%>%pull(score))
## P-values reported in table 1
### Interest Surprised
t_surp <- t.test(t1_data%>%filter(time == "Time 2", measure == "Surprised")%>%pull(score),t1_data%>%filter(time == "Time 1", measure == "Surprised")%>%pull(score), paired = T)
### Interest Meaningfulness
t_mean <- t.test(t1_data%>%filter(time == "Time 2", measure == "Meaningful")%>%pull(score),t1_data%>%filter(time == "Time 1", measure == "Meaningful")%>%pull(score), paired = T)
### Interest Interesting
t_int <- t.test(t1_data%>%filter(time == "Time 2", measure == "Interesting")%>%pull(score),t1_data%>%filter(time == "Time 1", measure == "Interesting")%>%pull(score), paired = T)# Curiosity
## df
reportObject <- reproCheck(reportedValue = "105", obtainedValue = t_cur$parameter, valueType = "df")## [1] "MATCH for df. The reported value (105) and the obtained value (105) differed by 0%. Note that the obtained value was rounded to 0 decimal places to match the reported value."## t
reportObject <- reproCheck(reportedValue = "2.88", obtainedValue = t_cur$statistic, valueType = "t")## [1] "MATCH for t. The reported value (2.88) and the obtained value (2.88) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."## p
reportObject <- reproCheck(reportedValue = ".005", obtainedValue = t_cur$p.value, valueType = "p")## [1] "MATCH for p. The reported value (0.005) and the obtained value (0.005) differed by 0%. Note that the obtained value was rounded to 3 decimal places to match the reported value."## d
reportObject <- reproCheck(reportedValue = "0.27", obtainedValue = d_cur, valueType = "d")## [1] "MATCH for d. The reported value (0.27) and the obtained value (0.27) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."# Interest
## df
reportObject <- reproCheck(reportedValue = "105", obtainedValue = t_int_comp$parameter, valueType = "df")## [1] "MATCH for df. The reported value (105) and the obtained value (105) differed by 0%. Note that the obtained value was rounded to 0 decimal places to match the reported value."## t
reportObject <- reproCheck(reportedValue = "3.10", obtainedValue = t_int_comp$statistic, valueType = "t")## [1] "MATCH for t. The reported value (3.1) and the obtained value (3.1) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."## p
reportObject <- reproCheck(reportedValue = ".003", obtainedValue = t_int_comp$p.value, valueType = "p")## [1] "MATCH for p. The reported value (0.003) and the obtained value (0.003) differed by 0%. Note that the obtained value was rounded to 3 decimal places to match the reported value."## d
reportObject <- reproCheck(reportedValue = "0.29", obtainedValue = d_int_comp, valueType = "d")## [1] "MINOR_ERROR for d. The reported value (0.29) and the obtained value (0.3) differed by 3.45%. Note that the obtained value was rounded to 2 decimal places to match the reported value."# Surprise
## p
reportObject <- reproCheck(reportedValue = "<.001", obtainedValue = t_surp$p.value, valueType = "p",eyeballCheck = TRUE)## [1] "MATCH for p. Eyeball comparison only."# Interest Surprised
## p
reportObject <- reproCheck(reportedValue = "<.001", obtainedValue = t_surp$p.value, valueType = "p",eyeballCheck = TRUE)## [1] "MATCH for p. Eyeball comparison only."# Interest Meaningfulness
## p
reportObject <- reproCheck(reportedValue = ".02", obtainedValue = t_mean$p.value, valueType = "p")## [1] "MATCH for p. The reported value (0.02) and the obtained value (0.02) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."# Interest Interesting
## p
reportObject <- reproCheck(reportedValue = ".03", obtainedValue = t_int$p.value, valueType = "p")## [1] "MATCH for p. The reported value (0.03) and the obtained value (0.03) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."Step 5: Conclusion
The manuscript does not specify the way that confidence intervals were computed and it was necessary to make an educated guess. All target outcomes were reproduced successfully.
Author_Assistance = FALSE # was author assistance provided? (if so, enter TRUE)
Insufficient_Information_Errors <- 0 # how many discrete insufficient information issues did you encounter?
# Assess the causal locus (discrete reproducibility issues) of any reproducibility errors. Note that there doesn't necessarily have to be a one-to-one correspondance between discrete reproducibility issues and reproducibility errors. For example, it could be that the original article neglects to mention that a Greenhouse-Geisser correct was applied to ANOVA outcomes. This might result in multiple reproducibility errors, but there is a single causal locus (discrete reproducibility issue).
locus_typo <- NA # how many discrete issues did you encounter that related to typographical errors?
locus_specification <- NA # how many discrete issues did you encounter that related to incomplete, incorrect, or unclear specification of the original analyses?
locus_analysis <- NA # how many discrete issues did you encounter that related to errors in the authors' original analyses?
locus_data <- NA # how many discrete issues did you encounter that related to errors in the data files shared by the authors?
locus_unidentified <- NA # how many discrete issues were there for which you could not identify the cause
# How many of the above issues were resolved through author assistance?
locus_typo_resolved <- NA # how many discrete issues did you encounter that related to typographical errors?
locus_specification_resolved <- NA # how many discrete issues did you encounter that related to incomplete, incorrect, or unclear specification of the original analyses?
locus_analysis_resolved <- NA # how many discrete issues did you encounter that related to errors in the authors' original analyses?
locus_data_resolved <- NA # how many discrete issues did you encounter that related to errors in the data files shared by the authors?
locus_unidentified_resolved <- NA # how many discrete issues were there for which you could not identify the cause
Affects_Conclusion <- NA # Do any reproducibility issues encounter appear to affect the conclusions made in the original article? TRUE, FALSE, or NA. This is a subjective judgement, but you should taking into account multiple factors, such as the presence/absence of decision errors, the number of target outcomes that could not be reproduced, the type of outcomes that could or could not be reproduced, the difference in magnitude of effect sizes, and the predictions of the specific hypothesis under scrutiny.reportObject <- reportObject %>%
filter(dummyRow == FALSE) %>% # remove the dummy row
select(-dummyRow) %>% # remove dummy row designation
mutate(articleID = articleID) %>% # add variables to report
select(articleID, everything()) # make articleID first column
# decide on final outcome
if(any(!(reportObject$comparisonOutcome %in% c("MATCH", "MINOR_ERROR"))) | Insufficient_Information_Errors > 0){
finalOutcome <- "Failure without author assistance"
if(Author_Assistance == T){
finalOutcome <- "Failure despite author assistance"
}
}else{
finalOutcome <- "Success without author assistance"
if(Author_Assistance == T){
finalOutcome <- "Success with author assistance"
}
}
# collate report extra details
reportExtras <- data.frame(articleID, pilotNames, copilotNames, pilotTTC, copilotTTC, pilotStartDate, copilotStartDate, completionDate, Author_Assistance, finalOutcome, Insufficient_Information_Errors, locus_typo, locus_specification, locus_analysis, locus_data, locus_unidentified, locus_typo_resolved, locus_specification_resolved, locus_analysis_resolved, locus_data_resolved, locus_unidentified_resolved)
# save report objects
if(reportType == "pilot"){
write_csv(reportObject, "pilotReportDetailed.csv")
write_csv(reportExtras, "pilotReportExtras.csv")
}
if(reportType == "final"){
write_csv(reportObject, "finalReportDetailed.csv")
write_csv(reportExtras, "finalReportExtras.csv")
}Session information
devtools::session_info()## ─ Session info ───────────────────────────────────────────────────────────────
## setting value
## version R version 4.0.0 (2020-04-24)
## os macOS Catalina 10.15.4
## system x86_64, darwin17.0
## ui X11
## language (EN)
## collate en_US.UTF-8
## ctype en_US.UTF-8
## tz Europe/London
## date 2020-05-18
##
## ─ Packages ───────────────────────────────────────────────────────────────────
## package * version date lib
## assertthat 0.2.1 2019-03-21 [1]
## backports 1.1.6 2020-04-05 [1]
## broom 0.5.6 2020-04-20 [1]
## callr 3.4.3 2020-03-28 [1]
## cellranger 1.1.0 2016-07-27 [1]
## cli 2.0.2 2020-02-28 [1]
## colorspace 1.4-1 2019-03-18 [1]
## crayon 1.3.4 2017-09-16 [1]
## DBI 1.1.0 2019-12-15 [1]
## dbplyr 1.4.3 2020-04-19 [1]
## desc 1.2.0 2018-05-01 [1]
## devtools 2.3.0 2020-04-10 [1]
## digest 0.6.25 2020-02-23 [1]
## dplyr * 0.8.5 2020-03-07 [1]
## ellipsis 0.3.0 2019-09-20 [1]
## evaluate 0.14 2019-05-28 [1]
## fansi 0.4.1 2020-01-08 [1]
## forcats * 0.5.0 2020-03-01 [1]
## fs 1.4.1 2020-04-04 [1]
## generics 0.0.2 2018-11-29 [1]
## ggplot2 * 3.3.0 2020-03-05 [1]
## glue 1.4.0 2020-04-03 [1]
## gtable 0.3.0 2019-03-25 [1]
## haven * 2.2.0 2019-11-08 [1]
## hms 0.5.3 2020-01-08 [1]
## htmltools 0.4.0 2019-10-04 [1]
## httr 1.4.1 2019-08-05 [1]
## jsonlite 1.6.1 2020-02-02 [1]
## knitr * 1.28 2020-02-06 [1]
## lattice 0.20-41 2020-04-02 [1]
## lifecycle 0.2.0 2020-03-06 [1]
## lsr * 0.5 2015-03-02 [1]
## lubridate 1.7.8 2020-04-06 [1]
## magrittr 1.5 2014-11-22 [1]
## memoise 1.1.0 2017-04-21 [1]
## modelr 0.1.7 2020-04-30 [1]
## munsell 0.5.0 2018-06-12 [1]
## nlme 3.1-147 2020-04-13 [1]
## pillar 1.4.4 2020-05-05 [1]
## pkgbuild 1.0.7 2020-04-25 [1]
## pkgconfig 2.0.3 2019-09-22 [1]
## pkgload 1.0.2 2018-10-29 [1]
## prettyunits 1.1.1 2020-01-24 [1]
## processx 3.4.2 2020-02-09 [1]
## ps 1.3.2 2020-02-13 [1]
## purrr * 0.3.4 2020-04-17 [1]
## R6 2.4.1 2019-11-12 [1]
## Rcpp 1.0.4.6 2020-04-09 [1]
## readr * 1.3.1 2018-12-21 [1]
## readxl * 1.3.1 2019-03-13 [1]
## remotes 2.1.1 2020-02-15 [1]
## reprex 0.3.0 2019-05-16 [1]
## ReproReports * 0.1 2020-05-06 [1]
## rlang 0.4.6 2020-05-02 [1]
## rmarkdown 2.1 2020-01-20 [1]
## rprojroot 1.3-2 2018-01-03 [1]
## rstudioapi 0.11 2020-02-07 [1]
## rvest 0.3.5 2019-11-08 [1]
## scales 1.1.0 2019-11-18 [1]
## sessioninfo 1.1.1 2018-11-05 [1]
## stringi 1.4.6 2020-02-17 [1]
## stringr * 1.4.0 2019-02-10 [1]
## testthat 2.3.2 2020-03-02 [1]
## tibble * 3.0.1 2020-04-20 [1]
## tidyr * 1.0.2 2020-01-24 [1]
## tidyselect 1.0.0 2020-01-27 [1]
## tidyverse * 1.3.0 2019-11-21 [1]
## usethis 1.6.1 2020-04-29 [1]
## vctrs 0.2.4 2020-03-10 [1]
## withr 2.2.0 2020-04-20 [1]
## xfun 0.13 2020-04-13 [1]
## xml2 1.3.2 2020-04-23 [1]
## yaml 2.2.1 2020-02-01 [1]
## source
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## Github (METRICS-CARPS/CARPSreports@3277f85)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
## CRAN (R 4.0.0)
##
## [1] /Library/Frameworks/R.framework/Versions/4.0/Resources/library