get— title: “Reproducibility Report” output: html_document: toc: true toc_float: true —
articleID <- "2-10-2014" # insert the article ID code here e.g., "10-3-2015"
reportType <- 'pilot' # specify whether this is the 'pilot' report or 'copilot' report
pilotNames <- "Elizabeth Mortenson" # insert the pilot's name here e.g., "Tom Hardwicke".
copilotNames <- "Nicky" # # insert the co-pilot's name here e.g., "Michael Frank".
pilotTTC <- 650 # insert the pilot's estimated time to complete (in minutes, it is fine to approximate) e.g., 120
copilotTTC <- NA # insert the co-pilot's estimated time to complete (in minutes, it is fine to approximate) e.g., 120
pilotStartDate <- as.Date("11/03/2019") # insert the piloting start date in US format e.g., as.Date("01/25/18", format = "%m/%d/%y")
copilotStartDate <- as.Date("11/10/2019") # insert the co-piloting start date in US format e.g., as.Date("01/25/18", format = "%m/%d/%y")
completionDate <- as.Date("11/10/2019") # insert the date of final report completion in US format e.g., as.Date("01/25/18", format = "%m/%d/%y")Study 1 investigates how accurately participants estimate their future feelings about previous, routine events. At Time 1 (beginning of summer), individuals rated how curious and interested they thought they would be to revisit nine events (e.g., an inside joke, a recent status they had posted on their Facebook profile). Survey responses were used to create a “time-capsule.” Three months later, at Time 2 and prior to reviewing their time-capsules, people rated their curiousity before being presented with their answers from Time 1. Next, participants reported their real-time feelings, including levels of surprise, meaningfulness, and interest for each experience (1 = not at all, 7 = extremely). Finally, predicted feelings at Time 1 were compared with those experienced at Time 2. Answers to three constructs–surprise, meaningfulness and interest–were averaged to create a composite interest score.
For this article you should focus on the findings reported in the results section for Study 1 (and Table 1).
Specifically, you should attempt to reproduce all descriptive and inferential analyses reported in the text below and associated tables/figures:
Table 1 provides descriptive statistics for each measure for Study 1.
Participants’ Time 1 predictions of their curiosity (M = 3.99, SD = 1.32) were lower than their actual curiosity ratings at Time 2, immediately before reading their responses (M = 4.34, SD = 1.25), t(105) = 2.88, p = .005, d = 0.27. Participants also underestimated how interesting they would find their responses. Predictions of interest at Time 1 (M = 3.54, SD = 1.01) were lower than ratings of actual interest experienced at Time 2 (M = 3.82, SD = 0.89), t(105) = 3.10, p = .003, d = 0.29.
# load packages
library(tidyverse) # for data munging
library(knitr) # for kable table formating
library(haven) # import and export 'SPSS', 'Stata' and 'SAS' Files
library(readxl) # import excel files
library(ReproReports) # custom reporting functions
library(psych) # for calculating alphas
library(lsr) # calculating cohen's d
library(rcompanion) #calculating confidence intervals# Prepare report object. This will be updated automatically by the reproCheck function each time values are compared
reportObject <- data.frame(dummyRow = TRUE, reportedValue = NA, obtainedValue = NA, valueType = NA, percentageError = NA, comparisonOutcome = NA, eyeballCheck = NA)data <- read_sav("data/Study1_Data.sav")#N
data <- data %>%
filter(!is.na(Order)) # remove NA casesHow many participants in the dataset? From the Participants section:
One hundred thirty-five undergraduates (65.9% female, 33.3% male, 0.7% unreported; mean age = 20.4 years, SD = 1.0) in the northeastern United States completed the first part of this online study in exchange for $5, knowing that they would be contacted later for a follow-up.
Three months later, 106 of these students (78.5% response rate; 67.0% female, 32.1% male, 0.9% unreported; mean age = 20.4 years, SD = 1.0) completed a follow-up survey in exchange for an additional $20."
Caption from Table 1 (Study 1)
Across the nine prompts, participants’ ratings of their curiosity and interest were highly intercorrelated (αcuriosity = .93, αinterest = .90). We therefore present results collapsed across the prompts.
The values in square brackets are 95% confidence intervals. The table presents results for the composite measure of interest as well as for the specific scales. For the measure of memory, participants indicated how much of their written summary of their conversation they expected to remember at Time 2 or they had remembered at Time 2.
This column presents p values from t tests comparing predictions with actual experience.
#Check alphas before collapsing measures
a.curiosity <- select(data, 1, 5, 9, 13, 17, 21, 25, 29, 33)
a.interest <- select(data, 2, 6, 10, 14, 18, 22, 26, 30, 34)
# Reported curiosity alpha = .93
alpha.c <- alpha(a.curiosity)
alpha.c$total$std.alpha #Note: variable is the standard alpha (.89) but upper value from my analysis was higher (0.93); not sure how to pull or if needed## [1] 0.8987716# Reported interest alpha = .90
alpha.a <- alpha(a.interest)
alpha.a$total$std.alpha #Note: variable is is the standard alpha (.86) but upper value from my analysis was higher (0.90); not sure how to pull or if needed## [1] 0.8670095#Running value comparisons on alphas
reportObject <- reproCheck(reportedValue = '.93', obtainedValue = alpha.c$total$std.alpha, valueType = 'coeff') ## [1] "MINOR_ERROR for coeff. The reported value (0.93) and the obtained value (0.9) differed by 3.23%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = '.90', obtainedValue = alpha.a$total$std.alpha, valueType = 'coeff')## [1] "MINOR_ERROR for coeff. The reported value (0.9) and the obtained value (0.87) differed by 3.33%. Note that the obtained value was rounded to 2 decimal places to match the reported value."#T1 Means/Standard deviations (for Curiosity & Interest Composite only)
#Curiosity T1 mean = 3.99 [3.74, 4.24], SD = 1.32
T1.curious <- groupwiseMean(T1_Curious ~ 1,
data = data,
conf = 0.95,
digits = 3)
T1sd.curious <- sd(data$T1_Curious, na.rm = TRUE) # getting SD of T1 curious
#Interest Composite T1 mean = 3.54 [3.34, 3.73], SD = 1.01
T1.interestcomp <- groupwiseMean(T1_Interest_Composite ~ 1,
data = data,
conf = 0.95,
digits = 3)
T1sd.interestcomp <- sd(data$T1_Interest_Composite, na.rm = TRUE) # getting SD of T1 interest comp
#Surprise T1 mean = 2.84 [2.64, 3.05]
T1.surprise <- groupwiseMean(T1_Surprised ~ 1,
data = data,
conf = 0.95,
digits = 3)
#Meaningfulness T1 mean = 3.81 [3.60, 4.03]
T1.meaningful <- groupwiseMean(T1_Meaningful ~ 1,
data = data,
conf = 0.95,
digits = 3)
#Interesting T1 mean = 3.95 [3.73, 4.18]
T1.interest <- groupwiseMean(T1_Interesting ~ 1,
data = data,
conf = 0.95,
digits = 3)
#Running value comparisons on T1 means
reportObject <- reproCheck(reportedValue = '3.99', obtainedValue = T1.curious$Mean, valueType = 'mean')## [1] "MATCH for mean. The reported value (3.99) and the obtained value (3.99) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = '3.54', obtainedValue = T1.interestcomp$Mean, valueType = 'mean')## [1] "MATCH for mean. The reported value (3.54) and the obtained value (3.54) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = '2.84', obtainedValue = T1.surprise$Mean, valueType = 'mean')## [1] "MATCH for mean. The reported value (2.84) and the obtained value (2.84) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = '3.81', obtainedValue = T1.meaningful$Mean, valueType = 'mean')## [1] "MATCH for mean. The reported value (3.81) and the obtained value (3.81) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = '3.95', obtainedValue = T1.interest$Mean, valueType = 'mean')## [1] "MATCH for mean. The reported value (3.95) and the obtained value (3.95) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."#Running value comparisons on T1 SDs
reportObject <- reproCheck(reportedValue = '1.32', obtainedValue = T1sd.curious, valueType = 'sd')## [1] "MATCH for sd. The reported value (1.32) and the obtained value (1.32) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = '1.01', obtainedValue = T1sd.interestcomp, valueType = 'sd')## [1] "MATCH for sd. The reported value (1.01) and the obtained value (1.01) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."#Running value comparisons on lower T1 CIs
reportObject <- reproCheck(reportedValue = '3.74', obtainedValue = T1.curious$Trad.lower, valueType = 'ci')## [1] "MATCH for ci. The reported value (3.74) and the obtained value (3.74) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = '3.34', obtainedValue = T1.interestcomp$Trad.lower, valueType = 'ci')## [1] "MATCH for ci. The reported value (3.34) and the obtained value (3.34) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = '2.64', obtainedValue = T1.surprise$Trad.lower, valueType = 'ci')## [1] "MATCH for ci. The reported value (2.64) and the obtained value (2.64) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = '3.60', obtainedValue = T1.meaningful$Trad.lower, valueType = 'ci')## [1] "MATCH for ci. The reported value (3.6) and the obtained value (3.6) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = '3.73', obtainedValue = T1.interest$Trad.lower, valueType = 'ci')## [1] "MATCH for ci. The reported value (3.73) and the obtained value (3.73) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."#Running value comparisons on upper T1 CIs
reportObject <- reproCheck(reportedValue = '4.24', obtainedValue = T1.curious$Trad.upper, valueType = 'ci')## [1] "MATCH for ci. The reported value (4.24) and the obtained value (4.24) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = '3.73', obtainedValue = T1.interestcomp$Trad.upper, valueType = 'ci')## [1] "MATCH for ci. The reported value (3.73) and the obtained value (3.73) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = '3.05', obtainedValue = T1.surprise$Trad.upper, valueType = 'ci')## [1] "MATCH for ci. The reported value (3.05) and the obtained value (3.05) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = '4.03', obtainedValue = T1.meaningful$Trad.upper, valueType = 'ci')## [1] "MATCH for ci. The reported value (4.03) and the obtained value (4.03) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = '4.18', obtainedValue = T1.interest$Trad.upper, valueType = 'ci')## [1] "MATCH for ci. The reported value (4.18) and the obtained value (4.18) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."#T2 Means/Standard deviations (for Curiosity & Interest Composite only)
#Curiosity T2 mean = 4.34 [4.10, 4.58], SD = 1.25
T2.curious <- groupwiseMean(T2_Curious ~ 1,
data = data,
conf = 0.95,
digits = 3)
T2sd.curious <- sd(data$T2_Curious, na.rm = TRUE) # getting SD of T2 curious
#Interest Composite T2 mean = 3.82 [3.65, 4.00], SD = 0.89
T2.interestcomp <- groupwiseMean(T2_Interest_Composite ~ 1,
data = data,
conf = 0.95,
digits = 3)
T2sd.interestcomp <- sd(data$T2_Interest_Composite, na.rm = TRUE) # getting SD of interest comp
#Surprise T2 mean = 3.25 [3.06, 3.44]
T2.surprise <- groupwiseMean(T2_Surprised ~ 1,
data = data,
conf = 0.95,
digits = 3)
#Meaningfulness T2 mean = 4.04 [3.84, 4.23]
T2.meaningful <- groupwiseMean(T2_Meaningful ~ 1,
data = data,
conf = 0.95,
digits = 3)
#Interesting T2 mean = 4.19 [4.00, 4.38]
T2.interest <- groupwiseMean(T2_Interesting ~ 1,
data = data,
conf = 0.95,
digits = 3)
#Running value comparisons on T2 means
reportObject <- reproCheck(reportedValue = '4.34', obtainedValue = T2.curious$Mean, valueType = 'mean')## [1] "MATCH for mean. The reported value (4.34) and the obtained value (4.34) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = '3.82', obtainedValue = T2.interestcomp$Mean, valueType = 'mean')## [1] "MATCH for mean. The reported value (3.82) and the obtained value (3.82) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = '3.25', obtainedValue = T2.surprise$Mean, valueType = 'mean')## [1] "MATCH for mean. The reported value (3.25) and the obtained value (3.25) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = '4.04', obtainedValue = T2.meaningful$Mean, valueType = 'mean')## [1] "MATCH for mean. The reported value (4.04) and the obtained value (4.04) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = '4.19', obtainedValue = T2.interest$Mean, valueType = 'mean')## [1] "MATCH for mean. The reported value (4.19) and the obtained value (4.19) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."#Running value comparisons on T2 SDs
reportObject <- reproCheck(reportedValue = '1.25', obtainedValue = T2sd.curious, valueType = 'sd')## [1] "MATCH for sd. The reported value (1.25) and the obtained value (1.25) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = '0.89', obtainedValue = T2sd.interestcomp, valueType = 'sd')## [1] "MATCH for sd. The reported value (0.89) and the obtained value (0.89) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."#Running value comparisons on lower T2 CIs
reportObject <- reproCheck(reportedValue = '4.10', obtainedValue = T2.curious$Trad.lower, valueType = 'ci')## [1] "MATCH for ci. The reported value (4.1) and the obtained value (4.1) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = '3.65', obtainedValue = T2.interestcomp$Trad.lower, valueType = 'ci')## [1] "MATCH for ci. The reported value (3.65) and the obtained value (3.65) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = '3.06', obtainedValue = T2.surprise$Trad.lower, valueType = 'ci')## [1] "MATCH for ci. The reported value (3.06) and the obtained value (3.06) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = '3.84', obtainedValue = T2.meaningful$Trad.lower, valueType = 'ci')## [1] "MATCH for ci. The reported value (3.84) and the obtained value (3.84) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = '4.00', obtainedValue = T2.interest$Trad.lower, valueType = 'ci')## [1] "MATCH for ci. The reported value (4) and the obtained value (4) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."#Running value comparisons on upper T2 CIs
reportObject <- reproCheck(reportedValue = '4.58', obtainedValue = T2.curious$Trad.upper, valueType = 'ci')## [1] "MATCH for ci. The reported value (4.58) and the obtained value (4.58) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = '4.00', obtainedValue = T2.interestcomp$Trad.upper, valueType = 'ci')## [1] "MATCH for ci. The reported value (4) and the obtained value (4) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = '3.44', obtainedValue = T2.surprise$Trad.upper, valueType = 'ci')## [1] "MATCH for ci. The reported value (3.44) and the obtained value (3.44) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = '4.23', obtainedValue = T2.meaningful$Trad.upper, valueType = 'ci')## [1] "MATCH for ci. The reported value (4.23) and the obtained value (4.23) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = '4.38', obtainedValue = T2.interest$Trad.upper, valueType = 'ci')## [1] "MATCH for ci. The reported value (4.38) and the obtained value (4.38) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."#Underestimates
#Curiosity underestimate = 0.35 [0.11, 0.59]
dif.curious <- groupwiseMean(Curious_diff ~ 1,
data = data,
conf = 0.95,
digits = 2)
#Interest Composite underestimate = 0.29 [0.10, 0.47]
dif.interestcomp <- groupwiseMean(Interest_composite_diff ~ 1,
data = data,
conf = 0.95,
digits = 2)
#Surprise underestimate = 0.40 [0.19, 0.62]
data$Surprised_diff <- data$T2_Surprised - data$T1_Surprised
dif.surprise <- groupwiseMean(Surprised_diff ~ 1,
data = data,
conf = 0.95,
digits = 2)
#Meaningfulness underestimate = 0.22 [0.03, 0.42]
data$Meaningful_diff <- data$T2_Meaningful - data$T1_Meaningful
dif.meaningful <- groupwiseMean(Meaningful_diff ~ 1,
data = data,
conf = 0.95,
digits = 2)
#Interesting underestimate = 0.23 [0.02, 0.45]
data$Interesting_diff <- data$T2_Interesting - data$T1_Interesting
dif.interest <- groupwiseMean(Interesting_diff ~ 1,
data = data,
conf = 0.95,
digits = 2)
#Running value comparisons on underestimate means
reportObject <- reproCheck(reportedValue = '0.35', obtainedValue = dif.curious$Mean, valueType = 'mean')## [1] "MATCH for mean. The reported value (0.35) and the obtained value (0.35) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = '0.29', obtainedValue = dif.interestcomp$Mean, valueType = 'mean')## [1] "MATCH for mean. The reported value (0.29) and the obtained value (0.29) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = '0.40', obtainedValue = dif.surprise$Mean, valueType = 'mean')## [1] "MATCH for mean. The reported value (0.4) and the obtained value (0.4) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = '0.22', obtainedValue = dif.meaningful$Mean, valueType = 'mean')## [1] "MATCH for mean. The reported value (0.22) and the obtained value (0.22) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = '0.23', obtainedValue = dif.interest$Mean, valueType = 'mean')## [1] "MATCH for mean. The reported value (0.23) and the obtained value (0.23) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."#Running value comparisons on lower underestimate CIs
reportObject <- reproCheck(reportedValue = '0.11', obtainedValue = dif.curious$Trad.lower, valueType = 'ci')## [1] "MATCH for ci. The reported value (0.11) and the obtained value (0.11) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = '0.10', obtainedValue = dif.interestcomp$Trad.lower, valueType = 'ci')## [1] "MATCH for ci. The reported value (0.1) and the obtained value (0.1) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = '0.19', obtainedValue = dif.surprise$Trad.lower, valueType = 'ci')## [1] "MATCH for ci. The reported value (0.19) and the obtained value (0.19) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = '0.03', obtainedValue = dif.meaningful$Trad.lower, valueType = 'ci')## [1] "MATCH for ci. The reported value (0.03) and the obtained value (0.03) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = '0.02', obtainedValue = dif.interest$Trad.lower, valueType = 'ci')## [1] "MATCH for ci. The reported value (0.02) and the obtained value (0.02) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."#Running value comparisons on upper underestimate CIs
reportObject <- reproCheck(reportedValue = '0.59', obtainedValue = dif.curious$Trad.upper, valueType = 'ci')## [1] "MATCH for ci. The reported value (0.59) and the obtained value (0.59) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = '0.47', obtainedValue = dif.interestcomp$Trad.upper, valueType = 'ci')## [1] "MATCH for ci. The reported value (0.47) and the obtained value (0.47) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = '0.62', obtainedValue = dif.surprise$Trad.upper, valueType = 'ci')## [1] "MATCH for ci. The reported value (0.62) and the obtained value (0.62) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = '0.42', obtainedValue = dif.meaningful$Trad.upper, valueType = 'ci')## [1] "MATCH for ci. The reported value (0.42) and the obtained value (0.42) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = '0.45', obtainedValue = dif.interest$Trad.upper, valueType = 'ci')## [1] "MATCH for ci. The reported value (0.45) and the obtained value (0.45) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."#Create vectors for columns (variable, T1 mean/CI, T2 mean/CI, Underestimate)
Measure <- c("curiosity", "interestcomp", "surpise", "meaningful", "interest")
T1Mean <- c(T1.curious$Mean, T1.interestcomp$Mean, T1.surprise$Mean, T1.meaningful$Mean, T1.interest$Mean)
T1LowerCI <- c(T1.curious$Trad.lower, T1.interestcomp$Trad.lower, T1.surprise$Trad.lower, T1.meaningful$Trad.lower, T1.interest$Trad.lower)
T1UpperCI <- c(T1.curious$Trad.upper, T1.interestcomp$Trad.upper, T1.surprise$Trad.upper, T1.meaningful$Trad.upper, T1.interest$Trad.upper)
T2Mean <- c(T2.curious$Mean, T2.interestcomp$Mean, T2.surprise$Mean, T2.meaningful$Mean, T2.interest$Mean)
T2LowerCI <- c(T2.curious$Trad.lower, T2.interestcomp$Trad.lower, T2.surprise$Trad.lower, T2.meaningful$Trad.lower, T2.interest$Trad.lower)
T2UpperCI <- c(T2.curious$Trad.upper, T2.interestcomp$Trad.upper, T2.surprise$Trad.upper, T2.meaningful$Trad.upper, T2.interest$Trad.upper)
Underestimate <- round(c(dif.curious$Mean, dif.interestcomp$Mean, dif.surprise$Mean, dif.meaningful$Mean, dif.interest$Mean), digits = 2)
UndLowerCI <- c(dif.curious$Trad.lower, dif.interestcomp$Trad.lower, dif.surprise$Trad.lower, dif.meaningful$Trad.lower, dif.interest$Trad.lower)
UndUpperCI <- c(dif.curious$Trad.upper, dif.interestcomp$Trad.upper, dif.surprise$Trad.upper, dif.meaningful$Trad.upper, dif.interest$Trad.upper)Participants’ Time 1 predictions of their curiosity (M = 3.99, SD = 1.32) were lower than their actual curiosity ratings at Time 2, immediately before reading their responses (M = 4.34, SD = 1.25), t(105) = 2.88, p = .005, d = 0.27. Participants also underestimated how interesting they would find their responses. Predictions of interest at Time 1 (M = 3.54, SD = 1.01) were lower than ratings of actual interest experienced at Time 2 (M = 3.82, SD = 0.89), t(105) = 3.10, p = .003, d = 0.29.
#Run t-tests
#Curious t = 2.879981 and p = 0.004821242
t.curious <- t.test(data$T2_Curious, data$T1_Curious, paired = TRUE)
t.curious$statistic # t for interest comp## t
## 2.879981t.curious$p.value # p value for curious## [1] 0.004821242#Interesting Composite t = 3.095256 and p = 0.002521152
t.interestcomp <- t.test(data$T2_Interest_Composite, data$T1_Interest_Composite, paired = TRUE)
t.interestcomp$statistic # t for interest comp## t
## 3.095256t.interestcomp$p.value # p value for interest comp## [1] 0.002521152# Running value comparisons on t-tests
reportObject <- reproCheck(reportedValue = '2.88', obtainedValue = t.curious$statistic, valueType = 't') # Curious## [1] "MATCH for t. The reported value (2.88) and the obtained value (2.88) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = '3.10', obtainedValue = t.interestcomp$statistic, valueType = 't') # Interest## [1] "MATCH for t. The reported value (3.1) and the obtained value (3.1) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."# Running value comparisons on p-value
reportObject <- reproCheck(reportedValue = '.005', obtainedValue = t.curious$p.value, valueType = 't') # Curious## [1] "MATCH for t. The reported value (0.005) and the obtained value (0.005) differed by 0%. Note that the obtained value was rounded to 3 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = '.003', obtainedValue = t.interestcomp$p.value, valueType = 't') # Interest## [1] "MATCH for t. The reported value (0.003) and the obtained value (0.003) differed by 0%. Note that the obtained value was rounded to 3 decimal places to match the reported value."#Calculating Cohen's d
#Cohen's d = 0.30 ******* off by a small bit again
interest.cd <- cohensD(data$T1_Interest_Composite, data$T2_Interest_Composite, method = "paired")
#Cohen's d = 0.28 ******* actual value 0.2797284... is this incorrect or rounding error?
curiosity.cd <- cohensD(data$T1_Curious, data$T2_Curious, method = "paired")
# Running value comparisons on Cohen's d
reportObject <- reproCheck(reportedValue = '0.27', obtainedValue = curiosity.cd, valueType = 'd')## [1] "MINOR_ERROR for d. The reported value (0.27) and the obtained value (0.28) differed by 3.7%. Note that the obtained value was rounded to 2 decimal places to match the reported value."reportObject <- reproCheck(reportedValue = '0.29', obtainedValue = interest.cd, valueType = 'd')## [1] "MINOR_ERROR for d. The reported value (0.29) and the obtained value (0.3) differed by 3.45%. Note that the obtained value was rounded to 2 decimal places to match the reported value."#Build a table
# Need to get the rest of p-values for the last column
t.surprise <- t.test(data$T1_Surprised, data$T2_Surprised, paired = TRUE) #Surprise
t.meaningful <- t.test(data$T1_Meaningful, data$T2_Meaningful, paired = TRUE) #Meaningful
t.interest <- t.test(data$T1_Interesting, data$T2_Interesting, paired = TRUE) #Interesting
#Significance column
P <- round(c(t.curious$p.value, t.interestcomp$p.value,
t.surprise$p.value, t.meaningful$p.value,
t.interest$p.value), digits = 3)
#Create table
table <- kable(cbind(Measure, T1Mean, T1LowerCI, T1UpperCI, T2Mean, T2LowerCI, T2UpperCI, Underestimate, UndLowerCI, UndUpperCI, P))
table| Measure | T1Mean | T1LowerCI | T1UpperCI | T2Mean | T2LowerCI | T2UpperCI | Underestimate | UndLowerCI | UndUpperCI | P |
|---|---|---|---|---|---|---|---|---|---|---|
| curiosity | 3.99 | 3.74 | 4.24 | 4.34 | 4.1 | 4.58 | 0.35 | 0.11 | 0.59 | 0.005 |
| interestcomp | 3.54 | 3.34 | 3.73 | 3.82 | 3.65 | 4 | 0.29 | 0.1 | 0.47 | 0.003 |
| surpise | 2.84 | 2.64 | 3.05 | 3.25 | 3.06 | 3.44 | 0.4 | 0.19 | 0.62 | 0 |
| meaningful | 3.81 | 3.6 | 4.03 | 4.04 | 3.84 | 4.23 | 0.22 | 0.031 | 0.42 | 0.023 |
| interest | 3.95 | 3.73 | 4.18 | 4.19 | 4 | 4.38 | 0.23 | 0.02 | 0.45 | 0.032 |
This reproducibiity check was a success: majority of the findings (inferential statistics and descriptive values) from the original study were reproduced here with 0% difference and of the four minor errors identified, two were because I wasn’t sure how to assign the upper alpha values to a variable, but had I been able to do that they would have reproduced as well.
reportObject <- reportObject %>%
filter(dummyRow == FALSE) %>% # remove the dummy row
select(-dummyRow) %>% # remove dummy row designation
mutate(articleID = articleID) %>% # add variables to report
select(articleID, everything()) # make articleID first column
# decide on final outcome
if(any(reportObject$comparisonOutcome %in% c("MAJOR_ERROR", "DECISION_ERROR"))){
finalOutcome <- "Failure"
}else{
finalOutcome <- "Success"
}
# collate report extra details
reportExtras <- data.frame(articleID, pilotNames, copilotNames, pilotTTC, copilotTTC, pilotStartDate, copilotStartDate, completionDate, finalOutcome)
# save report objects
if(reportType == "pilot"){
write_csv(reportObject, "pilotReportDetailed.csv")
write_csv(reportExtras, "pilotReportExtras.csv")
}
if(reportType == "copilot"){
write_csv(reportObject, "copilotReportDetailed.csv")
write_csv(reportExtras, "copilotReportExtras.csv")
}[This function will output information about the package versions used in this report:]
devtools::session_info()## ─ Session info ──────────────────────────────────────────────────────────
## setting value
## version R version 3.6.1 (2019-07-05)
## os macOS Mojave 10.14.6
## system x86_64, darwin15.6.0
## ui X11
## language (EN)
## collate en_US.UTF-8
## ctype en_US.UTF-8
## tz America/Los_Angeles
## date 2019-11-10
##
## ─ Packages ──────────────────────────────────────────────────────────────
## package * version date lib
## assertthat 0.2.1 2019-03-21 [1]
## backports 1.1.5 2019-10-02 [1]
## boot 1.3-22 2019-04-02 [1]
## broom 0.5.2 2019-04-07 [1]
## callr 3.3.2 2019-09-22 [1]
## cellranger 1.1.0 2016-07-27 [1]
## cli 1.1.0 2019-03-19 [1]
## codetools 0.2-16 2018-12-24 [1]
## coin 1.3-1 2019-08-28 [1]
## colorspace 1.4-1 2019-03-18 [1]
## crayon 1.3.4 2017-09-16 [1]
## desc 1.2.0 2018-05-01 [1]
## DescTools 0.99.30 2019-10-22 [1]
## devtools 2.2.1 2019-09-24 [1]
## digest 0.6.21 2019-09-20 [1]
## dplyr * 0.8.3 2019-07-04 [1]
## ellipsis 0.3.0 2019-09-20 [1]
## EMT 1.1 2013-01-29 [1]
## evaluate 0.14 2019-05-28 [1]
## expm 0.999-4 2019-03-21 [1]
## forcats * 0.4.0 2019-02-17 [1]
## foreign 0.8-71 2018-07-20 [1]
## fs 1.3.1 2019-05-06 [1]
## generics 0.0.2 2018-11-29 [1]
## ggplot2 * 3.2.1 2019-08-10 [1]
## glue 1.3.1 2019-03-12 [1]
## gtable 0.3.0 2019-03-25 [1]
## haven * 2.1.1 2019-07-04 [1]
## highr 0.8 2019-03-20 [1]
## hms 0.5.2 2019-10-30 [1]
## htmltools 0.4.0 2019-10-04 [1]
## httr 1.4.1 2019-08-05 [1]
## jsonlite 1.6 2018-12-07 [1]
## knitr * 1.25 2019-09-18 [1]
## lattice 0.20-38 2018-11-04 [1]
## lazyeval 0.2.2 2019-03-15 [1]
## libcoin 1.0-5 2019-08-27 [1]
## lifecycle 0.1.0 2019-08-01 [1]
## lmtest 0.9-37 2019-04-30 [1]
## lsr * 0.5 2015-03-02 [1]
## lubridate 1.7.4 2018-04-11 [1]
## magrittr 1.5 2014-11-22 [1]
## MASS 7.3-51.4 2019-03-31 [1]
## Matrix 1.2-17 2019-03-22 [1]
## matrixStats 0.55.0 2019-09-07 [1]
## memoise 1.1.0 2017-04-21 [1]
## mnormt 1.5-5 2016-10-15 [1]
## modelr 0.1.5 2019-08-08 [1]
## modeltools 0.2-22 2018-07-16 [1]
## multcomp 1.4-10 2019-03-05 [1]
## multcompView 0.1-7 2015-07-31 [1]
## munsell 0.5.0 2018-06-12 [1]
## mvtnorm 1.0-11 2019-06-19 [1]
## nlme 3.1-140 2019-05-12 [1]
## nortest 1.0-4 2015-07-30 [1]
## pillar 1.4.2 2019-06-29 [1]
## pkgbuild 1.0.6 2019-10-09 [1]
## pkgconfig 2.0.3 2019-09-22 [1]
## pkgload 1.0.2 2018-10-29 [1]
## plyr 1.8.4 2016-06-08 [1]
## prettyunits 1.0.2 2015-07-13 [1]
## processx 3.4.1 2019-07-18 [1]
## ps 1.3.0 2018-12-21 [1]
## psych * 1.8.12 2019-01-12 [1]
## purrr * 0.3.3 2019-10-18 [1]
## R6 2.4.0 2019-02-14 [1]
## rcompanion * 2.3.7 2019-09-29 [1]
## Rcpp 1.0.2 2019-07-25 [1]
## readr * 1.3.1 2018-12-21 [1]
## readxl * 1.3.1 2019-03-13 [1]
## remotes 2.1.0 2019-06-24 [1]
## ReproReports * 0.1 2019-11-04 [1]
## rlang 0.4.0 2019-06-25 [1]
## rmarkdown 1.16 2019-10-01 [1]
## rprojroot 1.3-2 2018-01-03 [1]
## rstudioapi 0.10 2019-03-19 [1]
## rvest 0.3.4 2019-05-15 [1]
## sandwich 2.5-1 2019-04-06 [1]
## scales 1.0.0 2018-08-09 [1]
## sessioninfo 1.1.1 2018-11-05 [1]
## stringi 1.4.3 2019-03-12 [1]
## stringr * 1.4.0 2019-02-10 [1]
## survival 2.44-1.1 2019-04-01 [1]
## testthat 2.2.1 2019-07-25 [1]
## TH.data 1.0-10 2019-01-21 [1]
## tibble * 2.1.3 2019-06-06 [1]
## tidyr * 1.0.0 2019-09-11 [1]
## tidyselect 0.2.5 2018-10-11 [1]
## tidyverse * 1.2.1 2017-11-14 [1]
## usethis 1.5.1 2019-07-04 [1]
## vctrs 0.2.0 2019-07-05 [1]
## withr 2.1.2 2018-03-15 [1]
## xfun 0.10 2019-10-01 [1]
## xml2 1.2.2 2019-08-09 [1]
## zeallot 0.1.0 2018-01-28 [1]
## zoo 1.8-6 2019-05-28 [1]
## source
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.1)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.1)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.1)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.1)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.1)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.1)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.1)
## CRAN (R 3.6.1)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.1)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## Github (TomHardwicke/ReproReports@2ec6f60)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.1)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
##
## [1] /Library/Frameworks/R.framework/Versions/3.6/Resources/library