articleID <- "211-5-2015_CLASS"
# insert the article ID code here e.g. "10-3-2015_PS"
reportType <- "final" # specify whether this is the 'pilot' report or 'final' report
pilotNames <- "John Wilcox" # insert the pilot's name here e.g., "Tom Hardwicke". If there are multiple cpilots enter both names in a character string e.g., "Tom Hardwicke, Bob Dylan"
copilotNames <- "Kengthsagn Louis" # # insert the co-pilot's name here e.g., "Michael Frank". If there are multiple co-pilots enter both names in a character string e.g., "Tom Hardwicke, Bob Dylan"
pilotTTC <- "150" # insert the pilot's estimated time to complete (in minutes, fine to approximate) e.g., 120
copilotTTC <- "10" # insert the co-pilot's estimated time to complete (in minutes, fine to approximate) e.g., 120
pilotStartDate <- "11/4/2018" # insert the pilot's start date in US format e.g., as.Date("01/25/18", format = "%m/%d/%y")
copilotStartDate <- "11/4/2018" # insert the co-pilot's start date in US format e.g., as.Date("01/25/18", format = "%m/%d/%y")
completionDate <- "11/5/2018" # copilot insert the date of final report completion (after any necessary rounds of author assistance) in US format e.g., as.Date("01/25/18", format = "%m/%d/%y")In this study, the researchers aimed to determine the extent to which the brain’s right hemisphere is sensitive to syntactic aspects of language. To do this, they recruited 64 right-handed university students to participate in their study. These participants were shown 184 word pairs, half of which conformed to proper grammatical rules whereas the other half did not. The participants were asked to judge the grammaticality of the each phrase. The phrases were presented randomly to either the left visual field (LVF) or the right visual field (RVF) to ensure that phrases were presented in a lateralized manner to the subjects. Other measures were also taken to ensure a lateralized presentation of the phrases. EEGs were used to monitor brain and eye activity. This setup enabled to researchers to measure laterialized brain activity in response grammatically correct or incorrect word pairs.
For this article you should focus on the findings reported in the results section for behavioural data only.
Specifically, you should attempt to reproduce all descriptive and inferential analyses reported in the text below and associated tables/figures:
Behavioral data.
For all participants as a group, accuracy was higher for RVF than for LVF words, although better-than-chance accuracy was seen in both visual fields (RVF: 81%; LVF: 75%).
The same pattern held regardless of familial sinistrality: for FS– participants, RVF accuracy was 80% and LVF accuracy was 73%; for FS+ participants, RVF accuracy was 82% and LVF accuracy was 77%.
Results of the ANOVA with factors of familial sinistrality, visual field, and grammaticality showed main effects of visual field, F (1, 62) = 37.39, p < .001, and grammaticality, F (1, 62) = 18.51, p < .001, but no effect of familial sinistrality (p = .3).
There was also no Visual Field × Grammaticality interaction (p = .8) or familial sinistrality interactions (ps > .3).
# load packages
library(tidyverse) # for data munging
library(knitr) # for kable table formating
library(haven) # import and export 'SPSS', 'Stata' and 'SAS' Files
library(readxl) # import excel files
library(CARPSreports) # custom report functions
library(ez) # for anova analysis
library(readr) # easier to load data# Prepare report object. This will be updated automatically by the reproCheck function each time values are compared
reportObject <- data.frame(dummyRow = TRUE, reportedValue = NA, obtainedValue = NA, valueType = NA, percentageError = NA, comparisonOutcome = NA, eyeballCheck = NA)BehavioralData_OSF <- read_csv("data/BehavioralData_OSF.csv")
df <- BehavioralData_OSF #makes it easier to refer to data in code# no data tidying required for this specific study# 1. For all participants as a group, accuracy was higher for RVF than for LVF words, although better-than-chance accuracy was seen in both visual fields (RVF: 81%; LVF: 75%).
## Results were reproduced.
aggregate(cbind(Accuracy) ~ VisualField, data=df, mean)## VisualField Accuracy
## 1 LVF 0.7491508
## 2 RVF 0.8067255# 2. The same pattern held regardless of familial sinistrality: for FS– participants, RVF accuracy was 80% and LVF accuracy was 73%; for FS+ participants, RVF accuracy was 82% and LVF accuracy was 77%.
## Results were reproduced.
aggregate(cbind(Accuracy) ~ group + VisualField, data=df, mean)## group VisualField Accuracy
## 1 FS- LVF 0.7326087
## 2 FS+ LVF 0.7691154
## 3 FS- RVF 0.7984472
## 4 FS+ RVF 0.8167166# 3. Results of the ANOVA with factors of familial sinistrality, visual field, and grammaticality showed main effects of visual field, F (1, 62) = 37.39, p < .001, and grammaticality, F (1, 62) = 18.51, p < .001, but no effect of familial sinistrality (p = .3).
## Results were reproduced.
## visual field, F (1, 62) = 37.38543627 p = 7.125528e-08
## grammaticality, F (1, 62) = 18.50922966 p = 6.106040e-05
## no effect of familial sinistrality p = 2.712845e-01
## (Note that "group" denotes familial sinistrality.)
ezANOVA(data = df, dv = Accuracy, wid = Subject, within = .(VisualField, Grammaticality), between = group, type = 3)## $ANOVA
## Effect DFn DFd F p p<.05
## 2 group 1 62 1.23210582 2.712845e-01
## 3 VisualField 1 62 37.38543627 7.125528e-08 *
## 5 Grammaticality 1 62 18.50922966 6.106040e-05 *
## 4 group:VisualField 1 62 0.96625957 3.294353e-01
## 6 group:Grammaticality 1 62 0.49414734 4.847143e-01
## 7 VisualField:Grammaticality 1 62 0.03892194 8.442487e-01
## 8 group:VisualField:Grammaticality 1 62 0.02396118 8.774866e-01
## ges
## 2 1.202025e-02
## 3 4.959315e-02
## 5 6.241787e-02
## 4 1.346847e-03
## 6 1.774175e-03
## 7 4.911562e-05
## 8 3.023719e-05# 4. There was also no Visual Field × Grammaticality interaction (p = .8) or familial sinistrality interactions (ps > .3).
## Results were reproduced: Visual Field x Grammaticality interaction p = 8.442487e-01, and all p-values for familial sinistrality interactions were above .3. (See above.)All findings were reproduced without any detected errors. A minor point, though: it was not obvious to me what code could be used to reproduce their ANOVA statistics since the analysis requires an ANOVA involving both within and between subjects conditions for the same analysis. However, perhaps that just reflects nothing more than my lack of familiarity with producing ANOVAs in R.
Author_Assistance = FALSE # was author assistance provided? (if so, enter TRUE)
Insufficient_Information_Errors <- 0 # how many discrete insufficient information issues did you encounter?
# Assess the causal locus (discrete reproducibility issues) of any reproducibility errors. Note that there doesn't necessarily have to be a one-to-one correspondance between discrete reproducibility issues and reproducibility errors. For example, it could be that the original article neglects to mention that a Greenhouse-Geisser correct was applied to ANOVA outcomes. This might result in multiple reproducibility errors, but there is a single causal locus (discrete reproducibility issue).
locus_typo <- NA # how many discrete issues did you encounter that related to typographical errors?
locus_specification <- NA # how many discrete issues did you encounter that related to incomplete, incorrect, or unclear specification of the original analyses?
locus_analysis <- NA # how many discrete issues did you encounter that related to errors in the authors' original analyses?
locus_data <- NA # how many discrete issues did you encounter that related to errors in the data files shared by the authors?
locus_unidentified <- NA # how many discrete issues were there for which you could not identify the cause
# How many of the above issues were resolved through author assistance?
locus_typo_resolved <- NA # how many discrete issues did you encounter that related to typographical errors?
locus_specification_resolved <- NA # how many discrete issues did you encounter that related to incomplete, incorrect, or unclear specification of the original analyses?
locus_analysis_resolved <- NA # how many discrete issues did you encounter that related to errors in the authors' original analyses?
locus_data_resolved <- NA # how many discrete issues did you encounter that related to errors in the data files shared by the authors?
locus_unidentified_resolved <- NA # how many discrete issues were there for which you could not identify the cause
Affects_Conclusion <- NA # Do any reproducibility issues encounter appear to affect the conclusions made in the original article? TRUE, FALSE, or NA. This is a subjective judgement, but you should taking into account multiple factors, such as the presence/absence of decision errors, the number of target outcomes that could not be reproduced, the type of outcomes that could or could not be reproduced, the difference in magnitude of effect sizes, and the predictions of the specific hypothesis under scrutiny.reportObject <- reportObject %>%
filter(dummyRow == FALSE) %>% # remove the dummy row
select(-dummyRow) %>% # remove dummy row designation
mutate(articleID = articleID) %>% # add variables to report
select(articleID, everything()) # make articleID first column
# decide on final outcome
if(any(reportObject$comparisonOutcome %in% c("MAJOR_ERROR", "DECISION_ERROR")) | Insufficient_Information_Errors > 0){
finalOutcome <- "Failure without author assistance"
if(Author_Assistance == T){
finalOutcome <- "Failure despite author assistance"
}
}else{
finalOutcome <- "Success without author assistance"
if(Author_Assistance == T){
finalOutcome <- "Success with author assistance"
}
}
# collate report extra details
reportExtras <- data.frame(articleID, pilotNames, copilotNames, pilotTTC, copilotTTC, pilotStartDate, copilotStartDate, completionDate, Author_Assistance, finalOutcome, Insufficient_Information_Errors, locus_typo, locus_specification, locus_analysis, locus_data, locus_unidentified, locus_typo_resolved, locus_specification_resolved, locus_analysis_resolved, locus_data_resolved, locus_unidentified_resolved)
# save report objects
if(reportType == "pilot"){
write_csv(reportObject, "pilotReportDetailed.csv")
write_csv(reportExtras, "pilotReportExtras.csv")
}
if(reportType == "final"){
write_csv(reportObject, "finalReportDetailed.csv")
write_csv(reportExtras, "finalReportExtras.csv")
}devtools::session_info()## - Session info ----------------------------------------------------------
## setting value
## version R version 3.5.1 (2018-07-02)
## os Windows 10 x64
## system x86_64, mingw32
## ui RTerm
## language (EN)
## collate English_United States.1252
## ctype English_United States.1252
## tz America/Los_Angeles
## date 2018-11-05
##
## - Packages --------------------------------------------------------------
## package * version date lib
## abind 1.4-5 2016-07-21 [1]
## assertthat 0.2.0 2017-04-11 [1]
## backports 1.1.2 2017-12-13 [1]
## base64enc 0.1-3 2015-07-28 [1]
## bindr 0.1.1 2018-03-13 [1]
## bindrcpp * 0.2.2 2018-03-29 [1]
## broom 0.5.0 2018-07-17 [1]
## callr 3.0.0 2018-08-24 [1]
## car 3.0-2 2018-08-23 [1]
## carData 3.0-2 2018-09-30 [1]
## CARPSreports * 0.1 2018-11-04 [1]
## cellranger 1.1.0 2016-07-27 [1]
## cli 1.0.1 2018-09-25 [1]
## colorspace 1.3-2 2016-12-14 [1]
## crayon 1.3.4 2017-09-16 [1]
## curl 3.2 2018-03-28 [1]
## data.table 1.11.8 2018-09-30 [1]
## desc 1.2.0 2018-05-01 [1]
## devtools 2.0.1 2018-10-26 [1]
## digest 0.6.17 2018-09-12 [1]
## dplyr * 0.7.7 2018-10-16 [1]
## evaluate 0.11 2018-07-17 [1]
## ez * 4.4-0 2016-11-02 [1]
## forcats * 0.3.0 2018-02-19 [1]
## foreign 0.8-71 2018-07-20 [1]
## fs 1.2.6 2018-08-23 [1]
## ggplot2 * 3.1.0 2018-10-25 [1]
## glue 1.3.0 2018-07-17 [1]
## gtable 0.2.0 2016-02-26 [1]
## haven * 1.1.2 2018-06-27 [1]
## hms 0.4.2 2018-03-10 [1]
## htmltools 0.3.6 2017-04-28 [1]
## httr 1.3.1 2017-08-20 [1]
## jsonlite 1.5 2017-06-01 [1]
## knitr * 1.20 2018-02-20 [1]
## lattice 0.20-35 2017-03-25 [2]
## lazyeval 0.2.1 2017-10-29 [1]
## lme4 1.1-18-1 2018-08-17 [1]
## lubridate 1.7.4 2018-04-11 [1]
## magrittr 1.5 2014-11-22 [1]
## MASS 7.3-50 2018-04-30 [2]
## Matrix 1.2-14 2018-04-13 [2]
## memoise 1.1.0 2017-04-21 [1]
## mgcv 1.8-24 2018-06-23 [2]
## minqa 1.2.4 2014-10-09 [1]
## modelr 0.1.2 2018-05-11 [1]
## munsell 0.5.0 2018-06-12 [1]
## nlme 3.1-137 2018-04-07 [2]
## nloptr 1.2.1 2018-10-03 [1]
## openxlsx 4.1.0 2018-05-26 [1]
## pillar 1.3.0 2018-07-14 [1]
## pkgbuild 1.0.2 2018-10-16 [1]
## pkgconfig 2.0.2 2018-08-16 [1]
## pkgload 1.0.2 2018-10-29 [1]
## plyr 1.8.4 2016-06-08 [1]
## prettyunits 1.0.2 2015-07-13 [1]
## processx 3.2.0 2018-08-16 [1]
## ps 1.1.0 2018-08-10 [1]
## purrr * 0.2.5 2018-05-29 [1]
## R6 2.2.2 2017-06-17 [1]
## Rcpp 0.12.18 2018-07-23 [1]
## readr * 1.1.1 2017-05-16 [1]
## readxl * 1.1.0 2018-04-20 [1]
## remotes 2.0.2 2018-10-30 [1]
## reshape2 1.4.3 2017-12-11 [1]
## rio 0.5.10 2018-03-29 [1]
## rlang 0.2.2 2018-08-16 [1]
## rmarkdown 1.10 2018-06-11 [1]
## rprojroot 1.3-2 2018-01-03 [1]
## rstudioapi 0.7 2017-09-07 [1]
## rvest 0.3.2 2016-06-17 [1]
## scales 1.0.0 2018-08-09 [1]
## sessioninfo 1.1.0 2018-09-25 [1]
## stringi 1.1.7 2018-03-12 [1]
## stringr * 1.3.1 2018-05-10 [1]
## tibble * 1.4.2 2018-01-22 [1]
## tidyr * 0.8.2 2018-10-28 [1]
## tidyselect 0.2.5 2018-10-11 [1]
## tidyverse * 1.2.1 2017-11-14 [1]
## usethis 1.4.0 2018-08-14 [1]
## withr 2.1.2 2018-03-15 [1]
## xml2 1.2.0 2018-01-24 [1]
## yaml 2.2.0 2018-07-25 [1]
## zip 1.0.0 2017-04-25 [1]
## source
## CRAN (R 3.5.0)
## CRAN (R 3.5.1)
## CRAN (R 3.5.0)
## CRAN (R 3.5.0)
## CRAN (R 3.5.1)
## CRAN (R 3.5.1)
## CRAN (R 3.5.1)
## CRAN (R 3.5.1)
## CRAN (R 3.5.1)
## CRAN (R 3.5.1)
## Github (METRICS-CARPS/CARPSreports@89db4a9)
## CRAN (R 3.5.1)
## CRAN (R 3.5.1)
## CRAN (R 3.5.1)
## CRAN (R 3.5.1)
## CRAN (R 3.5.1)
## CRAN (R 3.5.1)
## CRAN (R 3.5.1)
## CRAN (R 3.5.1)
## CRAN (R 3.5.1)
## CRAN (R 3.5.1)
## CRAN (R 3.5.1)
## CRAN (R 3.5.1)
## CRAN (R 3.5.1)
## CRAN (R 3.5.1)
## CRAN (R 3.5.1)
## CRAN (R 3.5.1)
## CRAN (R 3.5.1)
## CRAN (R 3.5.1)
## CRAN (R 3.5.1)
## CRAN (R 3.5.1)
## CRAN (R 3.5.1)
## CRAN (R 3.5.1)
## CRAN (R 3.5.1)
## CRAN (R 3.5.1)
## CRAN (R 3.5.1)
## CRAN (R 3.5.1)
## CRAN (R 3.5.1)
## CRAN (R 3.5.1)
## CRAN (R 3.5.1)
## CRAN (R 3.5.1)
## CRAN (R 3.5.1)
## CRAN (R 3.5.1)
## CRAN (R 3.5.1)
## CRAN (R 3.5.1)
## CRAN (R 3.5.1)
## CRAN (R 3.5.1)
## CRAN (R 3.5.1)
## CRAN (R 3.5.1)
## CRAN (R 3.5.1)
## CRAN (R 3.5.1)
## CRAN (R 3.5.1)
## CRAN (R 3.5.1)
## CRAN (R 3.5.1)
## CRAN (R 3.5.1)
## CRAN (R 3.5.1)
## CRAN (R 3.5.1)
## CRAN (R 3.5.1)
## CRAN (R 3.5.1)
## CRAN (R 3.5.1)
## CRAN (R 3.5.1)
## CRAN (R 3.5.1)
## CRAN (R 3.5.1)
## CRAN (R 3.5.1)
## CRAN (R 3.5.1)
## CRAN (R 3.5.1)
## CRAN (R 3.5.1)
## CRAN (R 3.5.1)
## CRAN (R 3.5.1)
## CRAN (R 3.5.1)
## CRAN (R 3.5.1)
## CRAN (R 3.5.1)
## CRAN (R 3.5.1)
## CRAN (R 3.5.0)
## CRAN (R 3.5.1)
## CRAN (R 3.5.1)
## CRAN (R 3.5.1)
## CRAN (R 3.5.1)
## CRAN (R 3.5.1)
## CRAN (R 3.5.1)
## CRAN (R 3.5.1)
## CRAN (R 3.5.1)
## CRAN (R 3.5.1)
## CRAN (R 3.5.1)
##
## [1] C:/Users/john-/Documents/R/win-library/3.5
## [2] C:/Program Files/R/R-3.5.1/library