Reproduction of study 1 from Chen, Sperandio, and Goodale 2015.
articleID <- 6-1-2015 # insert the article ID code here e.g., "10-3-2015"
reportType <- 'pilot' # specify whether this is the 'pilot' report or 'copilot' report
pilotNames <- "Adrian Apaza" # insert the pilot's name here e.g., "Tom Hardwicke".
copilotNames <- "Philip Hernandez" # # insert the co-pilot's name here e.g., "Michael Frank".
pilotTTC <- 250 # insert the pilot's estimated time to complete (in minutes, it is fine to approximate) e.g., 120
copilotTTC <- NA # insert the co-pilot's estimated time to complete (in minutes, it is fine to approximate) e.g., 120
pilotStartDate <- 11/1/19 # insert the piloting start date in US format e.g., as.Date("01/25/18", format = "%m/%d/%y")
copilotStartDate <- NA # insert the co-piloting start date in US format e.g., as.Date("01/25/18", format = "%m/%d/%y")
completionDate <- NA # insert the date of final report completion in US format e.g., as.Date("01/25/18", format = "%m/%d/%y")“10 participants were recruited. Each participant was seated at a table and wearing goggles that may be opened or closed, and the table varied in whether it was cluttered with other disks or not. Two main types of trials were performed. In action trials, participants tried to grasp (grip aperture is DV) the target disk as quickly as possible; in perception tasks they were required to indicate the size of the target disk with their had (manual estimate is DV). Additionally participants may have had the goggles closed (open loop) or open (closed loop) so that they may see thier hand and disks or not. In total for each participant”[t]here were 16 combinations of conditions and target sizes: 2 crowding conditions (crowded or uncrowded) × 2 tasks (grasping or manual estimation) × 2 viewing con- ditions (closed or open loop) × 2 target sizes (3.0 cm or 3.75 cm); each combination was presented 10 times (160 trials in total)."
“Experiment 1 was designed to explore the effects of crowding on perception and action, with a particular focus on whether participants could scale their grip aperture to the size of the target even when they could not consciously identify the size of the target. We carried out a four-way repeated measures ANOVA on the manual estimates and PGAs with task (estimation vs. grasping), crowding condition (uncrowded vs. crowded), viewing condition (closed- vs. open-loop), and target size (3.0 vs. 3.75 cm) as main factors. The significant interaction between task and crowding condition, F(1, 9) = 6.818, p = .028, suggested that crowding had different effects on performance of the grasping and manual estimation tasks. Not surprisingly, when the target was presented in isolation, participants were able to manually estimate the sizes of the two targets—and this was true for both closed-loop trials, t(9) = 7.23, p < .001, and open-loop trials, t(9) = 9.19, p < .001. Similarly, participants showed excellent grip scaling for targets presented in isolation on both closed-loop trials, t(9) = 4.29, p = .002, and openloop trials, t(9) = 4.79, p = .001 (Fig. 3). Things were quite different, however, when the target disks were surrounded by flankers. In this condition, participants could no longer discriminate between the two disk sizes using a manual estimate closed-loop trials: t(9) = 1.02, p = .334; open-loop trials: t(9) = 1.78, p = .108?presumably because the size of the target was perceptually invisible. (Note that we use the term invisible to refer to the fact that participants could not identify the size of the target, even though they were aware of its presence and position.) In contrast, when participants were asked to grasp the same targets, their PGAs were still scaled to target size - closed-loop trials: t(9) = 4.21, p = .002; open-loop trials: t(9) = 3.392, p = .008 (Fig. 3).”"
# load packages
library(tidyverse) # for data munging
library(dplyr)
library(readxl)
library(knitr) # for kable table formating
library(haven) # import and export 'SPSS', 'Stata' and 'SAS' Files
library(readxl) # import excel files
library(ReproReports) # custom reporting functions# Prepare report object. This will be updated automatically by the reproCheck function each time values are compared
reportObject <- data.frame(dummyRow = TRUE, reportedValue = NA, obtainedValue = NA, valueType = NA, percentageError = NA, comparisonOutcome = NA, eyeballCheck = NA)#read in the data
df <- read_excel("GroupA_6-1-2015/data/data_Exp1.xlsx",
sheet = "summary", col_names = FALSE,n_max=13)
#Notice that many cells are not filled in (since this is an excel table), lets fix that by assigning the conditions to appropriate columns
#but first drop empty rows
df<-df[,(colSums(is.na(df))!=13)]
#rename columns
colnames(df) <- c(1:17)
#rename columns for each condition
colnames(df) <- paste("Cond", c(1:17), sep = "_")##Create new variables (columns) for the various conditions: open/closed loop; crowded or not, small or large, estimation vs grasping
library(tidyr)
#data_long <- gather(df, df[,2], df[,3], df[,2:3], factor_key=TRUE)
#make into long data frame
data_long <- gather(df[4:13,], condition, measurement, Cond_2:Cond_17, factor_key=TRUE)#specify values of variables based upon values for each condition as seen in the orginal datframe (df)
data_long$closed<-F
#recode closed for conditions 2|3|4|5|10|11|12|13
data_long$closed[grepl("2|3|4|5|10|11|12|13",data_long$condition)]<-T
#recode 14,15 back to false
data_long$closed[grepl("14|15",data_long$condition)]<-F
#recode conditions c(3,5,7,9,11,13,15,17) as small
data_long$small<-T
data_long$small[grepl("3|5|7|9|11|13|15|17",data_long$condition)]<-F
#recode conditionsc(2,3,6,7,10,11,14,15)as small item conditions,
data_long$crowded<-T
data_long$crowded[grepl("2|3|6|7|10|11|14|15",data_long$condition)]<-F
#recode others back to false
data_long$crowded[grepl("12|13|16|17",data_long$condition)]<-T
#now identify estimation or not (task condition)
#estimation is 10 thru 17
data_long$estimation<-F
data_long$estimation[grepl("10|11|12|13|14|15|16|17",data_long$condition)]<-T
data_long$measurement<-as.numeric(data_long$measurement)
#rename the subject variable as such
colnames(data_long)[1]<-'Subject'#Lets look at means for various groups
means<-data_long %>% # Specify data frame
group_by(closed,crowded,small,estimation) %>% # Specify group indicator
summarise_at(vars(measurement), # Specify column
list(name = mean))
#rename the last column and view results
colnames(means)[5]<-"Group Mean"
print(means)## # A tibble: 16 x 5
## # Groups: closed, crowded, small [8]
## closed crowded small estimation `Group Mean`
## <lgl> <lgl> <lgl> <lgl> <dbl>
## 1 FALSE FALSE FALSE FALSE 6.48
## 2 FALSE FALSE FALSE TRUE 4.24
## 3 FALSE FALSE TRUE FALSE 5.66
## 4 FALSE FALSE TRUE TRUE 3.39
## 5 FALSE TRUE FALSE FALSE 5.52
## 6 FALSE TRUE FALSE TRUE 3.51
## 7 FALSE TRUE TRUE FALSE 5.02
## 8 FALSE TRUE TRUE TRUE 3.37
## 9 TRUE FALSE FALSE FALSE 6.36
## 10 TRUE FALSE FALSE TRUE 4.55
## 11 TRUE FALSE TRUE FALSE 5.61
## 12 TRUE FALSE TRUE TRUE 3.51
## 13 TRUE TRUE FALSE FALSE 5.42
## 14 TRUE TRUE FALSE TRUE 3.76
## 15 TRUE TRUE TRUE FALSE 5.18
## 16 TRUE TRUE TRUE TRUE 3.67“We carried out a four-way repeated measures ANOVA on the manual estimates and PGAs with task (estimation vs. grasping), crowding condition (uncrowded vs. crowded), viewing condition (closed- vs. open-loop), and target size (3.0 vs. 3.75 cm) as main factors. The significant interaction between task and crowding condition, F(1, 9) = 6.818, p = .028”
#Run the model then get the statistics
#
model22<-aov(measurement~closed*small+closed*crowded+closed*estimation+small*crowded+small*estimation+crowded*estimation+Subject,data=data_long)
anova(model22)## Analysis of Variance Table
##
## Response: measurement
## Df Sum Sq Mean Sq F value Pr(>F)
## closed 1 0.490 0.490 1.4668 0.2278917
## small 1 12.177 12.177 36.4274 1.346e-08 ***
## crowded 1 11.865 11.865 35.4928 1.971e-08 ***
## estimation 1 145.215 145.215 434.4080 < 2.2e-16 ***
## Subject 9 37.947 4.216 12.6129 1.564e-14 ***
## closed:small 1 0.019 0.019 0.0565 0.8124763
## closed:crowded 1 0.072 0.072 0.2163 0.6425739
## closed:estimation 1 0.740 0.740 2.2129 0.1391091
## small:crowded 1 3.936 3.936 11.7733 0.0007901 ***
## small:estimation 1 0.026 0.026 0.0767 0.7821596
## crowded:estimation 1 1.554 1.554 4.6489 0.0327830 *
## Residuals 140 46.800 0.334
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1#we get a significant result but not the same level of significance
reportObject <- reproCheck(reportedValue='6.818', anova(model22)[11,4], valueType = 'F')## [1] "MAJOR_ERROR for F. The reported value (6.818) and the obtained value (4.649) differed by 31.81%. Note that the obtained value was rounded to 3 decimal places to match the reported value."“in isolation, participants were able to manually estimate the sizes of the two targets—and this was true for both closed-loop trials, t(9) = 7.23, p < .001, and open-loop trials, t(9) = 9.19
#isoloation for closed loop crowded vs not are condtions"
#test for closed loop uncrowded estimation
t1<-t.test(data_long$measurement[data_long$closed==T & data_long$crowded==F & data_long$small==F & data_long$estimation==T],
data_long$measurement[data_long$closed==T & data_long$crowded==F &data_long$small==T & data_long$estimation==T],paired = T)
reportObject <- reproCheck(reportedValue='7.23', t1$statistic, valueType = 't')## [1] "MATCH for t. The reported value (7.23) and the obtained value (7.23) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."#test for open loop uncrowded estimation
t2<-t.test(data_long$measurement[data_long$closed==F & data_long$crowded==F & data_long$small==F & data_long$estimation==T],
data_long$measurement[data_long$closed==F & data_long$crowded==F &data_long$small==T & data_long$estimation==T],paired = T)
reportObject <- reproCheck(reportedValue='9.19', t2$statistic, valueType = 't')## [1] "MINOR_ERROR for t. The reported value (9.19) and the obtained value (9.2) differed by 0.11%. Note that the obtained value was rounded to 2 decimal places to match the reported value."“Similarly, participants showed excellent grip scaling for targets presented in isolation on both closed-loop trials, t(9) = 4.29, p = .002, and openloop trials, t(9) = 4.79, p = .001”
#test for closed loop grasp uncrowded
t3 <- t.test(data_long$measurement[data_long$closed==T & data_long$crowded==F & data_long$small==F & data_long$estimation==F],
data_long$measurement[data_long$closed==T & data_long$crowded==F &data_long$small==T & data_long$estimation==F],paired = T)
reportObject <- reproCheck(reportedValue='4.29', t3$statistic, valueType = 't')## [1] "MINOR_ERROR for t. The reported value (4.29) and the obtained value (4.3) differed by 0.23%. Note that the obtained value was rounded to 2 decimal places to match the reported value."#test for open loop grasp uncrowded
t4 <- t.test(data_long$measurement[data_long$closed==F & data_long$crowded==F & data_long$small==F & data_long$estimation==F],
data_long$measurement[data_long$closed==F & data_long$crowded==F &data_long$small==T & data_long$estimation==F],paired = T)
reportObject <- reproCheck(reportedValue='4.79', t4$statistic, valueType = 't')## [1] "MINOR_ERROR for t. The reported value (4.79) and the obtained value (4.8) differed by 0.21%. Note that the obtained value was rounded to 2 decimal places to match the reported value."“Things were quite different, however, when the target disks were surrounded by flankers. In this condition, participants could no longer discriminate between the two disk sizes using a manual estimate closed-loop trials: t(9) = 1.02, p = .334; open-loop trials: t(9) = 1.78, p = .108”
#Crowded
#test for closed loop crowded estimation
t5 <- t.test(data_long$measurement[data_long$closed==T & data_long$crowded==T & data_long$small==F & data_long$estimation==T],
data_long$measurement[data_long$closed==T & data_long$crowded==T &data_long$small==T & data_long$estimation==T],paired = T)
reportObject <- reproCheck(reportedValue='1.02', t5$statistic, valueType = 't')## [1] "MATCH for t. The reported value (1.02) and the obtained value (1.02) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."#test for open loop crowded estimation
t6 <- t.test(data_long$measurement[data_long$closed==F & data_long$crowded==T & data_long$small==F & data_long$estimation==T],
data_long$measurement[data_long$closed==F & data_long$crowded==T &data_long$small==T & data_long$estimation==T],paired = T)
reportObject <- reproCheck(reportedValue='1.78', t6$statistic, valueType = 't')## [1] "MATCH for t. The reported value (1.78) and the obtained value (1.78) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."“In contrast, when participants were asked to grasp the same targets, their PGAs were still scaled to target size?closed-loop trials: t(9) = 4.21, p = .002; open-loop trials: t(9) = 3.392, p = .008
#test for closed loop grasp crowded"
t7 <-t.test(data_long$measurement[data_long$closed==T & data_long$crowded==T & data_long$small==F & data_long$estimation==F],
data_long$measurement[data_long$closed==T & data_long$crowded==T &data_long$small==T & data_long$estimation==F],paired = T)
reportObject <- reproCheck(reportedValue='4.21', t7$statistic, valueType = 't')## [1] "MINOR_ERROR for t. The reported value (4.21) and the obtained value (4.22) differed by 0.24%. Note that the obtained value was rounded to 2 decimal places to match the reported value."#test for open loop grasp crowded
t8 <- t.test(data_long$measurement[data_long$closed==F & data_long$crowded==T & data_long$small==F & data_long$estimation==F],
data_long$measurement[data_long$closed==F & data_long$crowded==T &data_long$small==T & data_long$estimation==F],paired = T)
reportObject <- reproCheck(reportedValue='3.392', t8$statistic, valueType = 't')## [1] "MATCH for t. The reported value (3.392) and the obtained value (3.392) differed by 0%. Note that the obtained value was rounded to 3 decimal places to match the reported value."##All the t tests replicate!This reproducability check failed to generate the same F statistic for the four way repeated ANOVA with the interaction between task and crowding condition. This may be because it is not clear what interactions were all considered in the original authors’ ANOVA analysis to generate the F stat and so may not have been used here (e.g. did the authors consider all 4 way interactions? If so then that also failed to genereate the same F-statisitc). Additionally there were some minor differences in the t statistics generated. However the significance of the results in the anticipated directions is not in question. I do not think the original conclusions are seriously affected.
reportObject <- reportObject %>%
filter(dummyRow == FALSE) %>% # remove the dummy row
select(-dummyRow) %>% # remove dummy row designation
mutate(articleID = articleID) %>% # add variables to report
select(articleID, everything()) # make articleID first column
# decide on final outcome
if(any(reportObject$comparisonOutcome %in% c("MAJOR_ERROR", "DECISION_ERROR"))){
finalOutcome <- "Failure"
}else{
finalOutcome <- "Success"
}
# collate report extra details
reportExtras <- data.frame(articleID, pilotNames, copilotNames, pilotTTC, copilotTTC, pilotStartDate, copilotStartDate, completionDate, finalOutcome)
# save report objects
if(reportType == "pilot"){
write_csv(reportObject, "pilotReportDetailed.csv")
write_csv(reportExtras, "pilotReportExtras.csv")
}
if(reportType == "copilot"){
write_csv(reportObject, "copilotReportDetailed.csv")
write_csv(reportExtras, "copilotReportExtras.csv")
}devtools::session_info()## ─ Session info ──────────────────────────────────────────────────────────
## setting value
## version R version 3.6.1 (2019-07-05)
## os macOS Mojave 10.14.6
## system x86_64, darwin15.6.0
## ui X11
## language (EN)
## collate en_US.UTF-8
## ctype en_US.UTF-8
## tz America/Los_Angeles
## date 2019-11-11
##
## ─ Packages ──────────────────────────────────────────────────────────────
## package * version date lib
## assertthat 0.2.1 2019-03-21 [1]
## backports 1.1.5 2019-10-02 [1]
## broom 0.5.2 2019-04-07 [1]
## callr 3.3.2 2019-09-22 [1]
## cellranger 1.1.0 2016-07-27 [1]
## cli 1.1.0 2019-03-19 [1]
## colorspace 1.4-1 2019-03-18 [1]
## crayon 1.3.4 2017-09-16 [1]
## desc 1.2.0 2018-05-01 [1]
## devtools 2.2.1 2019-09-24 [1]
## digest 0.6.22 2019-10-21 [1]
## dplyr * 0.8.3 2019-07-04 [1]
## ellipsis 0.3.0 2019-09-20 [1]
## evaluate 0.14 2019-05-28 [1]
## fansi 0.4.0 2018-10-05 [1]
## forcats * 0.4.0 2019-02-17 [1]
## fs 1.3.1 2019-05-06 [1]
## generics 0.0.2 2018-11-29 [1]
## ggplot2 * 3.2.1 2019-08-10 [1]
## glue 1.3.1 2019-03-12 [1]
## gtable 0.3.0 2019-03-25 [1]
## haven * 2.2.0 2019-11-08 [1]
## hms 0.5.2 2019-10-30 [1]
## htmltools 0.4.0 2019-10-04 [1]
## httr 1.4.1 2019-08-05 [1]
## jsonlite 1.6 2018-12-07 [1]
## knitr * 1.25 2019-09-18 [1]
## lattice 0.20-38 2018-11-04 [1]
## lazyeval 0.2.2 2019-03-15 [1]
## lifecycle 0.1.0 2019-08-01 [1]
## lubridate 1.7.4 2018-04-11 [1]
## magrittr 1.5 2014-11-22 [1]
## memoise 1.1.0 2017-04-21 [1]
## modelr 0.1.5 2019-08-08 [1]
## munsell 0.5.0 2018-06-12 [1]
## nlme 3.1-140 2019-05-12 [1]
## pillar 1.4.2 2019-06-29 [1]
## pkgbuild 1.0.6 2019-10-09 [1]
## pkgconfig 2.0.3 2019-09-22 [1]
## pkgload 1.0.2 2018-10-29 [1]
## prettyunits 1.0.2 2015-07-13 [1]
## processx 3.4.1 2019-07-18 [1]
## ps 1.3.0 2018-12-21 [1]
## purrr * 0.3.3 2019-10-18 [1]
## R6 2.4.0 2019-02-14 [1]
## Rcpp 1.0.3 2019-11-08 [1]
## readr * 1.3.1 2018-12-21 [1]
## readxl * 1.3.1 2019-03-13 [1]
## remotes 2.1.0 2019-06-24 [1]
## ReproReports * 0.1 2019-11-12 [1]
## rlang 0.4.1 2019-10-24 [1]
## rmarkdown 1.16 2019-10-01 [1]
## rprojroot 1.3-2 2018-01-03 [1]
## rstudioapi 0.10 2019-03-19 [1]
## rvest 0.3.5 2019-11-08 [1]
## scales 1.0.0 2018-08-09 [1]
## sessioninfo 1.1.1 2018-11-05 [1]
## stringi 1.4.3 2019-03-12 [1]
## stringr * 1.4.0 2019-02-10 [1]
## testthat 2.3.0 2019-11-05 [1]
## tibble * 2.1.3 2019-06-06 [1]
## tidyr * 1.0.0 2019-09-11 [1]
## tidyselect 0.2.5 2018-10-11 [1]
## tidyverse * 1.2.1 2017-11-14 [1]
## usethis 1.5.1 2019-07-04 [1]
## utf8 1.1.4 2018-05-24 [1]
## vctrs 0.2.0 2019-07-05 [1]
## withr 2.1.2 2018-03-15 [1]
## xfun 0.10 2019-10-01 [1]
## xml2 1.2.2 2019-08-09 [1]
## yaml 2.2.0 2018-07-25 [1]
## zeallot 0.1.0 2018-01-28 [1]
## source
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.1)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.1)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## Github (TomHardwicke/ReproReports@2ec6f60)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
## CRAN (R 3.6.0)
##
## [1] /Library/Frameworks/R.framework/Versions/3.6/Resources/library