Reproducibility Report
Report Details
articleID <- 6-7-2014 # insert the article ID code here e.g., "10-3-2015"
reportType <- "copilot" # specify whether this is the 'pilot' report or 'copilot' report
pilotNames <- "Ashish Mehta" # insert the pilot's name here e.g., "Tom Hardwicke".
copilotNames <- "Kayla Good" # # insert the co-pilot's name here e.g., "Michael Frank".
pilotTTC <- 240 # insert the pilot's estimated time to complete (in minutes, it is fine to approximate) e.g., 120
copilotTTC <- 45 # insert the co-pilot's estimated time to complete (in minutes, it is fine to approximate) e.g., 120
pilotStartDate <- as.Date("11/01/19", format = "%m/%d/%y") # insert the piloting start date in US format e.g., as.Date("01/25/18", format = "%m/%d/%y")
copilotStartDate <- as.Date("11/09/19", format = "%m/%d/%y") # insert the co-piloting start date in US format e.g., as.Date("01/25/18", format = "%m/%d/%y")
completionDate <- as.Date("11/09/19", format = "%m/%d/%y") # insert the date of final report completion in US format e.g., as.Date("01/25/18", format = "%m/%d/%y")Methods summary:
Two-hundred-two volunteers at a subway station on either the westbound (n = 101) or eastbound (n = 101) platform were asked to rate the subjective distance of another subway station on the line they are traveling, (either coming up or just past). Participants were randomly assigned to conditions where they rate distance of stations either: 2 stops west, 1 stop west, 1 stop east, or 2 stops east. Participants were asked how far away each station felt on a 1-7 scale.
Target outcomes:
For this article you should focus on the findings reported in the results section of Experiment 1.
Specifically, you should attempt to reproduce all descriptive and inferential analyses reported in the text below and associated tables/figures:
Results
We carried out a 2 (orientation: toward, away from) × 4 (station: Spadina, St. George, Bloor-Yonge, Sherbourne) analysis of variance (ANOVA) on closeness ratings, which revealed no main effect of orientation, F < 1, and a main effect of station, F(3, 194) = 24.10, p < .001, ηp2 = .27. This main effect was qualified by the predicted interaction between orientation and station, F(3, 194) = 16.28, p < .001, ηp2 = .20. We decomposed this interaction by the subjective-distance ratings between participants traveling east and west for each of the four subway stations. Westbound participants rated the stations to the west of Bay Street as closer than did eastbound participants; this effect was obtained for both the station one stop to the west (St. George, p < .001, ηp2 = .28) and the station two stops to the west (Spadina, p = .001, ηp2 = .20). The opposite pattern held true for stations to the east of Bay Street. Eastbound participants rated the stations to the east of Bay Street as closer than did westbound participants; this effect was obtained for both the station one stop to the east (Bloor-Yonge, p = .053, ηp2 = .08) and the station two stops to the east (Sherbourne, p < .001, ηp2 = .24). Figure 1 summarizes these results. comparing
Note Make sure to use the original article for additional context and information about any necessary pre-processing steps. Also check for additional supplementary materials that may provide supporting documentation for analysis procedures.
Step 1: Load packages and prepare report object
# load packages
library(tidyverse) # for data munging
library(knitr) # for kable table formating
library(haven) # import and export 'SPSS', 'Stata' and 'SAS' Files
library(readxl) # import excel files
library(ReproReports) # custom reporting functions
library(lsr) # calculate partial eta squared
library(here) # root file path
library(pander)# Prepare report object. This will be updated automatically by the reproCheck function each time values are compared
reportObject <- data.frame(dummyRow = TRUE, reportedValue = NA, obtainedValue = NA, valueType = NA, percentageError = NA, comparisonOutcome = NA, eyeballCheck = NA)Step 2: Load data
df_raw <- read_excel(here("GroupB_6-7-2014/data/S1_Subway.xlsx"))Step 3: Tidy data
Reordering the factor to match the plot in the paper
EAST_STNS <- c("B-Y", "SHER")
WEST_STNS <- c("STG", "SPAD")
df <- df_raw %>%
mutate(STN_NAME = factor(STN_NAME, levels = c("SPAD", "STG", "B-Y", "SHER")))Step 4: Run analysis
Pre-processing
Creating a variable for orientation (whether participants are facing toward or away from the station they are responding about).
df <- df %>%
mutate(
ORIENTATION = case_when(
DIRECTION == "EAST" & STN_NAME %in% EAST_STNS ~ "TOWARD",
DIRECTION == "WEST" & STN_NAME %in% WEST_STNS ~ "TOWARD",
DIRECTION == "WEST" & STN_NAME %in% EAST_STNS ~ "AWAY",
DIRECTION == "EAST" & STN_NAME %in% WEST_STNS ~ "AWAY")
)Descriptive statistics
Grand mean and descriptives
df %>%
summarize(mean = mean(DISTANCE),
sd = sd(DISTANCE),
n = n(),
se = sd/sqrt(n)) %>%
kable| mean | sd | n | se |
|---|---|---|---|
| 2.67 | 1.29 | 202 | 0.091 |
Descriptives by station name
df %>%
group_by(STN_NAME) %>%
summarize(mean = mean(DISTANCE),
sd = sd(DISTANCE),
n = n(),
se = sd/sqrt(n))%>%
kable| STN_NAME | mean | sd | n | se |
|---|---|---|---|---|
| SPAD | 3.16 | 1.16 | 51 | 0.162 |
| STG | 2.22 | 1.08 | 51 | 0.152 |
| B-Y | 1.92 | 1.06 | 49 | 0.151 |
| SHER | 3.37 | 1.28 | 51 | 0.179 |
Descriptives by direction facing
df %>%
group_by(DIRECTION) %>%
summarize(mean = mean(DISTANCE),
sd = sd(DISTANCE),
n = n(),
se = sd/sqrt(n))%>%
kable| DIRECTION | mean | sd | n | se |
|---|---|---|---|---|
| EAST | 2.73 | 1.23 | 101 | 0.123 |
| WEST | 2.61 | 1.36 | 101 | 0.135 |
Descriptives by orientation in relation to target
df %>%
group_by(ORIENTATION) %>%
summarize(mean = mean(DISTANCE),
sd = sd(DISTANCE),
n = n(),
se = sd/sqrt(n))%>%
kable| ORIENTATION | mean | sd | n | se |
|---|---|---|---|---|
| AWAY | 3.15 | 1.32 | 103 | 0.130 |
| TOWARD | 2.18 | 1.07 | 99 | 0.108 |
Reproducing the plot from the paper
df %>%
ggplot(aes(x = STN_NAME, y = DISTANCE, color = DIRECTION)) +
stat_summary(fun.data="mean_se") +
scale_y_continuous(breaks = seq(0,5,.25)) +
theme_minimal()
Standard error bars look too small in the previous plot relative to the plot from the paper (see below), so now checking if maybe they’re supposed to be 95% CI bars.
df %>%
group_by(DIRECTION, STN_NAME) %>%
summarize(
N = n(),
SE = sd(DISTANCE)/sqrt(N),
CI = SE*qt(.025, N-1),
DISTANCE = mean(DISTANCE)
) %>%
ggplot(aes(x = STN_NAME, y = DISTANCE, color = DIRECTION)) +
geom_pointrange(aes(ymin = DISTANCE - CI, ymax = DISTANCE + CI),
position = position_jitter(.1)) +
scale_y_continuous(breaks = seq(0,5,.25)) +
theme_minimal()
These error bars almost looks like they could be right, but it’s hard to tell since the y-axis in the paper does not have tick marks.
Here is the original plot from the paper: 
Inferential Statistics
This is the main model from the study. Using a 2x2 Anova, the authors and I found the predicted interaction between direction and station. This was confusing because the authors mislabeled the their model terms. In the paper the authors state that they ran a “2 (orientation: toward, away from) × 4 (station: Spadina, St. George, Bloor-Yonge, Sherbourne) analysis of variance (ANOVA) on closeness ratings”, when in fact they ran a 2 (direction: east, west) × 4 (station: Spadina, St. George, Bloor-Yonge, Sherbourne) ANOVA on closeness ratings.
s_direction <- summary(fit_direction <- aov(DISTANCE ~ DIRECTION*STN_NAME, data = df))
pander(fit_direction)| Df | Sum Sq | Mean Sq | F value | Pr(>F) | |
|---|---|---|---|---|---|
| DIRECTION | 1 | 0.7129 | 0.7129 | 0.6644 | 0.416 |
| STN_NAME | 3 | 75.16 | 25.05 | 23.35 | 6.011e-13 |
| DIRECTION:STN_NAME | 3 | 52.41 | 17.47 | 16.28 | 1.765e-09 |
| Residuals | 194 | 208.2 | 1.073 | NA | NA |
pander((etasqrd <- etaSquared(fit_direction)))| eta.sq | eta.sq.part | |
|---|---|---|
| DIRECTION | 0.001196 | 0.001929 |
| STN_NAME | 0.2234 | 0.2653 |
| DIRECTION:STN_NAME | 0.1558 | 0.2012 |
When running this ANOVA, the results match for the most part although the F-value for the main effect of station is off by 0.751.
Now I have tried try running the model again using a type 3 ANOVA. This time I get the correct F-value from the paper.
fit_final <- car::Anova(lm(DISTANCE ~ DIRECTION * STN_NAME, data=df,
contrasts=list(DIRECTION=contr.sum, STN_NAME=contr.sum)), type=3)
pander(fit_final)| Sum Sq | Df | F value | Pr(>F) | |
|---|---|---|---|---|
| (Intercept) | 1426 | 1 | 1329 | 9.469e-89 |
| DIRECTION | 0.3405 | 1 | 0.3173 | 0.5739 |
| STN_NAME | 77.58 | 3 | 24.1 | 2.665e-13 |
| DIRECTION:STN_NAME | 52.41 | 3 | 16.28 | 1.765e-09 |
| Residuals | 208.2 | 194 | NA | NA |
Here is the ANOVA that they say they ran (using orientation instead of cardinal direction). The values do not match with the paper.
fit_orientation <- aov(DISTANCE ~ ORIENTATION*STN_NAME, data = df)
pander(etaSquared(fit_orientation))| eta.sq | eta.sq.part | |
|---|---|---|
| ORIENTATION | 0.148 | 0.1931 |
| STN_NAME | 0.233 | 0.2736 |
| ORIENTATION:STN_NAME | 0.008941 | 0.01424 |
pander(fit_orientation)| Df | Sum Sq | Mean Sq | F value | Pr(>F) | |
|---|---|---|---|---|---|
| ORIENTATION | 1 | 46.89 | 46.89 | 43.7 | 3.606e-10 |
| STN_NAME | 3 | 78.38 | 26.13 | 24.35 | 2.035e-13 |
| ORIENTATION:STN_NAME | 3 | 3.008 | 1.003 | 0.9345 | 0.4251 |
| Residuals | 194 | 208.2 | 1.073 | NA | NA |
Here are the pairwise comparisons for each station:
pairwise <- df %>%
split(.$STN_NAME) %>%
map(~ aov(DISTANCE ~ DIRECTION, data = .)) %>%
map(~ list(summary(.), etaSquared(.)))
pairwise %>%
map(~ kable(.[[1]][[1]])) # prints four tables with output of pairwise comparisons (east vs. west) for each station## $SPAD
##
##
## Df Sum Sq Mean Sq F value Pr(>F)
## ---------- --- ------- -------- -------- -------
## DIRECTION 1 13.1 13.10 12 0.001
## Residuals 49 53.6 1.09 NA NA
##
## $STG
##
##
## Df Sum Sq Mean Sq F value Pr(>F)
## ---------- --- ------- -------- -------- -------
## DIRECTION 1 16.3 16.252 18.8 0
## Residuals 49 42.4 0.865 NA NA
##
## $`B-Y`
##
##
## Df Sum Sq Mean Sq F value Pr(>F)
## ---------- --- ------- -------- -------- -------
## DIRECTION 1 4.16 4.16 3.94 0.053
## Residuals 47 49.52 1.05 NA NA
##
## $SHER
##
##
## Df Sum Sq Mean Sq F value Pr(>F)
## ---------- --- ------- -------- -------- -------
## DIRECTION 1 19.3 19.31 15.1 0
## Residuals 49 62.6 1.28 NA NApairwise %>%
map(~ kable(.[[2]])) # prings four tables with eta-squared (and partial eta-squared) values for east vs. west pairwise comparisons for each station## $SPAD
##
##
## eta.sq eta.sq.part
## ---------- ------- ------------
## DIRECTION 0.196 0.196
##
## $STG
##
##
## eta.sq eta.sq.part
## ---------- ------- ------------
## DIRECTION 0.277 0.277
##
## $`B-Y`
##
##
## eta.sq eta.sq.part
## ---------- ------- ------------
## DIRECTION 0.077 0.077
##
## $SHER
##
##
## eta.sq eta.sq.part
## ---------- ------- ------------
## DIRECTION 0.236 0.236Updating reportObject
Main ANOVA: main effect of station
# DF numerator
reportObject <- reproCheck("3", fit_final$Df[3], "df")## [1] "MATCH for df. The reported value (3) and the obtained value (3) differed by 0%. Note that the obtained value was rounded to 0 decimal places to match the reported value."# DF denominator
reportObject <- reproCheck("194", fit_final$Df[5], "df")## [1] "MATCH for df. The reported value (194) and the obtained value (194) differed by 0%. Note that the obtained value was rounded to 0 decimal places to match the reported value."# F value
reportObject <- reproCheck("24.10", fit_final$`F value`[3], "F")## [1] "MATCH for F. The reported value (24.1) and the obtained value (24.1) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."# P value
reportObject <- reproCheck("<.001", fit_final$`Pr(>F)`[3], "p", eyeballCheck = T)## [1] "MATCH for p. Eyeball comparison only."# Eta squared
reportObject <- reproCheck(".27", etasqrd[2,2], "other")## [1] "MATCH for other. The reported value (0.27) and the obtained value (0.27) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."Main ANOVA: interaction direction and station
# DF numerator
reportObject <- reproCheck("3", fit_final$Df[4], "df")## [1] "MATCH for df. The reported value (3) and the obtained value (3) differed by 0%. Note that the obtained value was rounded to 0 decimal places to match the reported value."# DF denominator
reportObject <- reproCheck("194", fit_final$Df[5], "df")## [1] "MATCH for df. The reported value (194) and the obtained value (194) differed by 0%. Note that the obtained value was rounded to 0 decimal places to match the reported value."# F value
reportObject <- reproCheck("16.28", fit_final$`F value`[4], "F")## [1] "MATCH for F. The reported value (16.28) and the obtained value (16.28) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."# P value
reportObject <- reproCheck("<.001", fit_final$`Pr(>F)`[4], "p", eyeballCheck = T)## [1] "MATCH for p. Eyeball comparison only."# Eta squared
reportObject <- reproCheck(".20", etasqrd[3,2], "other")## [1] "MATCH for other. The reported value (0.2) and the obtained value (0.2) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."Pairwise comparisons
# P value west 1 stop (stg)
reportObject <- reproCheck("<.001", pairwise$STG[[1]][[1]]$`Pr(>F)`[1], "p", T)## [1] "MATCH for p. Eyeball comparison only."# eta squared west 1 stop (stg)
reportObject <- reproCheck(".28", pairwise$STG[[2]][2], "other")## [1] "MATCH for other. The reported value (0.28) and the obtained value (0.28) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."# P value west 2 stops (spad)
reportObject <- reproCheck(".001", pairwise$SPAD[[1]][[1]]$`Pr(>F)`[1], "p")## [1] "MATCH for p. The reported value (0.001) and the obtained value (0.001) differed by 0%. Note that the obtained value was rounded to 3 decimal places to match the reported value."# eta squared west 2 stops (spad)
reportObject <- reproCheck(".20", pairwise$SPAD[[2]][2], "other")## [1] "MATCH for other. The reported value (0.2) and the obtained value (0.2) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."# P value east 1 stop (b-y)
reportObject <- reproCheck(".053", pairwise$`B-Y`[[1]][[1]]$`Pr(>F)`[1], "p")## [1] "MATCH for p. The reported value (0.053) and the obtained value (0.053) differed by 0%. Note that the obtained value was rounded to 3 decimal places to match the reported value."# eta squared east 1 stop(b-y)
reportObject <- reproCheck(".08", pairwise$`B-Y`[[2]][2], "other")## [1] "MATCH for other. The reported value (0.08) and the obtained value (0.08) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."# P value east 2 stops (sher)
reportObject <- reproCheck("<.001", pairwise$SHER[[1]][[1]]$`Pr(>F)`[1], "p", T)## [1] "MATCH for p. Eyeball comparison only."# eta squared east 2 stops (sher)
reportObject <- reproCheck(".24", pairwise$SHER[[2]][2], "other")## [1] "MATCH for other. The reported value (0.24) and the obtained value (0.24) differed by 0%. Note that the obtained value was rounded to 2 decimal places to match the reported value."Step 5: Conclusion
I found a significant interaction such that when you are facing towards the target you rate the target as closer than when you are facing away from the target. Additionally, the post-hoc comparison eta-squareds were equivalent to the original paper.
This reproducibility check was a partial success. I managed to get the same model values despite the paper reporting their ANOVA levels incorrectly and the fact that I had to determine which ANOVA type they used through trial and error. However, I am not sure why the standard error bars in the plot don’t match the ones from their paper. It’s possible that they are supposed to be 95% CI bars, however the STG station traveling West CI looks slightly small. It is hard to say for sure since the original paper does not include y-axis ticks on their plot.
Co-pilot note: No errors were found in the pilot’s reproducibility check. I also agree with the pilot’s note about Figure 1. It appears that what were labeled as standard error bars are actually 95% CIs (although it does appear that the bar associated with westbound participants’ distance judgments for St. George station might still be a bit off). My main changes to the report consisted of adding commentary to clarify the function of particular code chunks (specifically, those that were not obvious in what they would produce in the knitted document).
reportObject <- reportObject %>%
filter(dummyRow == FALSE) %>% # remove the dummy row
select(-dummyRow) %>% # remove dummy row designation
mutate(articleID = articleID) %>% # add variables to report
select(articleID, everything()) # make articleID first column
# decide on final outcome
if(any(reportObject$comparisonOutcome %in% c("MAJOR_ERROR", "DECISION_ERROR"))){
finalOutcome <- "Failure"
}else{
finalOutcome <- "Success"
}
# collate report extra details
reportExtras <- data.frame(articleID, pilotNames, copilotNames, pilotTTC, copilotTTC, pilotStartDate, copilotStartDate, completionDate, finalOutcome)
# save report objects
if(reportType == "pilot"){
write_csv(reportObject, "pilotReportDetailed.csv")
write_csv(reportExtras, "pilotReportExtras.csv")
}
if(reportType == "copilot"){
write_csv(reportObject, "copilotReportDetailed.csv")
write_csv(reportExtras, "copilotReportExtras.csv")
}Session information
devtools::session_info()## ─ Session info ──────────────────────────────────────────────────────────
## setting value
## version R version 3.5.3 (2019-03-11)
## os macOS Mojave 10.14.6
## system x86_64, darwin15.6.0
## ui X11
## language (EN)
## collate en_US.UTF-8
## ctype en_US.UTF-8
## tz America/Los_Angeles
## date 2019-11-10
##
## ─ Packages ──────────────────────────────────────────────────────────────
## package * version date lib
## abind 1.4-5 2016-07-21 [1]
## assertthat 0.2.1 2019-03-21 [1]
## backports 1.1.5 2019-10-02 [1]
## broom 0.5.2 2019-04-07 [1]
## callr 3.3.2 2019-09-22 [1]
## car 3.0-3 2019-05-27 [1]
## carData 3.0-2 2018-09-30 [1]
## cellranger 1.1.0 2016-07-27 [1]
## cli 1.1.0 2019-03-19 [1]
## colorspace 1.4-1 2019-03-18 [1]
## crayon 1.3.4 2017-09-16 [1]
## curl 4.2 2019-09-24 [1]
## data.table 1.12.2 2019-04-07 [1]
## desc 1.2.0 2018-05-01 [1]
## devtools 2.2.1 2019-09-24 [1]
## digest 0.6.22 2019-10-21 [1]
## dplyr * 0.8.3 2019-07-04 [1]
## ellipsis 0.3.0 2019-09-20 [1]
## evaluate 0.14 2019-05-28 [1]
## forcats * 0.4.0 2019-02-17 [1]
## foreign 0.8-71 2018-07-20 [1]
## fs 1.3.1 2019-05-06 [1]
## generics 0.0.2 2018-11-29 [1]
## ggplot2 * 3.2.1 2019-08-10 [1]
## glue 1.3.1 2019-03-12 [1]
## gtable 0.3.0 2019-03-25 [1]
## haven * 2.1.1 2019-07-04 [1]
## here * 0.1 2017-05-28 [1]
## highr 0.8 2019-03-20 [1]
## hms 0.5.2 2019-10-30 [1]
## htmltools 0.4.0 2019-10-04 [1]
## httr 1.4.1 2019-08-05 [1]
## jsonlite 1.6 2018-12-07 [1]
## knitr * 1.25 2019-09-18 [1]
## lattice 0.20-38 2018-11-04 [1]
## lazyeval 0.2.2 2019-03-15 [1]
## lifecycle 0.1.0 2019-08-01 [1]
## lsr * 0.5 2015-03-02 [1]
## lubridate 1.7.4 2018-04-11 [1]
## magrittr 1.5 2014-11-22 [1]
## memoise 1.1.0 2017-04-21 [1]
## modelr 0.1.5 2019-08-08 [1]
## munsell 0.5.0 2018-06-12 [1]
## nlme 3.1-137 2018-04-07 [1]
## openxlsx 4.1.0.1 2019-05-28 [1]
## pander * 0.6.3 2018-11-06 [1]
## pillar 1.4.2 2019-06-29 [1]
## pkgbuild 1.0.6 2019-10-09 [1]
## pkgconfig 2.0.3 2019-09-22 [1]
## pkgload 1.0.2 2018-10-29 [1]
## prettyunits 1.0.2 2015-07-13 [1]
## processx 3.4.1 2019-07-18 [1]
## ps 1.3.0 2018-12-21 [1]
## purrr * 0.3.3 2019-10-18 [1]
## R6 2.4.0 2019-02-14 [1]
## Rcpp 1.0.2 2019-07-25 [1]
## readr * 1.3.1 2018-12-21 [1]
## readxl * 1.3.1 2019-03-13 [1]
## remotes 2.1.0 2019-06-24 [1]
## ReproReports * 0.1 2019-11-01 [1]
## rio 0.5.16 2018-11-26 [1]
## rlang 0.4.1 2019-10-24 [1]
## rmarkdown 1.16 2019-10-01 [1]
## rprojroot 1.3-2 2018-01-03 [1]
## rstudioapi 0.10 2019-03-19 [1]
## rvest 0.3.4 2019-05-15 [1]
## scales 1.0.0 2018-08-09 [1]
## sessioninfo 1.1.1 2018-11-05 [1]
## stringi 1.4.3 2019-03-12 [1]
## stringr * 1.4.0 2019-02-10 [1]
## testthat 2.2.1 2019-07-25 [1]
## tibble * 2.1.3 2019-06-06 [1]
## tidyr * 1.0.0 2019-09-11 [1]
## tidyselect 0.2.5 2018-10-11 [1]
## tidyverse * 1.2.1 2017-11-14 [1]
## usethis 1.5.1 2019-07-04 [1]
## vctrs 0.2.0 2019-07-05 [1]
## withr 2.1.2 2018-03-15 [1]
## xfun 0.10 2019-10-01 [1]
## xml2 1.2.2 2019-08-09 [1]
## yaml 2.2.0 2018-07-25 [1]
## zeallot 0.1.0 2018-01-28 [1]
## zip 2.0.2 2019-05-13 [1]
## source
## CRAN (R 3.5.0)
## CRAN (R 3.5.2)
## CRAN (R 3.5.2)
## CRAN (R 3.5.2)
## CRAN (R 3.5.2)
## CRAN (R 3.5.2)
## CRAN (R 3.5.0)
## CRAN (R 3.5.0)
## CRAN (R 3.5.2)
## CRAN (R 3.5.2)
## CRAN (R 3.5.0)
## CRAN (R 3.5.2)
## CRAN (R 3.5.2)
## CRAN (R 3.5.0)
## CRAN (R 3.5.3)
## CRAN (R 3.5.3)
## CRAN (R 3.5.2)
## CRAN (R 3.5.2)
## CRAN (R 3.5.2)
## CRAN (R 3.5.2)
## CRAN (R 3.5.3)
## CRAN (R 3.5.2)
## CRAN (R 3.5.0)
## CRAN (R 3.5.2)
## CRAN (R 3.5.2)
## CRAN (R 3.5.2)
## CRAN (R 3.5.2)
## CRAN (R 3.5.0)
## CRAN (R 3.5.2)
## CRAN (R 3.5.3)
## CRAN (R 3.5.2)
## CRAN (R 3.5.2)
## CRAN (R 3.5.0)
## CRAN (R 3.5.2)
## CRAN (R 3.5.3)
## CRAN (R 3.5.2)
## CRAN (R 3.5.2)
## CRAN (R 3.5.0)
## CRAN (R 3.5.0)
## CRAN (R 3.5.0)
## CRAN (R 3.5.0)
## CRAN (R 3.5.2)
## CRAN (R 3.5.0)
## CRAN (R 3.5.3)
## CRAN (R 3.5.2)
## CRAN (R 3.5.0)
## CRAN (R 3.5.2)
## CRAN (R 3.5.2)
## CRAN (R 3.5.2)
## CRAN (R 3.5.0)
## CRAN (R 3.5.0)
## CRAN (R 3.5.2)
## CRAN (R 3.5.0)
## CRAN (R 3.5.2)
## CRAN (R 3.5.2)
## CRAN (R 3.5.2)
## CRAN (R 3.5.0)
## CRAN (R 3.5.2)
## CRAN (R 3.5.2)
## Github (TomHardwicke/ReproReports@2ec6f60)
## CRAN (R 3.5.0)
## CRAN (R 3.5.3)
## CRAN (R 3.5.2)
## CRAN (R 3.5.0)
## CRAN (R 3.5.2)
## CRAN (R 3.5.2)
## CRAN (R 3.5.0)
## CRAN (R 3.5.0)
## CRAN (R 3.5.2)
## CRAN (R 3.5.2)
## CRAN (R 3.5.2)
## CRAN (R 3.5.2)
## CRAN (R 3.5.2)
## CRAN (R 3.5.0)
## CRAN (R 3.5.0)
## CRAN (R 3.5.2)
## CRAN (R 3.5.2)
## CRAN (R 3.5.0)
## CRAN (R 3.5.2)
## CRAN (R 3.5.2)
## CRAN (R 3.5.0)
## CRAN (R 3.5.0)
## CRAN (R 3.5.2)
##
## [1] /Library/Frameworks/R.framework/Versions/3.5/Resources/library