Word Order and Spatial Postpositions in Japanese: Analyses on data collected and plans for ongoing research

KEY STUDY INFORMATION

Background: Successful language learning involves acquiring abstract structures that operate across particular linguistic items. Statistical learning approaches suggest that such generalization is driven by the linguistic input, with the prediction that generalization should be more likely when learners are exposed to more varied instances, since this allows linguistic structures to be disassociated from trained instances (Bybee 1995, LCP; Ramscar et al. 2010, Cognitive Science; Wonnacott et al. 2012, JML). The current work explores this hypothesis in a language training experiment with child learners.

Methods: 41 7-8 year olds were recruited and randomly assigned to one of two between-participant conditions. Children were all English speakers but some were bilingual, although we did not use data from children who spoke languages with post-positions. Our final data set had: high-variability conditon (N=21); low-variability condition (N=20).

Children were exposed to novel sentences in an unfamiliar language (Japanese) via a computerized game. All exposure sentences involved: (i) one of the case markers no ue ni (above) or no shita ni (below) (ii) the object marker o (iii) the verb oku (put). Half the sentences were of the form noun1-o noun2- no ue ni oku and half of the form noun1-o no ue ni noun2- no shit a ni oku . All nouns were all English cognates (avoiding lengthy vocabulary training).

Sentences were pre-recorded by a native speaker of Japanese. They served as instructions in a game where children were required to move pictures around a grid using a mouse (see figure 1). In each trial, they heard a Japanese instruction and responded by moving the pictures. If they gave an incorrect response, they would see a demonstration of the correct response (the correct picture would move above/below the other picture in the grid); this process was repeated until the child made the correct response for that trial. No additional explicit instruction was given about the nature of the sentence structures or the meaning of the spatial case-marker (post-positions). Critically the set of sentences used in training differed across two (between-participant) conditions: (1) high variability input: participants were trained a set of 28-above and 28-below sentences, each played once; (2) low variability condition: participants were trained on a restricted input set comprising just 2-above and 2-below sentences, each repeated 14 times (number of nouns (8) and total exposures matched across conditions).

Following training (over two 20-30 minute sessions) there was a genralization test in which children were tested on their comprehension of untrained sentences containing untrained nouns (more English cognates) in order to assess generalization. The test mirrored training except it was without feedback. The test was identical across participants.

In addition to the training and generalization test, there were the following stages in the experiment: (i) experimenter demonstration - here the experimenter demonstrated how the game worked in four trials (where s/he always showed a correct response first time; data are collected from this so we can check for experimenter error) (ii) vocabulary familiarization tests:-There were two of these, the first was administered immediately before the first traning session and the second immediately before the generalization test. The purpose was to familiarize participants with the vocabulary items that would be used in training (first test) and test (second test). Since cognates are used children are predicted to score well on this. Any child scoring less than 60% across the two tests should be excluded

Predictions and statistical approach

The key predictions concern variability: we predicted that: although (i) there wouldbe higher performance in LV condition than HV condition during training (due to repeatedly testing the same items), (ii) there would be higher performance in the HV than the LV condition in the generalization test (due to variability dissociating the structures from particular instances).

Other (less key) predictions were that (iii) there would be greater performance with no ue ni (above) than no shita ni (below) sentences (following what we had seen in a previous project with adults), which could show up in both training and testing (iv). In training, there would be greater performance in session2 than session1.

In the analyses below, we look at the training data (first attempt only) and testing data separately. Trials where they made an “illegal” response were excluded (less than 10% of the data; see below) so that there were four possible options (25% chance of correct response). For each of training and testing analsyes, we conduct both frequentist and BF statistics. For frequentist, we use glmer models predicting whethr they made a correct response, with the predictor factors as condition (hv,lv), postp (noueni(above), shitani(below)) and - for training only - phase (session1, session2). Although we don’t have clear predictions about interactions, all of these are also included in each model.

For the BF analsyes, which we consider key, we use the method advocated by Dienes (2008). We aimed to have a BF for every p (although key BFs are where we have predictions). Dienes method requires estimates and standard errors for the effect in question, which we take from the correspodning coefficient in the lmer. The method also requires us to specifiy the “theory” (h1). Throughought, we use a half normal for the main effects since here we have a clear prediction in each case (see above). For the interactions, we use two-tailed normal distribution. In each case, we set the mean of the thoery to be zero and the sd to be equal to either an an estimate of the predicted effect size OR HALF of the size of the maximum effect we suspect (since 2sd is around the maximum in the normal). (Note: we depart from Diennes 2008 in not using a uniform here, but believe the normal to be more appropriate in assuming smaller effects are more likely)

Ideally, estimates of H1 are based on prior data here. Since we don’t have any available, we generally obtain these estimates useing the scale/ or from elsewhere in the data.

For example, for the intercept for training, we assume a maximum effect of: logodds(.95) - logodds(.25), and set the sd to be equal to half of that (note we cannot set logodds(1) so use .95 as a reasonable estimate of perfect performance).

For main effects, assuming there is evidence for that the intercept is above chance (which there is in each case here), we set the sd to be the difference between the intercept and chance performance. The logic is that if all the learning happened in (e.g.) one condition (so other condition at chance), the value for the main effect of condition would be equal to 2x(intercept-chance), we set sd to be equal to half of this value.

For two way interactions, where we have evidence for learning of one of the contained main effects, we set sd to be equal to the estimate of the (main effect - chance). The logic is that if, for example, of the the effect of condition occured for one level of post-position, then the value of condition by post-position would be equal to 2xcondition, so we set sd to be equal to half of this value. Where we don’t have evidence for learning for one of the contained main effects, assuming there is evidence for that the intercept is above chance (which there is in each case here), we can set can set the value to be equal to 2x(intercept-chance), with the logic that if all the learning happened in (e.g.) condition for one of the post-position (other cells all at chance), the value of the interaction would be equal to 4x(intercept-chance), so we set sd to half of this value). For three way interactions, using similar logic to that above, if we have evidence of one of the contained two way interactions, we set sd to be equal to the estimate of that two way interactions; otherwise if we have evidence of learning of one of the contained main effects, we set sd to be equal to the 2* that effect; otherwise assuming there is evidence for that the intercept is above chance (which there is in each case here), we can set it to be 4x(intercept-chance).

In some cases, we have a more independent value with which to inform the h1, and wherever possible we use these. For example, where we look to see if the intercept is above chance in both conditions, if we have clear evidence that the intercept in conditionA is above chance, if we want to calcuate a BF for the intercept for conditionB, we can set sd to tbe the intercept-chance in the other condition.

In one case (for the effect of post-position) we use a value from training data to inform h1 for the equivalent effect in the test data.

In addition to the first analysis which looks at factors affecting the probability of producing a correct response out of total legal responses (chance=25%), for test data only, in order to further probe what aspect of learning is affected by variability, we did two follow up analyses (1) WORD ORDER: exploring whether they moved the first namedpicture (same data set as first analsyes, but for DV, trials where they moved the picture corresponding to the first noun coded as 1, those where they moved the picture corresponding to the second noun coded as 0) (2) SPATIAL TERMS SEMANTICS: exploring whether, given that they moved the correct picture, they moved it to the correct location (data set is the subset of legal responses where they moved the correct picture, for DV, trials where they moved the picture correctly (i.e. below for shitani, above for nounei) coded as 1, those where they moved the picture incorrectly (i.e. above for shitani, below for nounei) coded as 0.

We predicted that if there was an effect of variability condition, in either/both of these analyses, it would be in the direction of HV>LV. However we did not have a clear prediction as to whether this difference would show up in terms of their understanding of WORD ORDER (knowing to move the first noun) or knowledge of SPATIAL TERMS SEMANTICS (knowing that X shitani meant “below X” and X noueni meant “above X”)

DATA ANALYSES

Load Packages

rm(list=ls())
#library(languageR)
suppressMessages(library(lattice))
suppressMessages(library(plotrix))
suppressMessages(library(ggplot2))  #for graphs
suppressMessages(library(psych))
suppressMessages(library(plyr))
suppressMessages(library(doBy)) #to reshape
suppressMessages(library(cowplot))
suppressMessages(library(lme4))

suppressMessages(library(car))
suppressMessages(library(knitr))

Load Helper Functions

SummarySE

This function can be found on the website “Cookbook for R”.

http://www.cookbook-r.com/Manipulating_data/Summarizing_data/

It summarizes data, giving count, mean, standard deviation, standard error of the mean, and confidence interval (default 95%).

• data: a data frame
• measurevar: the name of a column that contains the variable to be summarized
• groupvars: a vector containing names of columns that contain grouping variables
• na.rm: a boolean that indicates whether to ignore NA’s
• conf.interval: the percent range of the confidence interval (default is 95%)

summarySE <- function(data=NULL, measurevar, groupvars=NULL, na.rm=FALSE,
                      conf.interval=.95, .drop=TRUE) {
    require(plyr)

    # New version of length which can handle NA's: if na.rm==T, don't count them
    length2 <- function (x, na.rm=FALSE) {
        if (na.rm) sum(!is.na(x))
        else       length(x)
    }

    # This does the summary. For each group's data frame, return a vector with
    # N, mean, and sd
    datac <- ddply(data, groupvars, .drop=.drop,
      .fun = function(xx, col) {
        c(N    = length2(xx[[col]], na.rm=na.rm),
          mean = mean   (xx[[col]], na.rm=na.rm),
          sd   = sd     (xx[[col]], na.rm=na.rm)
        )
      },
      measurevar
    )

    # Rename the "mean" column    
    datac <- rename(datac, c("mean" = measurevar))

    datac$se <- datac$sd / sqrt(datac$N)  # Calculate standard error of the mean

    # Confidence interval multiplier for standard error
    # Calculate t-statistic for confidence interval: 
    # e.g., if conf.interval is .95, use .975 (above/below), and use df=N-1
    ciMult <- qt(conf.interval/2 + .5, datac$N-1)
    datac$ci <- datac$se * ciMult

    return(datac)
}

SummarySEwithin

This function can be found on the website “Cookbook for R”.

http://www.cookbook-r.com/Graphs/Plotting_means_and_error_bars_(ggplot2)/#Helper functions

From that website:

Summarizes data, handling within-subjects variables by removing inter-subject variability. It will still work if there are no within-S variables. Gives count, un-normed mean, normed mean (with same between-group mean), standard deviation, standard error of the mean, and confidence interval. If there are within-subject variables, calculate adjusted values using method from Morey (2008).

• data: a data frame
• measurevar: the name of a column that contains the variable to be summarized
• betweenvars: a vector containing names of columns that are between-subjects variables
• withinvars: a vector containing names of columns that are within-subjects variables
• idvar: the name of a column that identifies each subject (or matched subjects)
• na.rm: a boolean that indicates whether to ignore NA’s
• conf.interval: the percent range of the confidence interval (default is 95%)

summarySEwithin <- function(data=NULL, measurevar, betweenvars=NULL, withinvars=NULL,
                            idvar=NULL, na.rm=FALSE, conf.interval=.95, .drop=TRUE) {

  # Ensure that the betweenvars and withinvars are factors
  factorvars <- vapply(data[, c(betweenvars, withinvars), drop=FALSE],
    FUN=is.factor, FUN.VALUE=logical(1))

  if (!all(factorvars)) {
    nonfactorvars <- names(factorvars)[!factorvars]
    message("Automatically converting the following non-factors to factors: ",
            paste(nonfactorvars, collapse = ", "))
    data[nonfactorvars] <- lapply(data[nonfactorvars], factor)
  }

  # Get the means from the un-normed data
  datac <- summarySE(data, measurevar, groupvars=c(betweenvars, withinvars),
                     na.rm=na.rm, conf.interval=conf.interval, .drop=.drop)

  # Drop all the unused columns (these will be calculated with normed data)
  datac$sd <- NULL
  datac$se <- NULL
  datac$ci <- NULL

  # Norm each subject's data
  ndata <- normDataWithin(data, idvar, measurevar, betweenvars, na.rm, .drop=.drop)

  # This is the name of the new column
  measurevar_n <- paste(measurevar, "_norm", sep="")

  # Collapse the normed data - now we can treat between and within vars the same
  ndatac <- summarySE(ndata, measurevar_n, groupvars=c(betweenvars, withinvars),
                      na.rm=na.rm, conf.interval=conf.interval, .drop=.drop)

  # Apply correction from Morey (2008) to the standard error and confidence interval
  #  Get the product of the number of conditions of within-S variables
  nWithinGroups    <- prod(vapply(ndatac[,withinvars, drop=FALSE], FUN=nlevels,
                           FUN.VALUE=numeric(1)))
  correctionFactor <- sqrt( nWithinGroups / (nWithinGroups-1) )

  # Apply the correction factor
  ndatac$sd <- ndatac$sd * correctionFactor
  ndatac$se <- ndatac$se * correctionFactor
  ndatac$ci <- ndatac$ci * correctionFactor

  # Combine the un-normed means with the normed results
  merge(datac, ndatac)
}

normDataWithin

This function is used by the SummarySEWithin fucntion above. It can be found on the website “Cookbook for R”.

http://www.cookbook-r.com/Graphs/Plotting_means_and_error_bars_(ggplot2)/#Helper functions

From that website: Norms the data within specified groups in a data frame; it normalizes each subject (identified by idvar) so that they have the same mean, within each group specified by betweenvars.

• data: a data frame
• idvar: the name of a column that identifies each subject (or matched subjects)
• measurevar: the name of a column that contains the variable to be summarized
• betweenvars: a vector containing names of columns that are between-subjects variables
• na.rm: a boolean that indicates whether to ignore NA’s

normDataWithin <- function(data=NULL, idvar, measurevar, betweenvars=NULL,
                           na.rm=FALSE, .drop=TRUE) {
    #library(plyr)

    # Measure var on left, idvar + between vars on right of formula.
    data.subjMean <- ddply(data, c(idvar, betweenvars), .drop=.drop,
     .fun = function(xx, col, na.rm) {
        c(subjMean = mean(xx[,col], na.rm=na.rm))
      },
      measurevar,
      na.rm
    )

    # Put the subject means with original data
    data <- merge(data, data.subjMean)

    # Get the normalized data in a new column
    measureNormedVar <- paste(measurevar, "_norm", sep="")
    data[,measureNormedVar] <- data[,measurevar] - data[,"subjMean"] +
                               mean(data[,measurevar], na.rm=na.rm)

    # Remove this subject mean column
    data$subjMean <- NULL

    return(data)
}

myCenter

This function ouputs the centered values of an variable, which can be a numeric variable, a factor, or a data frame. It was taken from Florian Jaegers blog

https://hlplab.wordpress.com/2009/04/27/centering-several-variables/.

From his blog:

• If the input is a numeric variable, the output is the centered variable
• If the input is a factor, the output is a numeric variable with centered factor level values. That is, the factor’s levels are converted into numerical values in their inherent order (if not specified otherwise, R defaults to alphanumerical order). More specifically, this centers any binary factor so that the value below 0 will be the 1st level of the original factor, and the value above 0 will be the 2nd level.
• If the input is a data frame or matrix, the output is a new matrix of the same dimension and with the centered values and column names that correspond to the colnames() of the input preceded by “c” (e.g. “Variable1” will be “cVariable1”).

myCenter= function(x) {
  if (is.numeric(x)) { return(x - mean(x, na.rm=T)) }
    if (is.factor(x)) {
        x= as.numeric(x)
        return(x - mean(x, na.rm=T))
    }
    if (is.data.frame(x) || is.matrix(x)) {
        m= matrix(nrow=nrow(x), ncol=ncol(x))
        colnames(m)= paste("c", colnames(x), sep="")
    
        for (i in 1:ncol(x)) {
        
            m[,i]= myCenter(x[,i])
        }
        return(as.data.frame(m))
    }
}

lizCenter

This function provides a wrapper around myCenter allowing you to center a specific list of variables from a dataframe.

• x: data frame
• listfname: a list of the variables to be centered (e.g. list(variable1,variable2))

The output is a copy of the data frame with a column (always a numeric variable) added for each of the centered variables. These columns are labelled with the each column’s previous name, but with “.ct” appended (e.g., “variable1” will become “variable1.ct”).

lizCenter= function(x, listfname) 
{
    for (i in 1:length(listfname)) 
    {
        fname = as.character(listfname[i])
        x[paste(fname,".ct", sep="")] = myCenter(x[fname])
    }
        
    return(x)
}

adjust_intercept_model

To be used for glmer models where the chance level isn’t 50% and so intercept (or intercepts) need to be adjusted.

Takes as input an glmer model and adjusts the estimate, z and p values for any coefficients which are intercepts. (Default is to assume just the one intercept which is so labelled, but can give a list for models where have fitted mutliple intercepts )

adjust_intercept_model<- function(model, chance, intercept_list = c("(Intercept)"))
{
  summary = summary(model)$coefficients
  
  for (i in 1:length(intercept_list)){
    summary[intercept_list[i], "Estimate"] = summary[intercept_list[i], "Estimate"]- chance
    summary[intercept_list[i], "z value"] =  summary[intercept_list[i], "Estimate"]/summary[intercept_list[i], "Std. Error"]
    summary[intercept_list[i], "Pr(>|z|)"] =  p =2*pnorm(-abs(summary[intercept_list[i], "z value"])) }
  
  
   summary[intercept_list[i], "Estimate"]
  
  
  return(summary)
  
  
}

Bf

This function is equivalent to the Dienes (2008) calculator which can be found here: http://www.lifesci.sussex.ac.uk/home/Zoltan_Dienes/inference/Bayes.htm.

The code was provided by Baguely and Kayne (2010) and can be found here: http://www.academia.edu/427288/Review_of_Understanding_psychology_as_a_science_An_introduction_to_scientific_and_statistical_inference

Bf<-function(sd, obtained, uniform, lower=0, upper=1, meanoftheory=0,sdtheory=1, tail=2){
 area <- 0
 if(identical(uniform, 1)){
  theta <- lower
  range <- upper - lower
  incr <- range / 2000
  for (A in -1000:1000){
     theta <- theta + incr
     dist_theta <- 1 / range
     height <- dist_theta * dnorm(obtained, theta, sd)
     area <- area + height * incr
  }
 }else
   {theta <- meanoftheory - 5 * sdtheory
    incr <- sdtheory / 200
    for (A in -1000:1000){
      theta <- theta + incr
      dist_theta <- dnorm(theta, meanoftheory, sdtheory)
      if(identical(tail, 1)){
        if (theta <= 0){
          dist_theta <- 0
        } else {
          dist_theta <- dist_theta * 2
        }
      }
      height <- dist_theta * dnorm(obtained, theta, sd)
      area <- area + height * incr
    }
 }
 LikelihoodTheory <- area
 Likelihoodnull <- dnorm(obtained, 0, sd)
 BayesFactor <- LikelihoodTheory / Likelihoodnull
 ret <- list("LikelihoodTheory" = LikelihoodTheory,"Likelihoodnull" = Likelihoodnull, "BayesFactor" = BayesFactor)
 ret
} 

Bf_model

This works by calling the Bf function above. Summary should be the coefficients summary from an lmer/gmer model (obtained via summary(model)$coefficients)

For each coefficient, the model uses the estimate as the obtained mean and the Std.Error as the measure of standard error.

The function allows for positive and negtive h1 values. If negative, since the BF function requires a positive value for sdtheory, both sdtheory and obtained are multiplied by -1.

Bf_model<-function(coeff_summary, coeff_list, h1_list,h1_motivation, tail_list)
  {
    Bfs = vector('double')
    estimates = vector('double')
    sdtheorys = vector('double')

    #summary = summary(model)$coefficients
    
  #  i=1
    for (i in 1:length(coeff_list)){
    
      sd_error = coeff_summary[coeff_list[i], "Std. Error"]
      obtained = coeff_summary[coeff_list[i], "Estimate"]
      stdtheory = h1_list[i]
           
          
          if(h1_list[i] <0) {
                stdtheory = h1_list[i]*-1
                obtained = (coeff_summary[coeff_list[i], "Estimate"]*-1)}
      
            
      Bfs[i] = Bf( sd_error, obtained, uniform = 0, meanoftheory = 0, sdtheory=stdtheory , tail = tail_list[i])$BayesFactor
      estimates[i] = obtained
      sdtheorys[i] = stdtheory
      
    }
    
    
    df = data.frame(cbind(coeff_list, round(estimates,3), round(sdtheorys,3),  h1_motivation, tail_list, round(Bfs,3)    )   )
    colnames(df) = c("coefficient", "estimate", "sdtheory",   "h1 motivation", "tail", "Bf" )
    return(df)
   
    
  }

Load Data, set factors, exclude participants

JapData.df <- read.csv("Jap_child_data.csv", header=TRUE)
JapData.df$condition = factor(JapData.df$condition)
JapData.df$phase = factor(JapData.df$phase)
JapData.df$participant= factor(JapData.df$participant)
JapData.df$postp= factor(JapData.df$postp)

Exclusions: Particpants are excluded because either (1) they didn’t get the full or correct training (due to absense, computer/experimenter error) (2) they spoke a language which used post-positions

JapData.df = subset(JapData.df,participant !="7")
# only completed session 1
JapData.df = subset(JapData.df, participant !="14")
# experimenter error, accidentally run in different conditions on differnt days 
JapData.df = subset(JapData.df, participant !="23")
# didn't do full set of training trials due to computer error (7 trials missing)
JapData.df = subset(JapData.df, participant !="38")
# only completed session1
JapData.df = subset(JapData.df, participant !="48")
# only completed session1
#JapData.df = subset(JapData.df, participant !="42")
# only got 9/32 training trials in s1 due to computer error - keeping it in, just recording error

JapData.df = subset(JapData.df, participant !="3")
# spoke language with postpositions
JapData.df = subset(JapData.df, participant !="8")
# spoke language with postpositions
JapData.df = subset(JapData.df, participant !="16")
# spoke language with postpositions
JapData.df = subset(JapData.df, participant !="27")
# spoke language with postpositions

JapData.df = droplevels(JapData.df)


levels(JapData.df$participant)

##  [1] "1"  "2"  "4"  "5"  "6"  "9"  "10" "11" "12" "13" "15" "17" "18" "19"
## [15] "20" "21" "22" "24" "25" "26" "28" "29" "30" "31" "32" "33" "34" "35"
## [29] "36" "37" "39" "40" "41" "42" "43" "44" "45" "46" "47" "49" "50"

Look at mising data per participant

a=count(droplevels(subset(JapData.df, attempt = 1 | attempt == "" )), c( "phase", "attempt", "participant"))

kable(table(a$freq, a$phase))

	demo	testing	training_1	training_2	vocab1	vocab2
1	0	0	15	10	0	0
2	0	0	5	5	0	0
3	0	0	2	6	0	0
4	41	0	2	1	0	0
5	0	0	0	3	0	0
6	0	0	1	1	0	0
7	0	0	3	1	3	0
8	0	0	3	0	38	0
9	0	0	2	1	0	0
10	0	0	0	1	0	41
11	0	0	1	1	0	0
12	0	0	1	4	0	0
13	0	0	3	2	0	0
14	0	0	4	2	0	0
15	0	0	2	1	0	0
16	0	0	2	2	0	0
17	0	0	2	6	0	0
18	0	0	4	4	0	0
19	0	0	2	2	0	0
20	0	41	3	1	0	0
21	0	0	2	0	0	0
22	0	0	4	0	0	0
23	0	0	2	1	0	0
24	0	0	0	39	0	0
32	0	0	40	0	0	0
34	0	0	1	0	0	0

vocab1: three participants 1,5,41 didn’t see one of the nouns training1: one participant (24) had two extra trials [these are the first two which are played 1,2,1,2 - computer error but minor so leave in]

Demo Phase Means

This part of the experiment was performed by the experimenter. We check here in case s/he made an error.

demo.df = droplevels(subset(JapData.df, phase =="demo"))
aggregate(data=demo.df,correct~ condition + phase, mean, na.rm=T)

##          condition phase correct
## 1 high_variability  demo       1
## 2  low_variability  demo       1

No errors.

Vocab Phase Means

We look at this data to make sure that participants are able to identify the vocabulary with good accuracy:

vocab.df = droplevels(subset(JapData.df, phase =="vocab1" |phase =="vocab2"))
min(aggregate(data=vocab.df,correct~ condition + participant, mean, na.rm=T)$correct)

## [1] 0.7222222

aggregate(data=vocab.df,correct~ condition + phase, mean, na.rm=T)

##          condition  phase   correct
## 1 high_variability vocab1 0.9454545
## 2  low_variability vocab1 0.8875000
## 3 high_variability vocab2 0.9409091
## 4  low_variability vocab2 0.9473684

Performance very high many particpants at ceiling.

No participant scored below 60% correct across the two tests so no need to exclude.

Training and Testing: set up and plot

set up

Set up data frame with appropriate data. Only include first attempt for training (since they keep going till they get it correct and that data has also been recorded). Check no missing responses.

traintest.df = droplevels(subset(JapData.df,  phase =="testing" |phase =="training_1" |phase =="training_2"))
traintest.df = droplevels(subset(traintest.df,  attempt == 1))
traintest.df = droplevels(subset(JapData.df,  phase =="testing" |phase =="training_1" |phase =="training_2"))
traintest.df = droplevels(subset(traintest.df,  attempt == 1))
sum(is.na(traintest.df$correct))

## [1] 0

No missing data.

Aggregate and get by participant confidence interavals (within-participant adjusted):

traintest.df = droplevels(subset(JapData.df,  phase =="testing" |phase =="training_1" |phase =="training_2"))
traintest.df = droplevels(subset(traintest.df,  attempt == 1))

                           
aggregated.traintest.df  = aggregate(correct ~ participant + postp + condition + phase, traintest.df, FUN=mean)

aggregated.traintest.df.2 = summarySEwithin(aggregated.traintest.df , measurevar= "correct", betweenvars = "condition", withinvar = c("postp", "phase"), idvar= "participant", na.rm = FALSE, conf.interval = .95)

kable(aggregated.traintest.df.2, dig = 3)

condition	postp	phase	N	correct	correct_norm	sd	se	ci
high_variability	noshitani	testing	21	0.476	0.512	0.194	0.042	0.088
high_variability	noshitani	training_1	21	0.372	0.408	0.118	0.026	0.054
high_variability	noshitani	training_2	21	0.395	0.431	0.163	0.036	0.074
high_variability	noueni	testing	21	0.476	0.512	0.174	0.038	0.079
high_variability	noueni	training_1	21	0.503	0.539	0.245	0.054	0.112
high_variability	noueni	training_2	21	0.476	0.512	0.123	0.027	0.056
low_variability	noshitani	testing	20	0.295	0.257	0.287	0.064	0.134
low_variability	noshitani	training_1	20	0.534	0.496	0.142	0.032	0.066
low_variability	noshitani	training_2	20	0.621	0.583	0.190	0.043	0.089
low_variability	noueni	testing	20	0.335	0.297	0.210	0.047	0.098
low_variability	noueni	training_1	20	0.689	0.651	0.171	0.038	0.080
low_variability	noueni	training_2	20	0.671	0.633	0.234	0.052	0.110

plot:

mytheme<-theme(axis.line = element_line(colour = "black"),
               text = element_text(size=12),
               panel.background = element_blank(),
               panel.border = element_blank(),
               panel.grid.major = element_blank(),
               panel.grid.minor = element_blank(),
               axis.ticks.x = element_blank(),
               strip.text.x = element_text(size=12, face="bold"),
               strip.background = element_rect(fill="white", colour =
                                                 "white"))
p = ggplot(aggregated.traintest.df.2, aes( x = condition, y = correct, fill = postp))
barchart= p + stat_summary(fun.y = mean, geom = "bar", position="dodge", colour="black", size=.3) + stat_summary(fun.data = mean_cl_boot, geom = "errorbar", width=.4, position=position_dodge(.9)) + xlab("") + ylab("proportion correct") + scale_x_discrete(labels=c("1" = "Low variability", "2" = "High variability"), expand=c(0, 0.6)) + scale_fill_manual(values=c("dodgerblue3", "orchid2"), name="Spatial Postposition", labels=c("noshitani", "noueni")) + scale_y_continuous(limits=c(0,1.0), expand = c(0, 0)) + mytheme  + theme(plot.title = element_text(hjust = -0.1, vjust=2.12, size=11))
barchart = barchart + geom_errorbar(aes(ymin=correct-ci, ymax= correct+ci), width=.4, position = position_dodge(.9))
barchart= barchart + facet_wrap("phase")
barchart= barchart+ scale_x_discrete(labels= c("low_variability" = "LV", "high_variability"= "HV"))

## Scale for 'x' is already present. Adding another scale for 'x', which
## will replace the existing scale.

print(barchart)

## Warning: Removed 12 rows containing missing values (geom_errorbar).

ggsave(file="Facet_barchart.jpeg", width=7, height=7, dpi=100)

## Warning: Removed 12 rows containing missing values (geom_errorbar).

Set up extra columns in training and testing data needed for analsyes

The experiment output coded which image they were supposed to move (target_image); which they actually moved (moved_image); the number of the square they were suposed to move the grid to (target_grid, values 1->6); the number of the square they actually moved the grid to (chosen_grid, values 1->6).

In order to look at different error types, we add a column which says whetehr they moved the correct image, and recode each of the the target_grid and moved_grid in terms of row and column co-ordinates.

traintest.df$moved_correct_image[traintest.df$target_image== traintest.df$moved_image] <- 1 
traintest.df$moved_correct_image[traintest.df$target_image!= traintest.df$moved_image] <- 0 

traintest.df$target_grid_row = recode(traintest.df$target_grid, "1:2=1;3:4=2;5:6=3; else=NA")
traintest.df$target_grid_column = recode(traintest.df$target_grid, "c(1,3,5)=1;c(2,4,6)=2; else=NA")
traintest.df$chosen_grid_row = recode(traintest.df$chosen_grid, "1:2=1;3:4=2;5:6=3; else=NA")
traintest.df$chosen_grid_column = recode(traintest.df$chosen_grid, "c(1,3,5)=1;c(2,4,6)=2; else=NA")

traintest.df$row_correct[traintest.df$target_grid_row == traintest.df$chosen_grid_row] <- 1 
traintest.df$row_correct[traintest.df$target_grid_row != traintest.df$chosen_grid_row] <- 0
traintest.df$column_correct[traintest.df$target_grid_column == traintest.df$chosen_grid_column] <- 1 
traintest.df$column_correct[traintest.df$target_grid_column != traintest.df$chosen_grid_column] <- 0

## note - if both values are NA, NA == NA returns true
traintest.df$row_correct[is.na(traintest.df$chosen_grid)]= NA
traintest.df$column_correct[is.na(traintest.df$chosen_grid)]= NA

However note there was a recording error and there are trials where the program recorded “na” for chosen_grid. For these trials, the participants in fact moved one picture into the same square as the other (which should NOT have been possible in the program)

sum(is.na(traintest.df$moved_image))

## [1] 0

sum(is.na(traintest.df$chosen_grid))

## [1] 22

There are 22 of these - later in the code, in the below, these are recorded as “illegal” responses

Code response type (gives error types).

The codes are:

l1: legal final board (i.e. one picture directly above/below the other) where they moved the correct picture to correct location (i.e. completely correct). l2: legal final board where they moved the correct image to the incorrect location (above when it should be below, or vice versa) l3a:legal final board where they moved the wrong picture and it ended up to a square with the correct row and incorrrect column (means moved it above when should be above/ below when should be below) l3b:legal final board where they moved the wrong picture and it ended up to a square with the incorrect row and incorrrect column (means moved it above when should be below/ below when should be above)

i1a: illegal final board, the target has been moved to the correct row and incorrect column (it has been moved up/down appropriately, but on a diagonal) i1b: illegal final board, the target has been moved to the incorrect row and incorrect column (it has been moved up/down inappropriately, and is on a diagonal) i1c: illegal final board, wrong picture been moved to where the target should be (so has been moved immediate up/down, in direction that would be appropriate for the target, and thus ends up on diagonal to target)

i1d: illegal final board, wrong picture been moved to where the target should be (so has been moved immediate up/down, in direction that would be appropriate for the target, and thus ends up on diagonal to target)

i2a: move target into same square as the other picture (so they end up on top of each other) i2b: move wrong picture into same square as the target (so they end up on top of each other)

traintest.df$response_type = NULL
traintest.df$response_type[traintest.df$correct==1] = "l1"

traintest.df$response_type[(traintest.df$moved_correct_image==1) & (traintest.df$row_correct==0)  * (traintest.df$column_correct==1)]  = "l2"
traintest.df$response_type[(traintest.df$moved_correct_image==0) & (traintest.df$row_correct==1)  * (traintest.df$column_correct==0)]  = "l3a"
traintest.df$response_type[(traintest.df$moved_correct_image==0) & (traintest.df$row_correct==0)  * (traintest.df$column_correct==0)]  = "l3b"

traintest.df$response_type[(traintest.df$moved_correct_image==1) & (traintest.df$row_correct==1)  * (traintest.df$column_correct==0)]  = "ia"
traintest.df$response_type[(traintest.df$moved_correct_image==1) & (traintest.df$row_correct==0)  * (traintest.df$column_correct==0)]  = "ib"
traintest.df$response_type[(traintest.df$moved_correct_image==0) & (traintest.df$row_correct==1)  * (traintest.df$column_correct==1)]  = "ic"
traintest.df$response_type[(traintest.df$moved_correct_image==0) & (traintest.df$row_correct==0)  * (traintest.df$column_correct==1)]  = "id"

traintest.df$response_type[(is.na(traintest.df$chosen_grid)) &  (traintest.df$moved_correct_image==1)] = "i2a"
traintest.df$response_type[(is.na(traintest.df$chosen_grid)) &  (traintest.df$moved_correct_image==0)] = "i2b"

traintest.df$response_type <- factor(traintest.df$response_type, levels = c("l1", "l2", "l3a", "l3b", "ia", "ib", "ic" ,"id"))

Statistical Analsyes: Training

Check proportions of illegal trials

train.df = subset(traintest.df, phase != "testing")
train.df$session = train.df$phase

train.df$legal = substr(train.df$response_type,1,1)
train.df$legal = recode(substr(train.df$response_type,1,1), "'l'=1;else=0")

round(tapply(train.df$legal, train.df$condition, mean, na.rm=T),2)

## high_variability  low_variability 
##             0.98             0.91

Very few “illegal” moves. Makes sense to remove these from the analyses (makes chance = .25). Both conditions near ceiling so little data will be removed in each case.

Analsysis predicting likliehood of correct response (illegal responses excluded)

main glmer

Run an lmer predicting proportion correct (chance = .25)

Note that each effect has two levels. They are coded such that: a positive effect of condition means LV> HV; positive effect of session means session2 > session1; positive effect of postp means above> below (these are the directions of predictions -see above).

Note - in the output printed - a function has been used to adjust estimate of the intercept (it is set to be the original intercept - chance), as well as the z and p values

# get appropriate df and center factors
train.df.legal = droplevels(subset(train.df, legal == 1))
train.df.legal = lizCenter(train.df.legal, list("condition", "postp", "session"))
# run model

train.lmer = glmer(correct ~ condition.ct*postp.ct * session.ct + (postp.ct*session.ct|participant), family = binomial, data = train.df.legal, control=glmerControl(optimizer = "bobyqa"))
#round(summary(train.lmer)$coeff,4)

adjusted.coeffs.train.lmer = adjust_intercept_model(train.lmer, chance = log(0.25/(1-0.25)))

round(adjusted.coeffs.train.lmer,3)

##                                  Estimate Std. Error z value Pr(>|z|)
## (Intercept)                         1.473      0.138  10.695    0.000
## condition.ct                        1.309      0.276   4.747    0.000
## postp.ct                            0.645      0.155   4.171    0.000
## session.ct                          0.204      0.171   1.190    0.234
## condition.ct:postp.ct               0.303      0.310   0.980    0.327
## condition.ct:session.ct             0.478      0.343   1.392    0.164
## postp.ct:session.ct                -0.218      0.215  -1.012    0.312
## condition.ct:postp.ct:session.ct   -0.173      0.429  -0.403    0.687

Bf corresponding to main glmer

chance = log(0.25/(1-0.25))
coeff_list = c("(Intercept)", "condition.ct","postp.ct","session.ct",
               "condition.ct:postp.ct", "condition.ct:postp.ct",
               "condition.ct:session.ct",
               "postp.ct:session.ct",
               "condition.ct:postp.ct:session.ct",
               "condition.ct:postp.ct:session.ct"
                             )

half_maximum_minus_chance = (log((.95)/(1-(.95))) - chance)/2



h1_list = c((log((.95)/(1-(.95))) - chance)/2,
            adjusted.coeffs.train.lmer["(Intercept)", "Estimate" ],
            adjusted.coeffs.train.lmer["(Intercept)", "Estimate" ],
            adjusted.coeffs.train.lmer["(Intercept)", "Estimate" ],
            summary(train.lmer)$coefficients["condition.ct", "Estimate" ]-chance,
            summary(train.lmer)$coefficients["postp.ct", "Estimate" ]-chance,
            summary(train.lmer)$coefficients["condition.ct", "Estimate" ]-chance,
           summary(train.lmer)$coefficients["postp.ct", "Estimate" ]-chance,
            (summary(train.lmer)$coefficients["condition.ct", "Estimate" ]-chance)*2,
           (summary(train.lmer)$coefficients["condition.ct", "Estimate" ]-chance)*2
            
             )

h1_motivation = c(
          "half maximum minus chance", 
          "adjusted intercept (i.e. minus chance)", 
           "adjusted intercept (i.e. minus chance)",
           "adjusted intercept (i.e. minus chance)",
           "condition minus chance",
           "postp minus chance",
           "condition minus chance",
           "postp minus chance",
           "twice condition minus chance",
            "twice postp minus chance"
            
             )

tail_list = c(1,1,1,1,2,2,2,2,2,2)



Bayes_Factor_table_train = Bf_model(coeff_summary= adjusted.coeffs.train.lmer, coeff_list, h1_list, h1_motivation, tail_list)
                        
kable(Bayes_Factor_table_train)

coefficient	estimate	sdtheory	h1 motivation	tail	Bf
(Intercept)	1.473	2.022	half maximum minus chance	1	7.20864927750681e+23
condition.ct	1.309	1.473	adjusted intercept (i.e. minus chance)	1	19659.333
postp.ct	0.645	1.473	adjusted intercept (i.e. minus chance)	1	1136.282
session.ct	0.204	1.473	adjusted intercept (i.e. minus chance)	1	0.408
condition.ct:postp.ct	0.303	2.408	condition minus chance	2	0.205
condition.ct:postp.ct	0.303	1.744	postp minus chance	2	0.279
condition.ct:session.ct	0.478	2.408	condition minus chance	2	0.365
postp.ct:session.ct	-0.218	1.744	postp minus chance	2	0.203
condition.ct:postp.ct:session.ct	-0.173	4.815	twice condition minus chance	2	0.096
condition.ct:postp.ct:session.ct	-0.173	4.815	twice postp minus chance	2	0.096

glmer to follow up main effect of condition

Fit separate interecepts for each condition

train.lmer.a = glmer(correct ~ -1 + condition + (condition.ct*postp.ct * session.ct)- condition.ct + (postp.ct*session.ct|participant), family = binomial, data = train.df.legal, control=glmerControl(optimizer = "bobyqa"))

# check is same model
anova(train.lmer.a,train.lmer)

## Data: train.df.legal
## Models:
## train.lmer.a: correct ~ -1 + condition + (condition.ct * postp.ct * session.ct) - 
## train.lmer.a:     condition.ct + (postp.ct * session.ct | participant)
## train.lmer: correct ~ condition.ct * postp.ct * session.ct + (postp.ct * 
## train.lmer:     session.ct | participant)
##              Df    AIC    BIC  logLik deviance Chisq Chi Df Pr(>Chisq)    
## train.lmer.a 18 2614.6 2716.8 -1289.3   2578.6                            
## train.lmer   18 2614.6 2716.8 -1289.3   2578.6     0      0  < 2.2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

#round(summary(train.lmer.a)$coeff,4)

adjusted.coeffs.train.lmer.a = adjust_intercept_model(train.lmer.a, chance = log(0.25/(1-0.25)), intercept_list = c("conditionhigh_variability", "conditionlow_variability"))

round(adjusted.coeffs.train.lmer.a,3)

##                                  Estimate Std. Error z value Pr(>|z|)
## conditionhigh_variability           0.865      0.189   4.588    0.000
## conditionlow_variability            2.174      0.201  10.792    0.000
## postp.ct                            0.645      0.155   4.170    0.000
## session.ct                          0.204      0.171   1.190    0.234
## condition.ct:postp.ct               0.303      0.310   0.980    0.327
## condition.ct:session.ct             0.478      0.343   1.392    0.164
## postp.ct:session.ct                -0.218      0.215  -1.012    0.312
## condition.ct:postp.ct:session.ct   -0.173      0.429  -0.403    0.687

Bf corresponding to glmer to follow up main effect of condition

for each condition, inform h1 using the estimate for the intercept from the other from the other condition

chance = log(0.25/(1-0.25))
coeff_list = c("conditionhigh_variability",
               "conditionlow_variability")
h1_list = c(adjusted.coeffs.train.lmer.a["conditionlow_variability", "Estimate" ] ,
            adjusted.coeffs.train.lmer.a["conditionhigh_variability", "Estimate" ] )
h1_motivation = c("adjusted lv intercept", "adjusted hv intercept" )
tail_list = c(1,1)
Bayes_Factor_table_train_a = Bf_model(coeff_summary=adjusted.coeffs.train.lmer.a, coeff_list, h1_list, h1_motivation, tail_list)
kable(Bayes_Factor_table_train_a)

coefficient	estimate	sdtheory	h1 motivation	tail	Bf
conditionhigh_variability	0.865	2.174	adjusted lv intercept	1	5938.32
conditionlow_variability	2.174	0.865	adjusted hv intercept	1	4.42133165845635e+23

glmer to follow up main effect of Spatial Postposition

Fit separate interecepts for each level of Spatial Postposition.

train.lmer.b = glmer(correct ~ -1 + postp + (condition.ct*postp.ct * session.ct)- postp.ct + (postp.ct*session.ct|participant), family = binomial, data = train.df.legal, control=glmerControl(optimizer = "bobyqa"))

# check is same model
anova(train.lmer.b,train.lmer)

## Data: train.df.legal
## Models:
## train.lmer.b: correct ~ -1 + postp + (condition.ct * postp.ct * session.ct) - 
## train.lmer.b:     postp.ct + (postp.ct * session.ct | participant)
## train.lmer: correct ~ condition.ct * postp.ct * session.ct + (postp.ct * 
## train.lmer:     session.ct | participant)
##              Df    AIC    BIC  logLik deviance Chisq Chi Df Pr(>Chisq)    
## train.lmer.b 18 2614.6 2716.8 -1289.3   2578.6                            
## train.lmer   18 2614.6 2716.8 -1289.3   2578.6     0      0  < 2.2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

#round(summary(train.lmer.b)$coeff,4)

adjusted.coeffs.train.lmer.b = adjust_intercept_model(train.lmer.b, chance = log(0.25/(1-0.25)), intercept_list = c("postpnoshitani", "postpnoueni"))

round(adjusted.coeffs.train.lmer.b,3)

##                                  Estimate Std. Error z value Pr(>|z|)
## postpnoshitani                      1.151      0.155   7.431    0.000
## postpnoueni                         1.796      0.161  11.162    0.000
## condition.ct                        1.309      0.276   4.748    0.000
## session.ct                          0.204      0.171   1.190    0.234
## condition.ct:postp.ct               0.303      0.310   0.980    0.327
## condition.ct:session.ct             0.478      0.343   1.393    0.164
## postp.ct:session.ct                -0.218      0.215  -1.012    0.312
## condition.ct:postp.ct:session.ct   -0.173      0.429  -0.404    0.687

Bf corresponding to glmer to follow up main effect ofSpatial Postposition

Note here for each condition, we inform h1 using the relevant value for the intercept from the other from the other condition

chance = log(0.25/(1-0.25))
coeff_list = c("postpnoshitani",
               "postpnoueni")
h1_list = c(adjusted.coeffs.train.lmer.b["postpnoueni", "Estimate" ]  ,
            adjusted.coeffs.train.lmer.b["postpnoueni", "Estimate" ]  )
h1_motivation = c("adjusted noeuni intercept", "adjusted shitani intercept" )
tail_list = c(1,1)
Bayes_Factor_table_train_b = Bf_model(coeff_summary=adjusted.coeffs.train.lmer.b, coeff_list, h1_list, h1_motivation, tail_list)
kable(Bayes_Factor_table_train_b)

coefficient	estimate	sdtheory	h1 motivation	tail	Bf
postpnoshitani	1.151	1.796	adjusted noeuni intercept	1	137592804603.573
postpnoueni	1.796	1.796	adjusted shitani intercept	1	1.23528017502442e+26

Sustantial/strong evidence that they are above chance with each condition and each Spatial Postposition separately

Training Data Analyses Summary

Strong evidence that low variability outperform the high variability group, although also strong evidence that both condtions above chance. Substantial evidence that this is not qualified by an interaction with Spatial Postposition. Some evidence that not qualified by an interaction with session (though not quite BF= .365 susbtantial).

Strong evidence of stronger peformance with nonueni(above) than shitani(below) sentences and substantial evidence that this is not qualified by an interaction with Spatial Postposition or with session.

No evidence that they perform better in s2 than s1, though don’t yet have substantial evidence (BF=.408)

Substantial evidence for no three way interaction.

Statistical Analsyes: Test

check proportions of illegal trials

test.df = droplevels(subset(traintest.df, phase == "testing"))

test.df$legal = substr(test.df$response_type,1,1)
test.df$legal = recode(substr(test.df$response_type,1,1), "'l'=1;else=0")

round(tapply(test.df$legal, test.df$condition, mean, na.rm=T),2)

## high_variability  low_variability 
##             0.98             0.90

Very few illegal moves. Makes sense to remove these from the analyses (makes chance = .25). Both conditions near ceiling so little data will be removed in each case.

Analyses looking at likliehood of correct response (illegal responses excluded)

main glmer

Run an lmer predicting proportion correct (chance = .25)

Note that each effect has two levels. They are coded such that: a positive effect of condition means HV > LV [*NOTE: OPPOSITE WAY ROUND FROM TRAINING]; positive effect of session means session2 > session1; positive effect of postp means above > below (these are the directions of predictions -see above).

Note - in the output printed - a function has been used to adjust estimate of the intercept (it is set to be the original intercept - chance), as well as the z and p values

# get appropriate df and center factors
test.df.legal = droplevels(subset(test.df, legal == 1))
test.df.legal$condition = relevel(test.df.legal$condition, "low_variability")
test.df.legal = lizCenter(test.df.legal, list("postp", "condition"))
test.df.legal = lizCenter(test.df.legal, list("condition", "postp"))
# run model

test.lmer = glmer(correct ~ condition.ct*postp.ct  + (postp.ct|participant), family = binomial, data = test.df.legal, control=glmerControl(optimizer = "bobyqa"))
#round(summary(test.lmer)$coeff,4)

adjusted.coeffs.test.lmer = adjust_intercept_model(test.lmer, chance = log(0.25/(1-0.25)))

round(adjusted.coeffs.test.lmer,3)

##                       Estimate Std. Error z value Pr(>|z|)
## (Intercept)              0.761      0.187   4.079    0.000
## condition.ct             0.716      0.374   1.916    0.055
## postp.ct                 0.090      0.287   0.314    0.753
## condition.ct:postp.ct   -0.226      0.577  -0.392    0.695

Bf corresponding to main glmer

Note - we infom H1 for Spatial Postposition using the value from training data for Spatial Postposition

chance = log(0.25/(1-0.25))
coeff_list = c("(Intercept)", "condition.ct","postp.ct",
               "condition.ct:postp.ct")                             

h1_list = c((log((.95)/(1-(.95))) - chance)/2,
            adjusted.coeffs.test.lmer["(Intercept)", "Estimate" ],
            summary(train.lmer)$coefficients["postp.ct", "Estimate" ],
            summary(test.lmer)$coefficients["condition.ct", "Estimate" ]-chance)

h1_motivation = c(
          "half maximum minus chance", 
          "adjusted intercept (i.e. minus chance)",
           "estimate of Spatial Postposition in training ",
           "condition minus chance"
            
             )

tail_list = c(1,1,1,2)

Bayes_Factor_table_test = Bf_model(coeff_summary=adjusted.coeffs.test.lmer, coeff_list, h1_list, h1_motivation, tail_list)
kable(Bayes_Factor_table_test)

coefficient	estimate	sdtheory	h1 motivation	tail	Bf
(Intercept)	0.761	2.022	half maximum minus chance	1	703.877
condition.ct	0.716	0.761	adjusted intercept (i.e. minus chance)	1	3.708
postp.ct	0.09	0.645	estimate of Spatial Postposition in training	1	0.517
condition.ct:postp.ct	-0.226	1.815	condition minus chance	2	0.325

glmer to follow up main effect of condition

Fit separate interecepts for each condition

test.lmer.a = glmer(correct ~ -1 + condition + (condition.ct*postp.ct )- condition.ct + (postp.ct|participant), family = binomial, data = test.df.legal, control=glmerControl(optimizer = "bobyqa"))

# check is same model
anova(test.lmer.a, test.lmer)

## Data: test.df.legal
## Models:
## test.lmer.a: correct ~ -1 + condition + (condition.ct * postp.ct) - condition.ct + 
## test.lmer.a:     (postp.ct | participant)
## test.lmer: correct ~ condition.ct * postp.ct + (postp.ct | participant)
##             Df   AIC    BIC logLik deviance Chisq Chi Df Pr(>Chisq)
## test.lmer.a  7 950.8 983.32 -468.4    936.8                        
## test.lmer    7 950.8 983.32 -468.4    936.8     0      0          1

#round(summary(test.lmer.a)$coeff,4)
adjusted.coeffs.test.lmer.a = adjust_intercept_model(test.lmer.a, chance = log(0.25/(1-0.25)), c("conditionlow_variability","conditionhigh_variability"))

round(adjusted.coeffs.test.lmer.a,3)

##                           Estimate Std. Error z value Pr(>|z|)
## conditionlow_variability     0.378      0.270   1.400    0.161
## conditionhigh_variability    1.094      0.258   4.239    0.000
## postp.ct                     0.090      0.287   0.314    0.753
## condition.ct:postp.ct       -0.226      0.577  -0.392    0.695

Bf corresponding to glmer to follow up main effect of condition

For lv condition we inform h1 using the estimate for hv; for hv we inform h1 using the maximum from the scale (as above)

chance = log(0.25/(1-0.25))
coeff_list = c("conditionhigh_variability",
               "conditionlow_variability")

  

h1_list = c((log((.95)/(1-(.95))) - chance)/2,
            adjusted.coeffs.test.lmer.a["conditionhigh_variability", "Estimate" ]     )

h1_motivation = c("adjusted lv intercept", "adjusted hv intercept" )
tail_list = c(1,1)
intercept_list = c(1,2)
Bayes_Factor_table_test_a = Bf_model(coeff_summary=adjusted.coeffs.test.lmer.a, coeff_list, h1_list, h1_motivation, tail_list)
kable(Bayes_Factor_table_test_a)

coefficient	estimate	sdtheory	h1 motivation	tail	Bf
conditionhigh_variability	1.094	2.022	adjusted lv intercept	1	1753.507
conditionlow_variability	0.378	1.094	adjusted hv intercept	1	1.1
### Follow up analysis 1: WO	RD ORDER- A	re they abo	ve chance in moving the	correct	picture?
#### main glmer

Predicting whether they move the correct picture (note chance now = 50% - i.e. log odds 0, no need to adjust intercept)

test.lmer.q2 = glmer(moved_correct_image ~ condition.ct*postp.ct  + (postp.ct|participant), family = binomial, data = test.df.legal, control=glmerControl(optimizer = "bobyqa"))

round(summary(test.lmer.q2)$coeff,3)

##                       Estimate Std. Error z value Pr(>|z|)
## (Intercept)              1.621      0.262   6.184    0.000
## condition.ct             0.181      0.505   0.359    0.720
## postp.ct                -0.066      0.259  -0.255    0.799
## condition.ct:postp.ct    0.559      0.396   1.410    0.158

Bf corresponding to main glmer

coeff_list = c("(Intercept)", "condition.ct","postp.ct",
               "condition.ct:postp.ct")

half_maximum_minus_chance = (log((.95)/(1-(.95))) - chance)/2

h1_list = c((log((.95)/(1-(.95))))/2,
            summary( test.lmer.q2)$coefficients["(Intercept)", "Estimate" ],
            summary(train.lmer)$coefficients["postp.ct", "Estimate" ],
            summary( test.lmer.q2)$coefficients["(Intercept)", "Estimate" ]*2
            
             )
h1_motivation = c(
          "half maximum ", 
          "intercept", 
           "estimate of Spatial Postposition in training ",
           "condition"
            
           )

tail_list = c(1,1,1,2)

Bayes_Factor_table_test_q2 = Bf_model(coeff_summary= summary( test.lmer.q2)$coefficients, coeff_list, h1_list, h1_motivation, tail_list)

kable(Bayes_Factor_table_test_q2)

coefficient	estimate	sdtheory	h1 motivation	tail	Bf
(Intercept)	1.621	1.472	half maximum	1	39333741.556
condition.ct	0.181	1.621	intercept	1	0.402
postp.ct	-0.066	0.645	estimate of Spatial Postposition in training	1	0.309
condition.ct:postp.ct	0.559	3.241	condition	2	0.323

Follow up analysis 2: SPATIAL TERMS KNOWELDGE- Do they understand that X shitani means “below X” and X noeni means “above X”

main glmer

again chance = 50% (log odds 0) and don’t need to adjust intercept

test.df.legal.movedcorect = droplevels(subset(test.df.legal, moved_correct_image == 1))
test.df.legal.movedcorect= lizCenter(test.df.legal.movedcorect, list("condition", "postp"))
test.lmer.q3 = glmer(correct ~ condition.ct*postp.ct +(postp.ct|participant), family = binomial, data = test.df.legal.movedcorect, control=glmerControl(optimizer = "bobyqa"))

round(summary(test.lmer.q3)$coeff,3)

##                       Estimate Std. Error z value Pr(>|z|)
## (Intercept)              0.423      0.195   2.170    0.030
## condition.ct             0.806      0.388   2.080    0.038
## postp.ct                 0.340      0.434   0.783    0.434
## condition.ct:postp.ct   -0.839      0.864  -0.971    0.332

Bf corresponding to main glmer

coeff_list = c("(Intercept)", "condition.ct","postp.ct",
               "condition.ct:postp.ct")                             

half_maximum_minus_chance = (log((.95)/(1-(.95))) )/2

h1_list = c((log((.95)/(1-(.95))) )/2,
            summary( test.lmer.q3)$coefficients["(Intercept)", "Estimate" ]
,
           summary(train.lmer)$coefficients["postp.ct", "Estimate" ],
            summary(test.lmer.q3)$coefficients["condition.ct", "Estimate" ])
            
h1_motivation = c(
          "half maximum ",         "intercept",            "estimate of Spatial Postposition in training ",           "condition")
tail_list = c(1,1,1,2)

Bayes_Factor_table_test_q3 = Bf_model(coeff_summary=summary(test.lmer.q3)$coefficients, coeff_list, h1_list, h1_motivation, tail_list)
kable(Bayes_Factor_table_test_q3)

coefficient	estimate	sdtheory	h1 motivation	tail	Bf
(Intercept)	0.423	1.472	half maximum	1	2.608
condition.ct	0.806	0.423	intercept	1	4.107
postp.ct	0.34	0.645	estimate of Spatial Postposition in training	1	1.022
condition.ct:postp.ct	-0.839	0.806	condition	2	0.91

glmer to follow up main effect of condition

Fit separate interecepts for each condition

test.lmer.q3.a = glmer(correct ~ -1 + condition + (condition.ct*postp.ct )- condition.ct + (postp.ct|participant), family = binomial, data = test.df.legal.movedcorect, control=glmerControl(optimizer = "bobyqa"))

# check is same model
anova(test.lmer.q3.a,test.lmer.q3)

## Data: test.df.legal.movedcorect
## Models:
## test.lmer.q3.a: correct ~ -1 + condition + (condition.ct * postp.ct) - condition.ct + 
## test.lmer.q3.a:     (postp.ct | participant)
## test.lmer.q3: correct ~ condition.ct * postp.ct + (postp.ct | participant)
##                Df    AIC    BIC  logLik deviance Chisq Chi Df Pr(>Chisq)
## test.lmer.q3.a  7 706.54 737.11 -346.27   692.54                        
## test.lmer.q3    7 706.54 737.11 -346.27   692.54     0      0  < 2.2e-16
##                   
## test.lmer.q3.a    
## test.lmer.q3   ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

round(summary(test.lmer.q3.a)$coeff,4)

##                           Estimate Std. Error z value Pr(>|z|)
## conditionlow_variability   -0.0142     0.2798 -0.0506   0.9597
## conditionhigh_variability   0.7922     0.2698  2.9364   0.0033
## postp.ct                    0.3400     0.4343  0.7830   0.4336
## condition.ct:postp.ct      -0.8387     0.8638 -0.9709   0.3316

Bf corresponding to glmer to follow up main effect of condition

For lv we inform h1 using the maximum for hv; for hv we inform h1 using the maximum from the scale (as above)

coeff_list = c("conditionhigh_variability",
               "conditionlow_variability")


h1_list = c((log((.95)/(1-(.95))) )/2,
                       summary(test.lmer.q3.a)$coefficients["conditionhigh_variability", "Estimate" ] 
)


h1_motivation = c("estimate from lv", "estimate from hv" )

tail_list = c(1,1)

intercept_list = c(1,1)

Bayes_Factor_table_test_q3_a = Bf_model(coeff_summary=summary(test.lmer.q3.a)$coefficients , coeff_list, h1_list, h1_motivation, tail_list)
kable(Bayes_Factor_table_test_q3_a)

coefficient	estimate	sdtheory	h1 motivation	tail	Bf
conditionhigh_variability	0.792	1.472	estimate from lv	1	23.311
conditionlow_variability	-0.014	0.792	estimate from hv	1	0.319

Test Data Analyses Summary

For our intitial analyses, substantial evidnce evidence that high variability condition outperform low variability. Breaking this down, strong evidence that high variability condition are above chance whereas for lv condition evidence is ambiguous (BF=1.1). Substantial evidence that effect of condition isn’t qualified by an interaction with spatial postposition. Unlike in training, no evidence that noueni > shitani, but evidence is ambiguous (BF=.455)

In first additional analyses looking at knoweldge of WORD ORDER, we found no evidence high variability were better than low variability, although evidence was ambiguous (BF = .402), and substantial evidence that nounei not > noshitani and for no interaction between condition and Spatial Postposition.

In the second additional analysis looking at KNOWLEDGE OF SPATIAL TERMS, we found substantial evidence that the high variability condition were better than the low variability condition and breaking the data down we found that there was only evidence for learning for the high variability group (strong evidence) whereas for the low variability group there was evidence for the null. There was no evidence that this effect of condition was qualified by an interaction with postposition (but evidence for the interaction was ambiguous BF = .9). No evidence that noueni > shitani (but evidence is ambiguous, BF =1.1)

TAKE HOME SUMMARY OF FINDINGS

PRIMARY

For both training and testing our key hypotheses about variability were met: there is evidence of an LV benefit in training (strong) and for an HV benefit in the generalization test (substantial). Further analyses on the test data suggest that the benefit of hv for generalization stems from the fact that only the HV group learn the meanings of the spatial postpositionss.

SECONDARY

In line with our expectations, participants did better with noueni (above) sentences in training. However in contrast to expections, the same effect wasn’t found at test (although evidence was ambiguous).

In contrast to our expections, in training, we did not see greater performance in the second session than in the first session.

ON GOING PLANS

Description of planned experiment

The data collected so far support the key prediction that high variability training leads to greater peformance in the generalization test. However the sample was relatively small and the values of H1 which we used for our analyses were in some places based on aspects of the same data, rather than independent values.

Our plan is therefore to try and replicate the experiment, running new child participants in the HV and LV conditions. In addition, we will run a third (matched) skewed (sv) comprising a mixed of four highly frequent sentences and along with some unique ones. Hsu & Bishop 2014 had this type of intput in their training study and argue that the repeated items aided learning and generalization. We test whether this balance leads to greater generalization than non-skewed hv in our paradigm.

In the new skew condition: 32/56 are the LV sentences (played 8 times each) and 24/56 are unique sentences (number of nouns matched to other conditions). The sentences are ordered so that the first session training begins with a block of 16 sentences which are all from the lv set (so each of the four heard x4) followed by a block of 16 sentences comprising a mixture of the 4 lv sentences and 12 unique sentences (in intermixed random order). The second session begins with a block of 8 lv sentences (four lv sentences x2) followed by a block of 16 sentences where each of the four sentences followed by a block of 16 sentences comprising a mixture of the 4 lv sentences and 12 unique sentences (in intermixed random order).

All other details of the experiments are identical except for the following:

1. Instead of being a week apart, the two sessions will be on consequtive days. (This is partly for practical reasons working in the schools, however it also allows us to see if we get an effect of sessions when the sessions are further apart).

2. In the original experiment, the demonstration sentences were ordered such that the first two were “above” sentences and the second two “below” sentences; this is reversed in the current study. Reason: while - given what we saw in some previous adult work- we beleive that the “above” bias seen in training comes from iconicity (i.e. desire to put first named picture higher on the page) it is possible it comes from having heard those sentences first. Changing around this ordering gives us the opportunity to test this.

3. We have fixed a bug in the program which allowed childrren to put one picture into the same square as the others which led to recording errors (although they did this very rarely (less than 1% of the time)).

Replication of HV versus LV conditions

We will run the same set of analsyes as above with data from the two new sets of participants in these conditions, running the same set of glmer() followed by BF analsyes.

Glmers will have the same fixed effects and random effects above, although in the case of non converegence we will reduce the random slopes structure as follows, stopping when one is found that converged:

TRAINING:

FIRST: (postp.ct*session.ct|participant) THEN : (postp.ct*session.ct||participant) THEN: (postp.ct+session.ct||participant) THEN: (postp.ct||participant) AND/OR (session.ct||participant) THEN: (1|participant)

TESTING:

FIRST (postp.ct|participant) THEN: (postp.ct||participant)

Bayes factor analyses will be conducted as before, except that we will not inform using values from within the same data but instead values coming from the previous data (call it dataA). In general, the values we use to inform H1 will therefore be identical to those above - except that whererever we found clear evidence of an effect in in dataA the estimate of the effect from that model is the most direct estimate of our effect.

All the values are given in the table below (those with *** differ from those used above)

analysis = c(rep("Training", 12), rep("Testing Main", 6), rep("Testing (WORD ORDER)", 4),rep("Testing (SPATIAL)", 6))

estimate_of_sd =  c(             adjusted.coeffs.train.lmer["(Intercept)", "Estimate" ],
            summary(train.lmer)$coefficients["condition.ct", "Estimate" ],
            summary(train.lmer)$coefficients["postp.ct", "Estimate" ],
            adjusted.coeffs.train.lmer["(Intercept)", "Estimate" ],
            summary(train.lmer)$coefficients["condition.ct", "Estimate" ]-log(0.25/(1-0.25)),
            summary(train.lmer)$coefficients["postp.ct", "Estimate" ]-log(0.25/(1-0.25)),
            summary(train.lmer)$coefficients["condition.ct", "Estimate" ]-log(0.25/(1-0.25)),
            summary(train.lmer)$coefficients["postp.ct", "Estimate" ]-log(0.25/(1-0.25)),
            (summary(train.lmer)$coefficients["condition.ct", "Estimate" ]-log(0.25/(1-0.25)))*2,
            (summary(train.lmer)$coefficients["condition.ct", "Estimate" ]-log(0.25/(1-0.25)))*2,
            adjusted.coeffs.train.lmer.a["conditionhigh_variability","Estimate" ],
            adjusted.coeffs.train.lmer.a["conditionlow_variability","Estimate" ],
            
            adjusted.coeffs.test.lmer["(Intercept)", "Estimate" ],
            summary(test.lmer)$coefficients["condition.ct",  "Estimate" ],
            summary(train.lmer)$coefficients["postp.ct", "Estimate" ],
           
            summary(test.lmer)$coefficients["condition.ct",  "Estimate" ]- log(0.25/(1-0.25)),
            adjusted.coeffs.test.lmer.a["conditionhigh_variability", "Estimate" ] ,           
            adjusted.coeffs.test.lmer.a["conditionlow_variability", "Estimate" ]        , 
            summary(test.lmer.q2)$coefficients["(Intercept)", "Estimate"],
            summary( test.lmer.q2)$coefficients["(Intercept)", "Estimate" ],
            summary(train.lmer)$coefficients["postp.ct", "Estimate" ],
            summary( test.lmer.q2)$coefficients["(Intercept)", "Estimate" ]*2,
            summary(test.lmer.q3)$coefficients["(Intercept)", "Estimate"],
            summary(test.lmer.q3)$coefficients["condition.ct", "Estimate"],
            summary(train.lmer)$coefficients["postp.ct", "Estimate" ],
            summary(test.lmer.q3)$coefficients["condition.ct", "Estimate" ],
            summary(test.lmer.q3.a)$coefficients["conditionhigh_variability", "Estimate" ],
            summary(test.lmer.q3.a)$coefficients["conditionhigh_variability", "Estimate" ])


coefficient = c(  "(Intercept)",
              "condition.ct",
              "postp.ct",
              "session.ct",
               "condition.ct:postp.ct",
               "condition.ct:postp.ct",
               "condition.ct:session.ct",
               "postp.ct:session.ct",
               "condition.ct:postp.ct:session.ct",
               "condition.ct:postp.ct:session.ct",
              "conditionhigh_variability",
              "conditionlow_variability",
              "(Intercept)",
               "condition.ct",
              "postp.ct",
               "condition.ct:postp.ct",
              "conditionhigh_variability",
               "conditionlow_variability",
              "(Intercept)",
               "condition.ct",
              "postp.ct",
               "condition.ct:postp.ct",
              "(Intercept)",
               "condition.ct",
              "postp.ct",
               "condition.ct:postp.ct",
              "conditionhigh_variability",
               "conditionlow_variability")


h1_motivation = c(          "adjusted intercept from dataA model**", 
          "condition  from dataA model**", 
          "postp from dataA model**", 
"adjusted intercept (i.e. minus chance) from dataA model",
           "condition minus chance from dataA model",
           "postp minus chance from dataA model",
           "condition minus chance from dataA model",
           "postp minus chance from dataA model",
           "twice condition minus chance from dataA model",
            "twice postp minus chance",
"adjusted hv intercept from dataA model**",
"adjusted lv intercept from dataA model**",
          "adjusted intercept from dataA model",
          "condition  from dataA model**", 
           "estimate of Spatial Postposition in training from dataA model",
           "condition minus chance from dataA model",
"adjusted hv intercept from dataA model**",
"adjusted lv intercept from dataA model**",
          "intercept from dataA model**", 
          "intercept from dataA model",
           "estimate of Spatial Postposition in training from dataA model",
           "condition from dataA model",
          "intercept from dataA model**", 
          "condition  from dataA model**", 
           "estimate of Spatial Postposition in training from dataA model",
           "conditionfrom dataA model",
"adjusted hv intercept from dataA model**",
"adjusted lv intercept from dataA model**")


kable(data.frame(cbind(analysis, coefficient , estimate_of_sd, h1_motivation)))

analysis	coefficient	estimate_of_sd	h1_motivation
Training	(Intercept)	1.47292504032035	adjusted intercept from dataA model**
Training	condition.ct	1.30888780266562	condition from dataA model**
Training	postp.ct	0.644889129083052	postp from dataA model**
Training	session.ct	1.47292504032035	adjusted intercept (i.e. minus chance) from dataA model
Training	condition.ct:postp.ct	2.40750009133373	condition minus chance from dataA model
Training	condition.ct:postp.ct	1.74350141775116	postp minus chance from dataA model
Training	condition.ct:session.ct	2.40750009133373	condition minus chance from dataA model
Training	postp.ct:session.ct	1.74350141775116	postp minus chance from dataA model
Training	condition.ct:postp.ct:session.ct	4.81500018266746	twice condition minus chance from dataA model
Training	condition.ct:postp.ct:session.ct	4.81500018266746	twice postp minus chance
Training	conditionhigh_variability	0.864900778401958	adjusted hv intercept from dataA model**
Training	conditionlow_variability	2.17379023076295	adjusted lv intercept from dataA model**
Testing Main	(Intercept)	0.760963702361891	adjusted intercept from dataA model
Testing Main	condition.ct	0.716396139915893	condition from dataA model**
Testing Main	postp.ct	0.644889129083052	estimate of Spatial Postposition in training from dataA model
Testing Main	condition.ct:postp.ct	1.815008428584	condition minus chance from dataA model
Testing Main	conditionhigh_variability	1.0944746861169	adjusted hv intercept from dataA model**
Testing Main	conditionlow_variability	0.378077621797879	adjusted lv intercept from dataA model**
Testing (WORD ORDER)	(Intercept)	1.62064756070148	intercept from dataA model**
Testing (WORD ORDER)	condition.ct	1.62064756070148	intercept from dataA model
Testing (WORD ORDER)	postp.ct	0.644889129083052	estimate of Spatial Postposition in training from dataA model
Testing (WORD ORDER)	condition.ct:postp.ct	3.24129512140297	condition from dataA model
Testing (SPATIAL)	(Intercept)	0.422895916776066	intercept from dataA model**
Testing (SPATIAL)	condition.ct	0.806330045035369	condition from dataA model**
Testing (SPATIAL)	postp.ct	0.644889129083052	estimate of Spatial Postposition in training from dataA model
Testing (SPATIAL)	condition.ct:postp.ct	0.806330045035369	conditionfrom dataA model
Testing (SPATIAL)	conditionhigh_variability	0.792175673234309	adjusted hv intercept from dataA model**
Testing (SPATIAL)	conditionlow_variability	0.792175673234309	adjusted lv intercept from dataA model**

NOTE: if we see an effect of session in the model for just skewed data (decribed below), we will use this to inform our estimate of session.ct and the interactions with session. Note that since we don’t include skewed data in this model, this is still and indepdent estimate)

In addition to fitting separate intercepts for condition and spatial postposition, if we find effects ofsession we will fits separate intercepts for each session.

Following up effect of session

If we find evidence of an effect of session, we will fun a follow up model over the data from the new HV and LV conditions + that from the previous experiment. It will have the same structure as the model above except for an additional effect of “experiment” and all the interactions with that factor. The model will be inspect for an interaction between session and experiment and a corresponding BF will be calcualted. H1 for this will have to based on a value from the combined data. It will either be based on the effect of session (if there is an overall main effect of this in the combined data) OR on the intercept (see notes on how BF’s were determined in current analyses at the top of this document).

Analsyes of skewed data

Training

Here we look just at the data within this condition, comparing hv and lv within-participant. The prediction here is that they should do better with LV than with HV sentences. This will be a check that our manipulation is working.

We will first run a glmer() model with only the skew data with a similar structure to that used for the hv and lv conditions, except that we will have a within-participants slope for condition i.e. (postp.ct*session.ct*condition.ct|participant) (If this doesn’t converge, we will attempt the following modesl: (postp.ct*session.ct*condition.ct||participant) ((postp.ct*session.ct*condition.ct)-(postp.ct:session.ct:condition.ct)||participant) ((postp.ct*session.ct*condition.ct)-((postp.ct:session.ct:condition.ct)+(postp.ct:session.ct))||participant) ((postp.ct*session.ct*condition.ct)-((postp.ct:session.ct:condition.ct)+(postp.ct:session.ct)+ (session.ct:condition.ct))||participant) ((postp.ct+session.ct+condition.ct)||participant) ((postp.ct+condition.ct)||participant) ((condition.ct)||participant) (1|participant)

For the BFs, we will inform h1’s using the same set of estimates as for the comparison between the hv and lv conditions, except that for condition.ct we will set the sd to be equal to HALF of the value of condition.ct in the previous experiment (because for that contrast, each LV sentence had been heard more frequently than here) so since we use the half normal 2sd=max, so we set the sd to half this value; similarly when fitting a separate intercept for LV we will set the sd to be equal to HALF the value for that condition previously - i.e.

analysis = c(rep("Training", 12))

coefficient =  c("(Intercept)",
                 "condition.ct",
                "postp.ct",
                "session.ct",
                "condition.ct:postp.ct",
                 "condition.ct:postp.ct",
               "condition.ct:session.ct",
               "postp.ct:session.ct",
               "condition.ct:postp.ct:session.ct",
               "condition.ct:postp.ct:session.ct",
                "conditionhigh_variability",
                "conditionlow_variability")

estimate_of_sd = c(
            adjusted.coeffs.train.lmer["(Intercept)", "Estimate" ],
            adjusted.coeffs.train.lmer["condition.ct", "Estimate"]/2,
            summary(train.lmer)$coefficients["postp.ct", "Estimate" ],
            adjusted.coeffs.train.lmer["(Intercept)", "Estimate" ],
            summary(train.lmer)$coefficients["condition.ct", "Estimate" ]-chance,
            summary(train.lmer)$coefficients["postp.ct", "Estimate" ]-chance,
            summary(train.lmer)$coefficients["condition.ct", "Estimate" ]-chance,
           summary(train.lmer)$coefficients["postp.ct", "Estimate" ]-chance,
            (summary(train.lmer)$coefficients["condition.ct", "Estimate" ]-chance)*2,
           (summary(train.lmer)$coefficients["condition.ct", "Estimate" ]-chance)*2,
            adjusted.coeffs.train.lmer.a["conditionhigh_variability","Estimate"],
            adjusted.coeffs.train.lmer.a["conditionlow_variability","Estimate"]/2 )


h1_motivation = c(
            "adjusted intercept from dataA model**", 
            "half condition from dataA model **",
          "postp from dataA model**", 
            "adjusted intercept (i.e. minus chance) from dataA model",
           "condition minus chance from dataA model",
           "postp minus chance from dataA model",
           "condition minus chance from dataA model",
           "postp minus chance from dataA model",
           "twice condition minus chance from dataA model",
            "twice postp minus chance",
              "adjusted hv intercept from dataA model**",
              "HALF adjusted hv intercept from dataA model**")

kable(data.frame(cbind(analysis, coefficient , estimate_of_sd, h1_motivation)))

analysis	coefficient	estimate_of_sd	h1_motivation
Training	(Intercept)	1.47292504032035	adjusted intercept from dataA model**
Training	condition.ct	0.654443901332811	half condition from dataA model **
Training	postp.ct	0.644889129083052	postp from dataA model**
Training	session.ct	1.47292504032035	adjusted intercept (i.e. minus chance) from dataA model
Training	condition.ct:postp.ct	2.40750009133373	condition minus chance from dataA model
Training	condition.ct:postp.ct	1.74350141775116	postp minus chance from dataA model
Training	condition.ct:session.ct	2.40750009133373	condition minus chance from dataA model
Training	postp.ct:session.ct	1.74350141775116	postp minus chance from dataA model
Training	condition.ct:postp.ct:session.ct	4.81500018266746	twice condition minus chance from dataA model
Training	condition.ct:postp.ct:session.ct	4.81500018266746	twice postp minus chance
Training	conditionhigh_variability	0.864900778401958	adjusted hv intercept from dataA model**
Training	conditionlow_variability	1.08689511538147	HALF adjusted hv intercept from dataA model**

NOTE: if we see an effect of session in the model for just hv and lv data (decribed above), we will use this to inform our estimate of session.ct and the interactions with session. Note that since we don’t include skewed data in this model, this is still and indepdent estimate)

In addition to fitting separate intercepts for condition and spatial postposition, if we find effects ofsession we will fits separate intercepts for each session.

Generaliation test

Here our predictions concern contrasts with the HV and LV condition.

Specifically we predict that (1) there will be higher peformance performance in the skew condition than in than the LV conditions (2) there will be higher performance in the skew condition than in the HV condition. Of these. Note that the contrast with HV is critical in showing a benefit of skew and not just variability for generalization.

In order to look at this, we will fit three models akin to each of those fit for dataA above (i.e. the main model, followup1-WORDORDER, followup2-SPATIALKNOWELDGE)

However in each case, rather than apply the model to data from two conditions, we will apply it to data from three conditions. In place of the centered factor “condition.ct”, we will use the default factor coding for condition with “skew” as the baseline condition. This will ensure that the (Intercept) reported is just for the skew data and similarly that the effect of spatial postposition is just reported for skew only. In addition, the contrasts between skew and lv and skew and hv will also be reported (coefficients will be labelled “conditionlv” and “conditionhv”) (as well as interactions of these with spatial postposition, the main effect of spatial postposition will be reporteda the level of skew)

The values to inform BF’s will again all come from the previous dataA analyses except that for the main model and for followup2-SPATIALKNOWELDGE we use the effect of condition.ct for dataA (i.e. the contrast between hv and lv) as the estimate of h1 and set sd to be equal to this value in each case. We note that this likely underestimates the value for the contrast with LV (since we expect there to be an even bigger difference with skew than HV) but overesimates in the case of HV (since the difference of having some variability compared to none is likely to be a bigger difference). Note that the later is the key contrast of interest, and in erring in using a larger estimate we are taking a more conservative approach.

For the estimate of the intercept for skew, we base it on the intercept for the HV condition. All the BFs to be used are given below:

analysis = c(rep("Testing Main", 6), rep("Testing (WORD ORDER)", 6),rep("Testing (SPATIAL)", 6))

coefficient =  rep(c("(Intercept)", "conditionlv","conditionhv", "postp.ct", "conditionlv:postp.ct","conditionhv:postp.ct"),3)

interpreation_coefficient =  rep(c("intercept for skew", "contrast skew and lv ","contrast skew and lv", "postp for skew", "contrast skew and lv by postp","contrast skew and hv by postp"),3)

chance = log(0.25/(1-0.25))

estimate_of_sd = 
round(c(
   
      adjusted.coeffs.test.lmer.a["conditionhigh_variability", "Estimate" ], 
      summary(test.lmer)$coefficients["condition.ct", "Estimate"],
      summary(test.lmer)$coefficients["condition.ct", "Estimate"],
      
      summary(train.lmer)$coefficients["postp.ct", "Estimate"],
      
      summary(test.lmer)$coefficients["condition.ct", "Estimate" ]-log(0.25/(1-0.25)),
      summary(test.lmer)$coefficients["condition.ct", "Estimate" ]-log(0.25/(1-0.25)),
      
      summary(test.lmer.q2)$coefficients["(Intercept)", "Estimate" ], 
      summary(test.lmer.q2)$coefficients["(Intercept)", "Estimate" ],
      summary(test.lmer.q2)$coefficients["(Intercept)", "Estimate" ],
       summary(train.lmer)$coefficients["postp.ct", "Estimate"],
      summary(test.lmer.q2)$coefficients["condition.ct", "Estimate" ],
      summary(test.lmer.q2)$coefficients["condition.ct", "Estimate" ],
      
      summary(test.lmer.q3.a)$coefficients["conditionhigh_variability", "Estimate" ], 
      summary(test.lmer.q3)$coefficients["condition.ct", "Estimate"],
      summary(test.lmer.q3)$coefficients["condition.ct", "Estimate"],
       summary(train.lmer)$coefficients["postp.ct", "Estimate"],
      summary(test.lmer.q3)$coefficients["condition.ct", "Estimate" ],
      summary(test.lmer.q3)$coefficients["condition.ct", "Estimate" ]
      

     

),3)
   
h1_motivation = c(
          "adjusted intercept for hv for dataA model", 
          "condition  (i.e. hv versus lv contrast) from dataA model", 
          "condition  (i.e. hv versus lv contrast) from dataA model", 
           "estimate of Spatial Postposition in training from dataA model",
           "condition minus chance from dataA model",
           "condition minus chance from dataA model",
          "intercept for hv for dataA model", 
          "condition  (i.e. hv versus lv contrast) from dataA model", 
          "condition  (i.e. hv versus lv contrast) from dataA model", 
           "estimate of Spatial Postposition in training from dataA model",
           "condition minus chance from dataA model",
           "condition minus chance from dataA model",
          "intercept for hv for dataA model", 
          "condition  (i.e. hv versus lv contrast) from dataA model", 
          "condition  (i.e. hv versus lv contrast) from dataA model", 
           "estimate of Spatial Postposition in training from dataA model",
           "condition minus chance from dataA model",
           "condition minus chance from dataA model")




data.frame(cbind(analysis, coefficient ,interpreation_coefficient, estimate_of_sd))

##                analysis          coefficient     interpreation_coefficient
## 1          Testing Main          (Intercept)            intercept for skew
## 2          Testing Main          conditionlv         contrast skew and lv 
## 3          Testing Main          conditionhv          contrast skew and lv
## 4          Testing Main             postp.ct                postp for skew
## 5          Testing Main conditionlv:postp.ct contrast skew and lv by postp
## 6          Testing Main conditionhv:postp.ct contrast skew and hv by postp
## 7  Testing (WORD ORDER)          (Intercept)            intercept for skew
## 8  Testing (WORD ORDER)          conditionlv         contrast skew and lv 
## 9  Testing (WORD ORDER)          conditionhv          contrast skew and lv
## 10 Testing (WORD ORDER)             postp.ct                postp for skew
## 11 Testing (WORD ORDER) conditionlv:postp.ct contrast skew and lv by postp
## 12 Testing (WORD ORDER) conditionhv:postp.ct contrast skew and hv by postp
## 13    Testing (SPATIAL)          (Intercept)            intercept for skew
## 14    Testing (SPATIAL)          conditionlv         contrast skew and lv 
## 15    Testing (SPATIAL)          conditionhv          contrast skew and lv
## 16    Testing (SPATIAL)             postp.ct                postp for skew
## 17    Testing (SPATIAL) conditionlv:postp.ct contrast skew and lv by postp
## 18    Testing (SPATIAL) conditionhv:postp.ct contrast skew and hv by postp
##    estimate_of_sd
## 1           1.094
## 2           0.716
## 3           0.716
## 4           0.645
## 5           1.815
## 6           1.815
## 7           1.621
## 8           1.621
## 9           1.621
## 10          0.645
## 11          0.181
## 12          0.181
## 13          0.792
## 14          0.806
## 15          0.806
## 16          0.645
## 17          0.806
## 18          0.806

Data Recruitment Plan and Stoppping Requirement

Data collection starts in the week beginning 18/06/19 via a mixture of opt in and opt out schools recruiment. Children form yr 3 classes will be randomly assigned to one of the three conditions. The same exclusion criteria will be used as above.

Before the end of the school term (July 20th), we aim to collect 20 participants per condition. The data will be inspected at that point (NOTE: we will inspect the data even if we don’t have a suitable number of participants).

If we don’t have BF>3 OR BF<1/3 for our key predictions (i.e. hv>lv and skew>hv in overall analysis of generalization test and/or analsyis of generalization testing probing knowlege of spatial-prepositions we wil continue to collect participants, adding in 10 per condition, checking again etc.

One caveat: for the skew condition, an important check will be whether, in training, there is stronger performance with the repeated sentences than the non repeated sentences. If we do not find stronger performance with the repeated sentences, this might suggest that our skew manipulation may not be working. In that case, we may choose to stop this condition early and attempt a new design.

We acknowledge that increasing the sample in this way affects the interpreation of p values. For that reason, we regard the BF values as our key inferential statistic.