SOL BDA

SOL Experiment

• 29 kids complete ASL version of the LWL task (Fernald et al., 1998). Each kid is given 32 2-AFC trials. We want to analyze the relations between RT and age/productive ASL vocabulary.

• But RTs are only meaningful if they are generated by the underlying process of interest: speed of lexcial access. Put another way, you can be fast (or slow) in the VLP task for the wrong reasons, because you are just guessing.

• So we want some way to flag participants that we think are likely to be guessing on our task.

Data structure

Sub.Num	age_cdi_months	C_T_count	total_trials_shifting	mean_correct_rt	C_T_prop
30009	40	16	22	1028.571	0.73
30010	44	19	25	1187.000	0.76
30011	16	3	4	1633.000	0.75
30013	31	9	9	1140.000	1.00
30014	33	10	13	1283.300	0.77
30015	21	6	8	1258.250	0.75

Visualize

Model (Lee and Wagenmakers, 2013)

• We assume there are two different groups of people with different probabilities of success, and we want to know something about the overall rate of success and the unobserved group memberships.

• The probability of success on each question for each person is the rate θi. This rate is either ψ, if the person is in the guessing group, or φ if the person is in the knowledge group.

• Which group each person belongs to is determined by a binary indicator variable zi (0: guessing; 1: knowledge)

• We assume each of these indicator variables is equally likely to be 0 or 1 a priori, so they have the prior zi ∼ Bernoulli(􏰀1/2).

• For the guessing group, we assume that the rate is ψ = 1/2.

• For the knowledge group, we use a prior where all rate possibilities greater than 0.5 are equally likely, so that φ ∼ Uniform(􏰀0.5, 1􏰁.)

# Note: I adjusted the model to allow for different number of trials for each participants.
# change n to n[i] in the for loop.

cat('# Exam Scores
model{
  # Each Person Belongs To One Of Two Latent Groups
  for (i in 1:p){
    z[i] ~ dbern(0.5)
  }
  # First Group Guesses
  psi <- 0.5
  # Second Group Has Some Unknown Greater Rate Of Success
  phi ~ dbeta(1,1)I(0.5,1)
  # Data Follow Binomial With Rate Given By Each Persons Group Assignment
  for (i in 1:p){
    theta[i] <- equals(z[i],0)*psi+equals(z[i],1)*phi
    k[i] ~ dbin(theta[i],n[i])
  }
}', file={f<-tempfile()})

Initial values

1. Number of participants
2. Number correct for each participant
3. Number of trials for each participant

p <- nrow(df_correct)                         # number of participants
k <- df_correct$C_T_count              # number correct for each participant
n <- df_correct$total_trials_shifting  # number of trials for each participant

data <- list("p", "k", "n")                             # data passed on to JAGS
myinits <-  list(list(phi = 0.9, z = round(runif(p)))) # Initial group assignment

Parameters to be monitored

1. φ (the success rate of the knowledge group)
2. z (group membership: knowledge or guessing)

parameters <- c("phi","z") 

# get samples
samples <- jags(data, inits=myinits, parameters,
                model.file=f, n.chains=1, n.iter=1000, 
                n.burnin=1, n.thin=1, DIC=T)

## Compiling model graph
##    Resolving undeclared variables
##    Allocating nodes
##    Graph Size: 238
## 
## Initializing model

# extract values from model
df <- data.frame(phi = samples$BUGSoutput$sims.list$phi,
                 z = samples$BUGSoutput$sims.list$z)

kable(head(df))

phi	z.1	z.2	z.3	z.4	z.5	z.6	z.7	z.8	z.9	z.10	z.11	z.12	z.13	z.14	z.15	z.16	z.17	z.18	z.19	z.20	z.21	z.22	z.23	z.24	z.25	z.26	z.27	z.28	z.29
0.7527023	1	1	1	1	1	0	1	0	1	1	1	1	1	1	1	1	1	1	1	1	1	1	0	1	1	1	1	0	0
0.8160025	1	1	1	1	1	1	1	0	1	1	0	1	1	0	1	1	1	1	1	1	1	1	0	1	1	1	0	0	0
0.7847030	1	1	1	1	1	1	1	0	1	1	0	1	1	0	1	1	1	1	0	1	1	1	0	1	1	1	1	1	0
0.7858339	0	1	1	1	1	1	1	0	1	1	0	1	1	0	1	1	1	1	1	1	1	0	0	1	0	1	1	0	0
0.7676584	0	1	1	1	1	1	0	0	1	1	0	0	1	1	1	1	1	1	1	1	1	1	0	1	1	0	0	0	0
0.8074781	1	1	1	1	1	0	1	0	1	1	1	1	1	1	1	1	1	1	1	1	1	0	0	1	1	1	0	0	0

Visualize parameter values

Flag guessers

Need some criterion for flagging participant as guessing.

Sub.Num	posterior_mass	C_T_prop
30018	0.99	0.54
30028	0.78	0.58
30051	0.98	0.42
30086	0.97	0.29
30088	0.94	0.33

Compare to binomial test against chance

Sub.Num	binom_p	C_T_prop
30011	0.3125000	0.75
30015	0.1445313	0.75
30018	0.4252770	0.54
30024	0.1661530	0.65
30028	0.3872070	0.58
30050	0.3437500	0.67
30051	0.8061523	0.42
30085	0.2539063	0.67
30086	0.9375000	0.29
30088	0.8906250	0.33