1 Data

This analysis will test whether the relative distribution of response strategies varies across participants’ writing levels.

data <- read_delim("long_data_format.tsv", delim="\t")

The outcome variable has 6 possible response categories:

unique(data$strategy)

## [1] "Interactional" "Communication" "Affective"     "Approach"     
## [5] "Cognitive"     "Metacognitive"

We will test if the relative distribution of these strategies varies across the participants’ levels:

unique(data$level)

## [1] "L2" "L1" "L3" "L4"

2 Models

The strategy data were modelled in categorical mixed-effects models with random intercepts for participants. We compared a model with level as fixed effect (m1) against another model without level as fixed effect (m0).

m0 <- bf(strategy ~ 1 + (1|subno))
m1 <- bf(strategy ~ level + (1|subno))

Models were run in brms using the following specifications.

m0 <- brm(m0, 
          family = categorical(),
          cores = 3,
          chains = 3,
          iter = 4000,
          sample_prior = T,
          data = data)

m1 <- brm(m1, 
          family = categorical(), 
          cores = 3,
          chains = 3,
          iter = 4000,
          sample_prior = T,
          data = data)

3 Model comparison

Models were compared using leave-one-out cross-validation. The model comparison shows that adding level as a fixed effect did not increase the predictive performance. In other words, the distribution of strategies is similar across levels.

model_comparison <- loo(m0, m1)

##    elpd_diff se_diff elpd_loo se_elpd_loo
## m0      0.00    0.00 -1803.80       22.58
## m1     -4.33    3.01 -1808.13       23.26

4 Posterior

The posterior probability of response categories is summarised by levels in Figure 4.1. Similar distributions of response categories can be observed across levels with, in general, a larger probability to observe responses that are categorised as interactional and a small probability of observe responses that are categorised as affective.

Figure 4.1: Estimated cell means for response categories by levels with 95% PIs (probability intervals).