CHaPP
1 Lift duration
To account for lift durations that have to do with large displacements, we fitted a Generalized additive model on the log lift duration with log displacement as predictor and extracted the residual lift duration after accounting for displacement.
m <- mgcv::gam(log_y ~ s(ctr_log_disp), data = inair, method = 'REML')
inair <- mutate(inair, resid = as.vector(coef(m)[1]) + resid(m)) 1.1 Uni-modal mixed-effects model
Originally I was including a linear-mixed effects model as placeholder for the mixture model. I thought I’d just leave it for now, for comparison to the mixture model below. The mixed-effects model shows evidence for two main effects that seem to be associated with two different properties as uncovered by the mixture model. It’s save to jump ahead to the mixture model though.
For the linear-mixed effects model we used the following contrast matrix:
test condition interaction
post_control 1/4 1/4 1/4
post_intervention 1/4 -1/4 -1/4
pre_control -1/4 1/4 -1/4
pre_intervention -1/4 -1/4 1/4 The linear-mixed effects model on the residual lift duration after accounting for displacement suggests evidence for main effects of test and condition but not evidence for an interaction. Figure 1.1 shows longer lift duration in the pre test compared to the post test and for the intervention condition compared to the control condition.
# Linear mixed-effects model
m <- lme4::lmer(resid ~ x + (test|ppt), data = inair)
summary(m)$coef %>% round(2) Estimate Std. Error t value
(Intercept) 6.84 0.03 208.19
xtest -0.65 0.07 -9.64
xcondition -0.29 0.13 -2.17
xinteraction 0.07 0.07 1.10
Figure 1.1: Estimated marginal means for each condition with 95% CIs.
1.2 Bi-modal mixed-effects model (mixture model)
The lift duration was model as mixture process of two log-Gaussians. The model parameters are the brief residual lift duration, the slowdown for longer lift durations and the probability of longer lift durations. The brief residual lift duration was allowed to vary by participant to achieve partial pooling. All of these parameters were modeled by condition and by test.
The posterior estimates for each parameter by condition and test are shown in Figure 1.2. Evidence for main effects and their interaction will be determined next.
Figure 1.2: Posterior parameter estimates of the mixture model on the residual lift duration. Shown are the estimated cell means with 95% PIs.
Main effects and interaction of test and condition obtained from the mixture model can be found in Table 1.1. Evidence is shown by model parameter. The results show strong evidence for a main effect of test in the brief lift duration and in the probability of longer lift durations. Brief lift duration was shorter and the probability of longer lift durations was lower in the post test compared to the pre test. A main effect of condition was found for the slowdown for longer lift durations. The intervention group showed a larger slowdown for longer lift durations compared to the control group. In other words, brief lift durations were equivalent in the control and intervention group but when pauses occurred, pauses were longer in the intervention group. Also, there was weak evidence for a main effect of condition in the probability of longer lift durations, suggesting that the intervention group had a somewhat higher predisposition to pause during writing. Evidence for all other predictors and importantly interactions was negligible.
| Predictor | Estimate | H0 | H1 | Estimate | H0 | H1 | Estimate | H0 | H1 |
|---|---|---|---|---|---|---|---|---|---|
| Test | -0.25 [-0.28 – -0.21] | 0 | > 100 | 0.13 [0.02 – 0.23] | 0.57 | 1.76 | -0.39 [-0.54 – -0.25] | 0 | > 100 |
| Condition | 0.01 [-0.11 – 0.13] | 8.38 | 0.12 | -0.2 [-0.31 – -0.09] | 0.02 | 47.66 | -0.26 [-0.4 – -0.11] | 0.03 | 39.43 |
| Test : Condition | 0.06 [0 – 0.13] | 2.77 | 0.36 | 0.15 [-0.06 – 0.36] | 1.64 | 0.61 | -0.09 [-0.38 – 0.21] | 2.85 | 0.35 |
| Note: | |||||||||
| Colon indicates interaction. PI is the probability interval. H1 indicates the statistical support for the alternative hypothesis over the null corresponding (Bayes Factor). H0 indicates the evidence in support of the null hypothesis. |
The mixture components implied by the model are visualised in Figure 1.3. Note, the colour coding isn’t consistent with the other plots in this section. I had to do this, otherwise this visualisation wouldn’t really highlight anything meaningful. Generally, I think the relevant effects are harder to see in this plot.
Figure 1.3: Mixture model plot for residual lift duration. Shown are the distributions for both mixture components (short and longer lift durations) and their respective weightings implied by the mixture model.
2 snVP
Prior to analysis, we extracted the residual snVP after accounting for number of letters in the text using a generalised adaptive model with negative binomial likelihood function.
m <- gam(y ~ s(ctr_lett), data = snvp, family = nb())
snvp <- mutate(snvp, resid = as.vector(coef(m)[1]) + resid(m))A Bayesian mixed-effects model was fitted on snVP using a Gaussian probability model. Fixed effects were main effects and the interaction of test (levels: pretest, posttest) and condition (levels: intervention, control). Random intercepts were included for participants with by-participant slope adjustments for test.
# Load model
fit <- readRDS(str_c(path_stan,"snvp.rda"))
# Probability model
fit$family
Family: gaussian
Link function: identity # Model formula
fit$formularesid ~ x + (test | ppt) where x is the contrast matrix:
test condition interaction
post_control 1/4 1/4 1/4
post_intervention 1/4 -1/4 -1/4
pre_control -1/4 1/4 -1/4
pre_intervention -1/4 -1/4 1/4 Results can be found in Table 2.1. There is strong evidence for a main effect of test suggesting lower snVP in the post test compared to the pre test condition. Estimated cell means can be found in Figure 2.1.
| Predictor | Estimate | H0 | H1 |
|---|---|---|---|
| Test | -1.15 [-1.54 – -0.77] | 0 | > 100 |
| Condition | -0.18 [-0.79 – 0.44] | 5.55 | 0.18 |
| Test : Condition | 0.31 [-0.06 – 0.68] | 2.92 | 0.34 |
| Note: | |||
| Colon indicates interaction. PI is the probability interval. H1 indicates the statistical support for the alternative hypothesis over the null corresponding (Bayes Factor). H0 indicates the evidence in support of the null hypothesis. |
Figure 2.1: Posterior snVP. Shown are the estimated cell means for each condition with 95% PIs.
3 Stop duration
Below are the results of a frequentist mixed-effects model and a Bayesian bi-modal mixed-effects (mixture model). Again, I included the uni-modal results just as comparisons (as above), not because I’m suggesting that we use these.
3.1 Uni-modal mixed-effects model
Contrast matrix:
test condition interaction
post_control 1/4 1/4 1/4
post_intervention 1/4 -1/4 -1/4
pre_control -1/4 1/4 -1/4
pre_intervention -1/4 -1/4 1/4 A linear mixed-effects model on the log stop duration suggests evidence for main effects of test and condition but no evidence for an interaction.
# Linear mixed-effects model
m <- lme4::lmer(log_y ~ x + (test|ppt), data = stops)
summary(m)$coef %>% round(2) Estimate Std. Error t value
(Intercept) 3.95 0.01 338.81
xtest -0.20 0.02 -7.88
xcondition -0.15 0.05 -3.32
xinteraction 0.01 0.02 0.47Marginal estimated means can be found in Figure 3.1. Stop duration was found longer in the pre test compared to the post test and longer in the intervention condition compared to the control condition.
Figure 3.1: Estimated marginal means of stop duration for each condition with 95% CIs.
3.2 Bi-modal mixed-effects model (mixture model)
The stop duration was model as mixture process of two log-Gaussians. The model parameters are the brief stop duration, the slowdown for longer stop durations, and the probability of longer stop durations. The brief stop duration was allowed to vary by participant to achieve partial pooling. All of these three parameters were modeled by condition and by test.
The posterior estimates for each parameter by condition and test are shown in Figure 3.2. Evidence for main effects and their interaction will be determined next.
Figure 3.2: Posterior parameter estimates of the mixture model on the stop duration. Shown are the estimated cell means with 95% PIs.
Main effects and interaction of test and condition obtained from the mixture model can be found in Table 3.1. Evidence is shown by model parameter. The results show strong evidence for a main effect of test and a main effect of condition in the probability of longer stop durations. The probability of longer stop durations was lower in the post test compared to the pre test, and higher in the intervention condition compared to the control condition. Evidence for all other predictors and importantly interactions was negligible.
Note: The PI and the plot above suggest that the probability of longer stop duration was different between groups in the post test but not in the pre test (or there was a change from pre to post test in the control group but not in the intervention group). I can put numbers on this if needed but the BF doesn’t support an interaction so we might skip this. What do you think?
| Predictor | Estimate | H0 | H1 | Estimate | H0 | H1 | Estimate | H0 | H1 |
|---|---|---|---|---|---|---|---|---|---|
| Test | -0.03 [-0.05 – -0.01] | 2.4 | 0.42 | -0.03 [-0.06 – 0.01] | 9.03 | 0.11 | -0.2 [-0.32 – -0.09] | 0.01 | 80.1 |
| Condition | -0.01 [-0.05 – 0.02] | 24.69 | 0.04 | -0.02 [-0.06 – 0.01] | 11.87 | 0.08 | -0.19 [-0.3 – -0.08] | 0.02 | 51.14 |
| Test : Condition | 0.03 [-0.01 – 0.07] | 8.47 | 0.12 | 0.04 [-0.04 – 0.11] | 7.74 | 0.13 | -0.23 [-0.45 – -0.01] | 0.57 | 1.76 |
| Note: | |||||||||
| Colon indicates interaction. PI is the probability interval. H1 indicates the statistical support for the alternative hypothesis over the null corresponding (Bayes Factor). H0 indicates the evidence in support of the null hypothesis. |
The mixture components implied by the model are visualised in Figure 3.3. Note, the colour coding isn’t consistent with the other plots in this section. I had to do this, otherwise this visualisation wouldn’t really highlight anything meaningful. Generally, I think the relevant effects are harder or impossible to see in this plot.
Figure 3.3: Mixture model plot for stop duration (in msecs). Shown are the distributions for both mixture components (short and longer stop durations) and their respective weightings implied by the mixture model.