1 Data analysis

For all analyses, we report the most probable posterior (i.e. inferred) parameter value as well as the interval that contains the posterior parameter value with a 95% probability; 95% probability intervals (henceforth, PI). Also we calculated the statistical support for the effects of interest and the support for the alternative hypothesis over the null hypothesis. This evidence was obtained using Bayes Factors (henceforth, BF) calculated using the Savage-Dickey method (see, e.g., Dickey et al., 1970; Wagenmakers et al., 2010). A BF larger than 5 indicate moderate and larger than 10 strong evidence for a statistically meaningful effect compared to the null hypothesis (see, e.g., Jeffreys, 1961; Lee & Wagenmakers, 2014). For example BF=2 reflect that the alternative hypothesis is two times more likely than the null hypothesis given the evidence. Priors for all effects were weakly informative. We used these weakly informative priors favoring the null hypothesis over the alternative hypothesis for the slope parameters because BFs are sensitive to the distribution of the prior. Thus, our priors are not favoring the alternative hypothesis.

Data were analysed in Bayesian mixed effects models (Gelman et al., 2014; McElreath, 2016). The R (R Core Team, 2020) package rstan (Stan Development Team, 2024) was used to interface with the probabilistic programming language Stan (Carpenter et al., 2016) which was used to implement all models. Models were fitted with weakly informative priors (see McElreath, 2016), and run with at least 10,000 iterations on 3 chains with a warm-up of 5,000 iterations and no thinning. Model convergence was confirmed by the Rubin-Gelman statistic (\(\hat{R}\) = 1) (Gelman & Rubin, 1992) and inspection of the Markov chain Monte Carlo chains.

2 Sample overview

• N per gameplan / session etc
• text length per gameplan / session etc (or report as quality measures)
• number of transitions per gameplan / session etc

3 Do process measures predict text quality?

The text composition analysis involved overall ratings (by-participant median rating across rating criterion and rater rounded without decimals and stored as ordered factor), text length measured as number of characters, words, sentences and paragraphs, the mean word length and the mean sentence length, and informational density measured as the ratio of open to closed class words, and lexically diversity measured using the MTLD statistic (McCarthy, 2005) as a sensitive and text-length independent measure of lexical diversity (Torruella & Capsada, 2013).

As predictors we included the by-participant process measures reported above. All predictors were standardised (centred and scaled). Some predictors were log transformed before scaling to reduce positive skew.

Text composition data were modelled in a multi-variate mixed-effects models with the following specification and distribution families:

rating ~ ppause_post_sentence + ppause_post_word + ppause_within_word + pseq + n_edges + n_jumps + n_major_block + pedit_post_sentence + pedit_post_word + pedit_within_word + time_writing_mins + time_planning_mins + (session | ppt) 
rating_orga ~ ppause_post_sentence + ppause_post_word + ppause_within_word + pseq + n_edges + n_jumps + n_major_block + pedit_post_sentence + pedit_post_word + pedit_within_word + time_writing_mins + time_planning_mins + (session | ppt) 
rating_argu ~ ppause_post_sentence + ppause_post_word + ppause_within_word + pseq + n_edges + n_jumps + n_major_block + pedit_post_sentence + pedit_post_word + pedit_within_word + time_writing_mins + time_planning_mins + (session | ppt) 
rating_evid ~ ppause_post_sentence + ppause_post_word + ppause_within_word + pseq + n_edges + n_jumps + n_major_block + pedit_post_sentence + pedit_post_word + pedit_within_word + time_writing_mins + time_planning_mins + (session | ppt) 
oc_ratio ~ ppause_post_sentence + ppause_post_word + ppause_within_word + pseq + n_edges + n_jumps + n_major_block + pedit_post_sentence + pedit_post_word + pedit_within_word + time_writing_mins + time_planning_mins + (session | ppt) 
char_count ~ ppause_post_sentence + ppause_post_word + ppause_within_word + pseq + n_edges + n_jumps + n_major_block + pedit_post_sentence + pedit_post_word + pedit_within_word + time_writing_mins + time_planning_mins + (session | ppt) 
word_count ~ ppause_post_sentence + ppause_post_word + ppause_within_word + pseq + n_edges + n_jumps + n_major_block + pedit_post_sentence + pedit_post_word + pedit_within_word + time_writing_mins + time_planning_mins + (session | ppt) 
sentence_count ~ ppause_post_sentence + ppause_post_word + ppause_within_word + pseq + n_edges + n_jumps + n_major_block + pedit_post_sentence + pedit_post_word + pedit_within_word + time_writing_mins + time_planning_mins + (session | ppt) 
n_paragraph ~ ppause_post_sentence + ppause_post_word + ppause_within_word + pseq + n_edges + n_jumps + n_major_block + pedit_post_sentence + pedit_post_word + pedit_within_word + time_writing_mins + time_planning_mins + (session | ppt) 
M_word_len ~ ppause_post_sentence + ppause_post_word + ppause_within_word + pseq + n_edges + n_jumps + n_major_block + pedit_post_sentence + pedit_post_word + pedit_within_word + time_writing_mins + time_planning_mins + (session | ppt) 
M_sent_len ~ ppause_post_sentence + ppause_post_word + ppause_within_word + pseq + n_edges + n_jumps + n_major_block + pedit_post_sentence + pedit_post_word + pedit_within_word + time_writing_mins + time_planning_mins + (session | ppt) 
lexd ~ ppause_post_sentence + ppause_post_word + ppause_within_word + pseq + n_edges + n_jumps + n_major_block + pedit_post_sentence + pedit_post_word + pedit_within_word + time_writing_mins + time_planning_mins + (session | ppt) 

$rating

Family: cumulative 
Link function: logit 
Threshold: flexible 


$ratingorga

Family: cumulative 
Link function: logit 
Threshold: flexible 


$ratingargu

Family: cumulative 
Link function: logit 
Threshold: flexible 


$ratingevid

Family: cumulative 
Link function: logit 
Threshold: flexible 


$ocratio

Family: lognormal 
Link function: identity 


$charcount

Family: negbinomial 
Link function: log 


$wordcount

Family: negbinomial 
Link function: log 


$sentencecount

Family: poisson 
Link function: log 


$nparagraph

Family: poisson 
Link function: log 


$Mwordlen

Family: gaussian 
Link function: identity 


$Msentlen

Family: gaussian 
Link function: identity 


$lexd

Family: gaussian 
Link function: identity 

Session was included as by-participant random slopes adjustment.

The inferential results for main effects and interactions can be found in Table 3.1.

Posterior estimates for the text composition scores are visualised in Figure 3.1.

Figure 3.1: Estimated relationships between process measures and text composition scores with 95% PIs (probability intervals).

4 Did participant adhere to the assigned gameplan?

4.1 Primary gameplans

4.1.1 Do not edit (D2W)

4.1.1.1 Editing

Participants estimates were obtained using a binomial logistic model with the following specifications:

edits | trials(total) ~ 0 + condition + (condition | ppt) 

Family: binomial 
Link function: logit 

where condition consists of session type and transition location with the levels

[1] "diagnostic_post-sentence"   "diagnostic_post-word"      
[3] "diagnostic_within-word"     "followup_post-sentence"    
[5] "followup_post-word"         "followup_within-word"      
[7] "intervention_post-sentence" "intervention_post-word"    
[9] "intervention_within-word"  

The estimate for participants with the D2W gameplan or no gameplan are visualised in Figure 4.1 with different colouring for participants that showed process improvement vs participant that did not show evidence for process improvement.

Figure 4.1: Modelled mean by-ppt proportion of edits separated by gameplan and transition location.

The participant estimates were used to evaluate the intervention effect.

The model formula was the following:

pedit | trials(100) ~ condition + (session | ppt) 

Family: binomial 
Link function: logit 

where condition consists of gameplan (levels: D2W, no plan), session type and transition location with the levels

 [1] "D2W_diagnostic_post-sentence"      "D2W_diagnostic_post-word"         
 [3] "D2W_diagnostic_within-word"        "D2W_followup_post-sentence"       
 [5] "D2W_followup_post-word"            "D2W_followup_within-word"         
 [7] "D2W_intervention_post-sentence"    "D2W_intervention_post-word"       
 [9] "D2W_intervention_within-word"      "noplan_diagnostic_post-sentence"  
[11] "noplan_diagnostic_post-word"       "noplan_diagnostic_within-word"    
[13] "noplan_followup_post-sentence"     "noplan_followup_post-word"        
[15] "noplan_followup_within-word"       "noplan_intervention_post-sentence"
[17] "noplan_intervention_post-word"     "noplan_intervention_within-word"  

and as contrast we used sum coded main effects and interactions of gameplan, session type, and transition location.

Table 4.1: Editing frequency effects on logit scale.

Predictor	Est. with 95% PI	BF
Main effects
Gameplan (D2W, no plan)	-0.03 [-0.96 – 0.87]	0.84
Session 1 (diagnostic, intervention)	1.65 [1.34 – 1.96]	> 100
Session 2 (diagnostic, follow-up)	1.78 [1.39 – 2.16]	> 100
Location 1 (before word, sentence)	-0.42 [-0.7 – -0.14]	17.11
Location 2 (before word / sentence, within word)	1.16 [0.66 – 1.66]	> 100
Two-way interactions
Gameplan : Session 1	1.17 [0.86 – 1.47]	> 100
Gameplan : Session 2	0.59 [0.23 – 0.96]	37.51
Gameplan : Location 1	-0.01 [-0.29 – 0.27]	0.29
Gameplan : Location 2	-0.43 [-0.91 – 0.02]	2.27
Session 1 : Location 1	-0.12 [-0.36 – 0.11]	0.39
Session 1 : Location 2	-0.08 [-0.45 – 0.29]	0.39
Session 2 : Location 1	0.22 [-0.01 – 0.45]	1.42
Session 2 : Location 2	0.36 [-0.02 – 0.74]	1.97
Three-way interactions
Gameplan : Sess 1 : Loc 1	-0.26 [-0.5 – -0.04]	2.79
Gameplan : Sess 1 : Loc 2	-0.22 [-0.6 – 0.17]	0.67
Gameplan : Sess 2 : Loc 1	-0.2 [-0.43 – 0.03]	0.89
Gameplan : Sess 2 : Loc 2	-0.23 [-0.61 – 0.14]	0.74
Note:
Colon indicates interactions. BF is the evidence in favour of the alternative hypothesis over the null hypothesis

Figure 4.2: Estimated cell means for editing frequency with 95% PIs (probability intervals).

Cellmeans corresponding to Figure 4.2 can be found in Table 4.2 with pairwise comparisons between the diagnostic session and each the intervention session and the followup session.

Table 4.2: Editing frequency. Cell means for proportion of transitions terminated by an editing operation and their difference on logit scale shown with 95% PIs in brackets.

		Session type			Comparison
Gameplan	Transition location	diagnostic	intervention	followup	diagnostic vs intervention	diagnostic vs followup
D2W: do not edit	post-sentence	.17 [.15, .19]	.12 [.11, .14]	.12 [.10, .13]	-0.38 [-0.53, -0.23], BF>100	-0.47 [-0.63, -0.3], BF>100
D2W: do not edit	post-word	.18 [.16, .21]	.12 [.10, .13]	.13 [.12, .15]	-0.58 [-0.73, -0.44], BF>100	-0.45 [-0.61, -0.3], BF>100
D2W: do not edit	within-word	.17 [.15, .19]	.11 [.09, .12]	.12 [.11, .14]	-0.58 [-0.73, -0.43], BF>100	-0.43 [-0.59, -0.27], BF>100
no gameplan	post-sentence	.16 [.14, .17]	.14 [.13, .16]	.12 [.10, .13]	-0.13 [-0.22, -0.04], BF=3.4	-0.36 [-0.46, -0.26], BF>100
no gameplan	post-word	.15 [.14, .17]	.15 [.13, .16]	.14 [.12, .15]	-0.06 [-0.15, 0.03], BF=0.1	-0.15 [-0.24, -0.05], BF=3.39
no gameplan	within-word	.13 [.12, .14]	.13 [.11, .14]	.12 [.11, .13]	-0.05 [-0.14, 0.04], BF=0.08	-0.09 [-0.19, 0.01], BF=0.25
Note:
PIs are probability intervals. BF is the evidence in favour of the alternative hypothesis over the null hypothesis.

4.1.1.2 Length of production sequence

Participants estimates were obtained using a negative binomial model with the following specifications:

pseq_len ~ 0 + session + (session | ppt) 

Family: negbinomial 
Link function: log 

The estimate for participants with the D2W gameplan or no gameplan are visualised in Figure 4.3 with different colouring for participants that showed process improvement vs participant that did not show evidence for process improvement. Improvement was considered when both the intervention and the followup showed longer average production sequences.

Figure 4.3: Modelled mean by-ppt length of production sequence terminated by an editing operation by gameplan.

Data were modelled as a lognormal mixed effects models (because the production sequence length is no longer discrete but the modelled participant average) with the following specification:

pseq ~ condition + (session | ppt) 

Family: lognormal 
Link function: identity 

where condition consists of session type and gameplan

[1] "D2W_diagnostic"      "D2W_followup"        "D2W_intervention"   
[4] "noplan_diagnostic"   "noplan_followup"     "noplan_intervention"

which was sum coded to return main effects and interactions of session type and gameplan.

The inferential results for main effects and interactions can be found in Table 4.3.

Table 4.3: Main effects and interactions of the average production sequence length on log scale with 95% PIs in brackets.

Predictor	Est. with 95% PIs	BF
Main effects
Gameplan (D2W, no plan)	-0.13 [-0.53 – 0.27]	0.46
Session 1 (diagnostic, intervention)	-0.26 [-0.3 – -0.22]	> 100
Session 2 (diagnostic, follow-up)	-0.18 [-0.2 – -0.16]	> 100
Two-way interactions
Gameplan : Session 1	-0.14 [-0.18 – -0.1]	> 100
Gameplan : Session 2	-0.08 [-0.1 – -0.06]	> 100
Note:
Colon indicates interactions. BF is the evidence in favour of the alternative hypothesis over the null hypothesis

Posterior estimates for the process measures of production sequence length are visualised in Figure 4.4.

Figure 4.4: Estimated cell means for length of production sequence with 95% PIs (probability intervals).

Cellmeans corresponding to Figure 4.4 can be found in Table 4.4 with pairwise comparisons between the diagnostic session and each the intervention session and the followup session.

Table 4.4: Production sequence. Cell means for production sequence length and their difference on log scale shown with 95% PIs in brackets.

	Session type			Comparison
Gameplan	diagnostic	intervention	followup	diagnostic vs intervention	diagnostic vs followup
D2W: do not edit	19.07 [16.99, 21.48]	23.27 [20.70, 26.29]	21.75 [19.41, 24.51]	0.2 [0.17, 0.23], BF>100	0.13 [0.11, 0.15], BF>100
no gameplan	21.49 [19.90, 23.11]	22.77 [21.07, 24.56]	22.57 [20.90, 24.28]	0.06 [0.04, 0.08], BF>100	0.05 [0.04, 0.06], BF>100
Note:
PIs are probability intervals. BF is the evidence in favour of the alternative hypothesis over the null hypothesis.

4.1.2 Pause sentence-initially (PSS)

Pauses in the key-transition data were estimated for each participant by transition location (levels: post-sentence, post-word, within word) and sessions type (levels: diagnostic, intervention, followup) using Bayesian hierarchical mixture models as described in Roeser et al. (2021) using the the probabilistic programming language Stan (Carpenter et al., 2016; Hoffman & Gelman, 2014). The model that was used to estimated the pausing probability summarized in equation @ref{eq:mog}. The parameter of interest is the mixing probability of the distribution of long latencies, indicated as \(\theta\).

\[ \begin{align}\tag{4.1} \text{iki}_{i} \sim \ & \theta_\text{session[i], location[i]} \cdot \text{logN}(\beta + \delta_\text{location} + u_i, \sigma_\text{location}') \ +\\ & (1-\theta_\text{session[i], location[i]}) \cdot \text{logN}(\beta + u_i, \sigma_\text{location})\\ u_i \sim \ & \text{N}(0, \sigma_\text{ppt})\\ \text{constraints: } & \delta, \sigma_\text{ppt}, \sigma', \sigma > 0, \sigma' > \sigma\\ \end{align} \]

The estimate for participants with the PSS gameplan or no gameplan are visualised in Figure 4.5 with different colouring for participants that showed process improvement vs participant that did not show evidence for process improvement. A process was considered to show improvement when the proportion of sentence-initial pauses increased during both the intervention and followup session relative to the diagnostic session and then pauses at before word or within word locations decreased.

Figure 4.5: Modelled mean by-ppt proportion of pauses separated by gameplan and transition location.

The participant estimates obtained from the mixture model were used to evaluate the intervention effect.

The model formula was the

value | trials(100) ~ condition + (sess | ppt) 

Family: binomial 
Link function: logit 

where value is the by-participant hesitation / pausing probability and condition consists of gameplan (levels: PSS, no plan), session type and transition location with the levels

 [1] "noplan_diagnostic_post-sentence"   "noplan_diagnostic_post-word"      
 [3] "noplan_diagnostic_within-word"     "noplan_followup_post-sentence"    
 [5] "noplan_followup_post-word"         "noplan_followup_within-word"      
 [7] "noplan_intervention_post-sentence" "noplan_intervention_post-word"    
 [9] "noplan_intervention_within-word"   "PSS_diagnostic_post-sentence"     
[11] "PSS_diagnostic_post-word"          "PSS_diagnostic_within-word"       
[13] "PSS_followup_post-sentence"        "PSS_followup_post-word"           
[15] "PSS_followup_within-word"          "PSS_intervention_post-sentence"   
[17] "PSS_intervention_post-word"        "PSS_intervention_within-word"     

and as contrast we used sum coded main effects and interactions of gameplan, session type, and transition location.

The results of the intervention analysis on the pausing probability is summarised in Table 4.5.

Table 4.5: Pausing probability on logit scale.

Predictor	Est. with 95% PI	BF
Main effects
Gameplan (PSS, no plan)	-0.19 [-1.17 – 0.72]	0.89
Session 1 (diagnostic, intervention)	0.19 [-0.18 – 0.57]	0.61
Session 2 (diagnostic, follow-up)	0.75 [0.35 – 1.15]	> 100
Location 1 (before word, sentence)	5.01 [4.76 – 5.25]	> 100
Location 2 (before word / sentence, within word)	39.98 [39.32 – 40.66]	> 100
Two-way interactions
Gameplan : Session 1	0.03 [-0.34 – 0.4]	0.37
Gameplan : Session 2	0.03 [-0.35 – 0.42]	0.38
Gameplan : Location 1	1.02 [0.78 – 1.26]	> 100
Gameplan : Location 2	0.05 [-0.51 – 0.61]	0.55
Session 1 : Location 1	0.05 [-0.14 – 0.25]	0.21
Session 1 : Location 2	0.18 [-0.28 – 0.65]	0.6
Session 2 : Location 1	-0.15 [-0.35 – 0.04]	0.58
Session 2 : Location 2	0.3 [-0.16 – 0.78]	0.96
Three-way interactions
Gameplan : Sess 1 : Loc 1	0.21 [0.02 – 0.4]	1.91
Gameplan : Sess 1 : Loc 2	-0.16 [-0.62 – 0.29]	0.53
Gameplan : Sess 2 : Loc 1	0.28 [0.09 – 0.48]	9.03
Gameplan : Sess 2 : Loc 2	0.05 [-0.4 – 0.51]	0.44
Note:
Colon indicates interactions. BF is the evidence in favour of the alternative hypothesis over the null hypothesis

The posterior effects are visualised in Figure 4.6.

Figure 4.6: Estimated cell means for pausing probability with 95% PIs (probability intervals).

Cellmeans corresponding to Figure 4.6 can be found in Table 4.6 with pairwise comparisons between the diagnostic session and each the intervention session and the followup session.

Table 4.6: Pausing frequency. Cell means for proportion of transitions terminated by an editing operation and their difference on logit scale shown with 95% PIs in brackets.

		Session type			Comparison
Gameplan	Transition location	diagnostic	intervention	followup	diagnostic vs intervention	diagnostic vs followup
no gameplan	post-sentence	.76 [.74, .78]	.74 [.72, .76]	.73 [.71, .75]	-0.12 [-0.2, -0.03], BF=7.14	-0.21 [-0.29, -0.12], BF>100
no gameplan	post-word	.53 [.50, .55]	.53 [.51, .56]	.49 [.47, .52]	0.02 [-0.06, 0.1], BF=0.18	-0.14 [-0.21, -0.06], BF=35.67
no gameplan	within-word	.06 [.05, .07]	.06 [.05, .07]	.06 [.05, .06]	-0.04 [-0.18, 0.09], BF=0.34	-0.08 [-0.22, 0.06], BF=0.53
PSS: pause sentence-initially	post-sentence	.72 [.69, .75]	.72 [.68, .74]	.71 [.68, .74]	-0.03 [-0.19, 0.12], BF=0.34	-0.06 [-0.21, 0.1], BF=0.41
PSS: pause sentence-initially	post-word	.59 [.56, .63]	.57 [.53, .60]	.53 [.49, .57]	-0.13 [-0.27, 0.02], BF=1.31	-0.29 [-0.44, -0.14], BF>100
PSS: pause sentence-initially	within-word	.06 [.05, .07]	.06 [.05, .07]	.06 [.05, .07]	0.05 [-0.19, 0.28], BF=0.52	-0.08 [-0.33, 0.16], BF=0.62
Note:
PIs are probability intervals. BF is the evidence in favour of the alternative hypothesis over the null hypothesis.

4.1.3 Write linearly (NOJ)

Descriptive data by participant are shown in Figure 4.7. Data are shown for the total number of writing edges, the number of jumps within the text and the total number of major writing blocks.

Figure 4.7: Write linearily gameplan

Data were modelled in a multivariate mixed effects models with the following specification:

n_edges ~ condition + (session | p | ppt) 
n_jumps ~ condition + (session | p | ppt) 
n_major_block ~ condition + (session | p | ppt) 

$nedges

Family: negbinomial 
Link function: log 


$njumps

Family: negbinomial 
Link function: log 


$nmajorblock

Family: poisson 
Link function: log 

where condition consists of session type and gameplan

[1] "NOJ_diagnostic"      "NOJ_followup"        "NOJ_intervention"   
[4] "noplan_diagnostic"   "noplan_followup"     "noplan_intervention"

which was sum coded to return main effects and interactions of session type and gameplan.

The inferential results for main effects and interactions can be found in Table 4.7.

Table 4.7: Main effects and interactions of the linear writing measures on log scale with 95% PIs in brackets.

	No. of jumps		No. of writing edges		No. of major writing blocks
Predictor	Est. with 95% PIs	BF	Est. with 95% PIs	BF	Est. with 95% PIs	BF
Main effects
Gameplan (D2W, no plan)	0.44 [-0.59 – 1.49]	0.71	0.34 [-0.52 – 1.2]	0.59	0.34 [-0.37 – 1.08]	1.06
Session 1 (diagnostic, intervention)	1.44 [1.05 – 1.83]	> 100	1.07 [0.7 – 1.45]	> 100	0.23 [-0.32 – 0.8]	0.77
Session 2 (diagnostic, follow-up)	1.32 [0.89 – 1.76]	> 100	1.02 [0.61 – 1.43]	> 100	0.29 [-0.25 – 0.86]	0.88
Two-way interactions
Gameplan : Session 1	0.82 [0.44 – 1.19]	> 100	0.58 [0.22 – 0.95]	20.21	-0.18 [-0.72 – 0.37]	0.65
Gameplan : Session 2	0.41 [-0.02 – 0.84]	1.24	0.35 [-0.04 – 0.75]	0.92	-0.16 [-0.7 – 0.39]	0.63
Note:
Colon indicates interactions. BF is the evidence in favour of the alternative hypothesis over the null hypothesis

Posterior estimates for the process measures of writing linearity are visualised in Figure 4.8.

Figure 4.8: Estimated cell means for writing linearity with 95% PIs (probability intervals).

Cellmeans corresponding to Figure 4.8 can be found in Table 4.8 with pairwise comparisons between the diagnostic session and each the intervention session and the followup session.

Table 4.8: Linearity. Cell means for counts and their difference on log scale shown with 95% PIs in brackets.

	Session type			Comparison
Gameplan	diagnostic	intervention	followup	diagnostic vs intervention	diagnostic vs followup
No. of jumps
NOJ: write linearly	20.25 [14.71, 27.64]	8.89 [6.1, 12.74]	10.19 [6.96, 14.95]	-0.94 [-1.3, -0.58], BF>100	-0.81 [-1.21, -0.42], BF>100
no gameplan	13.22 [11.48, 15.2]	10.34 [8.81, 12.05]	9.49 [7.99, 11.21]	-0.26 [-0.4, -0.11], BF=16.3	-0.34 [-0.5, -0.19], BF>100
No. of writing edges
NOJ: write linearly	33.92 [24.36, 47.8]	10.96 [7.19, 16.76]	14.26 [9.17, 22.52]	-1.26 [-1.63, -0.88], BF>100	-1 [-1.42, -0.58], BF>100
no gameplan	19.46 [16.68, 22.64]	14.25 [11.87, 17.1]	12.34 [10.08, 14.98]	-0.32 [-0.47, -0.18], BF>100	-0.47 [-0.64, -0.31], BF>100
No. of major writing blocks
NOJ: write linearly	1.44 [1.01, 2.02]	1.4 [0.93, 2.06]	1.35 [0.89, 1.98]	-0.06 [-0.72, 0.59], BF=0.35	-0.12 [-0.79, 0.54], BF=0.37
no gameplan	1.43 [1.19, 1.7]	1.17 [0.96, 1.42]	1.15 [0.93, 1.39]	-0.25 [-0.53, 0.03], BF=0.65	-0.27 [-0.55, 0.01], BF=0.93
Note:
PIs are probability intervals. BF is the evidence in favour of the alternative hypothesis over the null hypothesis.

4.1.4 Revise periodically (PPS)

4.1.4.1 Editing

The participant estimates obtained from a binomial mixed-effects model of all participants described in the section “Do not edit (D2W)” were used to evaluate the intervention effect.

The model formula was the following:

pedit | trials(100) ~ condition + (session | ppt) 

Family: binomial 
Link function: logit 

where pedit is the by-participant proportion of key transitions that terminated in an editing operation and condition consists of gameplan (levels: PPS, no plan), session type and transition location with the levels

 [1] "noplan_diagnostic_post-sentence"   "noplan_diagnostic_post-word"      
 [3] "noplan_diagnostic_within-word"     "noplan_followup_post-sentence"    
 [5] "noplan_followup_post-word"         "noplan_followup_within-word"      
 [7] "noplan_intervention_post-sentence" "noplan_intervention_post-word"    
 [9] "noplan_intervention_within-word"   "PPS_diagnostic_post-sentence"     
[11] "PPS_diagnostic_post-word"          "PPS_diagnostic_within-word"       
[13] "PPS_followup_post-sentence"        "PPS_followup_post-word"           
[15] "PPS_followup_within-word"          "PPS_intervention_post-sentence"   
[17] "PPS_intervention_post-word"        "PPS_intervention_within-word"     

and as contrast we used sum coded main effects and interactions of gameplan, session type, and transition location.

Main effects and interactions are summarised in Table 4.9 for the PPS gameplan.

Table 4.9: Editing frequency effects on logit scale.

Predictor	Est. with 95% PI	BF
Main effects
Gameplan (PPS, no plan)	0.57 [-0.54 – 2.15]	1.55
Session 1 (diagnostic, intervention)	0.25 [-0.16 – 0.68]	0.82
Session 2 (diagnostic, follow-up)	0.72 [0.21 – 1.25]	17.72
Location 1 (before word, sentence)	-0.37 [-0.8 – 0.05]	1.75
Location 2 (before word / sentence, within word)	1.06 [0.35 – 1.81]	48.74
Two-way interactions
Gameplan : Session 1	0.17 [-0.26 – 0.6]	0.57
Gameplan : Session 2	0.36 [-0.15 – 0.88]	1.25
Gameplan : Location 1	-0.04 [-0.46 – 0.4]	0.43
Gameplan : Location 2	0.54 [-0.14 – 1.26]	2.07
Session 1 : Location 1	0.02 [-0.34 – 0.37]	0.35
Session 1 : Location 2	0.11 [-0.41 – 0.64]	0.59
Session 2 : Location 1	0.33 [-0.03 – 0.69]	1.72
Session 2 : Location 2	0.43 [-0.11 – 0.98]	1.92
Three-way interactions
Gameplan : Sess 1 : Loc 1	0.12 [-0.23 – 0.47]	0.45
Gameplan : Sess 1 : Loc 2	0.03 [-0.49 – 0.56]	0.52
Gameplan : Sess 2 : Loc 1	0.09 [-0.27 – 0.45]	0.42
Gameplan : Sess 2 : Loc 2	0.16 [-0.37 – 0.69]	0.64
Note:
Colon indicates interactions. BF is the evidence in favour of the alternative hypothesis over the null hypothesis

Editing estimates obtained from the interaction model are visualised in Figure 4.9 and summarised in Table 4.10 for each design cell.

Figure 4.9: Estimated cell means for editing frequency with 95% PIs (probability intervals).

Cellmeans corresponding to Figure 4.9 can be found in Table 4.10 with pairwise comparisons between the diagnostic session and each the intervention session and the followup session.

Table 4.10: Editing frequency. Cell means for proportion of transitions terminated by an editing operation and their difference on logit scale shown with 95% PIs in brackets.

		Session type			Comparison
Gameplan	Transition location	diagnostic	intervention	followup	diagnostic vs intervention	diagnostic vs followup
no gameplan	post-sentence	.15 [.14, .17]	.14 [.12, .15]	.11 [.10, .13]	-0.13 [-0.22, -0.04], BF=2.41	-0.36 [-0.46, -0.26], BF>100
no gameplan	post-word	.15 [.13, .16]	.14 [.13, .16]	.13 [.12, .15]	-0.05 [-0.14, 0.03], BF=0.09	-0.14 [-0.23, -0.04], BF=2.8
no gameplan	within-word	.13 [.11, .14]	.12 [.11, .14]	.12 [.11, .13]	-0.05 [-0.14, 0.05], BF=0.07	-0.09 [-0.19, 0.02], BF=0.21
PPS: revise periodically	post-sentence	.13 [.10, .15]	.13 [.10, .16]	.11 [.09, .14]	-0.03 [-0.31, 0.25], BF=0.14	-0.21 [-0.51, 0.1], BF=0.39
PPS: revise periodically	post-word	.13 [.11, .16]	.13 [.10, .15]	.13 [.10, .16]	-0.08 [-0.36, 0.2], BF=0.17	-0.05 [-0.34, 0.24], BF=0.16
PPS: revise periodically	within-word	.12 [.10, .15]	.12 [.10, .15]	.12 [.10, .15]	0.01 [-0.28, 0.3], BF=0.15	0.01 [-0.3, 0.31], BF=0.15
Note:
PIs are probability intervals. BF is the evidence in favour of the alternative hypothesis over the null hypothesis.

4.1.4.2 Length of production sequence

Participants estimates were obtained using a negative binomial model with the following specifications:

pseq_len ~ 0 + session + (session | ppt) 

Family: negbinomial 
Link function: log 

The estimate for participants with the PPS gameplan or no gameplan are visualised in Figure 4.10 with different colouring for participants that showed process improvement vs participant that did not show evidence for process improvement. Improvement was considered when both the intervention and the followup showed longer average production sequences.

NOTE: I’m not actually sure if the prediction is that production sequences should become longer or shorter for “revise periodically”

Figure 4.10: Modelled mean by-ppt length of production sequence terminated by an editing operation by gameplan.

Data were modelled as a lognormal mixed effects models (because the production sequence length is no longer discrete but the modelled participant average) with the following specification:

pseq ~ condition + (session | ppt) 

Family: lognormal 
Link function: identity 

where condition consists of session type and gameplan

[1] "noplan_diagnostic"   "noplan_followup"     "noplan_intervention"
[4] "PPS_diagnostic"      "PPS_followup"        "PPS_intervention"   

which was sum coded to return main effects and interactions of session type and gameplan.

The inferential results for main effects and interactions can be found in Table 4.11.

Table 4.11: Main effects and interactions of the average production sequence length on log scale with 95% PIs in brackets.

Predictor	Est. with 95% PIs	BF
Main effects
Gameplan (PP2, no plan)	-0.25 [-0.73 – 0.18]	1.32
Session 1 (diagnostic, intervention)	-0.12 [-0.18 – -0.07]	> 100
Session 2 (diagnostic, follow-up)	-0.09 [-0.13 – -0.06]	> 100
Two-way interactions
Gameplan : Session 1	0 [-0.05 – 0.06]	0.11
Gameplan : Session 2	0 [-0.04 – 0.03]	0.06
Note:
Colon indicates interactions. BF is the evidence in favour of the alternative hypothesis over the null hypothesis

Posterior estimates for the process measures of production sequence length are visualised in Figure 4.11.

Figure 4.11: Estimated cell means for length of production sequence with 95% PIs (probability intervals).

Cellmeans corresponding to Figure 4.11 can be found in Table 4.12 with pairwise comparisons between the diagnostic session and each the intervention session and the followup session.

Table 4.12: Revision. Cell means for production sequence length and their difference on log scale shown with 95% PIs in brackets.

	Session type			Comparison
Gameplan	diagnostic	intervention	followup	diagnostic vs intervention	diagnostic vs followup
no gameplan	21.85 [20.11, 23.66]	23.16 [21.29, 25.09]	22.94 [21.12, 24.81]	0.06 [0.04, 0.08], BF>100	0.05 [0.04, 0.06], BF>100
PPS: revise periodically	23.61 [20.36, 27.97]	25.16 [21.60, 29.79]	24.73 [21.35, 29.21]	0.07 [0.01, 0.12], BF=0.5	0.05 [0.02, 0.08], BF=1.51
Note:
PIs are probability intervals. BF is the evidence in favour of the alternative hypothesis over the null hypothesis.

4.1.4.3 Writing linearity

Descriptive data by participant are shown in Figure 4.12. Data are shown for the total number of writing edges, the number of jumps within the text and the total number of major writing blocks.

Figure 4.12: Write linearily for revision gameplan.

Data were modelled in a multivariate mixed effects models with the following specification:

n_edges ~ condition + (session | p | ppt) 
n_jumps ~ condition + (session | p | ppt) 
n_major_block ~ condition + (session | p | ppt) 

$nedges

Family: negbinomial 
Link function: log 


$njumps

Family: negbinomial 
Link function: log 


$nmajorblock

Family: poisson 
Link function: log 

where condition consists of session type and gameplan

[1] "noplan_diagnostic"   "noplan_followup"     "noplan_intervention"
[4] "PPS_diagnostic"      "PPS_followup"        "PPS_intervention"   

which was sum coded to return main effects and interactions of session type and gameplan.

The inferential results for main effects and interactions can be found in Table 4.13.

Table 4.13: Main effects and interactions of the linear writing measures for the PPS revise periodically gameplan on log scale with 95% PIs in brackets.

	No. of jumps		No. of writing edges		No. of major writing blocks
Predictor	Est. with 95% PIs	BF	Est. with 95% PIs	BF	Est. with 95% PIs	BF
Main effects
Gameplan (D2W, no plan)	1.06 [-0.03 – 2.19]	2.89	1.16 [0.24 – 2.13]	8.34	0.26 [-0.45 – 1.03]	0.87
Session 1 (diagnostic, intervention)	0.68 [0.25 – 1.1]	23.77	0.57 [0.15 – 1]	7.24	0.19 [-0.37 – 0.78]	0.67
Session 2 (diagnostic, follow-up)	1 [0.52 – 1.48]	> 100	0.8 [0.34 – 1.27]	56.84	0.27 [-0.3 – 0.86]	0.85
Two-way interactions
Gameplan : Session 1	-0.06 [-0.48 – 0.37]	0.21	-0.07 [-0.49 – 0.36]	0.22	0.22 [-0.36 – 0.8]	0.72
Gameplan : Session 2	-0.09 [-0.56 – 0.38]	0.24	-0.12 [-0.58 – 0.33]	0.26	0.18 [-0.4 – 0.76]	0.7
Note:
Colon indicates interactions. BF is the evidence in favour of the alternative hypothesis over the null hypothesis

Posterior estimates for the process measures of writing linearity are visualised in Figure 4.13.

Figure 4.13: Estimated cell means for writing linearity for the PPS revise periodically gameplan with 95% PIs (probability intervals).

Cellmeans corresponding to Figure 4.13 can be found in Table 4.14 with pairwise comparisons between the diagnostic session and each the intervention session and the followup session.

Table 4.14: PPS: revise periodically. Writing linearity measures. Cell means for counts and their difference on log scale shown with 95% PIs in brackets.

	Session type			Comparison
Gameplan	diagnostic	intervention	followup	diagnostic vs intervention	diagnostic vs followup
No. of jumps
no gameplan	12.85 [11.15, 14.69]	9.99 [8.55, 11.58]	9.16 [7.72, 10.75]	-0.26 [-0.4, -0.12], BF=50.81	-0.34 [-0.51, -0.18], BF>100
PPS: write periodically	9.3 [6.45, 13.15]	6.75 [4.48, 9.97]	5.86 [3.8, 8.98]	-0.42 [-0.83, 0], BF=1.56	-0.6 [-1.08, -0.14], BF=6.64
No. of writing edges
no gameplan	18.61 [15.83, 21.61]	13.65 [11.39, 16.29]	11.77 [9.67, 14.27]	-0.31 [-0.46, -0.17], BF>100	-0.46 [-0.63, -0.29], BF>100
PPS: write periodically	13.73 [9.2, 19.92]	9.47 [6.05, 14.79]	7.97 [4.87, 12.83]	-0.48 [-0.9, -0.07], BF=3	-0.7 [-1.2, -0.22], BF=14.23
No. of major writing blocks
no gameplan	1.4 [1.17, 1.67]	1.14 [0.93, 1.38]	1.12 [0.91, 1.36]	-0.25 [-0.52, 0.03], BF=0.69	-0.27 [-0.55, 0.01], BF=0.89
PPS: write periodically	1.13 [0.77, 1.61]	1.14 [0.74, 1.7]	1.09 [0.7, 1.62]	-0.01 [-0.82, 0.8], BF=0.41	-0.09 [-0.9, 0.7], BF=0.42
Note:
PIs are probability intervals. BF is the evidence in favour of the alternative hypothesis over the null hypothesis.

4.2 Secondary gameplans

4.2.1 Dedicate Time to Planning (PLN)

The planning and writing times are visualised in Figure 4.14 by participant. Colours indicate whether or not a text rating was improved in the intervention and followup session compared to the diagnostic session.

Figure 4.14: Planning and writing time visualised by participant.

Planning and writing time data were modelled in a multivariate mixed-effects models with the following specification:

time_planning_mins ~ condition + (session | ppt) 
time_writing_mins ~ condition + (session | ppt) 

$timeplanningmins

Family: lognormal 
Link function: identity 


$timewritingmins

Family: lognormal 
Link function: identity 

where condition consists of session type and gameplan

[1] "no gameplan_diagnostic"   "no gameplan_followup"    
[3] "no gameplan_intervention" "pln_diagnostic"          
[5] "pln_followup"             "pln_intervention"        

which was sum coded to return main effects and interactions of session type and gameplan.

The inferential results for main effects and interactions can be found in Table 4.15.

Table 4.15: Main effects and interactions of the planning and writing time on log scale with 95% PIs in brackets.

	Planning time		Writing time
Predictor	Est. with 95% PIs	BF	Est. with 95% PIs	BF
Main effects
Gameplan (PLN, no plan)	0.04 [-0.93 – 1]	0.47	0.32 [-0.12 – 0.77]	0.6
Session 1 (diagnostic, intervention)	-0.37 [-0.91 – 0.16]	0.68	0.23 [0.09 – 0.37]	12.08
Session 2 (diagnostic, follow-up)	0 [-0.49 – 0.5]	0.24	0.38 [0.23 – 0.53]	> 100
Two-way interactions
Gameplan : Session 1	0.34 [-0.19 – 0.87]	0.56	0.04 [-0.1 – 0.18]	0.08
Gameplan : Session 2	0.22 [-0.27 – 0.73]	0.36	0.02 [-0.13 – 0.16]	0.08
Note:
Colon indicates interactions. BF is the evidence in favour of the alternative hypothesis over the null hypothesis

Posterior estimates for the process measures of planning and writing time are visualised in Figure 4.15.

Figure 4.15: Estimated cell means for planning and writing timey with 95% PIs (probability intervals).

Cellmeans corresponding to Figure 4.15 can be found in Table 4.16 with pairwise comparisons between the diagnostic session and each the intervention session and the followup session.

Table 4.16: PLN gameplan. Cell means for time in mins and their difference on log scale shown with 95% PIs in brackets.

	Session type			Comparison
Gameplan	diagnostic	intervention	followup	diagnostic vs intervention	diagnostic vs followup
Planning time
no gameplan	.45 [.39, .53]	.46 [.38, .55]	.40 [.34, .48]	0.02 [-0.17, 0.21], BF=0.1	-0.11 [-0.29, 0.07], BF=0.19
PLN: dedicate time to planning	.37 [.25, .54]	.52 [.33, .84]	.41 [.27, .64]	0.41 [-0.12, 0.95], BF=0.85	0.15 [-0.35, 0.65], BF=0.31
Writing time
no gameplan	24.04 [22.78, 25.36]	21.06 [19.88, 22.29]	19.72 [18.47, 21.06]	-0.13 [-0.18, -0.09], BF>100	-0.2 [-0.25, -0.15], BF>100
PLN: dedicate time to planning	21.20 [18.41, 24.57]	19.30 [16.53, 22.59]	17.68 [14.76, 21.21]	-0.1 [-0.23, 0.03], BF=0.19	-0.19 [-0.33, -0.04], BF=2.04
Note:
PIs are probability intervals. BF is the evidence in favour of the alternative hypothesis over the null hypothesis.

4.2.2 Dedicate Time to Final Evaluation (FIN)

Whether or not there was a final revision period in participants with the final evaluation gameplan was analysed in a binomial regression model with the following specifications:

has_revision ~ 0 + session + (session | ppt) 

Family: bernoulli 
Link function: logit 

Cellmeans corresponding and pairwise comparisons can be found in Table 4.17.

Table 4.17: FIN gameplan. Cell means for proportion of revisions and their difference on logit scale shown with 95% PIs in brackets.

				Comparison
Session	Number of final revisions	Number of ppts	Prop. of revisions with 95% PIs	diagnostic vs intervention	diagnostic vs followup
diagnostic	5	12	.44 [.18, .72]	-0.03 [-2.09, 2], BF=0.97	0.35 [-1.71, 2.47], BF=1.07
intervention	5	12	.44 [.11, .80]
followup	6	12	.52 [.16, .86]
Note:
PIs are probability intervals. BF is the evidence in favour of the alternative hypothesis over the null hypothesis.

5 Does the gameplan intervention increase the overall quality of the produced text?

5.1 Text composition (product data)

The text composition analysis involved overall ratings, text length measured as number of characters, words, sentences and paragraphs, the mean word length and the mean sentence length, and informational density measured as the ratio of open to closed class words, and lexically diversity measured using the MTLD statistic (McCarthy, 2005) as a sensitive and text-length independent measure of lexical diversity (Torruella & Capsada, 2013).

The text composition quality ratings are visualised in Figure 5.1 by participant. Colours indicate whether or not a measure was improved in the intervention and followup session compared to the diagnostic session.

Figure 5.1: Text composition quality measures visualised by participant.

Text composition data were modelled in a multi-variate mixed-effects models with the following specification and distribution families:

rating ~ condition + (session | ppt) 
rating_orga ~ condition + (session | ppt) 
rating_evid ~ condition + (session | ppt) 
rating_argu ~ condition + (session | ppt) 
oc_ratio ~ condition + (session | ppt) 
char_count ~ condition + (session | ppt) 
word_count ~ condition + (session | ppt) 
sentence_count ~ condition + (session | ppt) 
n_paragraph ~ condition + (session | ppt) 
M_word_len ~ condition + (session | ppt) 
M_sent_len ~ condition + (session | ppt) 
lexd ~ condition + (session | ppt) 

$rating

Family: cumulative 
Link function: logit 
Threshold: flexible 


$ratingorga

Family: cumulative 
Link function: logit 
Threshold: flexible 


$ratingevid

Family: cumulative 
Link function: logit 
Threshold: flexible 


$ratingargu

Family: cumulative 
Link function: logit 
Threshold: flexible 


$ocratio

Family: lognormal 
Link function: identity 


$charcount

Family: negbinomial 
Link function: log 


$wordcount

Family: negbinomial 
Link function: log 


$sentencecount

Family: poisson 
Link function: log 


$nparagraph

Family: poisson 
Link function: log 


$Mwordlen

Family: gaussian 
Link function: identity 


$Msentlen

Family: gaussian 
Link function: identity 


$lexd

Family: gaussian 
Link function: identity 

Session was included as by-participant random slopes adjustment.

The fixed effect condition consists of all combinations of gameplan (divided into “gameplan”, “no gameplan”) and session type

[1] "gameplan_diagnostic"      "gameplan_followup"       
[3] "gameplan_intervention"    "no gameplan_diagnostic"  
[5] "no gameplan_followup"     "no gameplan_intervention"

which was sum coded to return main effects and interactions of gameplan and session type.

The inferential results for main effects and interactions can be found in Table 5.1.

Table 5.1: Main effects and interactions of rating on logit scale with 95% PIs in brackets.

	Text quality rating		Rating (argument)		Rating (evidence)		Rating (organisation)		Lexical diversity		Open-to-closed class words		Mean sentence length		Mean word length		No. of paragraphs		No. of sentences		No. of words		No. of characters
Predictor	Est. with PIs	BF	Est. with PIs	BF	Est. with PIs	BF	Est. with PIs	BF	Est. with PIs	BF	Est. with PIs	BF	Est. with PIs	BF	Est. with PIs	BF	Est. with PIs	BF	Est. with PIs	BF	Est. with PIs	BF	Est. with PIs	BF
Main effects
Gameplan (gameplan, no plan)	0.26 [-1.1 – 1.68]	0.7	0.46 [-0.8 – 1.78]	0.82	0.08 [-1.3 – 1.47]	0.65	-0.4 [-1.95 – 1.04]	0.82	5.83 [-5.54 – 17.06]	0.63	0.1 [-0.11 – 0.31]	0.16	-0.18 [-2.09 – 1.68]	0.89	0.1 [-0.09 – 0.3]	0.17	-0.02 [-0.26 – 0.23]	0.12	0.03 [-0.24 – 0.3]	0.13	-0.02 [-0.28 – 0.24]	0.01	0 [-0.24 – 0.25]	0.01
Session 1 (diagnostic, intervention)	0.09 [-0.72 – 0.91]	0.4	-0.05 [-0.84 – 0.73]	0.38	-0.14 [-1.05 – 0.76]	0.45	-0.18 [-0.97 – 0.61]	0.42	4.89 [-0.97 – 10.76]	0.69	-0.09 [-0.23 – 0.06]	0.14	0.19 [-0.84 – 1.21]	0.53	-0.01 [-0.1 – 0.07]	0.04	0.02 [-0.18 – 0.22]	0.1	0.12 [0.02 – 0.22]	0.6	0.13 [0.04 – 0.22]	0.25	0.13 [0.04 – 0.21]	0.17
Session 2 (diagnostic, follow-up)	-0.53 [-1.34 – 0.27]	0.93	-0.71 [-1.52 – 0.07]	1.78	-0.26 [-1.16 – 0.61]	0.51	-0.17 [-0.95 – 0.61]	0.43	1.68 [-4.27 – 7.68]	0.23	0.07 [-0.08 – 0.22]	0.11	-0.51 [-1.61 – 0.55]	0.81	0.04 [-0.05 – 0.12]	0.06	0.03 [-0.17 – 0.24]	0.11	0.19 [0.09 – 0.3]	> 100	0.16 [0.07 – 0.25]	4.25	0.17 [0.08 – 0.26]	5.02
Two-way interactions
Gameplan : Session 1	-0.04 [-0.86 – 0.77]	0.4	-0.16 [-0.94 – 0.63]	0.41	0.3 [-0.56 – 1.18]	0.53	-0.39 [-1.17 – 0.39]	0.62	-0.48 [-6.39 – 5.44]	0.19	0.12 [-0.02 – 0.27]	0.28	0.01 [-1.01 – 1.02]	0.52	0.06 [-0.02 – 0.14]	0.12	-0.03 [-0.23 – 0.17]	0.1	-0.03 [-0.13 – 0.07]	0.06	-0.04 [-0.12 – 0.05]	0	-0.02 [-0.11 – 0.06]	0
Gameplan : Session 2	0.44 [-0.37 – 1.26]	0.72	0.07 [-0.71 – 0.85]	0.39	0.41 [-0.45 – 1.28]	0.62	-0.09 [-0.87 – 0.69]	0.39	-0.29 [-6.39 – 5.64]	0.2	0.07 [-0.08 – 0.23]	0.12	0.59 [-0.48 – 1.7]	0.93	0.11 [0.02 – 0.19]	0.87	-0.03 [-0.23 – 0.17]	0.1	-0.03 [-0.13 – 0.07]	0.06	-0.01 [-0.1 – 0.08]	0	0.01 [-0.08 – 0.1]	0
Note:
Colon indicates interactions. BF is the evidence in favour of the alternative hypothesis over the null hypothesis

Posterior estimates for the text composition scores are visualised in Figure 5.2.

Figure 5.2: Estimated cellmeans for text composition scores with 95% PIs (probability intervals).

Pairwise comparisons can be found in Table 5.2 with differences between the diagnostic session and each the intervention session and the followup session.

Table 5.2: Text quality measures. Cellmeans and difference for text quality measures with 95% PIs in brackets. Differences are shown on the scale used for modelling (see distribution family of outcome measure).

			Differences with 95% PIs
Gameplan	Session	Est. with 95% PIs	diagnosis vs. intervention	diagnosis vs. followup
Text quality rating
gameplan	diagnostic	4.03 [3.69, 4.34]	0.02 [-0.6, 0.63], BF = 0.29	-0.05 [-0.66, 0.56], BF = 0.29
gameplan	intervention	4.01 [3.65, 4.35]
gameplan	followup	4.05 [3.71, 4.38]
no gameplan	diagnostic	3.88 [3.55, 4.2]	0.05 [-0.56, 0.67], BF = 0.3	-0.53 [-1.15, 0.06], BF = 1.31
no gameplan	intervention	3.84 [3.48, 4.18]
no gameplan	followup	4.19 [3.86, 4.5]
Rating (argument)
gameplan	diagnostic	4.03 [3.61, 4.43]	-0.13 [-0.72, 0.46], BF = 0.32	-0.36 [-0.96, 0.23], BF = 0.59
gameplan	intervention	4.12 [3.69, 4.52]
gameplan	followup	4.3 [3.89, 4.71]
no gameplan	diagnostic	3.92 [3.5, 4.33]	0.05 [-0.54, 0.64], BF = 0.3	-0.42 [-1.02, 0.16], BF = 0.8
no gameplan	intervention	3.87 [3.44, 4.29]
no gameplan	followup	4.25 [3.84, 4.65]
Rating (evidence)
gameplan	diagnostic	4.1 [3.79, 4.39]	0.09 [-0.59, 0.77], BF = 0.34	0.08 [-0.59, 0.75], BF = 0.33
gameplan	intervention	4.05 [3.73, 4.36]
gameplan	followup	4.06 [3.74, 4.36]
no gameplan	diagnostic	3.95 [3.63, 4.25]	-0.27 [-0.94, 0.38], BF = 0.43	-0.38 [-1.06, 0.25], BF = 0.61
no gameplan	intervention	4.08 [3.75, 4.38]
no gameplan	followup	4.14 [3.82, 4.43]
Rating (organisation)
gameplan	diagnostic	3.72 [3.24, 4.23]	-0.32 [-0.91, 0.27], BF = 0.5	-0.15 [-0.74, 0.43], BF = 0.33
gameplan	intervention	3.98 [3.47, 4.54]
gameplan	followup	3.84 [3.33, 4.38]
no gameplan	diagnostic	3.99 [3.49, 4.53]	0.11 [-0.46, 0.7], BF = 0.3	-0.04 [-0.61, 0.53], BF = 0.29
no gameplan	intervention	3.89 [3.39, 4.46]
no gameplan	followup	4.03 [3.53, 4.57]
Lexical diversity
gameplan	diagnostic	82.71 [78.99, 86.38]	2.28 [-1.99, 6.51], BF = 0.24	0.72 [-3.71, 5.14], BF = 0.15
gameplan	intervention	80.5 [76.87, 84.15]
gameplan	followup	82.01 [78.17, 85.84]
no gameplan	diagnostic	81.03 [77.37, 84.67]	2.72 [-1.46, 6.93], BF = 0.31	1.03 [-3.24, 5.25], BF = 0.15
no gameplan	intervention	78.33 [74.78, 81.93]
no gameplan	followup	80.02 [76.36, 83.76]
Open-to-closed class words
gameplan	diagnostic	1.05 [0.97, 1.13]	0.02 [-0.09, 0.12], BF = 0.05	0.07 [-0.04, 0.18], BF = 0.12
gameplan	intervention	1.03 [0.95, 1.11]
gameplan	followup	0.97 [0.9, 1.06]
no gameplan	diagnostic	0.95 [0.87, 1.02]	-0.11 [-0.21, 0], BF = 0.41	0 [-0.11, 0.11], BF = 0.05
no gameplan	intervention	1.05 [0.97, 1.14]
no gameplan	followup	0.95 [0.87, 1.03]
Mean sentence length
gameplan	diagnostic	20.39 [19.61, 21.18]	0.12 [-0.67, 0.91], BF = 0.39	0.06 [-0.76, 0.88], BF = 0.39
gameplan	intervention	20.3 [19.48, 21.11]
gameplan	followup	20.35 [19.53, 21.17]
no gameplan	diagnostic	20.25 [19.46, 21.04]	0.07 [-0.72, 0.85], BF = 0.39	-0.64 [-1.47, 0.16], BF = 1.23
no gameplan	intervention	20.16 [19.35, 20.98]
no gameplan	followup	20.8 [19.99, 21.62]
Mean word length
gameplan	diagnostic	4.56 [4.5, 4.62]	0.02 [-0.04, 0.08], BF = 0.04	0.07 [0.01, 0.13], BF = 0.5
gameplan	intervention	4.54 [4.48, 4.59]
gameplan	followup	4.49 [4.43, 4.55]
no gameplan	diagnostic	4.47 [4.42, 4.53]	-0.04 [-0.1, 0.02], BF = 0.07	-0.04 [-0.09, 0.02], BF = 0.05
no gameplan	intervention	4.51 [4.45, 4.56]
no gameplan	followup	4.51 [4.45, 4.57]
No. of paragraphs
gameplan	diagnostic	4.86 [4.39, 5.36]	-0.01 [-0.15, 0.14], BF = 0.07	0 [-0.14, 0.15], BF = 0.07
gameplan	intervention	4.9 [4.4, 5.42]
gameplan	followup	4.85 [4.36, 5.36]
no gameplan	diagnostic	4.99 [4.52, 5.5]	0.03 [-0.11, 0.17], BF = 0.07	0.03 [-0.11, 0.17], BF = 0.08
no gameplan	intervention	4.86 [4.39, 5.36]
no gameplan	followup	4.84 [4.36, 5.34]
No. of sentences
gameplan	diagnostic	20.7 [19.17, 22.29]	0.04 [-0.03, 0.11], BF = 0.07	0.08 [0.01, 0.15], BF = 0.46
gameplan	intervention	19.8 [18.33, 21.41]
gameplan	followup	19.05 [17.61, 20.61]
no gameplan	diagnostic	20.88 [19.38, 22.45]	0.07 [0, 0.14], BF = 0.25	0.11 [0.04, 0.18], BF = 4.79
no gameplan	intervention	19.42 [18.01, 20.96]
no gameplan	followup	18.69 [17.34, 20.14]
No. of words
gameplan	diagnostic	405.29 [377.92, 434.68]	0.05 [-0.02, 0.11], BF = 0.01	0.08 [0.01, 0.14], BF = 0.03
gameplan	intervention	386.26 [359.68, 414.86]
gameplan	followup	375.66 [349.21, 403.9]
no gameplan	diagnostic	414.23 [386.77, 444.49]	0.08 [0.02, 0.14], BF = 0.08	0.09 [0.02, 0.15], BF = 0.07
no gameplan	intervention	381.19 [355.22, 408.9]
no gameplan	followup	380.24 [354.73, 407.76]
No. of characters
gameplan	diagnostic	1845.51 [1722.12, 1974.76]	0.05 [-0.01, 0.11], BF = 0.01	0.09 [0.03, 0.15], BF = 0.1
gameplan	intervention	1750.31 [1633.45, 1875.93]
gameplan	followup	1684.21 [1566.36, 1809.38]
no gameplan	diagnostic	1848.32 [1727.99, 1977.57]	0.07 [0.01, 0.13], BF = 0.03	0.08 [0.02, 0.14], BF = 0.05
no gameplan	intervention	1716.66 [1603.59, 1835.74]
no gameplan	followup	1708.43 [1592.12, 1829.6]
Note:
PIs are probability intervals. BF is the evidence in favour of the alternative hypothesis over the null hypothesis.

5.2 Product goals

The product goal rating data are visualised in Figure 5.3 by participant. Colours indicate whether or not a text rating was improved in the intervention and followup session compared to the diagnostic session.

Figure 5.3: Product goal ratings visualised by participant.

Rating data were modelled in a ordinal multivariate mixed-effects models with the following specification:

organization ~ condition + (session * rater | p | ppt) 
evidence ~ condition + (session * rater | p | ppt) 
argument ~ condition + (session * rater | p | ppt) 

$organization

Family: cumulative 
Link function: logit 
Threshold: flexible 


$evidence

Family: cumulative 
Link function: logit 
Threshold: flexible 


$argument

Family: cumulative 
Link function: logit 
Threshold: flexible 

Rater was included as by-participant random slopes adjustment.

The fixed effect condition consists of all combinations of gameplan, session type, and product goal

 [1] "gameplan_diagnostic_ARG"      "gameplan_diagnostic_DEV"     
 [3] "gameplan_diagnostic_INT"      "gameplan_followup_ARG"       
 [5] "gameplan_followup_DEV"        "gameplan_followup_INT"       
 [7] "gameplan_intervention_ARG"    "gameplan_intervention_DEV"   
 [9] "gameplan_intervention_INT"    "no gameplan_diagnostic_ARG"  
[11] "no gameplan_diagnostic_DEV"   "no gameplan_diagnostic_INT"  
[13] "no gameplan_followup_ARG"     "no gameplan_followup_DEV"    
[15] "no gameplan_followup_INT"     "no gameplan_intervention_ARG"
[17] "no gameplan_intervention_DEV" "no gameplan_intervention_INT"

which was sum coded to return main effects and interactions of gameplan, session type and product goal.

The inferential results for main effects and interactions can be found in Table 5.3.

Table 5.3: Main effects and interactions of rating on logit scale with 95% PIs in brackets.

	Organization		Evidence		Argument
Predictor	Rating est. with 95% PIs	BF	Rating est. with 95% PIs	BF	Rating est. with 95% PIs	BF
Main effects
Gameplan (gameplan, no plan)	-0.18 [-2.58, 2.05]	1.04	-0.04 [-2.24, 2.11]	0.95	0.35 [-1.81, 2.75]	1.02
Session 1 (diagnostic, intervention)	-0.01 [-2.06, 2]	0.93	-0.36 [-2.48, 1.55]	0.92	0.36 [-1.6, 2.49]	0.95
Session 2 (diagnostic, follow-up)	0.12 [-1.87, 2.17]	0.91	0.07 [-1.89, 2.03]	0.84	-1.04 [-3.59, 0.95]	1.47
Prod. goal 1 (ARG, DEV)	0.7 [-1.59, 3.78]	1.19	0.06 [-2.07, 2.27]	0.94	-0.25 [-2.55, 1.91]	0.97
Prod. goal 2 (DEV, INT)	10.2 [0.1, 23.48]	9.53	-2.27 [-7.15, 0.48]	3.06	-0.1 [-2.35, 2.05]	0.98
Two-way interactions
Gameplan : Session 1	-0.36 [-2.54, 1.6]	0.98	0.54 [-1.31, 2.68]	1	-0.51 [-2.69, 1.37]	1.01
Gameplan : Session 2	-0.17 [-2.23, 1.83]	0.92	0.3 [-1.56, 2.29]	0.92	0.35 [-1.55, 2.43]	0.93
Gameplan : Prod. goal 1	0.06 [-2.19, 2.34]	0.98	-0.8 [-3.49, 1.3]	1.21	0.17 [-1.96, 2.4]	0.95
Gameplan : Prod. goal 2	-0.19 [-2.46, 1.91]	0.96	-0.43 [-2.63, 1.51]	0.96	-0.23 [-2.38, 1.8]	0.95
Session 1 : Prod. goal 1	0.22 [-1.66, 2.25]	0.89	0.03 [-1.81, 1.91]	0.83	-0.03 [-1.88, 1.81]	0.82
Session 1 : Prod. goal 2	0.28 [-1.5, 2.18]	0.87	-0.33 [-2.11, 1.33]	0.83	-0.44 [-2.28, 1.23]	0.9
Session 2 : Prod. goal 1	-0.03 [-1.97, 1.89]	0.9	0.76 [-1.04, 2.84]	1.14	0.11 [-1.74, 2]	0.88
Session 2 : Prod. goal 2	0.66 [-1.09, 2.64]	1.06	0.04 [-1.59, 1.66]	0.77	0.96 [-0.79, 2.98]	1.35
Three-way interactions
Gameplan : Session 1 : Prod. goal 1	-0.11 [-2.08, 1.79]	0.89	-0.89 [-3.1, 0.95]	1.25	0.26 [-1.56, 2.17]	0.91
Gameplan : Session 1 : Prod. goal 2	0.08 [-1.7, 1.87]	0.83	0.12 [-1.53, 1.82]	0.78	0.52 [-1.16, 2.34]	0.96
Gameplan : Session 2 : Prod. goal 1	-1.1 [-3.62, 0.88]	1.51	-0.48 [-2.44, 1.28]	0.96	0.34 [-1.51, 2.28]	0.89
Gameplan : Session 2 : Prod. goal 2	-0.42 [-2.29, 1.31]	0.93	-0.77 [-2.61, 0.89]	1.16	0.26 [-1.4, 1.96]	0.81
Note:
Colon indicates interactions. BF is the evidence in favour of the alternative hypothesis over the null hypothesis

Posterior estimates for the rating scores are visualised in Figure 5.4.

Figure 5.4: Estimated cell means for product goal ratings with 95% PIs (probability intervals).

Pairwise comparisons can be found in Table 5.4 with differences between the diagnostic session and each the intervention session and the followup session.

Table 5.4: Product goal effects. Difference for the product goal effects with 95% PIs in brackets.

			Rating estimate with 95% PIs
gp	pg	comp	organization	evidence	argument
gameplan	ARG	diagnosis vs. followup	-0.79 [-2.73, 0.9], BF=1.12	0.24 [-1.24, 1.8], BF=0.76	0.09 [-1.49, 1.68], BF=0.76
gameplan	ARG	diagnosis vs. intervention	-0.21 [-1.95, 1.49], BF=0.81	-0.44 [-2.09, 1.12], BF=0.86	0.02 [-1.58, 1.64], BF=0.78
gameplan	DEV	diagnosis vs. followup	0.77 [-0.9, 2.64], BF=1.12	-0.28 [-1.83, 1.21], BF=0.75	0.07 [-1.56, 1.71], BF=0.75
gameplan	DEV	diagnosis vs. intervention	0.01 [-1.69, 1.73], BF=0.78	0.35 [-1.18, 1.93], BF=0.81	-0.14 [-1.78, 1.44], BF=0.78
gameplan	INT	diagnosis vs. followup	-0.08 [-1.25, 1.09], BF=0.54	0.37 [-0.68, 1.45], BF=0.65	-0.85 [-2.04, 0.26], BF=1.58
gameplan	INT	diagnosis vs. intervention	-0.39 [-1.59, 0.82], BF=0.69	0.28 [-0.8, 1.39], BF=0.6	-0.19 [-1.32, 0.91], BF=0.57
no gamplan	ARG	diagnosis vs. followup	0.84 [-0.62, 2.46], BF=1.21	0.57 [-0.77, 1.97], BF=0.92	-0.6 [-2.12, 0.84], BF=0.95
no gamplan	ARG	diagnosis vs. intervention	0.51 [-1, 2.15], BF=0.86	0.11 [-1.31, 1.52], BF=0.67	0.04 [-1.4, 1.49], BF=0.69
no gamplan	DEV	diagnosis vs. followup	0.27 [-1.28, 1.87], BF=0.76	-0.46 [-1.91, 0.89], BF=0.82	-0.24 [-1.75, 1.25], BF=0.75
no gamplan	DEV	diagnosis vs. intervention	0.15 [-1.45, 1.82], BF=0.75	-1.26 [-2.98, 0.27], BF=2.53	-0.11 [-1.61, 1.41], BF=0.72
no gamplan	INT	diagnosis vs. followup	-0.63 [-1.95, 0.6], BF=0.92	-0.73 [-1.9, 0.38], BF=1.22	-0.76 [-2.03, 0.44], BF=1.21
no gamplan	INT	diagnosis vs. intervention	0.03 [-1.26, 1.34], BF=0.61	-0.2 [-1.35, 0.94], BF=0.59	0.77 [-0.43, 2.04], BF=1.27
Note:
PIs are probability intervals. BF is the evidence in favour of the alternative hypothesis over the null hypothesis.

6 Do product scores increase for participants that adhere to the allocated gameplane?

Rating data were modelled in multivariate mixed-effects models with the following specification:

rating ~ condition * (d2w_pseq + d2w_edit_post_sentence + d2w_edit_post_word + d2w_edit_within_word + noj_jumps + noj_edges + pss) + (session | ppt) 
rating_orga ~ condition * (d2w_pseq + d2w_edit_post_sentence + d2w_edit_post_word + d2w_edit_within_word + noj_jumps + noj_edges + pss) + (session | ppt) 
rating_argu ~ condition * (d2w_pseq + d2w_edit_post_sentence + d2w_edit_post_word + d2w_edit_within_word + noj_jumps + noj_edges + pss) + (session | ppt) 
rating_evid ~ condition * (d2w_pseq + d2w_edit_post_sentence + d2w_edit_post_word + d2w_edit_within_word + noj_jumps + noj_edges + pss) + (session | ppt) 
oc_ratio ~ condition * (d2w_pseq + d2w_edit_post_sentence + d2w_edit_post_word + d2w_edit_within_word + noj_jumps + noj_edges + pss) + (session | ppt) 
char_count ~ condition * (d2w_pseq + d2w_edit_post_sentence + d2w_edit_post_word + d2w_edit_within_word + noj_jumps + noj_edges + pss) + (session | ppt) 
word_count ~ condition * (d2w_pseq + d2w_edit_post_sentence + d2w_edit_post_word + d2w_edit_within_word + noj_jumps + noj_edges + pss) + (session | ppt) 
sentence_count ~ condition * (d2w_pseq + d2w_edit_post_sentence + d2w_edit_post_word + d2w_edit_within_word + noj_jumps + noj_edges + pss) + (session | ppt) 
n_paragraph ~ condition * (d2w_pseq + d2w_edit_post_sentence + d2w_edit_post_word + d2w_edit_within_word + noj_jumps + noj_edges + pss) + (session | ppt) 
M_word_len ~ condition * (d2w_pseq + d2w_edit_post_sentence + d2w_edit_post_word + d2w_edit_within_word + noj_jumps + noj_edges + pss) + (session | ppt) 
M_sent_len ~ condition * (d2w_pseq + d2w_edit_post_sentence + d2w_edit_post_word + d2w_edit_within_word + noj_jumps + noj_edges + pss) + (session | ppt) 
lexd ~ condition * (d2w_pseq + d2w_edit_post_sentence + d2w_edit_post_word + d2w_edit_within_word + noj_jumps + noj_edges + pss) + (session | ppt) 

$rating

Family: cumulative 
Link function: logit 
Threshold: flexible 


$ratingorga

Family: cumulative 
Link function: logit 
Threshold: flexible 


$ratingargu

Family: cumulative 
Link function: logit 
Threshold: flexible 


$ratingevid

Family: cumulative 
Link function: logit 
Threshold: flexible 


$ocratio

Family: lognormal 
Link function: identity 


$charcount

Family: negbinomial 
Link function: log 


$wordcount

Family: negbinomial 
Link function: log 


$sentencecount

Family: poisson 
Link function: log 


$nparagraph

Family: poisson 
Link function: log 


$Mwordlen

Family: gaussian 
Link function: identity 


$Msentlen

Family: gaussian 
Link function: identity 


$lexd

Family: gaussian 
Link function: identity 

Session was included as by-participant random slopes adjustment.

The fixed effect condition consists of all combinations of gameplan and session type.

[1] "0_diagnostic"   "0_followup"     "0_intervention" "1_diagnostic"  
[5] "1_followup"     "1_intervention"

which was sum coded to return main effects and interactions of gameplan (0 = no gameplan, 1 = gameplan), session type and product goal. Continuous predictors were included for behavioural measures specific to the assigned gameplan (for other gameplans the behavioural measure was set to 0). Skewed behavioural predictors were log transformed and all predictors were centred and scaled for model fit.

The inferential results for main effects and interactions can be found in Table 6.1.

Posterior estimates of the effect of process measures on product scores are visualised in Figure 6.1. We extracted visualisation only for those product and process measures that were found to be affected by the gameplan intervention as evidenced by the three-way interaction in Table 6.1.

Figure 6.1: Estimated effects of process measures on product scores with 95% PIs (probability intervals).

7 Does the gameplan intervention affect participants self-efficacy?

The self-efficacy scores by-participant are shown in Figure 7.1.

Figure 7.1: Self-efficacy scores visualised by participant.

The participant self-efficacy scores were analysis in binomial mixed-effects model. The model formula was the following:

response | trials(100) ~ condition + (test | participant) + (condition | item) 

Family: binomial 
Link function: logit 

where response response to what extent participants agreed with a statement on a slider scale from 0 to 100 (0 = disagree; 100 = agree) and condition consists of group (levels: experimental, control) and session (levels: pre-test, post-test). Random intercepts were added for participants and items with random by-participant slope adjustments for session and random by-item slope adjustments for session by group.

condition had the following levels:

[1] "post_control"      "post_experimental" "pre_control"      
[4] "pre_experimental" 

which were sum-coded to obtain main effects and interactions of group and session.

Main effects and interactions are summarised in Table 7.1 for the self-efficacy scores.

Editing estimates obtained from the interaction model are visualised in Figure 7.2 and summarised in Table 7.2 for each design cell.

Figure 7.2: Estimated cell means for self-efficacy with 95% PIs (probability intervals).

Cellmeans corresponding to Figure 4.9 can be found in Table 4.10 with pairwise comparisons between pre-test and post-test for each group.

8 Posthoc: Does gameplan help to improve textquality for dysfunctional writers?

8.1 Writers separated by overall text quality

For this analysis we separate participants into strong and poor writers but dividing them on the basis of the overall rating achieved during the diagnostic session. Poor writers were those who achieved an overall rating less than 3 and strong writers were those who received a rating larger than 3 (participants that received a rating of 3 were excluded).

    text          gp  n
1 strong    gameplan 40
2 strong no gameplan 32
3   weak    gameplan 34
4   weak no gameplan 42

As above, the text product analysis involved the overall rating, as well as the product goal specific ratings (evidence, organisation, argument), text length measured as number of characters, words, sentences and paragraphs, the mean word length and the mean sentence length, and informational density measured as the ratio of open to closed class words, and lexically diversity measured using the MTLD statistic (McCarthy, 2005) as a sensitive and text-length independent measure of lexical diversity (Torruella & Capsada, 2013).

Text composition data were modelled in a multi-variate mixed-effects models with the following specification and distribution families:

rating ~ condition + (session | ppt) 
rating_orga ~ condition + (session | ppt) 
rating_evid ~ condition + (session | ppt) 
rating_argu ~ condition + (session | ppt) 
oc_ratio ~ condition + (session | ppt) 
char_count ~ condition + (session | ppt) 
word_count ~ condition + (session | ppt) 
sentence_count ~ condition + (session | ppt) 
n_paragraph ~ condition + (session | ppt) 
M_word_len ~ condition + (session | ppt) 
M_sent_len ~ condition + (session | ppt) 
lexd ~ condition + (session | ppt) 

$rating

Family: cumulative 
Link function: logit 
Threshold: flexible 


$ratingorga

Family: cumulative 
Link function: logit 
Threshold: flexible 


$ratingevid

Family: cumulative 
Link function: logit 
Threshold: flexible 


$ratingargu

Family: cumulative 
Link function: logit 
Threshold: flexible 


$ocratio

Family: lognormal 
Link function: identity 


$charcount

Family: negbinomial 
Link function: log 


$wordcount

Family: negbinomial 
Link function: log 


$sentencecount

Family: poisson 
Link function: log 


$nparagraph

Family: poisson 
Link function: log 


$Mwordlen

Family: gaussian 
Link function: identity 


$Msentlen

Family: gaussian 
Link function: identity 


$lexd

Family: gaussian 
Link function: identity 

Session was included as by-participant random slopes adjustment.

The fixed effect condition consists of all combinations of gameplan (divided into “gameplan”, “no gameplan”), session type, and writer rating (levels: high [strong writers], low [poor writers])

 [1] "high_gameplan_diagnostic"      "high_gameplan_followup"       
 [3] "high_gameplan_intervention"    "high_no gameplan_diagnostic"  
 [5] "high_no gameplan_followup"     "high_no gameplan_intervention"
 [7] "low_gameplan_diagnostic"       "low_gameplan_followup"        
 [9] "low_gameplan_intervention"     "low_no gameplan_diagnostic"   
[11] "low_no gameplan_followup"      "low_no gameplan_intervention" 

which was sum coded to return main effects and interactions of gameplan, session type, and writer rating.

The inferential results for main effects and interactions can be found in Table 8.1.

Table 8.1: Main effects and interactions with 95% PIs in brackets.

	Text quality rating		Rating (argument)		Rating (evidence)		Rating (organisation)		Lexical diversity		Open-to-closed class words		Mean sentence length		Mean word length		No. of paragraphs		No. of sentences		No. of words		No. of characters
Predictor	Est. with PIs	BF	Est. with PIs	BF	Est. with PIs	BF	Est. with PIs	BF	Est. with PIs	BF	Est. with PIs	BF	Est. with PIs	BF	Est. with PIs	BF	Est. with PIs	BF	Est. with PIs	BF	Est. with PIs	BF	Est. with PIs	BF
Main effects
Gameplan	-0.24 [-2.21 – 1.64]	0.9	0.11 [-1.67 – 1.91]	0.84	-0.17 [-1.99 – 1.63]	0.86	-0.7 [-2.93 – 1.17]	1.14	1.98 [-7.82 – 13.13]	1	0.24 [-0.17 – 0.66]	0.42	-0.15 [-2.38 – 1.98]	0.98	0.18 [-0.22 – 0.59]	0.29	-0.07 [-0.56 – 0.43]	0.24	0.01 [-0.53 – 0.55]	0.26	-0.15 [-0.64 – 0.35]	0.06	-0.1 [-0.6 – 0.38]	0.05
Session 1	0.46 [-1.17 – 2.2]	0.89	0.13 [-1.32 – 1.62]	0.71	0.06 [-1.49 – 1.54]	0.75	-0.19 [-1.5 – 1.09]	0.66	3.97 [-3.7 – 12.47]	1.21	-0.15 [-0.45 – 0.14]	0.24	0.31 [-1.24 – 1.96]	0.8	-0.05 [-0.21 – 0.12]	0.1	0.01 [-0.4 – 0.41]	0.19	0.22 [0.01 – 0.42]	0.88	0.25 [0.07 – 0.43]	0.71	0.24 [0.06 – 0.42]	0.58
Session 2	-0.84 [-2.65 – 0.79]	1.26	-1.3 [-3.07 – 0.24]	2.69	-0.06 [-1.55 – 1.35]	0.72	-0.29 [-1.6 – 0.98]	0.7	-0.22 [-8.1 – 7.61]	0.75	0.21 [-0.1 – 0.51]	0.37	-0.73 [-2.54 – 0.86]	1.12	0.04 [-0.13 – 0.21]	0.09	0.06 [-0.35 – 0.46]	0.2	0.37 [0.16 – 0.58]	29.51	0.3 [0.12 – 0.49]	2.91	0.31 [0.13 – 0.49]	5.47
Writer	24.26 [17.66 – 32.44]	> 100	16.69 [11.85 – 22.38]	> 100	10.48 [6.55 – 14.74]	> 100	5.73 [1.08 – 10.98]	45.43	-3.14 [-14.88 – 6.92]	1.12	-0.38 [-0.79 – 0.04]	1.05	0.87 [-1.3 – 3.82]	1.28	0.22 [-0.18 – 0.63]	0.37	0.25 [-0.24 – 0.75]	0.38	0.66 [0.13 – 1.19]	5.18	1.19 [0.71 – 1.68]	> 100	1.25 [0.76 – 1.73]	> 100
Two-way interactions
Gameplan : Session 1	-0.59 [-2.26 – 0.92]	1.02	-0.48 [-1.98 – 0.94]	0.84	0.14 [-1.29 – 1.58]	0.69	-0.57 [-1.91 – 0.73]	0.89	-1 [-8.73 – 6.54]	0.77	0.19 [-0.1 – 0.49]	0.32	-0.04 [-1.6 – 1.52]	0.76	0.13 [-0.04 – 0.29]	0.24	-0.08 [-0.48 – 0.33]	0.21	-0.09 [-0.29 – 0.11]	0.15	-0.11 [-0.29 – 0.07]	0.04	-0.08 [-0.26 – 0.09]	0.03
Gameplan : Session 2	0.28 [-1.25 – 1.85]	0.8	-0.09 [-1.55 – 1.35]	0.69	0.38 [-1 – 1.82]	0.78	-0.23 [-1.51 – 1.05]	0.66	0.08 [-7.62 – 7.79]	0.75	0.12 [-0.19 – 0.42]	0.2	0.56 [-1.01 – 2.3]	0.97	0.23 [0.05 – 0.4]	1.97	-0.06 [-0.47 – 0.34]	0.21	-0.08 [-0.28 – 0.13]	0.13	-0.06 [-0.24 – 0.12]	0.02	-0.02 [-0.2 – 0.16]	0.02
Gameplan : Writer	-0.04 [-1.98 – 1.87]	0.88	-0.29 [-2.15 – 1.46]	0.86	-0.04 [-1.81 – 1.73]	0.84	-0.32 [-2.34 – 1.55]	0.92	0.99 [-9.04 – 11.58]	0.94	0.22 [-0.19 – 0.64]	0.37	0.21 [-1.94 – 2.5]	1.01	-0.04 [-0.45 – 0.37]	0.2	0.06 [-0.44 – 0.56]	0.24	-0.07 [-0.6 – 0.46]	0.27	0.03 [-0.46 – 0.53]	0.05	0.03 [-0.47 – 0.51]	0.05
Writer : Session 1	6.34 [2.69 – 10.35]	> 100	4.54 [1.97 – 7.27]	> 100	4.25 [1.9 – 6.72]	> 100	0.67 [-0.62 – 2.06]	1.06	4.55 [-3.12 – 13.33]	1.36	0.24 [-0.06 – 0.54]	0.5	0.57 [-1 – 2.26]	0.96	0.03 [-0.14 – 0.2]	0.09	0.1 [-0.3 – 0.5]	0.21	0.18 [-0.02 – 0.37]	0.41	0.25 [0.07 – 0.42]	0.58	0.26 [0.08 – 0.43]	1.19
Writer : Session 2	6.63 [3.1 – 10.52]	> 100	5.08 [2.44 – 7.87]	> 100	3.85 [1.62 – 6.27]	> 100	0.7 [-0.59 – 2.07]	1.09	-0.87 [-8.89 – 6.82]	0.74	0.14 [-0.17 – 0.44]	0.22	-0.09 [-1.7 – 1.48]	0.81	0.01 [-0.15 – 0.18]	0.08	0.03 [-0.37 – 0.43]	0.19	0.14 [-0.07 – 0.34]	0.24	0.15 [-0.03 – 0.34]	0.07	0.16 [-0.02 – 0.34]	0.08
Three-way interactions
Writer : Gameplan : Session 1	-0.44 [-2.12 – 1.11]	0.88	-0.62 [-2.14 – 0.83]	0.96	0.05 [-1.38 – 1.48]	0.68	-0.79 [-2.19 – 0.53]	1.25	4.63 [-3.26 – 13.64]	1.47	0.08 [-0.22 – 0.37]	0.17	0.61 [-0.95 – 2.31]	1.03	0.11 [-0.06 – 0.28]	0.18	0.18 [-0.22 – 0.58]	0.28	0.01 [-0.19 – 0.22]	0.1	0.09 [-0.09 – 0.27]	0.03	0.11 [-0.06 – 0.29]	0.04
Writer : Gameplan : Session 2	0.05 [-1.48 – 1.59]	0.74	-0.23 [-1.71 – 1.17]	0.71	-0.59 [-2.1 – 0.8]	0.93	0.03 [-1.25 – 1.35]	0.66	0.27 [-7.46 – 8.25]	0.75	-0.15 [-0.46 – 0.15]	0.25	0.17 [-1.42 – 1.81]	0.79	0.04 [-0.13 – 0.21]	0.09	0.17 [-0.24 – 0.57]	0.27	0.1 [-0.1 – 0.3]	0.16	0.13 [-0.05 – 0.31]	0.05	0.14 [-0.04 – 0.32]	0.06
Note:
Colon indicates interactions. BF is the evidence in favour of the alternative hypothesis over the null hypothesis

Posterior estimates for the text composition scores are visualised in Figure 8.1.

Figure 8.1: Estimated cellmeans with 95% PIs (probability intervals).

8.2 Writers separated by editing behavior

For this analysis we separate participants on the basis of their editing behavior during the diagnostic session into editors and writers (using a median split) rendering the following group sizes for the gameplan group only (we removed the control group from this analysis).

     edit  n
1 editors 23
2 writers 51

As above, the text product analysis involved the overall rating, as well as the product goal specific ratings (evidence, organisation, argument), text length measured as number of characters, words, sentences and paragraphs, the mean word length and the mean sentence length, and informational density measured as the ratio of open to closed class words, and lexically diversity measured using the MTLD statistic (McCarthy, 2005) as a sensitive and text-length independent measure of lexical diversity (Torruella & Capsada, 2013).

Text composition data were modelled in a multi-variate mixed-effects models with the following specification and distribution families:

rating ~ condition + (session | ppt) 
rating_orga ~ condition + (session | ppt) 
rating_evid ~ condition + (session | ppt) 
rating_argu ~ condition + (session | ppt) 
oc_ratio ~ condition + (session | ppt) 
char_count ~ condition + (session | ppt) 
word_count ~ condition + (session | ppt) 
sentence_count ~ condition + (session | ppt) 
n_paragraph ~ condition + (session | ppt) 
M_word_len ~ condition + (session | ppt) 
M_sent_len ~ condition + (session | ppt) 
lexd ~ condition + (session | ppt) 

$rating

Family: cumulative 
Link function: logit 
Threshold: flexible 


$ratingorga

Family: cumulative 
Link function: logit 
Threshold: flexible 


$ratingevid

Family: cumulative 
Link function: logit 
Threshold: flexible 


$ratingargu

Family: cumulative 
Link function: logit 
Threshold: flexible 


$ocratio

Family: lognormal 
Link function: identity 


$charcount

Family: negbinomial 
Link function: log 


$wordcount

Family: negbinomial 
Link function: log 


$sentencecount

Family: poisson 
Link function: log 


$nparagraph

Family: poisson 
Link function: log 


$Mwordlen

Family: gaussian 
Link function: identity 


$Msentlen

Family: gaussian 
Link function: identity 


$lexd

Family: gaussian 
Link function: identity 

Session was included as by-participant random slopes adjustment.

The fixed effect condition consists of all combinations of editing behavior (levels: editors, writers), and session type as above

[1] "editors_diagnostic"   "editors_followup"     "editors_intervention"
[4] "writers_diagnostic"   "writers_followup"     "writers_intervention"

which was sum coded to return main effects and interactions of editing behavior, and session type.

The inferential results for main effects and interactions can be found in Table 8.2.

Table 8.2: Main effects and interactions with 95% PIs in brackets.

	Text quality rating		Rating (argument)		Rating (evidence)		Rating (organisation)		Lexical diversity		Open-to-closed class words		Mean sentence length		Mean word length		No. of paragraphs		No. of sentences		No. of words		No. of characters
Predictor	Est. with PIs	BF	Est. with PIs	BF	Est. with PIs	BF	Est. with PIs	BF	Est. with PIs	BF	Est. with PIs	BF	Est. with PIs	BF	Est. with PIs	BF	Est. with PIs	BF	Est. with PIs	BF	Est. with PIs	BF	Est. with PIs	BF
Main effects
Edit	-0.44 [-2.45 – 1.39]	0.92	-0.51 [-2.31 – 1.14]	0.92	-0.36 [-2.22 – 1.39]	0.89	-0.6 [-2.67 – 1.21]	1.01	2.66 [-6.41 – 13.06]	0.96	-0.02 [-0.4 – 0.35]	0.19	-0.46 [-2.91 – 1.66]	1.02	0.36 [0.08 – 0.64]	3.02	-0.15 [-0.53 – 0.23]	0.26	-0.32 [-0.71 – 0.06]	0.74	-0.48 [-0.86 – -0.1]	0.74	-0.4 [-0.78 – -0.02]	0.3
Session 1	-0.16 [-1.31 – 0.99]	0.56	-0.25 [-1.36 – 0.83]	0.58	-0.13 [-1.51 – 1.17]	0.63	-0.37 [-1.52 – 0.75]	0.66	1.87 [-4.83 – 8.98]	0.74	0.02 [-0.24 – 0.27]	0.13	0.08 [-1.34 – 1.53]	0.67	0.07 [-0.06 – 0.2]	0.12	-0.03 [-0.35 – 0.28]	0.16	0.08 [-0.08 – 0.24]	0.13	0.09 [-0.05 – 0.23]	0.03	0.11 [-0.04 – 0.25]	0.04
Session 2	-0.44 [-1.6 – 0.7]	0.72	-0.91 [-2.07 – 0.2]	1.88	-0.17 [-1.41 – 1.04]	0.62	-0.43 [-1.56 – 0.68]	0.72	0.28 [-6.64 – 7.23]	0.66	0.17 [-0.09 – 0.42]	0.3	0.07 [-1.39 – 1.55]	0.72	0.13 [0 – 0.26]	0.41	0.01 [-0.31 – 0.33]	0.16	0.1 [-0.06 – 0.26]	0.16	0.09 [-0.06 – 0.23]	0.03	0.12 [-0.03 – 0.26]	0.05
Two-way interactions
Edit : Session 1	-0.36 [-1.56 – 0.8]	0.67	0.04 [-1.05 – 1.14]	0.52	-0.69 [-2.06 – 0.59]	1.06	0.42 [-0.69 – 1.56]	0.69	-0.8 [-7.56 – 5.85]	0.66	-0.04 [-0.28 – 0.21]	0.13	-0.34 [-1.83 – 1.05]	0.74	0.07 [-0.06 – 0.2]	0.1	-0.03 [-0.34 – 0.27]	0.16	0 [-0.16 – 0.16]	0.08	0 [-0.14 – 0.14]	0.01	0.02 [-0.13 – 0.16]	0.01
Edit : Session 2	-0.68 [-1.88 – 0.47]	1.08	-0.53 [-1.65 – 0.55]	0.77	-1.06 [-2.43 – 0.19]	2.21	-0.33 [-1.45 – 0.76]	0.64	1.29 [-5.43 – 8.31]	0.7	0.01 [-0.25 – 0.26]	0.13	0.79 [-0.65 – 2.39]	1.15	-0.02 [-0.15 – 0.11]	0.07	0 [-0.32 – 0.31]	0.16	-0.16 [-0.32 – 0]	0.48	-0.12 [-0.27 – 0.02]	0.05	-0.13 [-0.27 – 0.02]	0.06
Note:
Colon indicates interactions. BF is the evidence in favour of the alternative hypothesis over the null hypothesis

Posterior estimates for the text composition scores are visualised in Figure 8.2. These are the estimates for the gameplan group.

Figure 8.2: Estimated cellmeans with 95% PIs (probability intervals).

8.3 Writers separated by overall text quality and editing behavior

For this analysis we separate participants into strong and poor writers but dividing them on the basis of the overall rating achieved during the diagnostic session. Poor writers were those who achieved an overall rating less than 3 and strong writers were those who received a rating larger than 3 (participants that received a rating of 3 were excluded). In contrast to the previous section we also divided writers on the basis of their editing behavior during the diagnostic session into editors and writers (using a median split) rendering the following group sizes for the gameplan group only (we removed the control group from this analysis).

    text    edit  n
1 strong editors  9
2 strong writers 31
3   weak editors 14
4   weak writers 20

As above, the text product analysis involved the overall rating, as well as the product goal specific ratings (evidence, organisation, argument), text length measured as number of characters, words, sentences and paragraphs, the mean word length and the mean sentence length, and informational density measured as the ratio of open to closed class words, and lexically diversity measured using the MTLD statistic (McCarthy, 2005) as a sensitive and text-length independent measure of lexical diversity (Torruella & Capsada, 2013).

Text composition data were modelled as a multi-variate mixed-effects models with the following specification and distribution families:

rating ~ condition + (session | ppt) 
rating_orga ~ condition + (session | ppt) 
rating_evid ~ condition + (session | ppt) 
rating_argu ~ condition + (session | ppt) 
oc_ratio ~ condition + (session | ppt) 
char_count ~ condition + (session | ppt) 
word_count ~ condition + (session | ppt) 
sentence_count ~ condition + (session | ppt) 
n_paragraph ~ condition + (session | ppt) 
M_word_len ~ condition + (session | ppt) 
M_sent_len ~ condition + (session | ppt) 
lexd ~ condition + (session | ppt) 

$rating

Family: cumulative 
Link function: logit 
Threshold: flexible 


$ratingorga

Family: cumulative 
Link function: logit 
Threshold: flexible 


$ratingevid

Family: cumulative 
Link function: logit 
Threshold: flexible 


$ratingargu

Family: cumulative 
Link function: logit 
Threshold: flexible 


$ocratio

Family: lognormal 
Link function: identity 


$charcount

Family: negbinomial 
Link function: log 


$wordcount

Family: negbinomial 
Link function: log 


$sentencecount

Family: poisson 
Link function: log 


$nparagraph

Family: poisson 
Link function: log 


$Mwordlen

Family: gaussian 
Link function: identity 


$Msentlen

Family: gaussian 
Link function: identity 


$lexd

Family: gaussian 
Link function: identity 

Session was included as by-participant random slopes adjustment.

The fixed effect condition consists of all combinations of gameplan text quality (levels: high [strong writers], low [poor writers]), editing behavior (levels: editors, writers), and session type as above

 [1] "high_editors_diagnostic"   "high_editors_followup"    
 [3] "high_editors_intervention" "high_writers_diagnostic"  
 [5] "high_writers_followup"     "high_writers_intervention"
 [7] "low_editors_diagnostic"    "low_editors_followup"     
 [9] "low_editors_intervention"  "low_writers_diagnostic"   
[11] "low_writers_followup"      "low_writers_intervention" 

which was sum coded to return main effects and interactions of text quality (of diagnostic session), editing behavior, and session type.

The inferential results for main effects and interactions can be found in Table 8.3.

Table 8.3: Main effects and interactions with 95% PIs in brackets.

	Text quality rating		Rating (argument)		Rating (evidence)		Rating (organisation)		Lexical diversity		Open-to-closed class words		Mean sentence length		Mean word length		No. of paragraphs		No. of sentences		No. of words		No. of characters
Predictor	Est. with PIs	BF	Est. with PIs	BF	Est. with PIs	BF	Est. with PIs	BF	Est. with PIs	BF	Est. with PIs	BF	Est. with PIs	BF	Est. with PIs	BF	Est. with PIs	BF	Est. with PIs	BF	Est. with PIs	BF	Est. with PIs	BF
Main effects
Edit	0.09 [-6.44 – 6.67]	0.62	0.11 [-5.11 – 5.37]	0.49	-0.12 [-6.08 – 5.67]	0.54	-1.68 [-7.56 – 3.98]	0.67	7.99 [-15.56 – 33.39]	0.9	-0.05 [-0.83 – 0.72]	0.08	-2.68 [-11.45 – 5.39]	0.93	0.83 [0.26 – 1.4]	3.15	-0.26 [-1.07 – 0.54]	0.1	-0.53 [-1.32 – 0.27]	0.18	-0.7 [-1.44 – 0.04]	0.13	-0.5 [-1.22 – 0.23]	0.06
Session 1	-1.09 [-5.38 – 3.01]	0.45	-1.3 [-4.82 – 2.16]	0.45	-0.89 [-5.3 – 3.04]	0.41	-1.04 [-4.05 – 1.93]	0.37	5.12 [-10.71 – 21.12]	0.66	0.05 [-0.49 – 0.57]	0.05	0.51 [-3.36 – 4.35]	0.4	0.14 [-0.12 – 0.41]	0.05	-0.08 [-0.72 – 0.56]	0.06	0.15 [-0.17 – 0.48]	0.05	0.18 [-0.1 – 0.47]	0.02	0.21 [-0.07 – 0.49]	0.03
Session 2	-1.91 [-6.17 – 2.13]	0.59	-3.43 [-7.02 – 0.04]	2.15	-0.45 [-4.38 – 3.18]	0.36	-1.24 [-4.18 – 1.61]	0.4	0.44 [-15.94 – 16.9]	0.55	0.33 [-0.21 – 0.86]	0.11	0.79 [-3.26 – 4.79]	0.44	0.26 [0 – 0.53]	0.16	0.01 [-0.64 – 0.69]	0.06	0.21 [-0.11 – 0.54]	0.07	0.21 [-0.07 – 0.5]	0.03	0.26 [-0.02 – 0.55]	0.05
Text	28.04 [17.3 – 41.46]	> 100	17.11 [10.29 – 25.19]	> 100	14.08 [7.09 – 22.24]	> 100	8.47 [2.22 – 15.3]	22.25	-4.91 [-28.65 – 17.58]	0.79	-0.02 [-0.8 – 0.75]	0.07	5.21 [-2.85 – 14.48]	1.59	0.45 [-0.11 – 1.02]	0.2	0.37 [-0.43 – 1.17]	0.12	0.7 [-0.09 – 1.48]	0.34	1.27 [0.54 – 2]	5.55	1.38 [0.64 – 2.1]	8.43
Two-way interactions
Edit : Session 1	-0.73 [-4.92 – 3.31]	0.42	1.14 [-2.33 – 4.65]	0.42	-2.41 [-6.35 – 1.44]	0.78	1.05 [-1.88 – 4.08]	0.38	1.06 [-14.87 – 16.95]	0.52	0 [-0.52 – 0.51]	0.05	-0.19 [-4.1 – 3.71]	0.37	0.17 [-0.1 – 0.44]	0.06	-0.02 [-0.67 – 0.63]	0.06	0.03 [-0.29 – 0.35]	0.03	0.07 [-0.22 – 0.35]	0.01	0.11 [-0.18 – 0.39]	0.01
Edit : Session 2	-1.61 [-5.62 – 2.34]	0.52	-0.49 [-3.86 – 2.9]	0.34	-3.54 [-7.35 – -0.03]	2.42	-0.61 [-3.48 – 2.21]	0.3	3.01 [-13.48 – 19.54]	0.57	0.01 [-0.52 – 0.55]	0.05	2.75 [-1.34 – 6.87]	0.99	-0.04 [-0.3 – 0.23]	0.03	0.04 [-0.62 – 0.69]	0.06	-0.28 [-0.61 – 0.04]	0.14	-0.18 [-0.47 – 0.1]	0.02	-0.2 [-0.48 – 0.09]	0.02
Edit : Text	1.99 [-4.37 – 8.53]	0.74	1.39 [-3.92 – 6.69]	0.57	1.55 [-4.24 – 7.53]	0.66	3.09 [-2.69 – 9.2]	1.03	1.58 [-21.59 – 25.25]	0.75	0.41 [-0.36 – 1.18]	0.13	-0.35 [-8.26 – 7.43]	0.73	0.3 [-0.28 – 0.87]	0.1	0.21 [-0.6 – 1]	0.09	0.42 [-0.36 – 1.22]	0.14	0.47 [-0.27 – 1.2]	0.05	0.53 [-0.2 – 1.26]	0.07
Text : Session 1	7.87 [3.25 – 13.34]	> 100	5.5 [1.86 – 9.53]	25.64	5.61 [1.59 – 10.24]	15.92	0.41 [-2.57 – 3.41]	0.3	15.76 [-0.75 – 33.05]	3.07	0.33 [-0.17 – 0.85]	0.11	2.72 [-1.2 – 6.7]	1	0.17 [-0.1 – 0.44]	0.06	0.29 [-0.36 – 0.94]	0.09	0.2 [-0.13 – 0.52]	0.06	0.38 [0.1 – 0.67]	0.28	0.42 [0.14 – 0.7]	0.56
Text : Session 2	9.52 [4.84 – 15.1]	> 100	6.86 [3.1 – 11.14]	> 100	4.31 [0.62 – 8.18]	5.16	1.55 [-1.31 – 4.51]	0.5	-0.02 [-16.43 – 16.6]	0.53	-0.03 [-0.57 – 0.49]	0.05	1.4 [-2.59 – 5.55]	0.51	0.06 [-0.21 – 0.33]	0.03	0.24 [-0.42 – 0.91]	0.08	0.24 [-0.09 – 0.57]	0.1	0.34 [0.05 – 0.62]	0.13	0.35 [0.06 – 0.63]	0.16
Three-way interactions
Text : Edit : Session 1	-2.07 [-6.35 – 2.03]	0.65	-1.69 [-5.23 – 1.75]	0.53	-3.76 [-7.95 – 0.12]	2.34	0.28 [-2.69 – 3.28]	0.3	1.7 [-14.25 – 17.76]	0.53	0.03 [-0.48 – 0.54]	0.05	1.3 [-2.57 – 5.22]	0.48	-0.01 [-0.28 – 0.26]	0.03	-0.01 [-0.66 – 0.63]	0.06	-0.01 [-0.34 – 0.32]	0.03	0.08 [-0.2 – 0.36]	0.01	0.09 [-0.2 – 0.36]	0.01
Text : Edit : Session 2	-0.57 [-4.58 – 3.4]	0.39	-0.7 [-4.01 – 2.65]	0.35	-1.81 [-5.44 – 1.73]	0.57	0.85 [-2.02 – 3.68]	0.34	-5.28 [-21.87 – 11.13]	0.66	-0.06 [-0.59 – 0.46]	0.05	1.17 [-2.83 – 5.21]	0.45	0.04 [-0.23 – 0.31]	0.03	0.06 [-0.59 – 0.71]	0.06	0.12 [-0.21 – 0.44]	0.04	0.22 [-0.06 – 0.51]	0.03	0.23 [-0.05 – 0.52]	0.03
Note:
Colon indicates interactions. BF is the evidence in favour of the alternative hypothesis over the null hypothesis

Posterior estimates for the text composition scores are visualised in Figure 8.3. Estimates are shown for the gameplan group only.

Figure 8.3: Estimated cellmeans with 95% PIs (probability intervals).

9 Session Info

R version 4.3.3 (2024-02-29)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 18.04.6 LTS

Matrix products: default
BLAS:   /usr/lib/x86_64-linux-gnu/openblas/libblas.so.3 
LAPACK: /usr/lib/x86_64-linux-gnu/libopenblasp-r0.2.20.so;  LAPACK version 3.7.1

locale:
 [1] LC_CTYPE=en_GB.UTF-8       LC_NUMERIC=C              
 [3] LC_TIME=en_GB.UTF-8        LC_COLLATE=en_GB.UTF-8    
 [5] LC_MONETARY=en_GB.UTF-8    LC_MESSAGES=en_GB.UTF-8   
 [7] LC_PAPER=en_GB.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_GB.UTF-8 LC_IDENTIFICATION=C       

time zone: Europe/London
tzcode source: system (glibc)

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] rmdformats_1.0.4      patchwork_1.2.0       polspline_1.1.24     
 [4] brms_2.20.4           Rcpp_1.0.12           kableExtra_1.3.4.9000
 [7] lubridate_1.9.3       forcats_1.0.0         stringr_1.5.1        
[10] dplyr_1.1.4           purrr_1.0.2           readr_2.1.5          
[13] tidyr_1.3.1           tibble_3.2.1          ggplot2_3.5.1        
[16] tidyverse_2.0.0      

loaded via a namespace (and not attached):
  [1] gridExtra_2.3        inline_0.3.19        rlang_1.1.3         
  [4] magrittr_2.0.3       matrixStats_1.2.0    compiler_4.3.3      
  [7] loo_2.6.0            systemfonts_1.0.6    vctrs_0.6.5         
 [10] reshape2_1.4.4       rvest_1.0.4          crayon_1.5.2        
 [13] pkgconfig_2.0.3      fastmap_1.1.1        backports_1.4.1     
 [16] ellipsis_0.3.2       labeling_0.4.3       utf8_1.2.4          
 [19] threejs_0.3.3        promises_1.2.1       rmarkdown_2.26      
 [22] markdown_1.1         tzdb_0.4.0           bit_4.0.5           
 [25] xfun_0.43            cachem_1.0.8         jsonlite_1.8.8      
 [28] highr_0.10           later_1.3.2          parallel_4.3.3      
 [31] R6_2.5.1             dygraphs_1.1.1.6     RColorBrewer_1.1-3  
 [34] StanHeaders_2.32.5   bslib_0.7.0          stringi_1.8.3       
 [37] estimability_1.4.1   jquerylib_0.1.4      bookdown_0.39       
 [40] rstan_2.32.5         knitr_1.46           zoo_1.8-12          
 [43] base64enc_0.1-3      bayesplot_1.10.0     httpuv_1.6.13       
 [46] Matrix_1.6-5         igraph_1.6.0         timechange_0.2.0    
 [49] tidyselect_1.2.1     rstudioapi_0.16.0    abind_1.4-5         
 [52] yaml_2.3.8           codetools_0.2-19     miniUI_0.1.1.1      
 [55] pkgbuild_1.4.3       lattice_0.22-5       plyr_1.8.6          
 [58] shiny_1.8.0          withr_3.0.0          bridgesampling_1.1-2
 [61] posterior_1.5.0      coda_0.19-4          evaluate_0.23       
 [64] RcppParallel_5.1.7   xts_0.13.1           xml2_1.3.6          
 [67] pillar_1.9.0         tensorA_0.36.2.1     stats4_4.3.3        
 [70] checkmate_2.3.1      DT_0.31              shinyjs_2.1.0       
 [73] distributional_0.3.2 generics_0.1.3       vroom_1.6.5         
 [76] hms_1.1.3            rstantools_2.3.1.1   munsell_0.5.1       
 [79] scales_1.3.0         gtools_3.9.5         xtable_1.8-4        
 [82] glue_1.7.0           emmeans_1.9.0        tools_4.3.3         
 [85] shinystan_2.6.0      colourpicker_1.3.0   webshot_0.5.5       
 [88] mvtnorm_1.2-4        grid_4.3.3           QuickJSR_1.0.9      
 [91] crosstalk_1.2.1      colorspace_2.1-0     nlme_3.1-163        
 [94] cli_3.6.2            fansi_1.0.6          viridisLite_0.4.2   
 [97] svglite_2.1.3        Brobdingnag_1.2-9    gtable_0.3.5        
[100] sass_0.4.9           digest_0.6.35        htmlwidgets_1.6.4   
[103] farver_2.1.1         htmltools_0.5.8.1    lifecycle_1.0.4     
[106] httr_1.4.7           mime_0.12            bit64_4.0.5         
[109] shinythemes_1.2.0