2.1 Raw data visualisation

Density plots of the raw keystroke intervals are shown in Figure 2.1 by copy-task component and language.

Figure 2.1: Density plots of inter-keystroke intervals (IKIs) by copy-task component.

2.2 Data reduction

This analysis focuses on transition duration. Transition duration shorter than (or equal to) 10 msecs were removed before analysis (M=1.18%, SE=0.12%).

Among the copy-task components participants had to copy 4 word-triplets (phrases) of which 3 word-triples consisted predominantly of high-frequency bigrams and 1 contained low-frequency bigrams. From the first 3 word triplets we removed bigrams that were not highly frequent and from the latter we used only low-frequency bigrams for analysis. Therefore, the HF-words task included only high-frequency bigrams, and the LF-words task included only low-frequency bigrams.

We randomly selected a subset of 50 observations per participant, language, and copy-task component to reduce the time and computational power necessary to run the statistical models. Therefore we used subsets of the data of (most) copy-task components (indicated is the by-participant average of observations used per copy-task component and language with standard errors [SE] indicating the variability of observations used across participants): consonants tasks (Norwegian: M=100%, SE=0%, English: M=100%, SE=0%), HF-words task (Norwegian: M=13.6%, SE=3.4%, English: M=13.4%, SE=3.4%), LF-words task (Norwegian: M=100%, SE=0%, English: M=100%, SE=0%), the sentence task (Norwegian: M=76.4%, SE=4.2%, English: M=74%, SE=4.4%), typing-speed task (Norwegian: M=40.7%, SE=4.9%, English: M=40.1%, SE=4.9%). The average number of bigrams used in the final sample is shown in Table 2.1.

2.3 Mixture model results

The mixture model was implemented in the probabilistic programming language Stan. The copy-typing process was constrained to be a mixture of two log-normal distributions of which one distribution represents fluent transitions and the other disfluencies.

\[ \begin{align}\tag{2.1} \text{iki}_{i} \sim \ & \theta_\text{condition[i]} \cdot \text{logN}(\beta_\text{condition} + \delta_\text{condition} + u_i, \sigma_\text{ctc}') \ +\\ & (1-\theta_\text{condition[i]}) \cdot \text{logN}(\beta_\text{condition} + u_i, \sigma_\text{ctc})\\ \text{constraints: } & \delta > 0, \sigma' > \sigma\\ \end{align} \]

The positively constrained parameter \(\delta\) achieves this: it ensures that the distribution on the first line of the equation is larger than the distribution on the second line as both distributions have the mean \(\beta\) but the first distribution is incremented by \(\delta\). Therefore, \(\beta\) is capturing the average transition duration of fluent keystroke pairs and the \(\delta\) the difference / slowdown for long keystroke pairs in addition to \(\beta\). The relative weighting between these two distributions is captured by the mixing proportion \(\theta\): \(\theta\) is parameterised so that it captures the relative weight of keystrokes that are disfluent and can, therefore, be understood as the pausing probability (Roeser et al., 2021). This is achieved by multiplying \(\theta\) with the distribution of slow key transitions but using the inverse \(1-\theta\) for the distribution of fluent transitions.

Each of these distributions was assumed to have unequal variances in two regards: first, disfluent keytransitions have a larger variability than fluent key transitions which is achieved by constraining \(\sigma'\) to be larger than \(\sigma\). Second each copy-task component (ctc) has its own variance component has the tasks are inherently different in terms of complexity.

The model is a mixed effects model because we included random effects for participants: first, we allowed the average typing speed to vary across participants. We assumed that some participants have a typing speed slower than the average \(\beta\) and others are on average faster. This is captured by the parameter \(u_i\) where \(i\) is indexing every participant. The typing speed difference between average and participant \(u\) was assumed to be distributed according to \(u \sim \text{N}(0, \sigma_u)\). Second, we assumed that the probability of long latencies varies across participants, hence the index \(i\) for \(\theta\). This variability in pausing frequency is frequenting different copy-typing styles and captures the extent to which participants use their memory or touch typing rather than pausing and looking up during copy typing.

Finally and importantly condition was included is an indicator for every combination of language (levels: Norwegian (L1), English (L2)) and copy-task component (in short ctc; levels: consonants task, HF words, LF words, sentence, typing speed). This allows us to estimate the model estimates for each of the three parameters of interesting, namely the fluent typing speed \(\beta\), the slowdown for long transitions \(\delta\), and the disfluency probability \(\theta\). From the posterior of the model we calculated then main effects and interactions as well as planned pairwise comparisons between language and estimated marginal cell means.

Note though that for the typing speed task we modeled \(\beta\) but not the pausing probability as disfluencies in the typing speed task have no cognitive interpretation but are assumed to be entirely random. This was achieved including an if-condition in the model that is modeling ikis of the typing speed task as

\[ \begin{align}\tag{2.2} \text{iki}_{i} \sim \ \text{logN}(\beta_\text{condition[i]} + u_i, \sigma_\text{ctc})\\ \end{align} \]

which corresponds to the second line of the model in equation (2.1).

The result of the mixed-effects mixture model are summarised in Table 2.2. Table 2.2 shows main effects and interactions for language and copy-task component for each model parameter.

Cell means wth 95% PIs correspoding to the main effects and interactions in Table 2.2 are illustrated in Figure 2.2 and again summarised as number along with planned by-language comparisons in Table 2.3.

Fluent transitions were found to be shorter in L2 in the HF words task, the LF words task, and the sentence task. As one would probably expect, the non-lexical tasks – the consonants task and the typing speed task – showed no language differences as these tasks are language independent and identical in both languages. No language difference was observed for the disfluency duration parameter: the slowdown participants experienced in either task was the same for Norwegian and English suggesting that pauses were not reflecting L2-related processing. Yet though, we observed a substantially higher probability of disfluencies in L2 copy-typing for the LF words task.

Figure 2.2: Estimated cell means of transition duration with 95% PIs (probability intervals). Shown are the estimates for the mixture component of long long transition durations in msecs (on log scale) and the probability of long transition durations.

3.1 Text features

As can be seen in Table 3.1, participants’ L2 texts were shorter in terms of number of words and characters but consisted of more sentences. Sentences were shorter while words were longer in L2, and were informationally more sparse (had lower ratio of open to closed class words), and lexically less diverse measured using the MTLD statistic (McCarthy, 2005) as a sensitive and text-length independent measure of lexical diversity (Torruella & Capsada, 2013).

3.2 Sample

The final sample of data comes from 85 Norwegian students of which each produced a text in their L1 (Norwegian) or their L2 (English). Seventeen participants were removed prior to analysis because they only contributed one text.

3.3 Number of transitions

# Run model
#source("../scripts/models/transition_counts.R")
# Load model posterior
fit <- readRDS(file = "../stanout/transition_counts.rda")

fit$formula

n_transitions ~ 1 + condition + (language | participant) 

In this section we analysed the number of transitions in writing data. condition was coded with main effects of Language (levels: L1, L2), Edit (levels: editing, no editing; i.e. whether a key transition terminated in an editing operation), Transition location 1 (levels: before sentence, before word), Transition location 2 (levels: before word, within word), Transition location 3 (levels: after word, within word), and all two and three-way interactions by-Transition location.

fit$family

Family: negbinomial 
Link function: log 

In addition we fitted a binomial model to account for the overall number of transitions by participant and language.

# Run model
#source("../scripts/models/transition_binomial.R")
# Load model posterior
fit_binom <- readRDS(file = "../stanout/transition_binomial.rda")

fit_binom$formula

n_transitions | trials(total) ~ condition + (language | participant) 

fit_binom$family

Family: binomial 
Link function: logit 

Main effects and interactions of the analysis of the transition counts are shown in Table 3.3. The results are illustrated in Figure 3.1 and with the corresponding cellmeans in Table 3.4 along with planned pairwise comparisons between language.

Figure 3.1: Estimated cell means for transition counts with 95% PIs (probability intervals).

3.4 Editing frequency

# Run model
#source("../scripts/models/editing_frequency.R")

# Load model posterior
fit <- readRDS(file = "../stanout/editing_frequency.rda")

fit$formula

edits | trials(total) ~ 1 + condition + (language | participant) 

This number concerns the frequency to which a key transition terminated an editing operation (revision) or not. condition was coded with main effects of Language (levels: L1, L2), Transition location 1 (levels: before sentence, before word), Transition location 2 (levels: before word, within word), Transition location 3 (levels: after word, within word) and all two-way interactions by-Transition location (i.e. no interactions that involve more than one Transition location).

fit$family

Family: binomial 
Link function: logit 

Main effects and interactions of the analysis of the editing frequency are shown in Table 3.5. The results are illustrated in Figure 3.2 and with the corresponding cellmeans in Table 3.6 along with planned pairwise comparisons between language. The results show that L1 and L2 writers edited their text equally frequent with one notable exception: L2 writers showed a higher proportion of transitions that terminated in an editing operation at after-word transition locations. There was substantial evidence for a difference at within-word locations as well but the size of the differences was too small to be meaningful.

Figure 3.2: Estimated cell means for editing frequency with 95% PIs (probability intervals).

3.5 Transition duration

3.5.1 Raw data visualisation

Density plots of the raw keystroke intervals are shown in Figure 3.3 by copy-task component and language. Among the copy-task components participants had to copy 4 word-triplets (phrases) of which 3 triples consisted predominantly of high-frequency bigrams and 1 contained low-frequency bigrams. From the first 3 triplets we removed bigrams that were not highly frequent and from the latter we used only low-frequency bigrams for analysis.

Figure 3.3: Density plots of inter-keystroke intervals (IKIs) by transition location and language.

3.5.2 Mixture model results

The model used to analyse transition durations in text composition is largely similar to the model for the copy-task data described equation (2.1). We provde a brief description for completness.

The mixture model was implemented in the probabilistic programming language Stan. The keystroke data resulting from the text production process were modelled as coming from a mixture of two log-normal distributions of which one distribution represents fluent transitions and the other disfluencies between keystroke intervals.

\[ \begin{align}\tag{3.1} \text{iki}_{i} \sim \ & \theta_\text{condition[i]} \cdot \text{logN}(\beta_\text{condition} + \delta_\text{condition} + u_i, \sigma_\text{location}') \ +\\ & (1-\theta_\text{condition[i]}) \cdot \text{logN}(\beta_\text{condition} + u_i, \sigma_\text{location})\\ \text{constraints: } & \delta > 0, \sigma' > \sigma\\ \end{align} \] Each distribution was assumed to have unequal variances in two regards: first, disfluent key transitions have a larger variability than fluent key transitions which is achieved by constraining \(\sigma'\) to be larger than \(\sigma\). Second each transition location (location) has its own variance component.

condition was included is an indicator for every combination of language (levels: Norwegian (L1), English (L2)) and transition location (levels: after words, before words, within words, before sentences). This allows us to estimate the model estimates for each of the three parameters of interesting, namely the fluent typing speed \(\beta\), the slowdown for long transitions \(\delta\), and the disfluency probability \(\theta\). From the posterior of the model we calculated then main effects and interactions as well as planned pairwise comparisons between language and estimated marginal cell means.

The model is a mixed effects model because we included random effects for participants: first, we allowed the average typing speed to vary across participants. We assumed that some participants have a typing speed slower than the average \(\beta\) and others are on average faster. This is captured by the parameter \(u_i\) where \(i\) is indexing every participant. The typing speed difference between average and participant \(u\) was assumed to be distributed according to \(u \sim \text{N}(0, \sigma_u)\). Second, we assumed that the probability of long latencies varies across participants, hence the index \(i\) for \(\theta\). This is capturing the idea that people vary in their writing styles as to how often they pause to plan upcoming utterances and plan in parallel to production.

Prior to analysis of the transition durations, we removed data that were shorter than (or equal to) 50 msecs (M=1.2%, SE=0.17%) or longer than 15 secs (M=0.2%, SE=0.03%) were removed before analysis. Values outside of this range were removed because they are unlikely to be representative for text composition and are more likely to be related to, for example, finger slips or the writer engaging in non-composition related activities (browsing, social media, extended reading), respectively. Also, removed key intervals that terminated in a revision of the text (M=5.1%, SE=2.39%).

We randomly selected a subset of 150 observations per participant, language, and transition location, where the number ob observations per participants exceed 150 observations, to reduce the time and computational power necessary to run the statistical models.

Therefore the final sample contained a subset of the full text production data set for (most) transition locations (indicated is the by-participant average of observations used per transition location and language with standard errors [SE] indicating the variability of observations used across participants): after-words transitions (L1: M=35.7%, SE=5.2%; L2: M=44.2%, SE=5.4%), before-words transitions (L1: M=34.6%, SE=5.2%; L2: M=41.8%, SE=5.3%), within-words transitions (L1: M=13.2%, SE=3.7%; L2: M=14.3%, SE=3.8%), before-sentences transitions (L1: M=100%, SE=0%; L2: M=100%, SE=0%).

As discussed above, we modelled transition durations (time between first and second keypress) as finite mixture models with fixed effects for language, for location in text, for keystrokes that represented ongoing production as opposed to keystrokes that were followed by a text revision; for an overview on mixture models see Gelman et al. (2014) p. 519 – 524 and Hall et al. (2022) and Roeser et al. (2021) for an application to keystroke data. We assumed two data-generating processes: one associated with fluent typing, generating a distribution of relatively short transition durations, and an one, associated with longer durations (i.e. disfluencies), in which the demands of upstream (higher-level) processing affect time to plan the next keystroke.

The result of the mixed-effects mixture model are summarised in Table 3.7. Table 3.7 shows main effects and interactions for language and transition location for each model parameter.

We found strong evidence for longer transition intervals and a larger proportion of hesitations at higher-level text locations (after word < within word < before word / before sentence) although in L2 writing transitions were longer before-sentences compared to before word-locations but there was no difference in L1 writing while there were fewer pauses in L1 writing before words.

There was strong evidence of a main effect of language. This represents a general tendency of longer fluent transitions and a larger probability of hesitant transitions in L2 while the slowdown for hesitations was similar across languages.

Table 3.7: Mixture model results of the transition duration with the predictor estimates for the distribution of fluent transitions and the slowdown for long transitions (on log scale) and the probability of long transitions / disfluencies (on logit scale). Estimates are shown with 95% PI.

	Fluent transitions		Slowdown for disfluencies		Probability of disfluencies
Predictor	Estimate	\(BF_{10}\)	Estimate	\(BF_{10}\)	Estimate	\(BF_{10}\)
Main effects
Language (L1, L2)	-0.07 [-0.09, -0.06]	>100	-0.04 [-0.08, 0]	0.16	-0.33 [-0.49, -0.17]	>100
Location 1 (before sentence, before word)	0.15 [0.06, 0.23]	6.45	1.05 [0.91, 1.16]	>100	-0.87 [-1.23, -0.46]	>100
Location 2 (before word, within word)	0.33 [0.31, 0.34]	>100	0.35 [0.3, 0.4]	>100	1.6 [1.36, 1.84]	>100
Location 3 (after word, within word)	-0.15 [-0.16, -0.14]	>100	0.3 [0.24, 0.36]	>100	0.25 [0.03, 0.48]	1.34
Two-way interactions
Language : Location 1	0.16 [0.1, 0.21]	>100	-0.05 [-0.16, 0.06]	0.08	0.28 [-0.2, 0.76]	0.48
Language : Location 2	-0.11 [-0.14, -0.08]	>100	0.15 [0.06, 0.24]	8.86	-0.11 [-0.55, 0.34]	0.25
Language : Location 3	0.05 [0.04, 0.07]	>100	0.08 [-0.03, 0.18]	0.15	0.13 [-0.31, 0.56]	0.26
Note:
Colon indicates interactions. PI is the probability interval. \(BF_{10}\) is the evidence in favour of the alternative hypothesis over the null hypothesis.

We unpacked these effects by calculating the effect of language by transition location. Cell means with 95% PIs corresponding to the main effects and interactions in Table 3.7 are illustrated in Figure 3.4 and again summarised as number along with planned by-language comparisons in Table 3.8. The following differences were found for writing in L2: Transition durations, in which we mean fluent transitions between keystrokes, were generally slower when writing in L2 at before-word locations and within-words. Also writing in L2 led to more pauses at the same locations. Interestingly, the slowdown for pauses, across all text locations was similar in L1 and L2. Also, we observed no changes related to L2 writing at before-sentence locations and at after-word locations. These results suggest that disfluency in L2 writing are associated with word-level difficulty, associated with lexical retrieval and spelling retrieval of which, persumably the latter, was not completed at word-production onset.

Figure 3.4: Estimated cell means of transition durations with 95% PIs (probability intervals). Shown are the estimates for each mixture components (short and long transitions) and the probability of disfluencies.

Table 3.8: Transition duration. Cell means for L1 and L2 with values in msecs for durations and probability of disfluencies. Language difference are shown on log scale (for durations) and logit scale for probability of disfluencies. 95% PIs in brackets.

Transition location	L1	L2	Language effect	\(BF_{10}\)
Short fixations
after words	160 [152, 167]	162 [155, 170]	-0.02 [-0.03, -0.01]	1.09
before sentences	299 [270, 326]	306 [276, 335]	-0.02 [-0.08, 0.03]	0.04
before words	237 [226, 249]	284 [270, 299]	-0.18 [-0.2, -0.16]	>100
within words	181 [173, 190]	194 [185, 203]	-0.07 [-0.08, -0.06]	>100
Slowdown for disfluencies
after words	453 [420, 488]	481 [449, 515]	-0.04 [-0.12, 0.03]	0.08
before sentences	2,584 [2,014, 3,136]	2,706 [2,181, 3,222]	-0.02 [-0.13, 0.08]	0.06
before words	735 [690, 782]	856 [807, 907]	0.03 [-0.02, 0.07]	0.05
within words	367 [340, 396]	443 [413, 476]	-0.12 [-0.2, -0.04]	4.83
Probability of disfluencies
after words	.20 [.17, .24]	.25 [.21, .29]	-0.26 [-0.55, 0.04]	0.63
before sentences	.30 [.23, .39]	.34 [.27, .43]	-0.2 [-0.56, 0.16]	0.34
before words	.47 [.41, .52]	.59 [.53, .64]	-0.49 [-0.8, -0.17]	16.82
within words	.16 [.13, .19]	.21 [.18, .25]	-0.38 [-0.69, -0.07]	2.78
Note:
PIs are probability intervals. \(BF_{10}\) is the evidence in favour of the alternative hypothesis over the null hypothesis.