Abbie’s Dissertation Project - Analysis Final
Sanshiroh Ogawa
2025-07-17
1 Pre-analysis
Set Working Directory:
Load necessary packages:
library(readr)
library(dplyr)
library(psych)
library(lme4)
library(lmerTest)
library(effects)
library(glmmTMB)
library(buildmer)
library(DHARMa)
library(performance)Import dataset:
dat <- read_csv("dataset_complete.csv", show_col_types = FALSE) # load data
dat$Participant <- as.factor(dat$Participant) # make sure "Participant" is a factor
is.factor(dat$Participant) # "TRUE"## [1] TRUEdat$Complexity <- as.factor(dat$Complexity) # make sure "Complexity" is a factor
is.factor(dat$Complexity) # "TRUE"## [1] TRUE## [1] TRUEdat$RSPANpartial_WM.C <- dat$RSPANpartial_WM - mean(dat$RSPANpartial_WM) # create a mean-centered variable for RSPANpartial_WM
colSums(is.na(dat)) # make sure there is no missing data ## Participant MasteryGoal_motivation
## 0 0
## AvoidanceGoal_motivation PerformanceGoal_motivation
## 0 0
## SelfEfficacy_motivation TaskValue_motivation
## 0 0
## Attribution_motivation Somatic_Anxiety
## 0 0
## Avoidance_Anxiety Cognitive_Anxiety
## 0 0
## OSPANabsolute_WM OSPANtotal_WM
## 0 0
## OSPANmath_WM RSPANpartial_WM
## 0 0
## RSPANtotal_WM MotEng
## 0 0
## AchievEng StAnx
## 0 0
## CogAnx Genre
## 0 0
## Complexity Task
## 0 0
## Subordination Coordination
## 0 0
## AccuracyGender AccuracyNumber
## 0 0
## AccuracyTense AccuracyAspect
## 0 0
## LexicalDensity LexicalDiversity
## 0 0
## FluencySyllablesperMinute mentaleffort
## 0 0
## difficulty contentplanning
## 0 0
## linguisticchallenges PTD_composite
## 0 0
## RSPANpartial_WM.C
## 0Descriptive statistics:
## vars n mean sd median trimmed mad min
## Participant* 1 632 79.50 45.65 79.50 79.50 58.56 1.00
## MasteryGoal_motivation 2 632 22.64 5.77 22.00 22.62 5.93 7.00
## AvoidanceGoal_motivation 3 632 40.27 8.40 41.00 40.33 8.90 17.00
## PerformanceGoal_motivation 4 632 23.74 6.65 24.00 23.73 5.93 3.00
## SelfEfficacy_motivation 5 632 39.70 6.43 39.50 39.47 5.93 24.00
## TaskValue_motivation 6 632 27.80 7.95 27.00 27.56 7.41 7.00
## Attribution_motivation 7 632 19.41 3.29 20.00 19.51 2.97 10.00
## Somatic_Anxiety 8 632 22.22 6.64 22.00 22.11 7.41 10.00
## Avoidance_Anxiety 9 632 17.18 3.99 17.00 17.11 4.45 7.00
## Cognitive_Anxiety 10 632 33.90 5.10 35.00 34.09 5.93 19.00
## OSPANabsolute_WM 11 632 38.30 15.58 39.00 38.79 14.08 0.00
## OSPANtotal_WM 12 632 52.32 15.04 55.00 54.16 11.86 1.00
## OSPANmath_WM* 13 632 15.51 10.44 16.00 15.55 14.83 1.00
## RSPANpartial_WM 14 632 15.92 6.29 15.50 15.75 5.19 1.00
## RSPANtotal_WM 15 632 29.03 8.92 29.00 28.86 7.41 7.00
## MotEng 16 632 -0.01 1.00 -0.01 -0.02 0.83 -2.65
## AchievEng 17 632 0.00 1.00 -0.05 -0.02 0.92 -2.98
## StAnx 18 632 0.00 1.00 -0.06 -0.02 1.09 -2.06
## CogAnx 19 632 -0.01 1.00 0.11 0.02 1.02 -3.36
## Genre* 20 632 1.50 0.50 1.50 1.50 0.74 1.00
## Complexity* 21 632 1.50 0.50 1.50 1.50 0.74 1.00
## Task* 22 632 2.50 1.12 2.50 2.50 1.48 1.00
## Subordination 23 632 1.53 1.59 1.00 1.26 1.48 0.00
## Coordination 24 632 2.37 2.04 2.00 2.12 1.48 0.00
## AccuracyGender 25 632 2.20 1.91 2.00 1.97 1.48 0.00
## AccuracyNumber 26 632 0.73 1.01 0.00 0.57 0.00 0.00
## AccuracyTense 27 632 1.56 1.52 1.00 1.37 1.48 0.00
## AccuracyAspect 28 632 0.62 1.59 0.00 0.19 0.00 0.00
## LexicalDensity* 29 632 120.97 68.36 126.00 121.43 87.47 1.00
## LexicalDiversity 30 632 0.75 2.06 0.64 0.64 0.15 0.06
## FluencySyllablesperMinute* 31 632 136.51 85.04 142.00 136.70 114.16 1.00
## mentaleffort 32 632 6.07 1.63 6.00 6.14 1.48 1.00
## difficulty 33 632 5.94 1.73 6.00 6.00 1.48 1.00
## contentplanning 34 632 5.32 1.96 5.00 5.32 2.22 1.00
## linguisticchallenges 35 632 6.48 1.72 7.00 6.56 1.48 1.00
## PTD_composite 36 632 5.95 1.47 6.00 5.99 1.48 1.50
## RSPANpartial_WM.C 37 632 0.00 6.29 -0.42 -0.17 5.19 -14.92
## max range skew kurtosis se
## Participant* 158.00 157.00 0.00 -1.21 1.82
## MasteryGoal_motivation 39.00 32.00 0.11 0.40 0.23
## AvoidanceGoal_motivation 59.00 42.00 -0.12 -0.45 0.33
## PerformanceGoal_motivation 40.00 37.00 -0.06 0.33 0.26
## SelfEfficacy_motivation 58.00 34.00 0.34 0.33 0.26
## TaskValue_motivation 50.00 43.00 0.23 0.18 0.32
## Attribution_motivation 30.00 20.00 -0.19 0.81 0.13
## Somatic_Anxiety 37.00 27.00 0.15 -0.82 0.26
## Avoidance_Anxiety 29.00 22.00 0.20 0.18 0.16
## Cognitive_Anxiety 44.00 25.00 -0.36 -0.60 0.20
## OSPANabsolute_WM 75.00 75.00 -0.28 0.04 0.62
## OSPANtotal_WM 78.00 77.00 -1.13 1.18 0.60
## OSPANmath_WM* 29.00 28.00 -0.16 -1.65 0.42
## RSPANpartial_WM 36.00 35.00 0.36 0.33 0.25
## RSPANtotal_WM 54.00 47.00 0.14 0.30 0.35
## MotEng 3.06 5.71 0.20 0.65 0.04
## AchievEng 2.73 5.70 0.06 0.50 0.04
## StAnx 2.36 4.42 0.22 -0.52 0.04
## CogAnx 2.23 5.59 -0.35 -0.05 0.04
## Genre* 2.00 1.00 0.00 -2.00 0.02
## Complexity* 2.00 1.00 0.00 -2.00 0.02
## Task* 4.00 3.00 0.00 -1.37 0.04
## Subordination 7.00 7.00 1.26 1.12 0.06
## Coordination 10.00 10.00 1.02 0.74 0.08
## AccuracyGender 10.00 10.00 1.08 1.21 0.08
## AccuracyNumber 7.00 7.00 1.50 2.89 0.04
## AccuracyTense 10.00 10.00 1.34 3.56 0.06
## AccuracyAspect 12.00 12.00 3.30 12.93 0.06
## LexicalDensity* 246.00 245.00 -0.10 -1.21 2.72
## LexicalDiversity 37.23 37.17 17.51 306.83 0.08
## FluencySyllablesperMinute* 273.00 272.00 -0.04 -1.41 3.38
## mentaleffort 9.00 8.00 -0.37 -0.10 0.06
## difficulty 9.00 8.00 -0.31 -0.38 0.07
## contentplanning 9.00 8.00 -0.04 -0.76 0.08
## linguisticchallenges 9.00 8.00 -0.44 -0.33 0.07
## PTD_composite 9.00 7.50 -0.24 -0.21 0.06
## RSPANpartial_WM.C 20.08 35.00 0.36 0.33 0.25CALF measures of interest:
1. Syntactic Complexity
- Subordination
- Coordination
2. Accuracy
- Number
- Gender
- Tense
- Aspect
3. Lexical Complexity
- Lexical Density
- Lexical Diversity
4. Fluency
- Syllables per minute
2 Research Question 1:
2.1 RQ1a:
- How does altering task complexity and genre — specifically argumentative and narrative — influence the quality of adolescent L2 writing?
2.2 RQ1b:
- To what extent do individual differences moderate the effect of task complexity and genre on writing quality?
3 Fit the models for RQ1a (i-ix) and RQ1b (i-ix) using Buildmer
3.1 RQ1a(i) & RQ1b(i): Syntactic Complexity (Subordination)
RQ1a(i):
- How does altering task complexity and genre — specifically argumentative and narrative — influence the syntactic subordination?
RQ1b(i):
- To what extent do individual differences moderate the effect of task complexity and genre on syntactic subordination?
First, check to see the distribution of “Subordination.”
## [1] 2 0 1 3 4 6 5 7## vars n mean sd median trimmed mad min max range skew kurtosis se
## X1 1 632 1.53 1.59 1 1.26 1.48 0 7 7 1.26 1.12 0.06
Things to note:
- Subordination includes 0.
- It is a count variable.
Therefore, I will fit a negative binomial mixed-effects model.
## Family: nbinom2 ( log )
## Formula:
## Subordination ~ 1 + Genre + Complexity + Genre:Complexity + AchievEng +
## StAnx + MotEng + Complexity:MotEng + RSPANpartial_WM.C +
## (1 | Participant)
## Data: dat
##
## AIC BIC logLik -2*log(L) df.resid
## 1887.8 1936.7 -932.9 1865.8 621
##
## Random effects:
##
## Conditional model:
## Groups Name Variance Std.Dev.
## Participant (Intercept) 0.4474 0.6689
## Number of obs: 632, groups: Participant, 158
##
## Dispersion parameter for nbinom2 family (): 6.19e+07
##
## Conditional model:
## Estimate Std. Error z value Pr(>|z|)
## (Intercept) 0.449106 0.081934 5.481 4.22e-08 ***
## GenreNAR -0.273932 0.086003 -3.185 0.001447 **
## ComplexitySIMPLE -0.087650 0.081857 -1.071 0.284276
## AchievEng -0.068519 0.066511 -1.030 0.302927
## StAnx -0.130722 0.064607 -2.023 0.043038 *
## MotEng 0.085732 0.073696 1.163 0.244698
## RSPANpartial_WM.C 0.003546 0.010465 0.339 0.734681
## GenreNAR:ComplexitySIMPLE -0.514525 0.136328 -3.774 0.000161 ***
## ComplexitySIMPLE:MotEng -0.071217 0.066402 -1.073 0.283488
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1Results:
- Subordination ~ 1 + Genre + Complexity + Genre:Complexity + StAnx + MotEng + Complexity:MotEng + (1 | Participant) is the best fitting model.
- Simple effect of Genre significant
- Simple effect of Stress Anxiety (StAnx) significant
- Interaction between Genre and Complexity significant
Visualization
# See the predicted values
allEffects(mod.sub@model,
xlevels = list(StAnx=c(-3,-2,-1,0,1,2,3),
MotEng=c(-3,-2,-1,0,1,2,3))) ## Warning in Effect.glmmTMB(predictors, mod, vcov. = vcov., ...): overriding
## variance function for effects/dev.resids: computed variances may be incorrect
## Warning in Effect.glmmTMB(predictors, mod, vcov. = vcov., ...): overriding
## variance function for effects/dev.resids: computed variances may be incorrect
## Warning in Effect.glmmTMB(predictors, mod, vcov. = vcov., ...): overriding
## variance function for effects/dev.resids: computed variances may be incorrect
## Warning in Effect.glmmTMB(predictors, mod, vcov. = vcov., ...): overriding
## variance function for effects/dev.resids: computed variances may be incorrect
## Warning in Effect.glmmTMB(predictors, mod, vcov. = vcov., ...): overriding
## variance function for effects/dev.resids: computed variances may be incorrect## model: Subordination ~ 1 + Genre + Complexity + Genre:Complexity + AchievEng +
## StAnx + MotEng + Complexity:MotEng + RSPANpartial_WM.C
##
## AchievEng effect
## AchievEng
## -3 -2 -0.1 1 3
## 1.4116042 1.3181222 1.1572225 1.0732079 0.9357705
##
## StAnx effect
## StAnx
## -3 -2 -1 0 1 2 3
## 1.7014604 1.4929658 1.3100199 1.1494920 1.0086349 0.8850382 0.7765869
##
## RSPANpartial_WM.C effect
## RSPANpartial_WM.C
## -10 -6 3 10 20
## 1.109293 1.125142 1.161634 1.190833 1.233823
##
## Genre*Complexity effect
## Complexity
## Genre COMPLEX SIMPLE
## ARG 1.565711 1.4350944
## NAR 1.190541 0.6523157
##
## Complexity*MotEng effect
## MotEng
## Complexity -3 -2 -1 0 1 2 3
## COMPLEX 1.0563589 1.1509183 1.2539421 1.366188 1.4884816 1.621722 1.766890
## SIMPLE 0.9264158 0.9399605 0.9537033 0.967647 0.9817946 0.996149 1.010713# Generate plots
allEffects(mod.sub@model,
xlevels = list(StAnx=c(-3,-2,-1,0,1,2,3),
MotEng=c(-3,-2,-1,0,1,2,3))) %>% plot(multiline=TRUE) ## Warning in Effect.glmmTMB(predictors, mod, vcov. = vcov., ...): overriding
## variance function for effects/dev.resids: computed variances may be incorrect
## Warning in Effect.glmmTMB(predictors, mod, vcov. = vcov., ...): overriding
## variance function for effects/dev.resids: computed variances may be incorrect
## Warning in Effect.glmmTMB(predictors, mod, vcov. = vcov., ...): overriding
## variance function for effects/dev.resids: computed variances may be incorrect
## Warning in Effect.glmmTMB(predictors, mod, vcov. = vcov., ...): overriding
## variance function for effects/dev.resids: computed variances may be incorrect
## Warning in Effect.glmmTMB(predictors, mod, vcov. = vcov., ...): overriding
## variance function for effects/dev.resids: computed variances may be incorrect
Interpretation:
RQ1a(i):
- There is a significant effect of Genre such that Subordination decreases for the Narrative condition in both Simple (predicted 0.65 times) and Complex conditions (predicted 1.19 times) compared to the Argumentative Condition (predicted 1.43 and 1.57 times, respectively).
- There is also a significant interaction such that there is even less Subordination for Narrative in the Simple condition than in the Complex condition.
- There is no significant difference of complexity for the Argumentative Condition (1.57 vs. 1.43 predicted counts).
Note to Abbie
I’m indicating what is significant or not based on the model, but it’ll be up to you to determine what is a meaningful effect size or not (for example, I’m noting that nothing is strongly predicted to happen more than once or twice).
RQ1b(i):
- There is significant simple effect of StAnx, such that higher StAnx resulted in fewer Subordination clauses. At the low end of StAnx in the sample, participants produced about 1.51 Subordination clauses, and at the high end of StAnx, participants produced about 0.88 Subordination clauses, on average, controlling for Complexity, Genre, and MotEng
- Despite the inclusion of MotEng and and the interaction of MotEng and Complexity in the final model of best fit, they were not statistically significant and only kept by the algorithm to retain model convergence. (Note to Abbie: Even though the plot for Complexity x MotEng was produced, there are no significant effects to interpret there.)
3.2 RQ1a(ii) & RQ1b(ii): Syntactic Complexity (Coordination)
RQ1a(ii):
- How does altering task complexity and genre — specifically argumentative and narrative — influence the syntactic coordination?
RQ1b(ii):
- To what extent do individual differences moderate the effect of task complexity and genre on syntactic coordination?
First, check to see the distribution of “Coordination.”
## [1] 1 3 2 6 8 4 7 5 0 10 9##
## 0 1 2 3 4 5 6 7 8 9 10
## 118 127 140 100 52 47 12 14 20 1 1## vars n mean sd median trimmed mad min max range skew kurtosis se
## X1 1 632 2.37 2.04 2 2.12 1.48 0 10 10 1.02 0.74 0.08
Things to note:
- Coordination includes 0.
- It is a count variable.
Therefore, I will fit a Poisson mixed-effects model (Note: negative binomial mixed-effects model had convergence issues).
## Family: poisson ( log )
## Formula:
## Coordination ~ 1 + Genre + Complexity + Genre:Complexity + (1 |
## Participant)
## Data: dat
##
## AIC BIC logLik -2*log(L) df.resid
## 2311.4 2333.6 -1150.7 2301.4 627
##
## Random effects:
##
## Conditional model:
## Groups Name Variance Std.Dev.
## Participant (Intercept) 0.4238 0.651
## Number of obs: 632, groups: Participant, 158
##
## Conditional model:
## Estimate Std. Error z value Pr(>|z|)
## (Intercept) 0.57888 0.07732 7.487 7.07e-14 ***
## GenreNAR 0.10216 0.07438 1.374 0.170
## ComplexitySIMPLE 0.03989 0.07550 0.528 0.597
## GenreNAR:ComplexitySIMPLE 0.04799 0.10358 0.463 0.643
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1Results:
- Nothing is significant.
Visualization
## model: Coordination ~ 1 + Genre + Complexity + Genre:Complexity
##
## Genre*Complexity effect
## Complexity
## Genre COMPLEX SIMPLE
## ARG 1.784031 1.856640
## NAR 1.975920 2.157434
Interpretation:
- Altering Genre and Complexity does not have a significant effect on Coordination, particularly when the interaction effect of Genre and Complexity is taken into account (i.e., when the interaction of Genre and Complexity is incorporated in the model) (Note to Abbie: Remember that even though the plot for Genre x Complexity was produced, there are no significant effects to interpret there.)
3.3 RQ1a(iii) & RQ1b(iii): Accuracy (Number)
RQ1a(iii):
- How does altering task complexity and genre — specifically argumentative and narrative — influence the accuracy in number?
RQ1b(iii):
- To what extent do individual differences moderate the effect of task complexity and genre on accuracy in number?
First, check to see the distribution of “AccuracyNumber.”
## [1] 0 1 2 5 3 7 4##
## 0 1 2 3 4 5 7
## 361 127 113 20 9 1 1## vars n mean sd median trimmed mad min max range skew kurtosis se
## X1 1 632 0.73 1.01 0 0.57 0 0 7 7 1.5 2.89 0.04
Things to note:
- AccuracyNumber includes 0.
- It is a count variable.
Therefore, I will fit a zero-inflated negative binomial mixed-effects hurdle model.
## Family: truncated_nbinom2 ( log )
## Formula: AccuracyNumber ~ 1 + Genre + Complexity + Genre:Complexity +
## (1 | Participant)
## Zero inflation: ~1
## Data: dat
##
## AIC BIC logLik -2*log(L) df.resid
## 1463.9 1495.0 -724.9 1449.9 625
##
## Random effects:
##
## Conditional model:
## Groups Name Variance Std.Dev.
## Participant (Intercept) 0.06059 0.2461
## Number of obs: 632, groups: Participant, 158
##
## Dispersion parameter for truncated_nbinom2 family (): 3.16e+07
##
## Conditional model:
## Estimate Std. Error z value Pr(>|z|)
## (Intercept) -0.3727 0.1839 -2.026 0.0428 *
## GenreNAR 0.6542 0.2095 3.122 0.0018 **
## ComplexitySIMPLE 0.6197 0.2181 2.841 0.0045 **
## GenreNAR:ComplexitySIMPLE -0.7084 0.2829 -2.504 0.0123 *
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Zero-inflation model:
## Estimate Std. Error z value Pr(>|z|)
## (Intercept) 0.28676 0.08037 3.568 0.00036 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1Results:
RQ1a(iii)
- There is a significant effect of Genre such that AccuracyNumber increases for the Narrative condition in Complex condition (predicted 1.33 times) compared to the Argumentative condition (predicted 0.69 times).
- There is a significant effect of Complexity such that AccuracyNumber increases for the Simple condition in the Argumentative condition (predicted 1.28 times) compared to the Complex Condition (predicted 0.69 times).
- There is also a significant interaction of Genre and Complexity such that the effect of Genre becomes negative in the Simple condition. AccuracyNumber increases for the Narrative condition in Complex condition (predicted 1.33 times) compared to the Argumentative condition (predicted 0.69 times), but AccuracyNumber decreases for the Narrative (predicted 1.21 times) condition in Simple condition compared to the Argumentative condition (predicted 1.28 times).
RQ1b(iii)
- Individual difference factors did not significantly influence AccuracyNumber.
3.4 RQ1a(iv) & RQ1b(iv): Accuracy (Gender)
RQ1a(iv):
- How does altering task complexity and genre — specifically argumentative and narrative — influence the accuracy in gender?
RQ1b(iv):
- To what extent do individual differences moderate the effect of task complexity and genre on accuracy in gender?
First, check to see the distribution of “AccuracyGender.”
## [1] 1 0 3 2 5 4 7 6 10 8 9##
## 0 1 2 3 4 5 6 7 8 9 10
## 127 134 143 97 52 42 14 12 5 5 1## vars n mean sd median trimmed mad min max range skew kurtosis se
## X1 1 632 2.2 1.91 2 1.97 1.48 0 10 10 1.08 1.21 0.08
Things to note:
- AccuracyGender includes 0.
- It is a count variable.
Therefore, I will fit a negative binomial mixed-effects model.
## Family: nbinom2 ( log )
## Formula: AccuracyGender ~ 1 + Genre + Complexity + Genre:Complexity +
## CogAnx + (1 | Participant)
## Data: dat
##
## AIC BIC logLik -2*log(L) df.resid
## 2370.0 2401.2 -1178.0 2356.0 625
##
## Random effects:
##
## Conditional model:
## Groups Name Variance Std.Dev.
## Participant (Intercept) 0.0122 0.1104
## Number of obs: 632, groups: Participant, 158
##
## Dispersion parameter for nbinom2 family (): 4.7
##
## Conditional model:
## Estimate Std. Error z value Pr(>|z|)
## (Intercept) 1.09068 0.06071 17.966 < 2e-16 ***
## GenreNAR -0.45531 0.09042 -5.036 4.77e-07 ***
## ComplexitySIMPLE -0.19521 0.08608 -2.268 0.0233 *
## CogAnx -0.06905 0.03377 -2.045 0.0409 *
## GenreNAR:ComplexitySIMPLE -0.08789 0.13370 -0.657 0.5109
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1Results:
- Simple effect of Genre significant
- Simple effect of Complexity significant
- Simple effect of Cognitive Anxiety (CogAnx) significant
Visualization
## Warning in Effect.glmmTMB(predictors, mod, vcov. = vcov., ...): overriding
## variance function for effects/dev.resids: computed variances may be incorrect
## Warning in Effect.glmmTMB(predictors, mod, vcov. = vcov., ...): overriding
## variance function for effects/dev.resids: computed variances may be incorrect## model: AccuracyGender ~ 1 + Genre + Complexity + Genre:Complexity +
## CogAnx
##
## CogAnx effect
## CogAnx
## -3 -2 -1 0 1 2 3
## 2.587304 2.414667 2.253550 2.103184 1.962850 1.831880 1.709649
##
## Genre*Complexity effect
## Complexity
## Genre COMPLEX SIMPLE
## ARG 2.977639 2.449588
## NAR 1.888580 1.422937# Generate plots
allEffects(mod.gen@model,
xlevels = list(CogAnx=c(-3,-2,-1,0,1,2,3))) %>% plot(multiline=TRUE) ## Warning in Effect.glmmTMB(predictors, mod, vcov. = vcov., ...): overriding
## variance function for effects/dev.resids: computed variances may be incorrect
## Warning in Effect.glmmTMB(predictors, mod, vcov. = vcov., ...): overriding
## variance function for effects/dev.resids: computed variances may be incorrect
Interpretation:
RQ1a(iv):
- There is a significant effect of Genre such that AccuracyGender decreases for the Narrative condition in both Simple (predicted 1.42 times) and Complex conditions (predicted 1.89 times) compared to the Argumentative Condition (predicted 2.45 and 2.98 times, respectively).
- There is a significant effect of Complexity such that AccuracyGender decreases for the Simple condition in both Narrative (predicted 1.42 times) and Argumentative conditions (predicted 2.45 times) compared to the Complex Condition (predicted 1.89 and 2.98 times, respectively).
- The interaction of the Genre and Complexity did not have a significant effect on AccuracyGender.
Note to Abbie
I’m indicating what is significant or not based on the model, but it’ll be up to you to determine what is a meaningful effect size or not.
RQ1b(iv):
- There is a significant effect of Cognitive Anxiety such that higher CogAnx resulted in lower AccuracyGender. At the low end of CogAnx in the sample, participants produced about 2.41 accurate instances of gender, and at the high end of CogAnx, participants produced about 1.83 accurate instances of gender on average, controlling for Complexity and Genre.
3.5 RQ1a(v) & RQ1b(v): Accuracy (Tense)
RQ1a(v):
- How does altering task complexity and genre — specifically argumentative and narrative — influence the accuracy in tense?
RQ1b(v):
- To what extent do individual differences moderate the effect of task complexity and genre on accuracy in tense?
First, check to see the distribution of “AccuracyTense.”
## [1] 1 0 3 4 2 5 6 10 7 8 9##
## 0 1 2 3 4 5 6 7 8 9 10
## 192 146 144 87 45 8 5 1 1 1 2## vars n mean sd median trimmed mad min max range skew kurtosis se
## X1 1 632 1.56 1.52 1 1.37 1.48 0 10 10 1.34 3.56 0.06
Things to note:
- AccuracyTense includes 0.
- It is a count variable.
Therefore, I will fit a negative binomial mixed-effects model.
## Family: nbinom2 ( log )
## Formula:
## AccuracyTense ~ 1 + Genre + Complexity + Genre:Complexity + AchievEng +
## RSPANpartial_WM.C + Complexity:RSPANpartial_WM.C + (1 | Participant)
## Data: dat
##
## AIC BIC logLik -2*log(L) df.resid
## 2027.4 2067.5 -1004.7 2009.4 623
##
## Random effects:
##
## Conditional model:
## Groups Name Variance Std.Dev.
## Participant (Intercept) 0.1526 0.3906
## Number of obs: 632, groups: Participant, 158
##
## Dispersion parameter for nbinom2 family (): 193
##
## Conditional model:
## Estimate Std. Error z value Pr(>|z|)
## (Intercept) 0.067685 0.081874 0.827 0.40841
## GenreNAR 0.515843 0.093746 5.503 3.74e-08 ***
## ComplexitySIMPLE -0.033251 0.105534 -0.315 0.75271
## AchievEng 0.097096 0.044930 2.161 0.03069 *
## RSPANpartial_WM.C 0.007901 0.008749 0.903 0.36651
## GenreNAR:ComplexitySIMPLE 0.059253 0.132571 0.447 0.65491
## ComplexitySIMPLE:RSPANpartial_WM.C -0.027523 0.010188 -2.701 0.00691 **
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1Results:
- Simple effect of Genre is significant.
- Simple effects of Achievement Engagement (AchievEng) is significant.
- The interaction of Complexity and RSPANpartial is significant.
Visualization
# See the predicted values
allEffects(mod.ten@model,
xlevels = list(AchievEng=c(-3,-2,-1,0,1,2,3),
RSPANpartial_WM.C=c(-20,-15,-10,-5,0,5,10,15,20))) ## Warning in Effect.glmmTMB(predictors, mod, vcov. = vcov., ...): overriding
## variance function for effects/dev.resids: computed variances may be incorrect
## Warning in Effect.glmmTMB(predictors, mod, vcov. = vcov., ...): overriding
## variance function for effects/dev.resids: computed variances may be incorrect
## Warning in Effect.glmmTMB(predictors, mod, vcov. = vcov., ...): overriding
## variance function for effects/dev.resids: computed variances may be incorrect## model: AccuracyTense ~ 1 + Genre + Complexity + Genre:Complexity + AchievEng +
## RSPANpartial_WM.C + Complexity:RSPANpartial_WM.C
##
## AchievEng effect
## AchievEng
## -3 -2 -1 0 1 2 3
## 1.033039 1.138374 1.254451 1.382363 1.523318 1.678645 1.849811
##
## Genre*Complexity effect
## Complexity
## Genre COMPLEX SIMPLE
## ARG 1.070002 1.035009
## NAR 1.792307 1.839522
##
## Complexity*RSPANpartial_WM.C effect
## RSPANpartial_WM.C
## Complexity -20 -15 -10 -5 0 5 10
## COMPLEX 1.182419 1.230065 1.279632 1.331195 1.384837 1.440640 1.498691
## SIMPLE 2.042942 1.852031 1.678960 1.522062 1.379827 1.250883 1.133989
## RSPANpartial_WM.C
## Complexity 15 20
## COMPLEX 1.559082 1.6219060
## SIMPLE 1.028018 0.9319506# Generate plots
allEffects(mod.ten@model,
xlevels = list(AchievEng=c(-3,-2,-1,0,1,2,3),
RSPANpartial_WM.C=c(-20,-15,-10,-5,0,5,10,15,20))) %>% plot(multiline=TRUE) ## Warning in Effect.glmmTMB(predictors, mod, vcov. = vcov., ...): overriding
## variance function for effects/dev.resids: computed variances may be incorrect
## Warning in Effect.glmmTMB(predictors, mod, vcov. = vcov., ...): overriding
## variance function for effects/dev.resids: computed variances may be incorrect
## Warning in Effect.glmmTMB(predictors, mod, vcov. = vcov., ...): overriding
## variance function for effects/dev.resids: computed variances may be incorrect
Interpretation:
RQ1a(v):
- There is a significant effect of Genre such that AccuracyTense increases for the Narrative condition in both Simple (predicted 1.84 times) and Complex conditions (predicted 1.79 times) compared to the Argumentative Condition (predicted 1.03 and 1.07 times, respectively).
- The interaction of the Genre and Complexity did not have a significant effect on AccuracyTense.
Note to Abbie
I’m indicating what is significant or not based on the model, but it’ll be up to you to determine what is a meaningful effect size or not.
RQ1b(v):
- There is a significant effect of Achievement Engagement such that higher AchievEng resulted in higher AccuracyTense. At the low end of AchievEng in the sample, participants produced about 1.14 accurate instances of tense, and at the high end of AchievEng, participants produced about 1.68 accurate instances of tense on average, controlling for Complexity, Genre, and RSPANpartial_WM.
- There is a significant interaction effect of Complexity and RSPANpartial_WM such that higher RSPANpartial_WM is associated with a higher number of accurate instances of tense in the Complex condition, and higher RSPANpartial_WM is associated with a lower number of accurate instances of tense in the Simple condition. In the Simple condition, at the low end of RSPANpartial_WM in the sample, participants produced about 1.85 accurate instances of tense, and at the high end of RSPANpartial_WM, participants produced about 1.03 accurate instances of tense. In the Complex condition, at the low end of RSPANpartial_WM in the sample, participants produced about 1.23 accurate instances of tense, and at the high end of RSPANpartial_WM, participants produced about 1.56 accurate instances of tense.
3.6 RQ1a(vi) & RQ1b(vi): Accuracy (Aspect)
RQ1a(vi):
- How does altering task complexity and genre — specifically argumentative and narrative — influence the accuracy in aspect?
RQ1b(vi):
- To what extent do individual differences moderate the effect of task complexity and genre on accuracy in aspect?
First, check to see the distribution of “AccuracyAspect.”
## [1] 0 1 3 4 2 5 6 8 10 12 9 7 11##
## 0 1 2 3 4 5 6 7 8 9 10 11 12
## 514 23 24 33 11 12 5 5 1 1 1 1 1## vars n mean sd median trimmed mad min max range skew kurtosis se
## X1 1 632 0.62 1.59 0 0.19 0 0 12 12 3.3 12.93 0.06
Things to note:
- AccuracyAspect includes 0.
- There is a large number of 0s.
- It is a count variable.
Therefore, I will fit a zero-inflated negative binomial mixed-effects mixture model (zero-inflated negative binomial mixed-effects hurdle model had convergence issues).
## Family: nbinom1 ( log )
## Formula: AccuracyAspect ~ 1 + Genre + Complexity + Genre:Complexity +
## RSPANpartial_WM.C + Genre:RSPANpartial_WM.C + (1 | Participant)
## Zero inflation: ~1
## Data: dat
##
## AIC BIC logLik -2*log(L) df.resid
## 799.6 839.6 -390.8 781.6 623
##
## Random effects:
##
## Conditional model:
## Groups Name Variance Std.Dev.
## Participant (Intercept) 2.019e-09 4.493e-05
## Number of obs: 632, groups: Participant, 158
##
## Dispersion parameter for nbinom1 family (): 1.48
##
## Conditional model:
## Estimate Std. Error z value Pr(>|z|)
## (Intercept) -3.05385 0.57949 -5.270 1.36e-07 ***
## GenreNAR 4.11667 0.58330 7.058 1.69e-12 ***
## ComplexitySIMPLE -1.19375 0.84761 -1.408 0.15902
## RSPANpartial_WM.C -0.18230 0.07393 -2.466 0.01367 *
## GenreNAR:ComplexitySIMPLE -2.61401 0.96036 -2.722 0.00649 **
## GenreNAR:RSPANpartial_WM.C 0.17608 0.07463 2.359 0.01831 *
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Zero-inflation model:
## Estimate Std. Error z value Pr(>|z|)
## (Intercept) -1.3982 0.4328 -3.23 0.00124 **
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1Results:
- The simple effect of Genre is significant.
- The simple effect of RSPANpartial_WM is significant
- Interaction of Genre and Complexity is significant
- Interation of Genre and RSPANpartial_WM is significant
Visualization
# See the predicted values
allEffects(mod.asp@model,
xlevels = list(RSPANpartial_WM.C=c(-20,-15,-10,-5,0,5,10,15,20))) ## Warning in Effect.glmmTMB(predictors, mod, vcov. = vcov., ...): overriding
## variance function for effects/dev.resids: computed variances may be incorrect
## Warning in Effect.glmmTMB(predictors, mod, vcov. = vcov., ...): overriding
## variance function for effects/dev.resids: computed variances may be incorrect## model: AccuracyAspect ~ 1 + Genre + Complexity + Genre:Complexity +
## RSPANpartial_WM.C + Genre:RSPANpartial_WM.C
##
## Genre*Complexity effect
## Complexity
## Genre COMPLEX SIMPLE
## ARG 0.04717695 0.01429853
## NAR 2.89451174 0.06425178
##
## Genre*RSPANpartial_WM.C effect
## RSPANpartial_WM.C
## Genre -20 -15 -10 -5 0 5
## ARG 0.9953107 0.4000338 0.1607810 0.06462085 0.02597232 0.01043875
## NAR 0.4884043 0.4734424 0.4589389 0.44487966 0.43125113 0.41804010
## RSPANpartial_WM.C
## Genre 10 15 20
## ARG 0.004195528 0.00168626 0.0006777393
## NAR 0.405233775 0.39281976 0.3807860474# Generate plots
allEffects(mod.asp@model,
xlevels = list(RSPANpartial_WM.C=c(-20,-15,-10,-5,0,5,10,15,20))) %>% plot(multiline=TRUE) ## Warning in Effect.glmmTMB(predictors, mod, vcov. = vcov., ...): overriding
## variance function for effects/dev.resids: computed variances may be incorrect
## Warning in Effect.glmmTMB(predictors, mod, vcov. = vcov., ...): overriding
## variance function for effects/dev.resids: computed variances may be incorrect
Interpretation:
RQ1a(vi):
- There is a significant effect of Genre such that AccuracyAspect increases for the Narrative condition in both Simple (predicted 0.06 times) and Complex conditions (predicted 2.89 times) compared to the Argumentative Condition (predicted 0.01 and 0.05 times, respectively).
- There is a significant interaction effect of Complexity and Genre such that AccuracyAspect increases even more for the Narrative condition in the Complex condition (predicted 2.89 times) than in the Simple Condition (predicted 0.06 times).
RQ1b(vi):
- There is a significant effect of RSPANpartial_WM such that in the Argumentative condition, higher RSPANpartial_WM resulted in lower AccuracyTense in the Argumentative condition. At the low end of RSPANpartial_WM in the sample, participants produced about 0.40 accurate instances of tense, and at the high end of RSPANpartial_WM, participants produced about 0.002 accurate instances of tense on average, controlling for Complexity.
- There is a significant interaction effect of Genre and RSPANpartial_WM such that higher RSPANpartial_WM is associated with a lower number of accurate instances of aspect in the Complex condition, but the effect of RSPANpartial_WM is not significant in the Simple condition.
3.7 RQ1a(vii) & RQ1b(vii): Lexical Complexity (Lexical Density)
RQ1a(vii):
- How does altering task complexity and genre — specifically argumentative and narrative — influence the lexical density?
RQ1b(vii):
- To what extent do individual differences moderate the effect of task complexity and genre on lexical density??
First, check to see the distribution of “LexicalDensity.”
unique(dat$LexicalDensity) # Participant 141 has "%" in the cell ... I will remove for this analysis. ## [1] "0.28129999999999999" "0.42859999999999998" "0.31709999999999999"
## [4] "0.47499999999999998" "0.3448" "0.25690000000000002"
## [7] "0.28789999999999999" "0.39219999999999999" "0.5585"
## [10] "0.52629999999999999" "0.58460000000000001" "0.64"
## [13] "0.71430000000000005" "0.66669999999999996" "0.7"
## [16] "0.6" "0.6522" "0.73109999999999997"
## [19] "0.67290000000000005" "0.84550000000000003" "0.43590000000000001"
## [22] "0.55000000000000004" "0.47370000000000001" "0.60709999999999997"
## [25] "0.63639999999999997" "0.60609999999999997" "0.6552"
## [28] "0.55559999999999998" "0.73909999999999998" "0.50819999999999999"
## [31] "0.52" "0.51470000000000005" "0.5"
## [34] "0.64710000000000001" "0.375" "0.4375"
## [37] "0.46150000000000002" "0.59" "0.62"
## [40] "0.61" "0.28570000000000001" "0.38300000000000001"
## [43] "0.32140000000000002" "0.44190000000000002" "0.57279999999999998"
## [46] "0.59460000000000002" "0.54730000000000001" "0.51910000000000001"
## [49] "0.53849999999999998" "0.60699999999999998" "0.34620000000000001"
## [52] "0.56979999999999997" "0.48780000000000001" "0.55320000000000003"
## [55] "0.35899999999999999" "0.36170000000000002" "0.51280000000000003"
## [58] "0.33329999999999999" "0.25" "0.35"
## [61] "0.35289999999999999" "0.56100000000000005" "0.59260000000000002"
## [64] "0.45350000000000001" "0.47470000000000001" "0.44969999999999999"
## [67] "0.46289999999999998" "0.57140000000000002" "0.59379999999999999"
## [70] "0.4783" "0.44440000000000002" "0.44740000000000002"
## [73] "0.64859999999999995" "0.59240000000000004" "0.61539999999999995"
## [76] "0.56189999999999996" "0.54210000000000003" "0.55210000000000004"
## [79] "0.27160000000000001" "0.22770000000000001" "0.27850000000000003"
## [82] "0.40739999999999998" "0.50849999999999995" "0.47170000000000001"
## [85] "0.56520000000000004" "0.54239999999999999" "0.6512"
## [88] "0.61899999999999999" "0.3226" "0.42"
## [91] "0.4" "0.71150000000000002" "0.69"
## [94] "0.51129999999999998" "0.65310000000000001" "0.63200000000000001"
## [97] "0.71299999999999997" "0.51849999999999996" "0.4178"
## [100] "0.70669999999999999" "0.48699999999999999" "0.50449999999999995"
## [103] "0.45" "0.37209999999999999" "0.26669999999999999"
## [106] "0.42499999999999999" "0.37309999999999999" "0.42549999999999999"
## [109] "0.37780000000000002" "0.58620000000000005" "0.625"
## [112] "0.22439999999999999" "0.5161" "0.3846"
## [115] "0.44" "0.54549999999999998" "0.41670000000000001"
## [118] "0.51719999999999999" "0.36840000000000001" "0.36"
## [121] "0.5333" "0.45829999999999999" "0.63"
## [124] "0.65" "0.73960000000000004" "0.60199999999999998"
## [127] "0.70069999999999999" "0.65380000000000005" "0.69230000000000003"
## [130] "0.71699999999999997" "0.77780000000000005" "0.75"
## [133] "0.42109999999999997" "0.45710000000000001" "0.62860000000000005"
## [136] "0.62919999999999998" "0.56000000000000005" "0.49"
## [139] "0.57999999999999996" "0.63829999999999998" "0.67"
## [142] "0.66" "0.67649999999999999" "0.61699999999999999"
## [145] "0.60529999999999995" "0.60980000000000001" "0.56920000000000004"
## [148] "0.70450000000000002" "0.61729999999999996" "0.48649999999999999"
## [151] "0.48" "0.3019" "0.25929999999999997"
## [154] "0.44119999999999998" "0.32790000000000002" "0.53129999999999999"
## [157] "0.5625" "0.34210000000000002" "0.55879999999999996"
## [160] "0.34338000000000002" "0.3548" "0.29549999999999998"
## [163] "0.3095" "0.6573" "0.72919999999999996"
## [166] "0.69389999999999996" "0.2581" "0.2571"
## [169] "0.3478" "0.61109999999999998" "0.72"
## [172] "0.3256" "0.52939999999999998" "0.3947"
## [175] "0.38240000000000002" "0.36670000000000003" "0.34379999999999999"
## [178] "0.35420000000000001" "0.41860000000000003" "0.83299999999999996"
## [181] "0.63009999999999999" "0.71050000000000002" "0.4667"
## [184] "0.52380000000000004" "0.47220000000000001" "0.32079999999999997"
## [187] "0.78569999999999995" "0.5111" "0.44829999999999998"
## [190] "0.58819999999999995" "0.40029999999999999" "0.61160000000000003"
## [193] "0.72729999999999995" "0.6875" "0.6714"
## [196] "0.68689999999999996" "0.59040000000000004" "0.27589999999999998"
## [199] "0.39290000000000003" "0.30769999999999997" "0.63270000000000004"
## [202] "0.67569999999999997" "0.69769999999999999" "0.63890000000000002"
## [205] "0.63160000000000005" "0.69569999999999999" "0.68420000000000003"
## [208] "0.56759999999999999" "0.64439999999999997" "0.59299999999999997"
## [211] "0.49280000000000002" "0.45610000000000001" "0.59570000000000001"
## [214] "0.4516" "0.42420000000000002" "0.78949999999999998"
## [217] "0.67349999999999999" "0.72340000000000004" "0.43330000000000002"
## [220] "0.2545" "0.37930000000000003" "0.46510000000000001"
## [223] "0.68630000000000002" "0.68" "0.70730000000000004"
## [226] "0.81479999999999997" "0.69830000000000003" "0.83160000000000001"
## [229] "0.39579999999999999" "0.3962" "0.62070000000000003"
## [232] "0.27779999999999999" "56`%" "0.37140000000000001"
## [235] "0.67859999999999998" "0.65849999999999997" "0.6774"
## [238] "0.62370000000000003" "0.66269999999999996" "0.66069999999999995"
## [241] "0.72640000000000005" "0.43240000000000001" "0.2424"
## [244] "0.64839999999999998" "0.56999999999999995" "0.30499999999999999"dat_density <- dat %>%
filter(Participant != "141")
dat_density$LexicalDensity <- as.numeric(dat_density$LexicalDensity) # The values are stored as characters, change to numeric
is.numeric(dat_density$LexicalDensity)## [1] TRUE## vars n mean sd median trimmed mad min max range skew kurtosis se
## X1 1 628 0.52 0.14 0.55 0.52 0.16 0.22 0.85 0.62 -0.29 -0.97 0.01
Things to note:
- It is a continuous variable.
Therefore, I will fit a linear mixed-effects model.
## Linear mixed model fit by maximum likelihood (p-values based on Wald z-scores)
## [lmerMod]
## Formula: LexicalDensity ~ 1 + Genre + Complexity + Genre:Complexity +
## (1 | Participant)
## Data: dat_density
##
## AIC BIC logLik deviance df.resid
## -1119.3 -1092.7 565.7 -1131.3 622
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -3.0965 -0.5662 0.0441 0.4970 3.5040
##
## Random effects:
## Groups Name Variance Std.Dev.
## Participant (Intercept) 0.013834 0.1176
## Residual 0.005241 0.0724
## Number of obs: 628, groups: Participant, 157
##
## Fixed effects:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 0.524925 0.011023 47.623 <2e-16 ***
## GenreNAR 0.012968 0.008171 1.587 0.112
## ComplexitySIMPLE -0.017729 0.008171 -2.170 0.030 *
## GenreNAR:ComplexitySIMPLE -0.012870 0.011556 -1.114 0.265
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Correlation of Fixed Effects:
## (Intr) GnrNAR CSIMPL
## GenreNAR -0.371
## CmplxSIMPLE -0.371 0.500
## GNAR:CSIMPL 0.262 -0.707 -0.707
Note:
- The independence, linearity, normality, and homoscedasticity assumptions were checked based on visual inspection of residual plots. Some caution for interpretation may be necessary because the assumptions of linearity, normality, and homoscedasticity assumptions may not be met.
Results:
- Simple effect of Complexity is significant.
Visualization
## model: LexicalDensity ~ 1 + Genre + Complexity + Genre:Complexity
##
## Genre*Complexity effect
## Complexity
## Genre COMPLEX SIMPLE
## ARG 0.5249255 0.5071968
## NAR 0.5378936 0.5072954
Interpretation:
RQ1a(vii):
- There is a significant effect of Complexity such that LexicalDensity increases for the Complex condition in both Argumentative (predicted 0.52) and Narrative conditions (predicted 0.54) compared to the Simple condition (predicted 0.51 and 0.51, respectively).
- The interaction of the Genre and Complexity did not have a significant effect on LexicalDensity.
Note to Abbie
I’m indicating what is significant or not based on the model, but it’ll be up to you to determine what is a meaningful effect size or not.
RQ1b(vii):
- Individual difference factors did not significantly influence LexicalDensity.
3.8 RQ1a(viii) & RQ1b(viii): Lexical Complexity (Lexical Diversity)
RQ1a(vii):
- How does altering task complexity and genre — specifically argumentative and narrative — influence the lexical diversity?
RQ1b(vii):
- To what extent do individual differences moderate the effect of task complexity and genre on lexical diversity??
First, check to see the distribution of “LexicalDiversity.”
## vars n mean sd median trimmed mad min max range skew kurtosis se
## X1 1 632 0.75 2.06 0.64 0.64 0.15 0.06 37.23 37.17 17.51 306.83 0.08
unique(dat$LexicalDiversity) # Participants 157 and 158 have an extreme value (37.2300) ... remove for this analysis. ## [1] 0.5938 0.8095 0.5610 0.9000 0.4500 0.4900 0.4242 0.6078 0.3900
## [10] 0.4700 0.6494 0.6797 0.6923 0.7400 0.8571 0.9167 0.4957 0.6134
## [19] 0.5701 0.6182 0.4872 0.4103 0.5263 0.6429 0.6970 0.7333 0.7879
## [28] 0.6500 0.6207 0.6296 0.7391 0.4426 0.4800 0.4559 0.4231 0.8235
## [37] 0.2810 0.6875 0.9231 0.6900 0.6800 0.7100 0.7000 0.4400 0.6809
## [46] 0.5000 0.8140 0.4660 0.5405 0.4737 0.7692 0.7140 0.5556 0.4615
## [55] 0.6977 0.6585 0.6170 0.6316 0.3617 0.5897 0.6667 0.8500 0.7059
## [64] 0.8049 0.8000 0.7407 0.8519 0.6047 0.5886 0.6190 0.5943 0.5873
## [73] 0.6406 0.8261 0.8333 0.6053 0.6061 0.6486 0.6417 0.5091 0.4381
## [82] 0.6636 0.6250 0.5172 0.3761 0.2469 0.2079 0.2532 0.3529 0.5741
## [91] 0.6102 0.6226 0.6304 0.7627 0.6111 0.7619 0.8700 0.4839 0.7083
## [100] 0.5882 0.5100 0.7500 0.7600 0.4962 0.6224 0.5600 0.6731 0.6364
## [109] 0.7877 0.8533 0.7739 0.7928 0.6000 0.8182 0.8667 0.4250 0.3731
## [118] 0.4681 0.5102 0.6897 0.6222 0.7097 0.7308 0.4000 0.7300 0.6333
## [127] 0.8367 0.6087 0.7200 0.5500 0.5816 0.5839 0.7576 0.8462 0.7143
## [136] 0.6857 0.6538 0.5714 0.7547 0.8889 0.8750 0.6579 0.6571 0.8330
## [145] 0.7857 0.6067 0.5900 0.5278 0.8056 0.6892 0.7674 0.7667 0.7053
## [154] 0.7241 0.9091 0.7660 0.8185 0.5231 0.6591 0.4595 0.5309 0.5946
## [163] 0.9032 0.8400 0.9048 0.5660 0.4815 0.5385 0.5410 0.4490 0.7813
## [172] 0.7900 0.7105 0.6471 0.6563 0.5806 0.7045 0.6842 0.7381 0.4196
## [181] 0.7917 0.6774 0.6944 0.8900 0.6512 0.8250 0.7353 0.4688 0.6042
## [190] 0.8372 0.6905 0.6400 0.4658 0.5300 0.5640 0.5116 0.4468 0.8600
## [199] 0.9400 0.4038 0.7222 0.5094 0.4889 0.0986 0.5526 0.6164 0.6600
## [208] 0.6984 0.3429 0.3243 0.7647 0.5571 0.5812 0.7349 0.7544 0.7632
## [217] 0.6786 0.7234 0.7551 0.7368 0.6860 0.6377 0.7281 0.6383 0.4082
## [226] 0.5390 0.5143 0.7250 0.6458 0.5814 0.7561 0.7531 0.7895 0.5849
## [235] 0.8460 0.3700 0.6349 0.6118 0.7714 0.6154 0.7273 0.7317 0.7419
## [244] 0.5161 0.5904 0.5446 0.5283 0.5192 0.4697 0.0604 0.4018 37.2300dat_diversity <- dat %>%
filter(Participant != "157") %>%
filter(Participant != "158")
hist(dat_diversity$LexicalDiversity)
## vars n mean sd median trimmed mad min max range skew kurtosis se
## X1 1 624 0.63 0.14 0.64 0.64 0.15 0.06 0.94 0.88 -0.31 0.12 0.01Things to note:
- It is a continuous variable.
Therefore, I will fit a linear mixed-effects model.
## Linear mixed model fit by maximum likelihood (p-values based on Wald z-scores)
## [lmerMod]
## Formula: LexicalDiversity ~ 1 + Genre + Complexity + Genre:Complexity +
## (1 | Participant)
## Data: dat_diversity
##
## AIC BIC logLik deviance df.resid
## -890.2 -863.6 451.1 -902.2 618
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -4.2812 -0.4824 0.0283 0.4847 3.4035
##
## Random effects:
## Groups Name Variance Std.Dev.
## Participant (Intercept) 0.011668 0.10802
## Residual 0.008678 0.09316
## Number of obs: 624, groups: Participant, 156
##
## Fixed effects:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 0.62803 0.01142 54.992 <2e-16 ***
## GenreNAR 0.03443 0.01055 3.264 0.0011 **
## ComplexitySIMPLE -0.01341 0.01055 -1.271 0.2036
## GenreNAR:ComplexitySIMPLE -0.01646 0.01492 -1.103 0.2699
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Correlation of Fixed Effects:
## (Intr) GnrNAR CSIMPL
## GenreNAR -0.462
## CmplxSIMPLE -0.462 0.500
## GNAR:CSIMPL 0.327 -0.707 -0.707
Note:
- The independence, linearity, normality, and homoscedasticity assumptions were checked based on visual inspection of residual plots. Some caution for interpretation may be necessary because the assumptions of linearity, normality, and homoscedasticity assumptions may not be met.
Results:
- Main effect of Genre is significant.
Visualization
## model: LexicalDiversity ~ 1 + Genre + Complexity + Genre:Complexity
##
## Genre*Complexity effect
## Complexity
## Genre COMPLEX SIMPLE
## ARG 0.6280340 0.6146231
## NAR 0.6624673 0.6325974
Interpretation:
RQ1a(vii):
- There is a significant effect of Genre such that LexicalDiversity increases for the Narrative condition in both Simple (predicted 0.63) and Complex conditions (predicted 0.66) compared to the Argumentative condition (predicted 0.61 and 0.63, respectively).
- The interaction of the Genre and Complexity did not have a significant effect on LexicalDiversity.
Note to Abbie
I’m indicating what is significant or not based on the model, but it’ll be up to you to determine what is a meaningful effect size or not.
RQ1b(vii):
- Individual difference factors did not significantly influence LexicalDiversity.
3.9 RQ1a(ix) & RQ1b(ix): Fluency (Syllables per minute)
RQ1a(ix):
- How does altering task complexity and genre — specifically argumentative and narrative — influence the Fluency (Syllables per minute)?
RQ1b(ix):
- To what extent do individual differences moderate the effect of task complexity and genre on Fluency (Syllables per minute)??
First, check to see the distribution of “FluencySyllablesperMinute.”
unique(dat$FluencySyllablesperMinute) # Participant 72 has "7.3." (with a period after 3) ... remove this participant for the analysis. ## [1] "4.45" "9.14" "7.75"
## [4] "9" "10.33" "15"
## [7] "10" "14" "12"
## [10] "16" "11" "17"
## [13] "7.17" "13.2" "9.33"
## [16] "9.5" "14.64" "11.22"
## [19] "3.38" "3.67" "3.29"
## [22] "3.5" "32.39" "36"
## [25] "31.71" "43.2" "9.2899999999999991"
## [28] "6.86" "12.8" "1.97"
## [31] "10.57" "18.75" "3.64"
## [34] "6.25" "9.6" "10.86"
## [37] "9.8000000000000007" "13.67" "5.8"
## [40] "3.4" "5.33" "8.3000000000000007"
## [43] "8.98" "13.55" "10.42"
## [46] "13" "5.92" "5.54"
## [49] "8.1300000000000008" "6.75" "8.1999999999999993"
## [52] "10.36" "6.71" "6"
## [55] "4.0999999999999996" "9.0500000000000007" "19.440000000000001"
## [58] "11.64" "7.29" "9.7100000000000009"
## [61] "10.31" "6.63" "10.14"
## [64] "6.4" "5.4" "12.67"
## [67] "10.5" "11.7" "10.8"
## [70] "11.4" "8.57" "8.85"
## [73] "4.8" "4.58" "8.25"
## [76] "8.6300000000000008" "8.33" "3.17"
## [79] "5.88" "11.25" "18.89"
## [82] "19.64" "15.42" "9.75"
## [85] "14.82" "15.7" "11.8"
## [88] "12.6" "9.7799999999999994" "11.13"
## [91] "14.83" "13.5" "7.73"
## [94] "7.1" "10.6" "2.64"
## [97] "4.42" "2.8" "5"
## [100] "4.9400000000000004" "9.89" "5.07"
## [103] "7.36" "10.38" "17.329999999999998"
## [106] "15.5" "13.6" "14.67"
## [109] "7.63" "14.9" "15.4"
## [112] "18.329999999999998" "14.85" "2.83"
## [115] "5.9" "6.29" "3"
## [118] "1.86" "4" "2.11"
## [121] "6.22" "16.43" "7.71"
## [124] "5.5" "7.38" "5.25"
## [127] "8.8000000000000007" "4.7" "2.59"
## [130] "3.86" "6.38" "8.5"
## [133] "6.83" "2.93" "7.31"
## [136] "13.36" "10.83" "12.4"
## [139] "7" "8" "16.100000000000001"
## [142] "10.73" "14.13" "7.86"
## [145] "6.6" "8.8800000000000008" "11.17"
## [148] "3.27" "4.93" "3.63"
## [151] "7.2" "5.6" "2.86"
## [154] "5.75" "13.15" "23.22"
## [157] "15.79" "14.46" "4.3099999999999996"
## [160] "14.5" "6.78" "7.3"
## [163] "4.9000000000000004" "7.3." "9.1999999999999993"
## [166] "12.63" "12.43" "18.2"
## [169] "10.96" "10.85" "7.8"
## [172] "5.56" "6.49" "6.5"
## [175] "3.92" "4.88" "5.73"
## [178] "11.88" "7.13" "4.6399999999999997"
## [181] "3.1" "10.43" "11.3"
## [184] "15.55" "11.86" "9.43"
## [187] "12.5" "8.14" "27"
## [190] "5.29" "5.57" "6.92"
## [193] "6.67" "8.3800000000000008" "5.0999999999999996"
## [196] "6.64" "5.86" "7.33"
## [199] "1.9" "2" "1.2"
## [202] "8.9" "5.38" "18.38"
## [205] "16.8" "10.1" "4.71"
## [208] "2.89" "4.43" "9.4"
## [211] "11.71" "3.55" "4.5"
## [214] "7.88" "19.29" "12.25"
## [217] "4.13" "7.79" "16.5"
## [220] "15.56" "13.7" "7.9"
## [223] "9.44" "12.3" "12.9"
## [226] "17.43" "10.09" "9.25"
## [229] "4.4000000000000004" "7.14" "8.4"
## [232] "5.82" "15.2" "16.170000000000002"
## [235] "13.44" "14.17" "7.56"
## [238] "10.17" "11.57" "21.46"
## [241] "22.25" "8.67" "12.29"
## [244] "22.5" "7.67" "4.75"
## [247] "9.85" "16.600000000000001" "14.21"
## [250] "4.87" "7.5" "4.2"
## [253] "4.57" "6.98" "1.08"
## [256] "2.5" "3.6" "8.7100000000000009"
## [259] "11.2" "15.17" "5.17"
## [262] "4.55" "5.67" "9.4499999999999993"
## [265] "14.89" "14.77" "9.86"
## [268] "7.6" "21.33" "4.5999999999999996"
## [271] "8.56" "10.89" "30.5"dat_fluency <- dat %>%
filter(Participant != "72")
dat_fluency$FluencySyllablesperMinute <- as.numeric(dat_fluency$FluencySyllablesperMinute) # Store values as numeric
is.numeric(dat_fluency$FluencySyllablesperMinute) # Should be true## [1] TRUE
Things to note:
- It is a continuous variable.
Therefore, I will fit a linear mixed-effects model.
## Linear mixed model fit by maximum likelihood (p-values based on Wald z-scores)
## [lmerMod]
## Formula:
## FluencySyllablesperMinute ~ 1 + Genre + Complexity + Genre:Complexity +
## (1 | Participant)
## Data: dat_fluency
##
## AIC BIC logLik deviance df.resid
## 3406.0 3432.7 -1697.0 3394.0 622
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -3.1361 -0.5645 -0.0842 0.5227 5.9639
##
## Random effects:
## Groups Name Variance Std.Dev.
## Participant (Intercept) 13.639 3.693
## Residual 7.729 2.780
## Number of obs: 628, groups: Participant, 157
##
## Fixed effects:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 9.4454 0.3689 25.603 < 2e-16 ***
## GenreNAR 0.3648 0.3138 1.163 0.245024
## ComplexitySIMPLE -1.0564 0.3138 -3.367 0.000761 ***
## GenreNAR:ComplexitySIMPLE 1.8588 0.4438 4.189 2.8e-05 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Correlation of Fixed Effects:
## (Intr) GnrNAR CSIMPL
## GenreNAR -0.425
## CmplxSIMPLE -0.425 0.500
## GNAR:CSIMPL 0.301 -0.707 -0.707
Note:
- The independence, linearity, normality, and homoscedasticity assumptions were checked based on visual inspection of residual plots. Some caution for interpretation may be necessary because the assumptions of linearity, normality, and homoscedasticity assumptions may not be met.
Results:
- Simple effect of Complexity is significant.
- Interaction effect of Genre and Complexity is significant.
Visualization
## model: FluencySyllablesperMinute ~ 1 + Genre + Complexity + Genre:Complexity
##
## Genre*Complexity effect
## Complexity
## Genre COMPLEX SIMPLE
## ARG 9.445414 8.389045
## NAR 9.810191 10.612611
Interpretation:
RQ1a(ix):
- There is a significant effect of Complexity such that FluencySyllablesperMinute decreases for the Simple condition in the Argumentative (predicted 8.38) condition.
- There is a significant interaction effect of Genre and Complexity such that, in the Narrative condition, FluencySyllablesperMinute increases in the Simple condition from the Argumentative (predicted 8.39) to Narrative condition (predicted 10.61), while, in the Complex condition, the values are relatively similar for Argumentative (9.45) and Narrative (9.81) conditions.
RQ1b(ix):
- Individual difference factors did not significantly influence LexicalDiversity.
4 Research Questions 2:
4.1 RQ2a:
- How does altering task complexity and genre — specifically argumentative and narrative — influence the perceived difficulty of a task in L2 writing?
5 Fit the model for RQ2a and RQ2b using Buildmer
## Linear mixed model fit by maximum likelihood (p-values based on Wald z-scores)
## [lmerMod]
## Formula: PTD_composite ~ 1 + Genre + Complexity + Genre:Complexity + CogAnx +
## Complexity:CogAnx + (1 | Participant)
## Data: dat
##
## AIC BIC logLik deviance df.resid
## 2073.3 2108.9 -1028.6 2057.3 624
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -2.7445 -0.5777 0.0543 0.6690 3.3456
##
## Random effects:
## Groups Name Variance Std.Dev.
## Participant (Intercept) 0.9366 0.9678
## Residual 1.0354 1.0175
## Number of obs: 632, groups: Participant, 158
##
## Fixed effects:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 5.50120 0.11172 49.241 < 2e-16 ***
## GenreNAR 1.16297 0.11448 10.159 < 2e-16 ***
## ComplexitySIMPLE 0.29239 0.11448 2.554 0.0106 *
## CogAnx -0.05840 0.09598 -0.608 0.5429
## GenreNAR:ComplexitySIMPLE -1.11076 0.16190 -6.861 6.86e-12 ***
## ComplexitySIMPLE:CogAnx 0.19342 0.08099 2.388 0.0169 *
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Correlation of Fixed Effects:
## (Intr) GnrNAR CmSIMPLE CogAnx GNAR:C
## GenreNAR -0.512
## CmplxSIMPLE -0.512 0.500
## CogAnx 0.006 0.000 -0.002
## GNAR:CSIMPL 0.362 -0.707 -0.707 0.000
## CmSIMPLE:CA -0.002 0.000 0.005 -0.422 0.000
Note:
- The independence, linearity, normality, and homoscedasticity assumptions were checked based on visual inspection of residual plots. Some caution for interpretation may be necessary because the assumptions of linearity and homoscedasticity assumptions may not be met.
Results:
- Simple effect of Genre is significant
- Simple effect of Complexity is significant
- Interaction effect of Genre and Complexity is significant
- Interaction effect of Complexity and Cognitive Anxiety (CogAnx) is significant
Visualization
## model: PTD_composite ~ 1 + Genre + Complexity + Genre:Complexity + CogAnx +
## Complexity:CogAnx
##
## Genre*Complexity effect
## Complexity
## Genre COMPLEX SIMPLE
## ARG 5.501582 5.792722
## NAR 6.664557 5.844937
##
## Complexity*CogAnx effect
## CogAnx
## Complexity -3 -2 -1 0 1 2 3
## COMPLEX 6.257898 6.199496 6.141093 6.082691 6.024289 5.965886 5.907484
## SIMPLE 5.414661 5.549675 5.684690 5.819704 5.954718 6.089733 6.224747
Interpretation:
RQ2a:
- The significant simple effect of Genre indicates that Narrative produced higher PTD than Argumentative for the Complex condition only (6.66 vs. 5.50).
- The significant interaction of Genre and Complexity indicates that the effect of Genre weakens for the Simple condition (5.84 vs. 5.79) compared to the Complex condition (6.66 vs. 5.50).
RQ2b:
- For the Complex condition, there is no significant effects of CogAnx, but the significant interaction of Complexity and CogAnx indicates that, for the Simple condition, higher CogAnx is associated with higher PTD scores, and vice versa.