Model evaluation and violations

Dr Jens Roeser

Learning outcomes

After completing this lecture, the workshop and your own reading you should be able to …

explain what residuals are.
run regression models with categorical predictors.
evaluate statistical models on the basis of their unexplained (residual) variance.

Model evaluation

“Models are devices that connect theories to data. A model is an instantiation of a theory […]” (Rouder et al., 2016, p. 2)

Violations of model assumptions reduce reliability.
Possibility leading to incorrect conclusions about data.
Violations don’t render model results wrong; neither are models without violations necessarily correct.
Generally we need to be cautious with and conscious of violations when interpreting results.

Assumptions of parametric models

e.g. t-test, ANOVA, linear regression
Remember LINE
Linearity (for continuous predictor variables)
Independence
Normality
Equal of variance (also homogeneity)

Example data set

Blomkvist et al. (2017)

Age-related changes in cognitive performance through adolescence and adulthood in a real world task.

StarCraft 2

Real-time strategy video game

Blomkvist et al. (2017)

id	age	age_2	age_3	rt
1	84	old-ish	old-ish	701.67
2	37	young-ish	young-ish	470.67
3	62	old-ish	middle	638.67
4	85	old-ish	old-ish	708.00
5	73	old-ish	old-ish	607.33
6	65	old-ish	middle	541.67
7	30	young-ish	young-ish	570.67
8	49	young-ish	middle	509.33
9	83	old-ish	old-ish	736.67
10	58	young-ish	middle	549.67
11	25	young-ish	young-ish	548.00
12	88	old-ish	old-ish	889.00
13	62	old-ish	middle	883.67
14	88	old-ish	old-ish	831.67
15	27	young-ish	young-ish	536.33
16	60	old-ish	middle	671.67
17	80	old-ish	old-ish	930.00
18	24	young-ish	young-ish	454.67
19	27	young-ish	young-ish	531.33
20	52	young-ish	middle	539.33
21	27	young-ish	young-ish	436.67
23	74	old-ish	old-ish	696.33
24	57	young-ish	middle	611.67
25	23	young-ish	young-ish	523.67
26	39	young-ish	young-ish	443.33
27	51	young-ish	middle	628.67
28	59	young-ish	middle	586.33
29	24	young-ish	young-ish	450.33
30	27	young-ish	young-ish	532.67
31	26	young-ish	young-ish	494.00
32	76	old-ish	old-ish	566.33
33	62	old-ish	middle	485.00
34	39	young-ish	young-ish	518.00
35	24	young-ish	young-ish	547.67
36	57	young-ish	middle	517.67
37	64	old-ish	middle	582.00
38	64	old-ish	middle	552.67
39	88	old-ish	old-ish	727.00
40	37	young-ish	young-ish	425.33
41	52	young-ish	middle	592.00
42	58	young-ish	middle	557.67
43	57	young-ish	middle	548.00
44	89	old-ish	old-ish	739.00
45	21	young-ish	young-ish	616.67
46	27	young-ish	young-ish	435.00
47	62	old-ish	middle	537.00
48	70	old-ish	middle	650.33
49	39	young-ish	young-ish	556.67
50	85	old-ish	old-ish	918.67
51	44	young-ish	young-ish	449.67
52	73	old-ish	old-ish	519.33
53	44	young-ish	young-ish	630.33
54	31	young-ish	young-ish	541.00
55	71	old-ish	middle	670.67
56	27	young-ish	young-ish	518.33
57	23	young-ish	young-ish	430.00
58	54	young-ish	middle	502.33
59	83	old-ish	old-ish	522.33
60	46	young-ish	young-ish	661.00
61	30	young-ish	young-ish	516.33
62	42	young-ish	young-ish	475.67
63	55	young-ish	middle	701.00
64	33	young-ish	young-ish	583.67
65	71	old-ish	middle	526.00
66	81	old-ish	old-ish	656.67
67	63	old-ish	middle	617.33
68	33	young-ish	young-ish	480.33
69	53	young-ish	middle	680.33
70	85	old-ish	old-ish	1101.67
71	31	young-ish	young-ish	534.00
72	30	young-ish	young-ish	623.67
73	65	old-ish	middle	639.33
74	73	old-ish	old-ish	767.33
75	21	young-ish	young-ish	614.33
76	70	old-ish	middle	662.00
77	90	old-ish	old-ish	858.00
78	43	young-ish	young-ish	579.67
79	71	old-ish	middle	492.67
80	38	young-ish	young-ish	633.00
81	22	young-ish	young-ish	557.33
82	21	young-ish	young-ish	559.67
83	47	young-ish	middle	595.67
84	69	old-ish	middle	618.33
85	22	young-ish	young-ish	721.67
86	56	young-ish	middle	731.33
87	68	old-ish	middle	670.67
88	53	young-ish	middle	557.67
89	73	old-ish	old-ish	630.33
90	33	young-ish	young-ish	481.67
91	49	young-ish	middle	537.67
92	50	young-ish	middle	574.33
93	74	old-ish	old-ish	656.67
94	88	old-ish	old-ish	668.67
95	81	old-ish	old-ish	497.67
96	77	old-ish	old-ish	1241.33
97	76	old-ish	old-ish	571.33
98	49	young-ish	middle	537.00
99	72	old-ish	middle	597.67
100	51	young-ish	middle	784.00
101	52	young-ish	middle	441.00
102	38	young-ish	young-ish	663.00
103	69	old-ish	middle	725.33
104	50	young-ish	middle	534.00
105	21	young-ish	young-ish	409.67
106	22	young-ish	young-ish	441.67
107	74	old-ish	old-ish	550.00
108	56	young-ish	middle	614.67
109	70	old-ish	middle	835.00
110	74	old-ish	old-ish	1292.67
111	32	young-ish	young-ish	984.00
112	24	young-ish	young-ish	511.33
113	66	old-ish	middle	623.67
114	88	old-ish	old-ish	922.33
115	47	young-ish	middle	710.00
116	45	young-ish	young-ish	659.00
117	58	young-ish	middle	627.00
118	31	young-ish	young-ish	478.00
119	69	old-ish	middle	1148.33
120	86	old-ish	old-ish	994.33
121	21	young-ish	young-ish	561.33
122	23	young-ish	young-ish	669.67
123	48	young-ish	middle	619.67
124	20	young-ish	young-ish	520.33
125	59	young-ish	middle	619.33
126	79	old-ish	old-ish	894.00
127	96	old-ish	old-ish	1178.67
128	66	old-ish	middle	644.33
129	38	young-ish	young-ish	494.67
130	43	young-ish	young-ish	632.67
131	50	young-ish	middle	584.00
132	52	young-ish	middle	520.67
133	41	young-ish	young-ish	525.33
134	66	old-ish	middle	694.00
135	71	old-ish	middle	643.00
136	77	old-ish	old-ish	675.33
137	51	young-ish	middle	667.67
138	60	old-ish	middle	640.67
139	54	young-ish	middle	731.33
140	74	old-ish	old-ish	844.00
141	26	young-ish	young-ish	441.00
142	21	young-ish	young-ish	554.67
143	75	old-ish	old-ish	679.67
144	36	young-ish	young-ish	501.33
145	43	young-ish	young-ish	495.33
146	21	young-ish	young-ish	496.67
147	74	old-ish	old-ish	600.33
148	37	young-ish	young-ish	592.33
149	34	young-ish	young-ish	478.67
150	30	young-ish	young-ish	510.00
151	27	young-ish	young-ish	502.67
152	68	old-ish	middle	1030.33
153	27	young-ish	young-ish	471.67
154	21	young-ish	young-ish	544.67
155	34	young-ish	young-ish	536.33
156	67	old-ish	middle	859.00
157	64	old-ish	middle	666.00
158	34	young-ish	young-ish	431.67
159	75	old-ish	old-ish	1111.33
160	28	young-ish	young-ish	379.33
161	42	young-ish	young-ish	604.00
162	49	young-ish	middle	642.67
163	87	old-ish	old-ish	844.67
164	73	old-ish	old-ish	796.67
165	34	young-ish	young-ish	490.67
166	66	old-ish	middle	703.67
167	79	old-ish	old-ish	993.00
168	33	young-ish	young-ish	789.00
169	47	young-ish	middle	785.00
170	69	old-ish	middle	794.00
171	92	old-ish	old-ish	1024.67
172	74	old-ish	old-ish	554.67
173	84	old-ish	old-ish	630.33
174	71	old-ish	middle	711.00
175	54	young-ish	middle	627.00
176	82	old-ish	old-ish	657.67
177	89	old-ish	old-ish	897.67
178	26	young-ish	young-ish	466.00
179	64	old-ish	middle	560.67
180	59	young-ish	middle	521.33
181	69	old-ish	middle	1551.67
182	70	old-ish	middle	612.33
183	25	young-ish	young-ish	888.33
184	26	young-ish	young-ish	541.67
185	26	young-ish	young-ish	491.67
186	23	young-ish	young-ish	463.33
187	71	old-ish	middle	1000.33
188	71	old-ish	middle	587.67
189	28	young-ish	young-ish	386.33
190	33	young-ish	young-ish	510.33
191	23	young-ish	young-ish	468.67
192	70	old-ish	middle	744.67
193	30	young-ish	young-ish	594.00
194	73	old-ish	old-ish	667.67
195	58	young-ish	middle	616.00
196	44	young-ish	young-ish	460.33
197	45	young-ish	young-ish	479.33
198	66	old-ish	middle	431.33
199	48	young-ish	middle	558.00
200	28	young-ish	young-ish	469.33
201	34	young-ish	young-ish	544.33
202	48	young-ish	middle	508.67
203	71	old-ish	middle	519.33
204	77	old-ish	old-ish	928.33
205	49	young-ish	middle	530.00
206	28	young-ish	young-ish	520.67
207	24	young-ish	young-ish	430.33
208	58	young-ish	middle	477.00
209	69	old-ish	middle	684.00
210	69	old-ish	middle	545.00
211	58	young-ish	middle	563.33
212	31	young-ish	young-ish	451.33
213	74	old-ish	old-ish	832.00
214	88	old-ish	old-ish	711.33
215	72	old-ish	middle	952.67
216	69	old-ish	middle	617.33
217	48	young-ish	middle	591.00
218	64	old-ish	middle	574.00
219	57	young-ish	middle	511.00
220	49	young-ish	middle	471.00
221	30	young-ish	young-ish	459.33
222	75	old-ish	old-ish	538.67
223	31	young-ish	young-ish	542.67
224	59	young-ish	middle	614.00
226	57	young-ish	middle	593.33
227	45	young-ish	young-ish	536.67
228	25	young-ish	young-ish	436.00
229	63	old-ish	middle	607.67
230	59	young-ish	middle	628.00
231	53	young-ish	middle	427.33
232	25	young-ish	young-ish	477.33
233	25	young-ish	young-ish	482.67
234	74	old-ish	old-ish	547.33
235	38	young-ish	young-ish	452.00
236	27	young-ish	young-ish	537.67
237	45	young-ish	young-ish	548.33
238	53	young-ish	middle	474.67
239	24	young-ish	young-ish	583.67
240	36	young-ish	young-ish	461.67
241	39	young-ish	young-ish	532.33
242	32	young-ish	young-ish	397.67
243	64	old-ish	middle	991.33
244	75	old-ish	old-ish	755.67
245	61	old-ish	middle	877.33
246	77	old-ish	old-ish	841.67
247	40	young-ish	young-ish	538.33
248	65	old-ish	middle	566.67
249	69	old-ish	middle	629.00
250	73	old-ish	old-ish	605.67
251	30	young-ish	young-ish	524.00
252	83	old-ish	old-ish	475.67
253	31	young-ish	young-ish	407.33
254	78	old-ish	old-ish	645.33
255	67	old-ish	middle	646.33
256	79	old-ish	old-ish	389.67
257	53	young-ish	middle	505.33
258	53	young-ish	middle	589.33
259	67	old-ish	middle	466.00
260	86	old-ish	old-ish	1155.00
261	75	old-ish	old-ish	615.33
262	84	old-ish	old-ish	1595.33
263	67	old-ish	middle	549.67
264	64	old-ish	middle	1118.33
265	32	young-ish	young-ish	563.33
266	72	old-ish	middle	471.67
267	82	old-ish	old-ish	552.67
268	43	young-ish	young-ish	724.67
269	73	old-ish	old-ish	781.00
270	72	old-ish	middle	587.33
271	99	old-ish	old-ish	841.33
272	73	old-ish	old-ish	584.67
273	47	young-ish	middle	445.67
274	40	young-ish	young-ish	442.33
275	57	young-ish	middle	537.00
276	40	young-ish	young-ish	641.33
277	48	young-ish	middle	811.00
278	34	young-ish	young-ish	443.00
279	25	young-ish	young-ish	598.33
280	31	young-ish	young-ish	632.67
281	25	young-ish	young-ish	521.00
282	36	young-ish	young-ish	558.00
283	83	old-ish	old-ish	697.67
284	82	old-ish	old-ish	621.33
285	33	young-ish	young-ish	379.67
286	85	old-ish	old-ish	655.67
287	63	old-ish	middle	609.67
288	70	old-ish	middle	693.00
289	34	young-ish	young-ish	417.67
290	83	old-ish	old-ish	2076.00
291	27	young-ish	young-ish	521.67
292	67	old-ish	middle	457.67
293	26	young-ish	young-ish	485.00
294	43	young-ish	young-ish	483.33
295	36	young-ish	young-ish	475.67
296	70	old-ish	middle	768.67
297	72	old-ish	middle	696.33
298	85	old-ish	old-ish	916.33
299	27	young-ish	young-ish	493.33
300	67	old-ish	middle	724.33
301	79	old-ish	old-ish	649.00
302	66	old-ish	middle	623.00
303	68	old-ish	middle	718.67
304	69	old-ish	middle	767.67
305	86	old-ish	old-ish	1527.00
306	23	young-ish	young-ish	555.00
307	24	young-ish	young-ish	448.33
308	26	young-ish	young-ish	514.33
309	69	old-ish	middle	652.33
310	63	old-ish	middle	579.00
311	75	old-ish	old-ish	899.00
312	51	young-ish	middle	542.00
313	78	old-ish	old-ish	1270.67
314	54	young-ish	middle	602.33
315	83	old-ish	old-ish	1079.33
316	41	young-ish	young-ish	565.00
317	73	old-ish	old-ish	769.00
318	95	old-ish	old-ish	1341.33
319	66	old-ish	middle	746.00
320	71	old-ish	middle	550.33
321	70	old-ish	middle	797.33
322	88	old-ish	old-ish	814.67
323	55	young-ish	middle	597.33
324	67	old-ish	middle	605.00
325	41	young-ish	young-ish	578.67
326	50	young-ish	middle	545.00
327	83	old-ish	old-ish	660.67
328	81	old-ish	old-ish	604.67
329	68	old-ish	middle	601.67
330	79	old-ish	old-ish	1037.67
331	26	young-ish	young-ish	410.33
332	45	young-ish	young-ish	556.33
333	84	old-ish	old-ish	846.33
334	49	young-ish	middle	529.67
335	25	young-ish	young-ish	455.67
336	65	old-ish	middle	855.33
337	50	young-ish	middle	469.67
338	63	old-ish	middle	637.00
339	71	old-ish	middle	779.00
340	27	young-ish	young-ish	435.67
341	81	old-ish	old-ish	411.00
342	60	old-ish	middle	708.00
343	71	old-ish	middle	565.00
344	48	young-ish	middle	445.33
345	95	old-ish	old-ish	750.00
346	41	young-ish	young-ish	552.67
347	77	old-ish	old-ish	694.33
348	54	young-ish	middle	599.00
349	64	old-ish	middle	651.33
350	46	young-ish	young-ish	525.00
351	70	old-ish	middle	615.33
352	23	young-ish	young-ish	450.33
353	55	young-ish	middle	412.00
354	93	old-ish	old-ish	649.33
355	81	old-ish	old-ish	916.33

rt: mean reaction time of dominant hand in msecs
N=353 participants
Data available for non-dominant / dominant hand / feet, sex, smoker, etc.

Data modelling

Use scatterplot, tukeyboxplot, and histogram to inspect data.
Use lm for continuous predictor (linear regression), and categorical predictors (t-test, ANOVA):

lm(outcome ~ predictor, data)

Data visualisation and modelling

LMs can be evaluated on the basis of their residuals.
t-tests and ANOVAs can be translated into linear models.
Predictor can be
- continuous
- factor with two levels (t-test)
- factor with three levels or more (ANOVA).

lm(rt ~ age, data)

Age variable

summary(data$age) # continuous variable

   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
  20.00   36.00   57.00   55.03   72.00   99.00

Age variable

summary(data$age) # continuous variable

   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
  20.00   36.00   57.00   55.03   72.00   99.00

unique(data$age_2) # 2 groups

[1] old-ish   young-ish
Levels: young-ish < old-ish

Age variable

summary(data$age) # continuous variable

   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
  20.00   36.00   57.00   55.03   72.00   99.00

unique(data$age_2) # 2 groups

[1] old-ish   young-ish
Levels: young-ish < old-ish

unique(data$age_3) # 3 groups

[1] old-ish   young-ish middle   
Levels: young-ish < middle < old-ish

Histogram

Continuous predictor

model <- lm(rt ~ age, data = data)

predictor	estimate	std.error	t-value	p-value
(Intercept)	340.57	25.82	13.19	< 0.001
age	5.38	0.44	12.26	< 0.001

2 groups: t-test style model

model_2 <- lm(rt ~ age_2, data = data)

predictor	estimate	std.error	t-value	p-value
(Intercept)	644.53	9.43	68.32	< 0.001
age	146.00	13.34	10.94	< 0.001

3 groups: ANOVA-style model

model_3 <- lm(rt ~ age_3, data = data)
anova(model_3)

term	df	sumsq	meansq	F-value	p-value
age_3	2	4061908	2030954	66.81	< 0.001
Residuals	350	10640187	30401	–	–

Model evaluation

Using residuals to evaluate models:

Linearity: linear relationship between outcome and predictor(s)
Independence of residuals (remember iid)
Normality of residuals
Equality of variance of residuals

Outline

What are residuals?
Normality of residuals: skewness, kurtosis, and log-transformation
Independence of residuals: equality of variance, linearity

What are residuals?

Residuals are the unexplained (residual) variance: error in the modelling results.
Distance between observed (\(y\)) and predicted rt (\(\hat{y}\)): \(\epsilon = y - \hat{y}\)
The closer the residuals are to 0, the lower the prediction error.

What are residuals?

# Generate model predictions
data <- mutate(data, predicted = predict(model))

id	age	rt	predicted
1	84	701.67	792.46
2	37	470.67	539.62
3	62	638.67	674.11
4	85	708.00	797.84
5	73	607.33	733.29
6	65	541.67	690.25
7	30	570.67	501.96
8	49	509.33	604.18
9	83	736.67	787.08
10	58	549.67	652.59
11	25	548.00	475.07
12	88	889.00	813.98
13	62	883.67	674.11
14	88	831.67	813.98
15	27	536.33	485.82
16	60	671.67	663.35
17	80	930.00	770.95
18	24	454.67	469.69
19	27	531.33	485.82
20	52	539.33	620.32
21	27	436.67	485.82
23	74	696.33	738.67
24	57	611.67	647.21
25	23	523.67	464.31
26	39	443.33	550.38
27	51	628.67	614.94
28	59	586.33	657.97
29	24	450.33	469.69
30	27	532.67	485.82
31	26	494.00	480.44
32	76	566.33	749.43
33	62	485.00	674.11
34	39	518.00	550.38
35	24	547.67	469.69
36	57	517.67	647.21
37	64	582.00	684.87
38	64	552.67	684.87
39	88	727.00	813.98
40	37	425.33	539.62
41	52	592.00	620.32
42	58	557.67	652.59
43	57	548.00	647.21
44	89	739.00	819.36
45	21	616.67	453.55
46	27	435.00	485.82
47	62	537.00	674.11
48	70	650.33	717.15
49	39	556.67	550.38
50	85	918.67	797.84
51	44	449.67	577.28
52	73	519.33	733.29
53	44	630.33	577.28
54	31	541.00	507.34
55	71	670.67	722.53
56	27	518.33	485.82
57	23	430.00	464.31
58	54	502.33	631.07
59	83	522.33	787.08
60	46	661.00	588.04
61	30	516.33	501.96
62	42	475.67	566.52
63	55	701.00	636.45
64	33	583.67	518.10
65	71	526.00	722.53
66	81	656.67	776.33
67	63	617.33	679.49
68	33	480.33	518.10
69	53	680.33	625.70
70	85	1101.67	797.84
71	31	534.00	507.34
72	30	623.67	501.96
73	65	639.33	690.25
74	73	767.33	733.29
75	21	614.33	453.55
76	70	662.00	717.15
77	90	858.00	824.74
78	43	579.67	571.90
79	71	492.67	722.53
80	38	633.00	545.00
81	22	557.33	458.93
82	21	559.67	453.55
83	47	595.67	593.42
84	69	618.33	711.77
85	22	721.67	458.93
86	56	731.33	641.83
87	68	670.67	706.39
88	53	557.67	625.70
89	73	630.33	733.29
90	33	481.67	518.10
91	49	537.67	604.18
92	50	574.33	609.56
93	74	656.67	738.67
94	88	668.67	813.98
95	81	497.67	776.33
96	77	1241.33	754.81
97	76	571.33	749.43
98	49	537.00	604.18
99	72	597.67	727.91
100	51	784.00	614.94
101	52	441.00	620.32
102	38	663.00	545.00
103	69	725.33	711.77
104	50	534.00	609.56
105	21	409.67	453.55
106	22	441.67	458.93
107	74	550.00	738.67
108	56	614.67	641.83
109	70	835.00	717.15
110	74	1292.67	738.67
111	32	984.00	512.72
112	24	511.33	469.69
113	66	623.67	695.63
114	88	922.33	813.98
115	47	710.00	593.42
116	45	659.00	582.66
117	58	627.00	652.59
118	31	478.00	507.34
119	69	1148.33	711.77
120	86	994.33	803.22
121	21	561.33	453.55
122	23	669.67	464.31
123	48	619.67	598.80
124	20	520.33	448.17
125	59	619.33	657.97
126	79	894.00	765.57
127	96	1178.67	857.02
128	66	644.33	695.63
129	38	494.67	545.00
130	43	632.67	571.90
131	50	584.00	609.56
132	52	520.67	620.32
133	41	525.33	561.14
134	66	694.00	695.63
135	71	643.00	722.53
136	77	675.33	754.81
137	51	667.67	614.94
138	60	640.67	663.35
139	54	731.33	631.07
140	74	844.00	738.67
141	26	441.00	480.44
142	21	554.67	453.55
143	75	679.67	744.05
144	36	501.33	534.24
145	43	495.33	571.90
146	21	496.67	453.55
147	74	600.33	738.67
148	37	592.33	539.62
149	34	478.67	523.48
150	30	510.00	501.96
151	27	502.67	485.82
152	68	1030.33	706.39
153	27	471.67	485.82
154	21	544.67	453.55
155	34	536.33	523.48
156	67	859.00	701.01
157	64	666.00	684.87
158	34	431.67	523.48
159	75	1111.33	744.05
160	28	379.33	491.20
161	42	604.00	566.52
162	49	642.67	604.18
163	87	844.67	808.60
164	73	796.67	733.29
165	34	490.67	523.48
166	66	703.67	695.63
167	79	993.00	765.57
168	33	789.00	518.10
169	47	785.00	593.42
170	69	794.00	711.77
171	92	1024.67	835.50
172	74	554.67	738.67
173	84	630.33	792.46
174	71	711.00	722.53
175	54	627.00	631.07
176	82	657.67	781.71
177	89	897.67	819.36
178	26	466.00	480.44
179	64	560.67	684.87
180	59	521.33	657.97
181	69	1551.67	711.77
182	70	612.33	717.15
183	25	888.33	475.07
184	26	541.67	480.44
185	26	491.67	480.44
186	23	463.33	464.31
187	71	1000.33	722.53
188	71	587.67	722.53
189	28	386.33	491.20
190	33	510.33	518.10
191	23	468.67	464.31
192	70	744.67	717.15
193	30	594.00	501.96
194	73	667.67	733.29
195	58	616.00	652.59
196	44	460.33	577.28
197	45	479.33	582.66
198	66	431.33	695.63
199	48	558.00	598.80
200	28	469.33	491.20
201	34	544.33	523.48
202	48	508.67	598.80
203	71	519.33	722.53
204	77	928.33	754.81
205	49	530.00	604.18
206	28	520.67	491.20
207	24	430.33	469.69
208	58	477.00	652.59
209	69	684.00	711.77
210	69	545.00	711.77
211	58	563.33	652.59
212	31	451.33	507.34
213	74	832.00	738.67
214	88	711.33	813.98
215	72	952.67	727.91
216	69	617.33	711.77
217	48	591.00	598.80
218	64	574.00	684.87
219	57	511.00	647.21
220	49	471.00	604.18
221	30	459.33	501.96
222	75	538.67	744.05
223	31	542.67	507.34
224	59	614.00	657.97
226	57	593.33	647.21
227	45	536.67	582.66
228	25	436.00	475.07
229	63	607.67	679.49
230	59	628.00	657.97
231	53	427.33	625.70
232	25	477.33	475.07
233	25	482.67	475.07
234	74	547.33	738.67
235	38	452.00	545.00
236	27	537.67	485.82
237	45	548.33	582.66
238	53	474.67	625.70
239	24	583.67	469.69
240	36	461.67	534.24
241	39	532.33	550.38
242	32	397.67	512.72
243	64	991.33	684.87
244	75	755.67	744.05
245	61	877.33	668.73
246	77	841.67	754.81
247	40	538.33	555.76
248	65	566.67	690.25
249	69	629.00	711.77
250	73	605.67	733.29
251	30	524.00	501.96
252	83	475.67	787.08
253	31	407.33	507.34
254	78	645.33	760.19
255	67	646.33	701.01
256	79	389.67	765.57
257	53	505.33	625.70
258	53	589.33	625.70
259	67	466.00	701.01
260	86	1155.00	803.22
261	75	615.33	744.05
262	84	1595.33	792.46
263	67	549.67	701.01
264	64	1118.33	684.87
265	32	563.33	512.72
266	72	471.67	727.91
267	82	552.67	781.71
268	43	724.67	571.90
269	73	781.00	733.29
270	72	587.33	727.91
271	99	841.33	873.16
272	73	584.67	733.29
273	47	445.67	593.42
274	40	442.33	555.76
275	57	537.00	647.21
276	40	641.33	555.76
277	48	811.00	598.80
278	34	443.00	523.48
279	25	598.33	475.07
280	31	632.67	507.34
281	25	521.00	475.07
282	36	558.00	534.24
283	83	697.67	787.08
284	82	621.33	781.71
285	33	379.67	518.10
286	85	655.67	797.84
287	63	609.67	679.49
288	70	693.00	717.15
289	34	417.67	523.48
290	83	2076.00	787.08
291	27	521.67	485.82
292	67	457.67	701.01
293	26	485.00	480.44
294	43	483.33	571.90
295	36	475.67	534.24
296	70	768.67	717.15
297	72	696.33	727.91
298	85	916.33	797.84
299	27	493.33	485.82
300	67	724.33	701.01
301	79	649.00	765.57
302	66	623.00	695.63
303	68	718.67	706.39
304	69	767.67	711.77
305	86	1527.00	803.22
306	23	555.00	464.31
307	24	448.33	469.69
308	26	514.33	480.44
309	69	652.33	711.77
310	63	579.00	679.49
311	75	899.00	744.05
312	51	542.00	614.94
313	78	1270.67	760.19
314	54	602.33	631.07
315	83	1079.33	787.08
316	41	565.00	561.14
317	73	769.00	733.29
318	95	1341.33	851.64
319	66	746.00	695.63
320	71	550.33	722.53
321	70	797.33	717.15
322	88	814.67	813.98
323	55	597.33	636.45
324	67	605.00	701.01
325	41	578.67	561.14
326	50	545.00	609.56
327	83	660.67	787.08
328	81	604.67	776.33
329	68	601.67	706.39
330	79	1037.67	765.57
331	26	410.33	480.44
332	45	556.33	582.66
333	84	846.33	792.46
334	49	529.67	604.18
335	25	455.67	475.07
336	65	855.33	690.25
337	50	469.67	609.56
338	63	637.00	679.49
339	71	779.00	722.53
340	27	435.67	485.82
341	81	411.00	776.33
342	60	708.00	663.35
343	71	565.00	722.53
344	48	445.33	598.80
345	95	750.00	851.64
346	41	552.67	561.14
347	77	694.33	754.81
348	54	599.00	631.07
349	64	651.33	684.87
350	46	525.00	588.04
351	70	615.33	717.15
352	23	450.33	464.31
353	55	412.00	636.45
354	93	649.33	840.88
355	81	916.33	776.33

What are residuals?

# Difference between observed rt and predicted values
data <- mutate(data, residuals = rt - predicted)

id	age	rt	predicted	residuals
1	84	701.67	792.46	-90.80
2	37	470.67	539.62	-68.95
3	62	638.67	674.11	-35.45
4	85	708.00	797.84	-89.84
5	73	607.33	733.29	-125.95
6	65	541.67	690.25	-148.58
7	30	570.67	501.96	68.70
8	49	509.33	604.18	-94.84
9	83	736.67	787.08	-50.42
10	58	549.67	652.59	-102.93
11	25	548.00	475.07	72.93
12	88	889.00	813.98	75.02
13	62	883.67	674.11	209.55
14	88	831.67	813.98	17.68
15	27	536.33	485.82	50.51
16	60	671.67	663.35	8.31
17	80	930.00	770.95	159.05
18	24	454.67	469.69	-15.02
19	27	531.33	485.82	45.51
20	52	539.33	620.32	-80.98
21	27	436.67	485.82	-49.16
23	74	696.33	738.67	-42.33
24	57	611.67	647.21	-35.55
25	23	523.67	464.31	59.36
26	39	443.33	550.38	-107.05
27	51	628.67	614.94	13.73
28	59	586.33	657.97	-71.64
29	24	450.33	469.69	-19.35
30	27	532.67	485.82	46.84
31	26	494.00	480.44	13.56
32	76	566.33	749.43	-183.09
33	62	485.00	674.11	-189.11
34	39	518.00	550.38	-32.38
35	24	547.67	469.69	77.98
36	57	517.67	647.21	-129.55
37	64	582.00	684.87	-102.87
38	64	552.67	684.87	-132.20
39	88	727.00	813.98	-86.98
40	37	425.33	539.62	-114.29
41	52	592.00	620.32	-28.32
42	58	557.67	652.59	-94.93
43	57	548.00	647.21	-99.21
44	89	739.00	819.36	-80.36
45	21	616.67	453.55	163.12
46	27	435.00	485.82	-50.82
47	62	537.00	674.11	-137.11
48	70	650.33	717.15	-66.82
49	39	556.67	550.38	6.29
50	85	918.67	797.84	120.82
51	44	449.67	577.28	-127.61
52	73	519.33	733.29	-213.95
53	44	630.33	577.28	53.05
54	31	541.00	507.34	33.66
55	71	670.67	722.53	-51.86
56	27	518.33	485.82	32.51
57	23	430.00	464.31	-34.31
58	54	502.33	631.07	-128.74
59	83	522.33	787.08	-264.75
60	46	661.00	588.04	72.96
61	30	516.33	501.96	14.37
62	42	475.67	566.52	-90.85
63	55	701.00	636.45	64.55
64	33	583.67	518.10	65.56
65	71	526.00	722.53	-196.53
66	81	656.67	776.33	-119.66
67	63	617.33	679.49	-62.16
68	33	480.33	518.10	-37.77
69	53	680.33	625.70	54.64
70	85	1101.67	797.84	303.82
71	31	534.00	507.34	26.66
72	30	623.67	501.96	121.70
73	65	639.33	690.25	-50.92
74	73	767.33	733.29	34.05
75	21	614.33	453.55	160.79
76	70	662.00	717.15	-55.15
77	90	858.00	824.74	33.26
78	43	579.67	571.90	7.77
79	71	492.67	722.53	-229.86
80	38	633.00	545.00	88.00
81	22	557.33	458.93	98.41
82	21	559.67	453.55	106.12
83	47	595.67	593.42	2.25
84	69	618.33	711.77	-93.44
85	22	721.67	458.93	262.74
86	56	731.33	641.83	89.50
87	68	670.67	706.39	-35.72
88	53	557.67	625.70	-68.03
89	73	630.33	733.29	-102.95
90	33	481.67	518.10	-36.44
91	49	537.67	604.18	-66.51
92	50	574.33	609.56	-35.22
93	74	656.67	738.67	-82.00
94	88	668.67	813.98	-145.32
95	81	497.67	776.33	-278.66
96	77	1241.33	754.81	486.53
97	76	571.33	749.43	-178.09
98	49	537.00	604.18	-67.18
99	72	597.67	727.91	-130.24
100	51	784.00	614.94	169.06
101	52	441.00	620.32	-179.32
102	38	663.00	545.00	118.00
103	69	725.33	711.77	13.56
104	50	534.00	609.56	-75.56
105	21	409.67	453.55	-43.88
106	22	441.67	458.93	-17.26
107	74	550.00	738.67	-188.67
108	56	614.67	641.83	-27.17
109	70	835.00	717.15	117.85
110	74	1292.67	738.67	554.00
111	32	984.00	512.72	471.28
112	24	511.33	469.69	41.65
113	66	623.67	695.63	-71.96
114	88	922.33	813.98	108.35
115	47	710.00	593.42	116.58
116	45	659.00	582.66	76.34
117	58	627.00	652.59	-25.59
118	31	478.00	507.34	-29.34
119	69	1148.33	711.77	436.56
120	86	994.33	803.22	191.11
121	21	561.33	453.55	107.79
122	23	669.67	464.31	205.36
123	48	619.67	598.80	20.87
124	20	520.33	448.17	72.17
125	59	619.33	657.97	-38.64
126	79	894.00	765.57	128.43
127	96	1178.67	857.02	321.65
128	66	644.33	695.63	-51.30
129	38	494.67	545.00	-50.33
130	43	632.67	571.90	60.77
131	50	584.00	609.56	-25.56
132	52	520.67	620.32	-99.65
133	41	525.33	561.14	-35.81
134	66	694.00	695.63	-1.63
135	71	643.00	722.53	-79.53
136	77	675.33	754.81	-79.47
137	51	667.67	614.94	52.73
138	60	640.67	663.35	-22.69
139	54	731.33	631.07	100.26
140	74	844.00	738.67	105.33
141	26	441.00	480.44	-39.44
142	21	554.67	453.55	101.12
143	75	679.67	744.05	-64.38
144	36	501.33	534.24	-32.91
145	43	495.33	571.90	-76.57
146	21	496.67	453.55	43.12
147	74	600.33	738.67	-138.33
148	37	592.33	539.62	52.71
149	34	478.67	523.48	-44.82
150	30	510.00	501.96	8.04
151	27	502.67	485.82	16.84
152	68	1030.33	706.39	323.94
153	27	471.67	485.82	-14.16
154	21	544.67	453.55	91.12
155	34	536.33	523.48	12.85
156	67	859.00	701.01	157.99
157	64	666.00	684.87	-18.87
158	34	431.67	523.48	-91.82
159	75	1111.33	744.05	367.29
160	28	379.33	491.20	-111.87
161	42	604.00	566.52	37.48
162	49	642.67	604.18	38.49
163	87	844.67	808.60	36.06
164	73	796.67	733.29	63.38
165	34	490.67	523.48	-32.82
166	66	703.67	695.63	8.04
167	79	993.00	765.57	227.43
168	33	789.00	518.10	270.90
169	47	785.00	593.42	191.58
170	69	794.00	711.77	82.23
171	92	1024.67	835.50	189.17
172	74	554.67	738.67	-184.00
173	84	630.33	792.46	-162.13
174	71	711.00	722.53	-11.53
175	54	627.00	631.07	-4.07
176	82	657.67	781.71	-124.04
177	89	897.67	819.36	78.30
178	26	466.00	480.44	-14.44
179	64	560.67	684.87	-124.20
180	59	521.33	657.97	-136.64
181	69	1551.67	711.77	839.90
182	70	612.33	717.15	-104.82
183	25	888.33	475.07	413.27
184	26	541.67	480.44	61.22
185	26	491.67	480.44	11.22
186	23	463.33	464.31	-0.97
187	71	1000.33	722.53	277.80
188	71	587.67	722.53	-134.86
189	28	386.33	491.20	-104.87
190	33	510.33	518.10	-7.77
191	23	468.67	464.31	4.36
192	70	744.67	717.15	27.52
193	30	594.00	501.96	92.04
194	73	667.67	733.29	-65.62
195	58	616.00	652.59	-36.59
196	44	460.33	577.28	-116.95
197	45	479.33	582.66	-103.32
198	66	431.33	695.63	-264.30
199	48	558.00	598.80	-40.80
200	28	469.33	491.20	-21.87
201	34	544.33	523.48	20.85
202	48	508.67	598.80	-90.13
203	71	519.33	722.53	-203.20
204	77	928.33	754.81	173.53
205	49	530.00	604.18	-74.18
206	28	520.67	491.20	29.46
207	24	430.33	469.69	-39.35
208	58	477.00	652.59	-175.59
209	69	684.00	711.77	-27.77
210	69	545.00	711.77	-166.77
211	58	563.33	652.59	-89.26
212	31	451.33	507.34	-56.01
213	74	832.00	738.67	93.33
214	88	711.33	813.98	-102.65
215	72	952.67	727.91	224.76
216	69	617.33	711.77	-94.44
217	48	591.00	598.80	-7.80
218	64	574.00	684.87	-110.87
219	57	511.00	647.21	-136.21
220	49	471.00	604.18	-133.18
221	30	459.33	501.96	-42.63
222	75	538.67	744.05	-205.38
223	31	542.67	507.34	35.32
224	59	614.00	657.97	-43.97
226	57	593.33	647.21	-53.88
227	45	536.67	582.66	-45.99
228	25	436.00	475.07	-39.07
229	63	607.67	679.49	-71.83
230	59	628.00	657.97	-29.97
231	53	427.33	625.70	-198.36
232	25	477.33	475.07	2.27
233	25	482.67	475.07	7.60
234	74	547.33	738.67	-191.33
235	38	452.00	545.00	-93.00
236	27	537.67	485.82	51.84
237	45	548.33	582.66	-34.32
238	53	474.67	625.70	-151.03
239	24	583.67	469.69	113.98
240	36	461.67	534.24	-72.57
241	39	532.33	550.38	-18.05
242	32	397.67	512.72	-115.06
243	64	991.33	684.87	306.46
244	75	755.67	744.05	11.62
245	61	877.33	668.73	208.60
246	77	841.67	754.81	86.86
247	40	538.33	555.76	-17.43
248	65	566.67	690.25	-123.58
249	69	629.00	711.77	-82.77
250	73	605.67	733.29	-127.62
251	30	524.00	501.96	22.04
252	83	475.67	787.08	-311.42
253	31	407.33	507.34	-100.01
254	78	645.33	760.19	-114.85
255	67	646.33	701.01	-54.68
256	79	389.67	765.57	-375.90
257	53	505.33	625.70	-120.36
258	53	589.33	625.70	-36.36
259	67	466.00	701.01	-235.01
260	86	1155.00	803.22	351.78
261	75	615.33	744.05	-128.71
262	84	1595.33	792.46	802.87
263	67	549.67	701.01	-151.34
264	64	1118.33	684.87	433.46
265	32	563.33	512.72	50.61
266	72	471.67	727.91	-256.24
267	82	552.67	781.71	-229.04
268	43	724.67	571.90	152.77
269	73	781.00	733.29	47.71
270	72	587.33	727.91	-140.58
271	99	841.33	873.16	-31.83
272	73	584.67	733.29	-148.62
273	47	445.67	593.42	-147.75
274	40	442.33	555.76	-113.43
275	57	537.00	647.21	-110.21
276	40	641.33	555.76	85.57
277	48	811.00	598.80	212.20
278	34	443.00	523.48	-80.48
279	25	598.33	475.07	123.27
280	31	632.67	507.34	125.32
281	25	521.00	475.07	45.93
282	36	558.00	534.24	23.76
283	83	697.67	787.08	-89.42
284	82	621.33	781.71	-160.37
285	33	379.67	518.10	-138.44
286	85	655.67	797.84	-142.18
287	63	609.67	679.49	-69.83
288	70	693.00	717.15	-24.15
289	34	417.67	523.48	-105.82
290	83	2076.00	787.08	1288.92
291	27	521.67	485.82	35.84
292	67	457.67	701.01	-243.34
293	26	485.00	480.44	4.56
294	43	483.33	571.90	-88.57
295	36	475.67	534.24	-58.57
296	70	768.67	717.15	51.52
297	72	696.33	727.91	-31.58
298	85	916.33	797.84	118.49
299	27	493.33	485.82	7.51
300	67	724.33	701.01	23.32
301	79	649.00	765.57	-116.57
302	66	623.00	695.63	-72.63
303	68	718.67	706.39	12.28
304	69	767.67	711.77	55.90
305	86	1527.00	803.22	723.78
306	23	555.00	464.31	90.69
307	24	448.33	469.69	-21.35
308	26	514.33	480.44	33.89
309	69	652.33	711.77	-59.44
310	63	579.00	679.49	-100.49
311	75	899.00	744.05	154.95
312	51	542.00	614.94	-72.94
313	78	1270.67	760.19	510.48
314	54	602.33	631.07	-28.74
315	83	1079.33	787.08	292.25
316	41	565.00	561.14	3.86
317	73	769.00	733.29	35.71
318	95	1341.33	851.64	489.69
319	66	746.00	695.63	50.37
320	71	550.33	722.53	-172.20
321	70	797.33	717.15	80.18
322	88	814.67	813.98	0.68
323	55	597.33	636.45	-39.12
324	67	605.00	701.01	-96.01
325	41	578.67	561.14	17.53
326	50	545.00	609.56	-64.56
327	83	660.67	787.08	-126.42
328	81	604.67	776.33	-171.66
329	68	601.67	706.39	-104.72
330	79	1037.67	765.57	272.10
331	26	410.33	480.44	-70.11
332	45	556.33	582.66	-26.32
333	84	846.33	792.46	53.87
334	49	529.67	604.18	-74.51
335	25	455.67	475.07	-19.40
336	65	855.33	690.25	165.08
337	50	469.67	609.56	-139.89
338	63	637.00	679.49	-42.49
339	71	779.00	722.53	56.47
340	27	435.67	485.82	-50.16
341	81	411.00	776.33	-365.33
342	60	708.00	663.35	44.65
343	71	565.00	722.53	-157.53
344	48	445.33	598.80	-153.46
345	95	750.00	851.64	-101.64
346	41	552.67	561.14	-8.47
347	77	694.33	754.81	-60.47
348	54	599.00	631.07	-32.07
349	64	651.33	684.87	-33.54
350	46	525.00	588.04	-63.04
351	70	615.33	717.15	-101.82
352	23	450.33	464.31	-13.97
353	55	412.00	636.45	-224.45
354	93	649.33	840.88	-191.55
355	81	916.33	776.33	140.01

What are residuals?

# Difference between observed rt and predicted values
data <- mutate(data, residuals = rt - predicted)

# or simply do this to get the residuals
data <- mutate(data, residuals = residuals(model))

Normality of residuals

# Difference between observed rt and predicted values
data <- mutate(data, residuals = rt - predicted)

Distributed around 0
Right / positive skew shows some violation of normality assumption
rts are 0 bound and known to have a heavy right tail (see e.g. Baayen, 2008)

Normality: Skewness

Negatively skewed: skew < -2
Normal: skew \(\approx\) 0
Positively skewed: skew > 2

What can we do about positive skew?

Logarithmic transformation is routinely used in the literature, especially for rts, to correct positive skew (see e.g. Baayen, 2008).
Can also be used for non-linear relationship.

Linear / arithmetic scale

Distances between adjacent values must be the same, i.e. \(\pm 1\) on a linear scale.
Distances between 1 inch and 2 inch is 1 etc.

Logarithmic scale

Distance between units is going down as values go up.
Granularity for small while also including large numbers.
Basis 10 (\(\cdot\) or \(\div\) by 10): 0.1, 1, 10, 100

Linear vs logarithmic scale.

Logarithmic scale: example

Astronomy: Roen Kelly

Transform from rt to log rt

data <- mutate(data, log_rt = log(rt))

Refit model with log rt

# Old model on rt
model <- lm(rt ~ age, data = data)

predictor	estimate	std.error	t-value	p-value
(Intercept)	340.57	25.82	13.19	< 0.001
age	5.38	0.44	12.26	< 0.001

# New model on log rt
log_model <- lm(log_rt ~ age, data = data)

predictor	estimate	std.error	t-value	p-value
(Intercept)	5.99	0.03	187.07	< 0.001
age	0.01	0.00	14.12	< 0.001

Compare model residuals

Assess skewness of residuals

library(moments)
# Skew of rts
skewness(residuals(model))

[1] 2.53479

# Skew of log rts
skewness(residuals(log_model))

[1] 0.9311563

Normality: Kurtosis

Skewness is about horizontal shape of the distribution
Kurtosis is about vertical shape of the distribution

Normality: Kurtosis

library(moments)
# Kurtosis of rt model
kurtosis(residuals(model))

[1] 15.39679

# Kurtosis of log rt model
kurtosis(residuals(log_model))

[1] 5.538062

Independence of residuals

Must be independent and identically distributed (iid)
Observation must be independent of previous one.
No correlation between (adjacent) residuals.

id	age	rt	residuals
1	84	701.67	-90.80
2	37	470.67	-68.95
3	62	638.67	-35.45
4	85	708.00	-89.84
5	73	607.33	-125.95
6	65	541.67	-148.58
7	30	570.67	68.70
8	49	509.33	-94.84
9	83	736.67	-50.42
10	58	549.67	-102.93
11	25	548.00	72.93
12	88	889.00	75.02
13	62	883.67	209.55
14	88	831.67	17.68
15	27	536.33	50.51
16	60	671.67	8.31
17	80	930.00	159.05
18	24	454.67	-15.02
19	27	531.33	45.51
20	52	539.33	-80.98
21	27	436.67	-49.16
23	74	696.33	-42.33
24	57	611.67	-35.55
25	23	523.67	59.36
26	39	443.33	-107.05
27	51	628.67	13.73
28	59	586.33	-71.64
29	24	450.33	-19.35
30	27	532.67	46.84
31	26	494.00	13.56
32	76	566.33	-183.09
33	62	485.00	-189.11
34	39	518.00	-32.38
35	24	547.67	77.98
36	57	517.67	-129.55
37	64	582.00	-102.87
38	64	552.67	-132.20
39	88	727.00	-86.98
40	37	425.33	-114.29
41	52	592.00	-28.32
42	58	557.67	-94.93
43	57	548.00	-99.21
44	89	739.00	-80.36
45	21	616.67	163.12
46	27	435.00	-50.82
47	62	537.00	-137.11
48	70	650.33	-66.82
49	39	556.67	6.29
50	85	918.67	120.82
51	44	449.67	-127.61
52	73	519.33	-213.95
53	44	630.33	53.05
54	31	541.00	33.66
55	71	670.67	-51.86
56	27	518.33	32.51
57	23	430.00	-34.31
58	54	502.33	-128.74
59	83	522.33	-264.75
60	46	661.00	72.96
61	30	516.33	14.37
62	42	475.67	-90.85
63	55	701.00	64.55
64	33	583.67	65.56
65	71	526.00	-196.53
66	81	656.67	-119.66
67	63	617.33	-62.16
68	33	480.33	-37.77
69	53	680.33	54.64
70	85	1101.67	303.82
71	31	534.00	26.66
72	30	623.67	121.70
73	65	639.33	-50.92
74	73	767.33	34.05
75	21	614.33	160.79
76	70	662.00	-55.15
77	90	858.00	33.26
78	43	579.67	7.77
79	71	492.67	-229.86
80	38	633.00	88.00
81	22	557.33	98.41
82	21	559.67	106.12
83	47	595.67	2.25
84	69	618.33	-93.44
85	22	721.67	262.74
86	56	731.33	89.50
87	68	670.67	-35.72
88	53	557.67	-68.03
89	73	630.33	-102.95
90	33	481.67	-36.44
91	49	537.67	-66.51
92	50	574.33	-35.22
93	74	656.67	-82.00
94	88	668.67	-145.32
95	81	497.67	-278.66
96	77	1241.33	486.53
97	76	571.33	-178.09
98	49	537.00	-67.18
99	72	597.67	-130.24
100	51	784.00	169.06
101	52	441.00	-179.32
102	38	663.00	118.00
103	69	725.33	13.56
104	50	534.00	-75.56
105	21	409.67	-43.88
106	22	441.67	-17.26
107	74	550.00	-188.67
108	56	614.67	-27.17
109	70	835.00	117.85
110	74	1292.67	554.00
111	32	984.00	471.28
112	24	511.33	41.65
113	66	623.67	-71.96
114	88	922.33	108.35
115	47	710.00	116.58
116	45	659.00	76.34
117	58	627.00	-25.59
118	31	478.00	-29.34
119	69	1148.33	436.56
120	86	994.33	191.11
121	21	561.33	107.79
122	23	669.67	205.36
123	48	619.67	20.87
124	20	520.33	72.17
125	59	619.33	-38.64
126	79	894.00	128.43
127	96	1178.67	321.65
128	66	644.33	-51.30
129	38	494.67	-50.33
130	43	632.67	60.77
131	50	584.00	-25.56
132	52	520.67	-99.65
133	41	525.33	-35.81
134	66	694.00	-1.63
135	71	643.00	-79.53
136	77	675.33	-79.47
137	51	667.67	52.73
138	60	640.67	-22.69
139	54	731.33	100.26
140	74	844.00	105.33
141	26	441.00	-39.44
142	21	554.67	101.12
143	75	679.67	-64.38
144	36	501.33	-32.91
145	43	495.33	-76.57
146	21	496.67	43.12
147	74	600.33	-138.33
148	37	592.33	52.71
149	34	478.67	-44.82
150	30	510.00	8.04
151	27	502.67	16.84
152	68	1030.33	323.94
153	27	471.67	-14.16
154	21	544.67	91.12
155	34	536.33	12.85
156	67	859.00	157.99
157	64	666.00	-18.87
158	34	431.67	-91.82
159	75	1111.33	367.29
160	28	379.33	-111.87
161	42	604.00	37.48
162	49	642.67	38.49
163	87	844.67	36.06
164	73	796.67	63.38
165	34	490.67	-32.82
166	66	703.67	8.04
167	79	993.00	227.43
168	33	789.00	270.90
169	47	785.00	191.58
170	69	794.00	82.23
171	92	1024.67	189.17
172	74	554.67	-184.00
173	84	630.33	-162.13
174	71	711.00	-11.53
175	54	627.00	-4.07
176	82	657.67	-124.04
177	89	897.67	78.30
178	26	466.00	-14.44
179	64	560.67	-124.20
180	59	521.33	-136.64
181	69	1551.67	839.90
182	70	612.33	-104.82
183	25	888.33	413.27
184	26	541.67	61.22
185	26	491.67	11.22
186	23	463.33	-0.97
187	71	1000.33	277.80
188	71	587.67	-134.86
189	28	386.33	-104.87
190	33	510.33	-7.77
191	23	468.67	4.36
192	70	744.67	27.52
193	30	594.00	92.04
194	73	667.67	-65.62
195	58	616.00	-36.59
196	44	460.33	-116.95
197	45	479.33	-103.32
198	66	431.33	-264.30
199	48	558.00	-40.80
200	28	469.33	-21.87
201	34	544.33	20.85
202	48	508.67	-90.13
203	71	519.33	-203.20
204	77	928.33	173.53
205	49	530.00	-74.18
206	28	520.67	29.46
207	24	430.33	-39.35
208	58	477.00	-175.59
209	69	684.00	-27.77
210	69	545.00	-166.77
211	58	563.33	-89.26
212	31	451.33	-56.01
213	74	832.00	93.33
214	88	711.33	-102.65
215	72	952.67	224.76
216	69	617.33	-94.44
217	48	591.00	-7.80
218	64	574.00	-110.87
219	57	511.00	-136.21
220	49	471.00	-133.18
221	30	459.33	-42.63
222	75	538.67	-205.38
223	31	542.67	35.32
224	59	614.00	-43.97
226	57	593.33	-53.88
227	45	536.67	-45.99
228	25	436.00	-39.07
229	63	607.67	-71.83
230	59	628.00	-29.97
231	53	427.33	-198.36
232	25	477.33	2.27
233	25	482.67	7.60
234	74	547.33	-191.33
235	38	452.00	-93.00
236	27	537.67	51.84
237	45	548.33	-34.32
238	53	474.67	-151.03
239	24	583.67	113.98
240	36	461.67	-72.57
241	39	532.33	-18.05
242	32	397.67	-115.06
243	64	991.33	306.46
244	75	755.67	11.62
245	61	877.33	208.60
246	77	841.67	86.86
247	40	538.33	-17.43
248	65	566.67	-123.58
249	69	629.00	-82.77
250	73	605.67	-127.62
251	30	524.00	22.04
252	83	475.67	-311.42
253	31	407.33	-100.01
254	78	645.33	-114.85
255	67	646.33	-54.68
256	79	389.67	-375.90
257	53	505.33	-120.36
258	53	589.33	-36.36
259	67	466.00	-235.01
260	86	1155.00	351.78
261	75	615.33	-128.71
262	84	1595.33	802.87
263	67	549.67	-151.34
264	64	1118.33	433.46
265	32	563.33	50.61
266	72	471.67	-256.24
267	82	552.67	-229.04
268	43	724.67	152.77
269	73	781.00	47.71
270	72	587.33	-140.58
271	99	841.33	-31.83
272	73	584.67	-148.62
273	47	445.67	-147.75
274	40	442.33	-113.43
275	57	537.00	-110.21
276	40	641.33	85.57
277	48	811.00	212.20
278	34	443.00	-80.48
279	25	598.33	123.27
280	31	632.67	125.32
281	25	521.00	45.93
282	36	558.00	23.76
283	83	697.67	-89.42
284	82	621.33	-160.37
285	33	379.67	-138.44
286	85	655.67	-142.18
287	63	609.67	-69.83
288	70	693.00	-24.15
289	34	417.67	-105.82
290	83	2076.00	1288.92
291	27	521.67	35.84
292	67	457.67	-243.34
293	26	485.00	4.56
294	43	483.33	-88.57
295	36	475.67	-58.57
296	70	768.67	51.52
297	72	696.33	-31.58
298	85	916.33	118.49
299	27	493.33	7.51
300	67	724.33	23.32
301	79	649.00	-116.57
302	66	623.00	-72.63
303	68	718.67	12.28
304	69	767.67	55.90
305	86	1527.00	723.78
306	23	555.00	90.69
307	24	448.33	-21.35
308	26	514.33	33.89
309	69	652.33	-59.44
310	63	579.00	-100.49
311	75	899.00	154.95
312	51	542.00	-72.94
313	78	1270.67	510.48
314	54	602.33	-28.74
315	83	1079.33	292.25
316	41	565.00	3.86
317	73	769.00	35.71
318	95	1341.33	489.69
319	66	746.00	50.37
320	71	550.33	-172.20
321	70	797.33	80.18
322	88	814.67	0.68
323	55	597.33	-39.12
324	67	605.00	-96.01
325	41	578.67	17.53
326	50	545.00	-64.56
327	83	660.67	-126.42
328	81	604.67	-171.66
329	68	601.67	-104.72
330	79	1037.67	272.10
331	26	410.33	-70.11
332	45	556.33	-26.32
333	84	846.33	53.87
334	49	529.67	-74.51
335	25	455.67	-19.40
336	65	855.33	165.08
337	50	469.67	-139.89
338	63	637.00	-42.49
339	71	779.00	56.47
340	27	435.67	-50.16
341	81	411.00	-365.33
342	60	708.00	44.65
343	71	565.00	-157.53
344	48	445.33	-153.46
345	95	750.00	-101.64
346	41	552.67	-8.47
347	77	694.33	-60.47
348	54	599.00	-32.07
349	64	651.33	-33.54
350	46	525.00	-63.04
351	70	615.33	-101.82
352	23	450.33	-13.97
353	55	412.00	-224.45
354	93	649.33	-191.55
355	81	916.33	140.01

Independence of residuals (see Faraway, 2015, p. 74)

Independence of residuals

Plot residuals across ppt id, predictor, predicted data
What are we looking for?
Dependencies:
- linearity, independence violations
- For any value on the x-axis, mean of residuals should be roughly 0.
Equality of variance:
- For any value on the x-axis, the spread of the residuals should be about the same (constant variance).

Independence of residuals

Equality of variance

Homogeneity (same) vs heterogeneity (different)
Variance = SD\(^2\): deviations between observations and their mean
More diverse groups have a larger variance
Formal tests:
- Levene (between groups)
- Mauchly (within groups)
- Breusch–Pagan (continuous predictors)
We will use visual inspection (for now).

Equality of variance

# Refit t-test style lm with log rt
log_model_2 <- lm(log_rt ~ age_2, data = data)

Equality of variance

Linearity

Linear relationship between outcome and predictor variable
Relationship can be described with a straight line
Increase is additive.

Linearity: categorical predictor

No problem (ever): no reason to assume anything but a linear relationship.

Linearity: continuous predictor

Linearity: mean of residuals should be 0 at any age.
Divided age into 20 equal sized bins.
Calculated the mean of the residuals for each bin.

Linearity: continuous predictor

Epilogue

Summary

Our models on gaming data published in Blomkvist et al. (2017) showed some violations.
Linearity: potential linearity issues for age as continuous predictor
Independence: no real concerns
Normality: positive skew and kurtosis corrected using log transformation
Equality of variance: larger variance in older participants

Learning outcomes

What are residuals.
How do I fit regression models with categorical predictors (akin to t-tests and ANOVAs).
Evaluate statistical models on the basis of their unexplained variance.
We will get back to these three points and the Blomkvist et al. (2017) data set in the workshop.

Homework

Blomkvist et al. (2017) performed a range of analyses.
The authors addressed that the rts were not normal distributed in different ways.
But also, they performed a t-test on the raw rts.
Discuss on Teams:
- How did the authors address the violation of normality?
- What could be a consequence of the normality violation for the results of their t-test?

Useful textbook resources

Faraway (2015) chapter 6 (with R code)
Fox & Weisberg (2018) chapter 8.1 – 8.5 (with R code)
Chatterjee & Hadi (2012) chapter 4
Montgomery et al. (2012) chapter 4
Also check out this walkthrough, this online chapter, and Chapter 12.3 here.

R packages

library(psyntur): data visualisation
library(tidyverse): data processing
library(moments): testing skewness and kurtosis

References

Baayen, R. H. (2008). Analyzing linguistic data. A practical introduction to statistics using R. Cambridge University Press.

Blomkvist, A. W., Eika, F., Rahbek, M. T., Eikhof, K. D., Hansen, M. D., Søndergaard, M., Ryg, J., Andersen, S., & Jørgensen, M. G. (2017). Reference data on reaction time and aging using the Nintendo Wii Balance Board: A cross-sectional study of 354 subjects from 20 to 99 years of age. PLoS One, 12(12), e0189598. https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0189598

Chatterjee, S., & Hadi, A. S. (2012). Regression analysis by example (Vol. 5). John Wiley & Sons.

Faraway, J. J. (2015). Linear models with R (Vol. 2). CRC press.

Fox, J., & Weisberg, S. (2018). An R companion to applied regression (Vol. 3). Sage publications.

Montgomery, D. C., Peck, E. A., & Vining, G. G. (2012). Introduction to linear regression analysis (Vol. 5). John Wiley & Sons.

Rouder, J. N., Morey, R. D., & Wagenmakers, E.-J. (2016). The interplay between subjectivity, statistical practice, and psychological science. Collabra, 2(1), 1–12.