Here we analyze the by-item median correct RTs and accuracies provided from the original Chierchia et al. (2019) study of the MaRs-IB, in which 659 adults, adolescents, and children had 8 min to complete as many of the 80 MaRs-IB items as possible. Note that the items were presented in a fixed order, and many participants did not reach the end of the test, resulting in several items being tested very few times (e.g., <10 participants completed items 65-80, so we exclude these).

Accuracy and Correct Response Time by Age Group

16 easiest items in are near/at ceiling: 1, 2, 3, 4, 5, 7, 8, 9, 32, 33, 38, 41, 43, 48, 57, 68. Let’s include just a couple of these as practice trials?

item_age %>% filter(Group!="total", Item < 65) %>% group_by(Group) %>% 
  summarise(RT = mean(RT.median.corr, na.rm=T),
            RT.IQR = mean(RT.IQR.corr, na.rm=T), # IES = Inverse Efficiency
            accuracy = mean(Mean.correct, na.rm=T), 
            N = n())
## # A tibble: 4 × 5
##   Group    RT RT.IQR accuracy     N
##   <chr> <dbl>  <dbl>    <dbl> <int>
## 1 Ad    9285.  5487.     64.0    64
## 2 MA    7153.  5121.     50.9    64
## 3 OA    8205.  4779.     55.3    64
## 4 YA    6863.  5226.     44.1    64

Adolescents took ~9.3 seconds to correctly solve a problem, and had an overall accuracy of 64%. OAs were the next slowest, taking 8.2s (55% accuracy), while MAs took 7.2s (accuracy 59%) and YAs were the fastest (6.9s) with only a 44% accuracy.

Accuracy and Correct RTs by Item

Item accuracy ranges from 13-100 percent (for 58 items with >20 responses), with median correct RTs ranging from 1.3 to 18.9 seconds. The dimensionality score (range: 1-8) proposed by Chierchia et al. captures item difficulty fairly well, being associated with both accuracy (r=-.48) and with median correct RT (r=.26), while accuracy and median correct RT are not strongly associated (r=-.05).

item_shape <- item_shape %>%
  left_join(item_dims) 
## Joining with `by = join_by(Item)`
it_d <- item_shape %>% filter(Group=="total", N>20)
# cor.test(it_d$Mean.correct, it_d$dim_score) # -.48 higher dimensionality, lower accuracy
# cor.test(it_d$RT.median.corr, it_d$dim_score) # .26 higher dimensionality, slower RT
# cor.test(it_d$Mean.correct, it_d$RT.median.corr) # -.05

range(it_d$Mean.correct) # 13 to 100 percent accuracy
## [1]  13 100
range(it_d$RT.median.corr) # 1.3 to 18.9 seconds
## [1]  1286 18898
item_shape %>% filter(Group=="total", N>20) %>%
  group_by(Item, dim_score) %>% 
  summarise(RT = mean(RT.median.corr, na.rm=T),
            RT.IQR = mean(RT.IQR.corr, na.rm=T), # IES = Inverse Efficiency
            accuracy = mean(Mean.correct, na.rm=T)) %>% 
  arrange(desc(accuracy)) %>% kableExtra::kable()
## `summarise()` has grouped output by 'Item'. You can override using the
## `.groups` argument.
Item dim_score RT RT.IQR accuracy
5 1 2528 1381 100
4 1 3763 1710 99
3 1 2464 1188 98
7 1 5184 2923 98
1 2 2412 1579 97
2 1 4447 2513 97
9 1 5899 3017 94
16 4 7707 6052 89
8 1 7655 5596 85
41 1 4248 2618 84
38 1 4240 2509 83
19 2 7902 5100 82
22 2 7713 4963 82
23 4 9847 6226 81
6 2 9428 7129 78
25 2 9358 7050 77
20 2 12114 7182 67
29 7 11622 8504 62
30 7 9362 5374 60
33 1 6414 3698 60
11 3 13966 9770 59
17 4 13944 10089 59
28 2 13086 7615 57
15 3 13600 11373 56
32 1 8090 6122 56
31 3 11404 7901 55
27 3 11507 8290 54
10 3 15262 10196 53
13 3 17343 10754 49
21 7 16676 9658 49
49 2 4181 2674 47
40 4 5020 4839 44
57 1 4151 2261 43
58 3 3836 2070 43
35 8 8321 8830 42
18 3 15692 11098 41
24 7 15723 11576 41
39 2 5448 7342 41
46 6 4185 5728 38
14 5 18898 12920 37
34 3 8640 8548 37
50 2 4021 4195 37
52 5 1757 2072 37
26 5 17140 10786 36
36 6 6836 8263 36
37 3 8496 9960 36
55 6 2770 3133 36
43 1 4720 4510 33
45 7 6316 7412 33
48 1 2942 6569 33
51 2 5712 3462 29
42 4 5674 6414 27
12 4 16342 12690 25
47 2 5035 7364 21
53 4 1286 1058 20
56 3 1726 1443 20
44 6 2424 4126 19
54 6 1612 1382 13