g-loading by subtest

Introduction

To settle the online debate on which subtest of intelligence measures it the best, I collected 234 effect sizes, representing 430,000 individuals. I looked in several public datasets (e.g. NLSY) and a few meta-analyses (e.g.  Jensen 1985) to speed up the process.

I excluded any study that had less than 100 individuals or six subtests, had no g-loadings, did not use univariate models, or overlapped with other samples. I then searched for publications by typing (WAIS/Weschler/Woodcock Johnson/Stanford Binet) + factor analysis.

First, sample size + k

testn <- unique(jg %>% select(n, source))
sum(testn$n)

[1] 427596

nrow(testn)

[1] 18

Data-analysis

Median by study. No adjustments

means <- jg %>% group_by(Macrogroup) %>% summarise(median=median(g.loading, na.rm=T)) %>% arrange(-median)
print(means, n=43)

Mixed-effects meta-analysis to adjust for the effects of individual datasets. The effects of each dataset relative to the reference are fairly split between positive and negative, meaning that the arbitrary choice is probably not biasing the overall mean.

jg$se <- 1/sqrt(jg$n)
meta <- rma(data=jg, yi=g.loading, sei=se, mods = ~ source + Macrogroup)
summary(meta)

Mixed-Effects Model (k = 233; tau^2 estimator: REML)

   logLik   deviance        AIC        BIC       AICc   
 193.4114  -386.8228  -294.8228  -145.9465  -264.1561   

tau^2 (estimated amount of residual heterogeneity):     0.0066 (SE = 0.0008)
tau (square root of estimated tau^2 value):             0.0809
I^2 (residual heterogeneity / unaccounted variability): 99.30%
H^2 (unaccounted variability / sampling variability):   143.26
R^2 (amount of heterogeneity accounted for):            66.13%

Test for Residual Heterogeneity:
QE(df = 188) = 31568.8888, p-val < .0001

Test of Moderators (coefficients 2:45):
QM(df = 44) = 448.5662, p-val < .0001

Model Results:

---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Cleaning up the means and posting them.

testvector <- data.frame(Macrogroup=names(coefficients(meta)), gloadings=as.numeric(coefficients(meta))+0.6695, SE=meta$se)
testvector$Macrogroup[18] <- 'Macrogroup2D spatial ability'
testvector$gloadings[18] <- 0.6695
testvector$Macrogroup[2:length(testvector$Macrogroup)] <- sapply(testvector$Macrogroup[2:length(testvector$Macrogroup)], function(x) substr(x, start = 11, stop = nchar(x)))

testvector <- testvector[18:45, ] %>% arrange(-gloadings)
testvector[, 2:3] <- round(testvector[, 2:3], 3)

testvector

Discussion

It looks like reading comprehension came out on top. It makes sense, as it’s measuring a range of abilities indirectly (e.g. memory, verbal ability, ability to synthesize and connect information). Verbal ability seems to beat mathematical ability slightly, which is interesting, because mathematical ability is notoriously easy to practice, which actually suggests that mathematical ability may in fact be a truly better measurement of intelligence. Dexterity and processing speed stand out as rather weak measurements of intelligence.